2024-04-06T03:17:41.141 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-06T03:17:41.141 DEBUG:teuthology.run:Teuthology command: teuthology --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944 --name teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --verbose -- /home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944/orig.config.yaml 2024-04-06T03:17:41.192 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-06T03:17:41.290 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944 branch: squid description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7641944' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 584a67ccb9eb224c5312dccd96a82e67b0645518 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 5609 sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 sleep_before_teardown: 0 subset: 32/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_584a67ccb9eb224c5312dccd96a82e67b0645518/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 584a67ccb9eb224c5312dccd96a82e67b0645518 targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBP6SMWH+MXi6ihK37d/buMQ5Cj8s0JaeT5CJ0ALb5Dbkd42VTFVza4obvVBO52QC1WMGi8VAbe0+QrbvRwpL8= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5WSDxOO7mZqpUabI27URiunB+kqa+EEogb7FWfw54oyIZo6dzpfK23aXc6x9GGFPahLiwO82CZZOIRr99ALdU= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /home/teuthology/src/git.ceph.com_ceph_squid/qa/suites/orch/cephadm/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-05_21:08:03 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-06T03:17:41.290 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_584a67ccb9eb224c5312dccd96a82e67b0645518/qa; will attempt to use it 2024-04-06T03:17:41.291 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_584a67ccb9eb224c5312dccd96a82e67b0645518/qa/tasks 2024-04-06T03:17:41.291 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-06T03:17:41.292 INFO:teuthology.task.internal:Checking packages... 2024-04-06T03:17:41.315 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9f192350ba5fe6d0a365b251a8c2af61f56a5fa6' 2024-04-06T03:17:41.316 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-06T03:17:41.316 INFO:teuthology.packaging:ref: None 2024-04-06T03:17:41.316 INFO:teuthology.packaging:tag: None 2024-04-06T03:17:41.316 INFO:teuthology.packaging:branch: squid 2024-04-06T03:17:41.316 INFO:teuthology.packaging:sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:17:41.316 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-06T03:17:41.615 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2041.g584a67cc 2024-04-06T03:17:41.616 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-06T03:17:41.623 INFO:teuthology.task.internal:no buildpackages task found 2024-04-06T03:17:41.623 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-06T03:17:41.629 INFO:teuthology.task.internal:Saving configuration 2024-04-06T03:17:41.648 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-06T03:17:41.654 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-06T03:17:41.680 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-06 03:12:52.092289', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBP6SMWH+MXi6ihK37d/buMQ5Cj8s0JaeT5CJ0ALb5Dbkd42VTFVza4obvVBO52QC1WMGi8VAbe0+QrbvRwpL8='} 2024-04-06T03:17:41.704 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-06 03:12:52.090642', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5WSDxOO7mZqpUabI27URiunB+kqa+EEogb7FWfw54oyIZo6dzpfK23aXc6x9GGFPahLiwO82CZZOIRr99ALdU='} 2024-04-06T03:17:41.704 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-06T03:17:41.710 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-06T03:17:41.710 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-06T03:17:41.710 INFO:teuthology.run_tasks:Running task console_log... 2024-04-06T03:17:41.783 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7eff5c1380d0>, signals=[15]) 2024-04-06T03:17:41.784 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-06T03:17:41.791 INFO:teuthology.task.internal:Opening connections... 2024-04-06T03:17:41.792 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:17:41.793 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-06T03:17:41.885 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-04-06T03:17:41.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-06T03:17:41.972 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-06T03:17:41.979 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-04-06T03:17:41.996 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-04-06T03:17:41.997 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-04-06T03:17:42.056 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-04-06T03:17:42.056 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-04-06T03:17:42.056 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-04-06T03:17:42.056 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-04-06T03:17:42.056 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-04-06T03:17:42.056 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-04-06T03:17:42.056 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-06T03:17:42.056 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-04-06T03:17:42.057 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-04-06T03:17:42.057 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-06T03:17:42.057 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-04-06T03:17:42.057 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-06T03:17:42.057 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-06T03:17:42.057 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-06T03:17:42.057 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-04-06T03:17:42.085 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-04-06T03:17:42.103 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-04-06T03:17:42.103 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-04-06T03:17:42.159 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-06T03:17:42.160 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-06T03:17:42.161 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-04-06T03:17:42.184 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-06T03:17:42.193 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-06T03:17:42.198 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-06T03:17:42.198 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-04-06T03:17:42.200 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-04-06T03:17:42.216 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-06T03:17:42.222 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-06T03:17:42.222 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-04-06T03:17:42.260 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-04-06T03:17:42.515 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-06T03:17:42.575 INFO:teuthology.run_tasks:Running task kernel... 2024-04-06T03:17:42.588 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-06T03:17:42.588 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-06T03:17:42.589 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-06T03:17:42.589 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-06T03:17:42.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:17:42.606 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-06T03:17:42.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:17:42.612 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-04-06T03:17:42.667 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-361.el9.x86_64 2024-04-06T03:17:42.667 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.14.0-361.el9.x86_64 2024-04-06T03:17:42.667 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum install -y kernel 2024-04-06T03:17:42.668 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-361.el9.x86_64 2024-04-06T03:17:42.668 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.14.0-361.el9.x86_64 2024-04-06T03:17:42.669 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum install -y kernel 2024-04-06T03:17:44.776 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - AppStream 19 MB/s | 8.1 MB 00:00 2024-04-06T03:17:45.028 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-04-06T03:17:46.402 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - BaseOS 8.4 MB/s | 2.5 MB 00:00 2024-04-06T03:17:46.645 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - BaseOS 8.2 MB/s | 2.5 MB 00:00 2024-04-06T03:17:47.309 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-04-06T03:17:47.511 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.0 MB 00:00 2024-04-06T03:17:49.790 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-06T03:17:51.360 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-04-06T03:17:54.489 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.1 MB 00:01 2024-04-06T03:17:55.493 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-04-06T03:17:55.914 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-06T03:17:56.666 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 34 MB/s | 21 MB 00:00 2024-04-06T03:17:57.061 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-06T03:17:57.788 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 35 MB/s | 21 MB 00:00 2024-04-06T03:18:01.774 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-06T03:18:02.871 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-04-06T03:18:03.897 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:18:03.898 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.899 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:18:03.899 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.899 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:18:03.899 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.899 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:18:03.899 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.899 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:18:03.899 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.899 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:18:03.900 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.900 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:18:03.900 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.900 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:18:03.900 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.900 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:18:03.900 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.901 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:18:03.901 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.901 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:18:03.901 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.901 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:18:03.901 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.902 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:18:03.902 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.902 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:18:03.902 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.902 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:18:03.902 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.903 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:18:03.903 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.903 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:18:03.903 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.903 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:18:03.903 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.904 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:18:03.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.904 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:18:03.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.904 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:18:03.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:18:03.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:18:03.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:18:03.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:18:03.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.906 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:18:03.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.906 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:18:03.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.906 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:18:03.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.907 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:18:03.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.907 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:18:03.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.907 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:18:03.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.908 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:18:03.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.908 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:18:03.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.909 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:18:03.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.909 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:18:03.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.910 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:18:03.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.911 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.911 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:18:03.911 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.911 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.911 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.911 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.912 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:18:03.912 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.912 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.912 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.912 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.912 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:18:03.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.913 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:18:03.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.914 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:18:03.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.915 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:18:03.915 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.348 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-06T03:18:04.348 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-06T03:18:04.462 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:18:04.473 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:18:04.473 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-04-06T03:18:04.473 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:18:04.473 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:18:04.473 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-04-06T03:18:04.473 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-06T03:18:04.474 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-04-06T03:18:04.474 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-04-06T03:18:04.474 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-04-06T03:18:04.474 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:18:04.474 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:18:04.474 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:18:04.474 INFO:teuthology.orchestra.run.smithi094.stdout:Install 4 Packages 2024-04-06T03:18:04.474 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:18:04.475 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 76 M 2024-04-06T03:18:04.475 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 123 M 2024-04-06T03:18:04.475 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:18:04.685 INFO:teuthology.orchestra.run.smithi094.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 379 kB/s | 66 kB 00:00 2024-04-06T03:18:04.986 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:18:04.986 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:18:04.986 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:18:04.986 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.986 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:18:04.986 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.986 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:18:04.986 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.986 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.987 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:18:04.988 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:18:04.989 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.990 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.991 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.992 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:18:04.993 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.994 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.994 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:18:04.994 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:18:05.419 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-06T03:18:05.419 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-06T03:18:05.532 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:18:05.537 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:18:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2024-04-06T03:18:05.537 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:18:05.537 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:18:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-04-06T03:18:05.537 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-06T03:18:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-04-06T03:18:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-04-06T03:18:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-04-06T03:18:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:18:05.538 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:18:05.538 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:18:05.538 INFO:teuthology.orchestra.run.smithi136.stdout:Install 4 Packages 2024-04-06T03:18:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:18:05.538 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 76 M 2024-04-06T03:18:05.539 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 123 M 2024-04-06T03:18:05.539 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:18:06.038 INFO:teuthology.orchestra.run.smithi136.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 182 kB/s | 66 kB 00:00 2024-04-06T03:18:06.143 INFO:teuthology.orchestra.run.smithi094.stdout:(2/4): kernel-core-5.14.0-432.el9.x86_64.rpm 9.1 MB/s | 15 MB 00:01 2024-04-06T03:18:06.727 INFO:teuthology.orchestra.run.smithi094.stdout:(3/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 15 MB/s | 33 MB 00:02 2024-04-06T03:18:07.462 INFO:teuthology.orchestra.run.smithi094.stdout:(4/4): kernel-modules-core-5.14.0-432.el9.x86_6 9.9 MB/s | 27 MB 00:02 2024-04-06T03:18:07.462 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:18:07.463 INFO:teuthology.orchestra.run.smithi094.stdout:Total 25 MB/s | 76 MB 00:02 2024-04-06T03:18:08.123 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:18:08.493 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:18:08.493 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:18:09.531 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:18:09.531 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:18:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:18:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-04-06T03:18:12.808 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-04-06T03:18:15.559 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-04-06T03:18:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-04-06T03:18:19.936 INFO:teuthology.orchestra.run.smithi136.stdout:(2/4): kernel-core-5.14.0-432.el9.x86_64.rpm 1.0 MB/s | 15 MB 00:14 2024-04-06T03:18:20.425 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-04-06T03:18:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:18:25.289 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:18:26.352 INFO:teuthology.orchestra.run.smithi136.stdout:(3/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 1.6 MB/s | 33 MB 00:20 2024-04-06T03:18:31.210 INFO:teuthology.orchestra.run.smithi136.stdout:(4/4): kernel-modules-core-5.14.0-432.el9.x86_6 1.1 MB/s | 27 MB 00:25 2024-04-06T03:18:31.210 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:18:31.210 INFO:teuthology.orchestra.run.smithi136.stdout:Total 3.0 MB/s | 76 MB 00:25 2024-04-06T03:18:31.873 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:18:32.261 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:18:32.261 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:18:33.258 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:18:33.258 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:18:35.706 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:18:36.303 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-04-06T03:18:36.379 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-04-06T03:18:39.145 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-04-06T03:18:39.283 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-04-06T03:18:43.967 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-04-06T03:18:44.091 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:18:48.843 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:18:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:18:54.654 INFO:teuthology.orchestra.run.smithi094.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-06T03:18:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:18:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:18:55.130 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:18:55.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-04-06T03:18:55.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-04-06T03:18:55.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-04-06T03:18:55.581 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:18:55.581 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:18:55.581 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:18:55.581 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-5.14.0-432.el9.x86_64 2024-04-06T03:18:55.581 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-04-06T03:18:55.581 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-04-06T03:18:55.582 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-04-06T03:18:55.582 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:18:55.582 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:18:55.935 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-04-06T03:18:56.074 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-432.el9.x86_64 2024-04-06T03:18:56.074 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-04-06T03:18:56.074 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-04-06T03:18:56.074 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-06T03:18:56.075 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-06T03:18:56.075 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-04-06T03:18:56.075 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=None, version=distro) 2024-04-06T03:18:56.075 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-04-06T03:18:56.155 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-432.el9.x86_64 2024-04-06T03:18:56.155 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-04-06T03:18:56.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 68.el9 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8264254 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-06T03:18:56.189 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:18:56.190 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-04-06T03:18:56.191 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-06T03:18:56.191 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 5.14.0-432.el9.x86_64 2024-04-06T03:18:56.191 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-06T03:18:56.796 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-04-06T03:18:58.255 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-06T03:18:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-04-06T03:18:58.283 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-04-06T03:18:58.298 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.uDPAvcB0nl 2024-04-06T03:18:58.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uDPAvcB0nl 2024-04-06T03:18:58.367 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.uDPAvcB0nl 2024-04-06T03:18:58.509 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.uDPAvcB0nl is 6KB 2024-04-06T03:18:58.528 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.uDPAvcB0nl 2024-04-06T03:18:58.542 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-04-06T03:18:58.609 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-06T03:18:58.609 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-06T03:18:58.609 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-06T03:18:58.609 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-04-06T03:18:58.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-04-06T03:18:58.772 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-04-06T03:19:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:19:18.320 INFO:teuthology.orchestra.run.smithi136.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-06T03:19:18.320 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:19:18.365 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:19:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:19:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-04-06T03:19:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-04-06T03:19:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-04-06T03:19:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-04-06T03:19:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:19:19.164 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:19:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-5.14.0-432.el9.x86_64 2024-04-06T03:19:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-04-06T03:19:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-04-06T03:19:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-04-06T03:19:19.164 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:19:19.164 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:19:19.518 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2024-04-06T03:19:19.630 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-432.el9.x86_64 2024-04-06T03:19:19.631 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-04-06T03:19:19.631 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-04-06T03:19:19.631 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-06T03:19:19.631 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-06T03:19:19.631 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-04-06T03:19:19.632 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2024-04-06T03:19:19.632 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2024-04-06T03:19:19.712 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-432.el9.x86_64 2024-04-06T03:19:19.712 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-04-06T03:19:19.712 DEBUG:teuthology.orchestra.run.smithi136:> sudo rpm -qi grub2-tools 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Name : grub2-tools 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Epoch : 1 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Version : 2.06 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Release : 68.el9 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: x86_64 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Group : Unspecified 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Size : 8264254 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:License : GPLv3+ 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-06T03:19:19.753 INFO:teuthology.orchestra.run.smithi136.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:Packager : builder@centos.org 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:Vendor : CentOS 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:Summary : Support tools for GRUB. 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:Description : 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:hardware devices. 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:19:19.754 INFO:teuthology.orchestra.run.smithi136.stdout:This subpackage provides tools for support of all platforms. 2024-04-06T03:19:19.755 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-06T03:19:19.755 INFO:teuthology.task.kernel:Updating grub on smithi136 to boot 5.14.0-432.el9.x86_64 2024-04-06T03:19:19.755 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-06T03:19:20.333 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2024-04-06T03:19:21.767 INFO:teuthology.orchestra.run.smithi136.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-06T03:19:21.801 INFO:teuthology.orchestra.run.smithi136.stderr:done 2024-04-06T03:19:21.803 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-04-06T03:19:21.818 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.cs2U3CNiML 2024-04-06T03:19:21.819 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.cs2U3CNiML 2024-04-06T03:19:21.887 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.cs2U3CNiML 2024-04-06T03:19:22.032 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.cs2U3CNiML is 6KB 2024-04-06T03:19:22.079 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.cs2U3CNiML 2024-04-06T03:19:22.093 DEBUG:teuthology.orchestra.run.smithi136:> sudo /bin/ls /boot/loader/entries || true 2024-04-06T03:19:22.157 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-06T03:19:22.157 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-06T03:19:22.158 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-06T03:19:22.158 INFO:teuthology.orchestra.run.smithi136.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-04-06T03:19:22.159 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-04-06T03:19:22.314 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2024-04-06T03:19:28.782 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-04-06T03:19:28.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-06T03:19:28.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-06T03:19:52.338 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-04-06T03:19:52.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-06T03:19:52.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-06T03:20:28.821 DEBUG:teuthology.orchestra.remote:timed out 2024-04-06T03:20:37.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-06T03:20:37.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-06T03:20:38.314 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-06T03:20:38.714 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-06T03:20:38.714 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-432.el9.x86_64"... 2024-04-06T03:20:38.715 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-06T03:20:38.728 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-432.el9.x86_64 2024-04-06T03:20:38.729 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-04-06T03:20:38.729 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-06T03:20:38.729 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-06T03:20:39.730 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-06T03:20:39.730 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-06T03:20:39.796 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-04-06T03:20:39.838 DEBUG:teuthology.parallel:result is None 2024-04-06T03:20:52.343 DEBUG:teuthology.orchestra.remote:timed out 2024-04-06T03:21:01.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-06T03:21:01.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-06T03:21:09.117 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-04-06T03:21:09.524 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-06T03:21:09.524 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-432.el9.x86_64"... 2024-04-06T03:21:09.524 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-04-06T03:21:09.538 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-432.el9.x86_64 2024-04-06T03:21:09.539 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-04-06T03:21:09.539 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-06T03:21:09.539 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-06T03:21:10.540 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-06T03:21:10.540 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-06T03:21:10.645 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-04-06T03:21:10.671 DEBUG:teuthology.parallel:result is None 2024-04-06T03:21:10.671 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-06T03:21:10.679 INFO:teuthology.task.internal:Creating test directory... 2024-04-06T03:21:10.679 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-06T03:21:10.682 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-06T03:21:10.698 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-06T03:21:10.704 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-06T03:21:10.719 INFO:teuthology.task.internal:Creating archive directory... 2024-04-06T03:21:10.720 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-06T03:21:10.740 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-06T03:21:10.771 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-06T03:21:10.778 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-06T03:21:10.778 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-06T03:21:10.801 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-06T03:21:10.834 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-06T03:21:10.835 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-06T03:21:10.848 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-06T03:21:10.848 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-06T03:21:10.850 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-06T03:21:10.857 INFO:teuthology.task.internal:Configuring sudo... 2024-04-06T03:21:10.857 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-06T03:21:10.892 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-06T03:21:10.919 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-06T03:21:10.929 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-06T03:21:10.929 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-06T03:21:10.962 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-06T03:21:10.976 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-06T03:21:11.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-06T03:21:11.107 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-06T03:21:11.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-06T03:21:11.231 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:21:11.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-06T03:21:11.299 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-06T03:21:11.339 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-06T03:21:11.391 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-06T03:21:11.470 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-06T03:21:11.511 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:21:11.511 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-06T03:21:11.573 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-04-06T03:21:11.576 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-04-06T03:21:11.624 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-06T03:21:11.672 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-06T03:21:11.956 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-06T03:21:11.969 INFO:teuthology.task.internal:Starting timer... 2024-04-06T03:21:11.970 INFO:teuthology.run_tasks:Running task pcp... 2024-04-06T03:21:11.980 INFO:teuthology.run_tasks:Running task selinux... 2024-04-06T03:21:11.993 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-06T03:21:11.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-04-06T03:21:12.059 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-04-06T03:21:12.060 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-04-06T03:21:12.161 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-04-06T03:21:12.164 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-06T03:21:12.165 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-04-06T03:21:12.186 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-04-06T03:21:12.187 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-04-06T03:21:12.215 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-04-06T03:21:12.215 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2024-04-06T03:21:12.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-06T03:21:12.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:21:12.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-06T03:21:12.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:21:12.314 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-06T03:21:12.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-04-06T03:21:12.345 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-04-06T03:21:12.634 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-06T03:21:12.645 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-06T03:21:12.663 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-06T03:21:12.664 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-06T03:25:57.077 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-04-06T03:25:57.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-06T03:25:57.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-06T03:25:57.174 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-06T03:25:57.252 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-06T03:25:57.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-06T03:25:57.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-06T03:25:57.338 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-04-06T03:25:57.416 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-06T03:25:57.417 INFO:teuthology.run_tasks:Running task clock... 2024-04-06T03:25:57.429 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-06T03:25:57.429 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-06T03:25:57.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-06T03:25:57.431 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-06T03:25:57.432 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-06T03:25:57.460 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-06T03:25:57.477 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-06T03:25:57.487 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-06T03:25:57.506 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-06T03:25:57.506 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-04-06T03:25:57.519 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-04-06T03:25:57.533 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-04-06T03:25:57.535 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-06T03:25:57.547 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-04-06T03:25:57.551 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-06T03:25:57.563 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-06T03:25:57.579 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-06T03:25:57.603 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-06T03:25:57.606 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-06T03:25:57.606 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-06T03:25:57.606 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:25:57.606 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:25:57.606 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:25:57.606 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:25:57.634 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-04-06T03:25:57.637 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-06T03:25:57.637 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-04-06T03:25:57.637 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:25:57.637 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:25:57.637 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:25:57.637 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:25:57.638 INFO:teuthology.run_tasks:Running task install... 2024-04-06T03:25:57.647 DEBUG:teuthology.task.install:project ceph 2024-04-06T03:25:57.647 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9f192350ba5fe6d0a365b251a8c2af61f56a5fa6'}} 2024-04-06T03:25:57.647 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '9f192350ba5fe6d0a365b251a8c2af61f56a5fa6'} 2024-04-06T03:25:57.647 INFO:teuthology.task.install:Using flavor: default 2024-04-06T03:25:57.653 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-06T03:25:57.653 INFO:teuthology.task.install:extra packages: [] 2024-04-06T03:25:57.653 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9f192350ba5fe6d0a365b251a8c2af61f56a5fa6', 'tag': None, 'wait_for_package': False} 2024-04-06T03:25:57.653 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-06T03:25:57.653 INFO:teuthology.packaging:ref: None 2024-04-06T03:25:57.653 INFO:teuthology.packaging:tag: None 2024-04-06T03:25:57.654 INFO:teuthology.packaging:branch: quincy 2024-04-06T03:25:57.654 INFO:teuthology.packaging:sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:25:57.654 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-06T03:25:57.656 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9f192350ba5fe6d0a365b251a8c2af61f56a5fa6', 'tag': None, 'wait_for_package': False} 2024-04-06T03:25:57.656 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-06T03:25:57.656 INFO:teuthology.packaging:ref: None 2024-04-06T03:25:57.656 INFO:teuthology.packaging:tag: None 2024-04-06T03:25:57.656 INFO:teuthology.packaging:branch: quincy 2024-04-06T03:25:57.656 INFO:teuthology.packaging:sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:25:57.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-06T03:25:57.967 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/6699495ce2af437c1ff648f1736d2457f417a1aa/centos/9/flavors/default/ 2024-04-06T03:25:57.968 INFO:teuthology.task.install.rpm:Package version is 17.2.7-819.g6699495c 2024-04-06T03:25:57.989 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/6699495ce2af437c1ff648f1736d2457f417a1aa/centos/9/flavors/default/ 2024-04-06T03:25:57.990 INFO:teuthology.task.install.rpm:Package version is 17.2.7-819.g6699495c 2024-04-06T03:25:58.101 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/6699495ce2af437c1ff648f1736d2457f417a1aa/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/6699495ce2af437c1ff648f1736d2457f417a1aa/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/6699495ce2af437c1ff648f1736d2457f417a1aa/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-06T03:25:58.102 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:25:58.102 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-06T03:25:58.131 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-06T03:25:58.131 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-06T03:25:58.131 INFO:teuthology.packaging:ref: None 2024-04-06T03:25:58.131 INFO:teuthology.packaging:tag: None 2024-04-06T03:25:58.131 INFO:teuthology.packaging:branch: quincy 2024-04-06T03:25:58.132 INFO:teuthology.packaging:sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:25:58.132 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-06T03:25:58.139 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/6699495ce2af437c1ff648f1736d2457f417a1aa/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/6699495ce2af437c1ff648f1736d2457f417a1aa/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/6699495ce2af437c1ff648f1736d2457f417a1aa/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-06T03:25:58.139 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:25:58.139 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-06T03:25:58.169 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-06T03:25:58.169 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-06T03:25:58.169 INFO:teuthology.packaging:ref: None 2024-04-06T03:25:58.169 INFO:teuthology.packaging:tag: None 2024-04-06T03:25:58.169 INFO:teuthology.packaging:branch: quincy 2024-04-06T03:25:58.169 INFO:teuthology.packaging:sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:25:58.169 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-06T03:25:58.201 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-06T03:25:58.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-06T03:25:58.281 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-06T03:25:58.318 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-06T03:25:58.349 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-04-06T03:25:58.350 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-04-06T03:25:58.385 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-04-06T03:25:58.388 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-04-06T03:25:58.597 INFO:teuthology.orchestra.run.smithi136.stdout:74 files removed 2024-04-06T03:25:58.624 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw 2024-04-06T03:25:58.635 INFO:teuthology.orchestra.run.smithi094.stdout:74 files removed 2024-04-06T03:25:58.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw 2024-04-06T03:25:59.331 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 188 kB/s | 77 kB 00:00 2024-04-06T03:25:59.400 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 175 kB/s | 77 kB 00:00 2024-04-06T03:25:59.715 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 27 kB/s | 9.1 kB 00:00 2024-04-06T03:25:59.767 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 29 kB/s | 9.1 kB 00:00 2024-04-06T03:26:00.074 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 5.6 kB/s | 1.8 kB 00:00 2024-04-06T03:26:00.176 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 4.8 kB/s | 1.8 kB 00:00 2024-04-06T03:26:00.526 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - AppStream 20 MB/s | 8.1 MB 00:00 2024-04-06T03:26:00.611 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - AppStream 20 MB/s | 8.1 MB 00:00 2024-04-06T03:26:02.135 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS-9 - BaseOS 8.9 MB/s | 2.5 MB 00:00 2024-04-06T03:26:02.354 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.5 MB 00:00 2024-04-06T03:26:03.544 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-06T03:26:03.892 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.0 MB 00:01 2024-04-06T03:26:05.710 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-06T03:26:09.769 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-04-06T03:26:11.327 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-06T03:26:26.585 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 880 kB/s | 19 MB 00:21 2024-04-06T03:26:30.761 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.1 MB 00:00 2024-04-06T03:26:32.153 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-04-06T03:26:33.645 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 9.3 kB/s | 16 kB 00:01 2024-04-06T03:26:37.125 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-06T03:26:38.438 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 21 MB 00:04 2024-04-06T03:26:39.551 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:26:39.551 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:26:39.551 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:26:39.551 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.555 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:26:39.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.560 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:26:39.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:26:39.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:26:40.072 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:26:40.087 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:26:40.087 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-06T03:26:40.087 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:26:40.087 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:26:40.087 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:17.2.7-819.g6699495c.el9 ceph 14 M 2024-04-06T03:26:40.087 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-04-06T03:26:40.087 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:17.2.7-819.g6699495c.el9 ceph 3.1 M 2024-04-06T03:26:40.087 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:17.2.7-819.g6699495c.el9 ceph 2.9 M 2024-04-06T03:26:40.087 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:17.2.7-819.g6699495c.el9 ceph 5.0 M 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:17.2.7-819.g6699495c.el9 ceph 22 M 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:17.2.7-819.g6699495c.el9 ceph 25 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:17.2.7-819.g6699495c.el9 ceph 638 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:17.2.7-819.g6699495c.el9 ceph 457 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:17.2.7-819.g6699495c.el9 ceph 4.9 M 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-06T03:26:40.088 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:17.2.7-819.g6699495c.el9 ceph 45 k 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:17.2.7-819.g6699495c.el9 ceph 118 k 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:17.2.7-819.g6699495c.el9 ceph 156 k 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:17.2.7-819.g6699495c.el9 ceph 320 k 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:17.2.7-819.g6699495c.el9 ceph 293 k 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:17.2.7-819.g6699495c.el9 ceph 99 k 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout:Install 22 Packages 2024-04-06T03:26:40.089 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 2 Packages 2024-04-06T03:26:40.090 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:26:40.090 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 58 M 2024-04-06T03:26:40.090 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:26:42.067 INFO:teuthology.orchestra.run.smithi136.stdout:(1/24): ceph-base-17.2.7-819.g6699495c.el9.x86_ 3.7 MB/s | 5.0 MB 00:01 2024-04-06T03:26:42.117 INFO:teuthology.orchestra.run.smithi136.stdout:(2/24): ceph-selinux-17.2.7-819.g6699495c.el9.x 505 kB/s | 25 kB 00:00 2024-04-06T03:26:42.217 INFO:teuthology.orchestra.run.smithi136.stdout:(3/24): libcephfs2-17.2.7-819.g6699495c.el9.x86 6.2 MB/s | 638 kB 00:00 2024-04-06T03:26:42.301 INFO:teuthology.orchestra.run.smithi136.stdout:(4/24): libradosstriper1-17.2.7-819.g6699495c.e 5.4 MB/s | 457 kB 00:00 2024-04-06T03:26:42.785 INFO:teuthology.orchestra.run.smithi136.stdout:(5/24): librgw2-17.2.7-819.g6699495c.el9.x86_64 10 MB/s | 4.9 MB 00:00 2024-04-06T03:26:42.851 INFO:teuthology.orchestra.run.smithi136.stdout:(6/24): python3-ceph-argparse-17.2.7-819.g66994 680 kB/s | 45 kB 00:00 2024-04-06T03:26:42.918 INFO:teuthology.orchestra.run.smithi136.stdout:(7/24): python3-ceph-common-17.2.7-819.g6699495 1.7 MB/s | 118 kB 00:00 2024-04-06T03:26:42.985 INFO:teuthology.orchestra.run.smithi136.stdout:(8/24): python3-cephfs-17.2.7-819.g6699495c.el9 2.3 MB/s | 156 kB 00:00 2024-04-06T03:26:43.136 INFO:teuthology.orchestra.run.smithi136.stdout:(9/24): python3-rados-17.2.7-819.g6699495c.el9. 2.1 MB/s | 320 kB 00:00 2024-04-06T03:26:43.361 INFO:teuthology.orchestra.run.smithi136.stdout:(10/24): ceph-common-17.2.7-819.g6699495c.el9.x 8.5 MB/s | 22 MB 00:02 2024-04-06T03:26:43.386 INFO:teuthology.orchestra.run.smithi136.stdout:(11/24): python3-rbd-17.2.7-819.g6699495c.el9.x 1.1 MB/s | 293 kB 00:00 2024-04-06T03:26:43.419 INFO:teuthology.orchestra.run.smithi136.stdout:(12/24): python3-rgw-17.2.7-819.g6699495c.el9.x 1.7 MB/s | 99 kB 00:00 2024-04-06T03:26:43.638 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 7.2 kB/s | 1.7 kB 00:00 2024-04-06T03:26:43.786 INFO:teuthology.orchestra.run.smithi136.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 123 kB/s | 45 kB 00:00 2024-04-06T03:26:43.962 INFO:teuthology.orchestra.run.smithi136.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 278 kB/s | 160 kB 00:00 2024-04-06T03:26:44.204 INFO:teuthology.orchestra.run.smithi136.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-04-06T03:26:44.354 INFO:teuthology.orchestra.run.smithi136.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.1 MB/s | 662 kB 00:00 2024-04-06T03:26:44.371 INFO:teuthology.orchestra.run.smithi136.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 253 kB/s | 42 kB 00:00 2024-04-06T03:26:45.696 INFO:teuthology.orchestra.run.smithi136.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 37 kB/s | 49 kB 00:01 2024-04-06T03:26:45.829 INFO:teuthology.orchestra.run.smithi136.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 506 kB/s | 67 kB 00:00 2024-04-06T03:26:45.871 INFO:teuthology.orchestra.run.smithi136.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 204 kB/s | 309 kB 00:01 2024-04-06T03:26:46.079 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:26:46.080 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.081 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:26:46.082 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.083 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:26:46.084 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:26:46.085 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.086 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.087 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.088 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.089 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.089 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:26:46.180 INFO:teuthology.orchestra.run.smithi136.stdout:(21/24): librados2-17.2.7-819.g6699495c.el9.x86 10 MB/s | 3.1 MB 00:00 2024-04-06T03:26:46.364 INFO:teuthology.orchestra.run.smithi136.stdout:(22/24): librbd1-17.2.7-819.g6699495c.el9.x86_6 16 MB/s | 2.9 MB 00:00 2024-04-06T03:26:46.605 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:26:46.614 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:26:46.614 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:17.2.7-819.g6699495c.el9 ceph 14 M 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:17.2.7-819.g6699495c.el9 ceph 3.1 M 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:17.2.7-819.g6699495c.el9 ceph 2.9 M 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:17.2.7-819.g6699495c.el9 ceph 5.0 M 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:17.2.7-819.g6699495c.el9 ceph 22 M 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:17.2.7-819.g6699495c.el9 ceph 25 k 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:17.2.7-819.g6699495c.el9 ceph 638 k 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-06T03:26:46.615 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:17.2.7-819.g6699495c.el9 ceph 457 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:17.2.7-819.g6699495c.el9 ceph 4.9 M 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:17.2.7-819.g6699495c.el9 ceph 45 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:17.2.7-819.g6699495c.el9 ceph 118 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:17.2.7-819.g6699495c.el9 ceph 156 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:17.2.7-819.g6699495c.el9 ceph 320 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:17.2.7-819.g6699495c.el9 ceph 293 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:17.2.7-819.g6699495c.el9 ceph 99 k 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-06T03:26:46.616 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:26:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:26:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:26:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:Install 22 Packages 2024-04-06T03:26:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 2 Packages 2024-04-06T03:26:46.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:26:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 58 M 2024-04-06T03:26:46.617 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:26:47.355 INFO:teuthology.orchestra.run.smithi136.stdout:(23/24): ceph-radosgw-17.2.7-819.g6699495c.el9. 2.1 MB/s | 14 MB 00:06 2024-04-06T03:26:47.547 INFO:teuthology.orchestra.run.smithi136.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 943 kB/s | 1.6 MB 00:01 2024-04-06T03:26:47.552 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:26:47.552 INFO:teuthology.orchestra.run.smithi136.stdout:Total 7.7 MB/s | 58 MB 00:07 2024-04-06T03:26:47.615 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:26:47.648 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:26:47.648 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:26:48.117 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:26:48.117 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:26:49.272 INFO:teuthology.orchestra.run.smithi094.stdout:(1/24): ceph-common-17.2.7-819.g6699495c.el9.x8 9.6 MB/s | 22 MB 00:02 2024-04-06T03:26:49.306 INFO:teuthology.orchestra.run.smithi094.stdout:(2/24): ceph-selinux-17.2.7-819.g6699495c.el9.x 762 kB/s | 25 kB 00:00 2024-04-06T03:26:49.326 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:26:49.373 INFO:teuthology.orchestra.run.smithi094.stdout:(3/24): libcephfs2-17.2.7-819.g6699495c.el9.x86 9.4 MB/s | 638 kB 00:00 2024-04-06T03:26:49.431 INFO:teuthology.orchestra.run.smithi094.stdout:(4/24): libradosstriper1-17.2.7-819.g6699495c.e 7.7 MB/s | 457 kB 00:00 2024-04-06T03:26:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-06T03:26:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:17.2.7-819.g6699495c.el9.x86_64 2/26 2024-04-06T03:26:49.732 INFO:teuthology.orchestra.run.smithi094.stdout:(5/24): librgw2-17.2.7-819.g6699495c.el9.x86_64 16 MB/s | 4.9 MB 00:00 2024-04-06T03:26:49.799 INFO:teuthology.orchestra.run.smithi094.stdout:(6/24): python3-ceph-argparse-17.2.7-819.g66994 676 kB/s | 45 kB 00:00 2024-04-06T03:26:49.801 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:17.2.7-819.g6699495c.el9.x86_64 2/26 2024-04-06T03:26:49.849 INFO:teuthology.orchestra.run.smithi094.stdout:(7/24): python3-ceph-common-17.2.7-819.g6699495 2.3 MB/s | 118 kB 00:00 2024-04-06T03:26:49.878 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:17.2.7-819.g6699495c.el9.x86_64 3/26 2024-04-06T03:26:49.900 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 4/26 2024-04-06T03:26:49.916 INFO:teuthology.orchestra.run.smithi094.stdout:(8/24): ceph-base-17.2.7-819.g6699495c.el9.x86_ 1.7 MB/s | 5.0 MB 00:02 2024-04-06T03:26:49.932 INFO:teuthology.orchestra.run.smithi094.stdout:(9/24): python3-cephfs-17.2.7-819.g6699495c.el9 1.8 MB/s | 156 kB 00:00 2024-04-06T03:26:49.972 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 4/26 2024-04-06T03:26:49.983 INFO:teuthology.orchestra.run.smithi094.stdout:(10/24): python3-rbd-17.2.7-819.g6699495c.el9.x 5.7 MB/s | 293 kB 00:00 2024-04-06T03:26:50.020 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-06T03:26:50.033 INFO:teuthology.orchestra.run.smithi094.stdout:(11/24): python3-rgw-17.2.7-819.g6699495c.el9.x 1.9 MB/s | 99 kB 00:00 2024-04-06T03:26:50.055 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-06T03:26:50.092 INFO:teuthology.orchestra.run.smithi094.stdout:(12/24): python3-rados-17.2.7-819.g6699495c.el9 1.8 MB/s | 320 kB 00:00 2024-04-06T03:26:50.167 INFO:teuthology.orchestra.run.smithi094.stdout:(13/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-06T03:26:50.184 INFO:teuthology.orchestra.run.smithi094.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 494 kB/s | 45 kB 00:00 2024-04-06T03:26:50.260 INFO:teuthology.orchestra.run.smithi094.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-06T03:26:50.293 INFO:teuthology.orchestra.run.smithi094.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 42 kB 00:00 2024-04-06T03:26:50.318 INFO:teuthology.orchestra.run.smithi094.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-06T03:26:50.488 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-06T03:26:50.519 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:17.2.7-819.g6699495c.el9.x86_64 8/26 2024-04-06T03:26:50.535 INFO:teuthology.orchestra.run.smithi094.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 224 kB/s | 49 kB 00:00 2024-04-06T03:26:50.561 INFO:teuthology.orchestra.run.smithi094.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2024-04-06T03:26:50.565 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:17.2.7-819.g6699495c.el9.x86_64 8/26 2024-04-06T03:26:50.594 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-06T03:26:50.619 INFO:teuthology.orchestra.run.smithi094.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-04-06T03:26:50.637 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:17.2.7-819.g6699495c.el9.x 10/26 2024-04-06T03:26:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:17.2.7-819.g6699495c.el9.x86_64 11/26 2024-04-06T03:26:50.694 INFO:teuthology.orchestra.run.smithi094.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-06T03:26:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:17.2.7-819.g6699495c.el9.x86_64 12/26 2024-04-06T03:26:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:17.2.7-819.g6699495c.el9.x86_64 13/26 2024-04-06T03:26:50.821 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:17.2.7-819.g6699495c.el9.x86_64 13/26 2024-04-06T03:26:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-06T03:26:50.905 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-06T03:26:50.938 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-06T03:26:51.062 INFO:teuthology.orchestra.run.smithi094.stdout:(22/24): ceph-radosgw-17.2.7-819.g6699495c.el9. 3.4 MB/s | 14 MB 00:04 2024-04-06T03:26:51.145 INFO:teuthology.orchestra.run.smithi094.stdout:(23/24): librbd1-17.2.7-819.g6699495c.el9.x86_6 6.5 MB/s | 2.9 MB 00:00 2024-04-06T03:26:51.240 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-06T03:26:51.267 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:17.2.7-819.g6699495c.el9.x86_64 18/26 2024-04-06T03:26:51.320 INFO:teuthology.orchestra.run.smithi094.stdout:(24/24): librados2-17.2.7-819.g6699495c.el9.x86 4.4 MB/s | 3.1 MB 00:00 2024-04-06T03:26:51.323 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:26:51.324 INFO:teuthology.orchestra.run.smithi094.stdout:Total 12 MB/s | 58 MB 00:04 2024-04-06T03:26:51.357 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:17.2.7-819.g6699495c.el9.x86_64 18/26 2024-04-06T03:26:51.380 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:26:51.413 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:26:51.413 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:26:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:17.2.7-819.g6699495c.el9.x86_64 19/26 2024-04-06T03:26:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:17.2.7-819.g6699495c.el9.x86 20/26 2024-04-06T03:26:51.868 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:26:51.868 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:26:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:26:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-06T03:26:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:17.2.7-819.g6699495c.el9.x86_64 2/26 2024-04-06T03:26:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:17.2.7-819.g6699495c.el9.x86_64 2/26 2024-04-06T03:26:53.684 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:17.2.7-819.g6699495c.el9.x86_64 3/26 2024-04-06T03:26:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 4/26 2024-04-06T03:26:53.795 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 4/26 2024-04-06T03:26:53.850 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-06T03:26:53.856 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 21/26 2024-04-06T03:26:53.878 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-06T03:26:53.891 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 21/26 2024-04-06T03:26:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-06T03:26:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:17.2.7-819.g6699495c.el9.x86_64 8/26 2024-04-06T03:26:54.381 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:17.2.7-819.g6699495c.el9.x86_64 8/26 2024-04-06T03:26:54.409 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-06T03:26:54.440 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 21/26 2024-04-06T03:26:54.452 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:17.2.7-819.g6699495c.el9.x 10/26 2024-04-06T03:26:54.477 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 22/26 2024-04-06T03:26:54.489 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:17.2.7-819.g6699495c.el9.x86_64 11/26 2024-04-06T03:26:54.552 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:17.2.7-819.g6699495c.el9.x86_64 12/26 2024-04-06T03:26:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 22/26 2024-04-06T03:26:54.561 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-06T03:26:54.561 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-06T03:26:54.561 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:26:54.579 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:17.2.7-819.g6699495c.el9.x86_64 13/26 2024-04-06T03:26:54.587 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:17.2.7-819.g6699495c.el9.x86_64 23/26 2024-04-06T03:26:54.644 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:17.2.7-819.g6699495c.el9.x86_64 13/26 2024-04-06T03:26:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-06T03:26:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-06T03:26:54.753 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-06T03:26:55.051 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-06T03:26:55.082 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:17.2.7-819.g6699495c.el9.x86_64 18/26 2024-04-06T03:26:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:17.2.7-819.g6699495c.el9.x86_64 18/26 2024-04-06T03:26:55.245 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:17.2.7-819.g6699495c.el9.x86_64 19/26 2024-04-06T03:26:55.268 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:17.2.7-819.g6699495c.el9.x86 20/26 2024-04-06T03:26:57.678 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 21/26 2024-04-06T03:26:57.715 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 21/26 2024-04-06T03:26:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 21/26 2024-04-06T03:26:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 22/26 2024-04-06T03:26:58.365 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 22/26 2024-04-06T03:26:58.365 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-06T03:26:58.365 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-06T03:26:58.366 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:26:58.402 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:17.2.7-819.g6699495c.el9.x86_64 23/26 2024-04-06T03:27:04.031 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:17.2.7-819.g6699495c.el9.x86_64 23/26 2024-04-06T03:27:04.031 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-04-06T03:27:04.031 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-04-06T03:27:04.031 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-04-06T03:27:04.031 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-04-06T03:27:04.031 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-04-06T03:27:04.031 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-04-06T03:27:04.031 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-04-06T03:27:04.031 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:17.2.7-819.g6699495c.el9.x86_64 24/26 2024-04-06T03:27:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-819.g6699495c.el9.x86_64 24/26 2024-04-06T03:27:05.371 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:05.371 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-06T03:27:05.371 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-06T03:27:05.372 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-06T03:27:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:05.374 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-06T03:27:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-06T03:27:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-06T03:27:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 1/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 2/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:17.2.7-819.g6699495c.el9.x86_64 3/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:17.2.7-819.g6699495c.el9.x86_64 4/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 5/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:17.2.7-819.g6699495c.el9.x86_64 6/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:17.2.7-819.g6699495c.el9.x86_64 7/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:17.2.7-819.g6699495c.el9.x 8/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:17.2.7-819.g6699495c.el9.x86 9/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:17.2.7-819.g6699495c.el9.x86_64 10/26 2024-04-06T03:27:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:17.2.7-819.g6699495c.el9.x86_64 11/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:17.2.7-819.g6699495c.el9.x86_64 12/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:17.2.7-819.g6699495c.el9.x86_64 13/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:17.2.7-819.g6699495c.el9.x86_64 23/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-06T03:27:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:17.2.7-819.g6699495c.el9.x86_64 25/26 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.632 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-06T03:27:07.633 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.634 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:07.634 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-06T03:27:07.634 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:07.634 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:27:07.855 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:17.2.7-819.g6699495c.el9.x86_64 23/26 2024-04-06T03:27:07.855 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-04-06T03:27:07.855 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-04-06T03:27:07.855 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-04-06T03:27:07.855 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-04-06T03:27:07.855 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-04-06T03:27:07.855 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-04-06T03:27:07.855 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-04-06T03:27:07.855 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:08.017 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-test 2024-04-06T03:27:08.609 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:31 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:27:08.897 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:27:08.897 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:27:08.897 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:27:08.897 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.897 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:27:08.897 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.897 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:27:08.898 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.899 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.900 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:27:08.901 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:27:08.902 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:08.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:08.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:08.968 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:27:08.968 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:08.968 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-06T03:27:08.968 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:08.968 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:17.2.7-819.g6699495c.el9 ceph 48 M 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:17.2.7-819.g6699495c.el9 ceph 163 k 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout:Install 6 Packages 2024-04-06T03:27:08.969 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:08.970 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 49 M 2024-04-06T03:27:08.970 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 193 M 2024-04-06T03:27:08.970 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:27:09.143 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:17.2.7-819.g6699495c.el9.x86_64 24/26 2024-04-06T03:27:09.221 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-819.g6699495c.el9.x86_64 24/26 2024-04-06T03:27:09.221 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:09.222 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-06T03:27:09.222 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-06T03:27:09.222 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-06T03:27:09.222 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:09.223 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-06T03:27:09.288 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-06T03:27:09.288 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-06T03:27:09.442 INFO:teuthology.orchestra.run.smithi136.stdout:(1/6): jq-1.6-16.el9.x86_64.rpm 2.1 MB/s | 187 kB 00:00 2024-04-06T03:27:09.575 INFO:teuthology.orchestra.run.smithi136.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 218 kB 00:00 2024-04-06T03:27:09.701 INFO:teuthology.orchestra.run.smithi136.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-04-06T03:27:09.768 INFO:teuthology.orchestra.run.smithi136.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-04-06T03:27:09.793 INFO:teuthology.orchestra.run.smithi136.stdout:(5/6): libcephsqlite-17.2.7-819.g6699495c.el9.x 370 kB/s | 163 kB 00:00 2024-04-06T03:27:10.512 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-06T03:27:10.512 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 1/26 2024-04-06T03:27:10.512 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 2/26 2024-04-06T03:27:10.512 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:17.2.7-819.g6699495c.el9.x86_64 3/26 2024-04-06T03:27:10.512 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:17.2.7-819.g6699495c.el9.x86_64 4/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 5/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:17.2.7-819.g6699495c.el9.x86_64 6/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:17.2.7-819.g6699495c.el9.x86_64 7/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:17.2.7-819.g6699495c.el9.x 8/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:17.2.7-819.g6699495c.el9.x86 9/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:17.2.7-819.g6699495c.el9.x86_64 10/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:17.2.7-819.g6699495c.el9.x86_64 11/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:17.2.7-819.g6699495c.el9.x86_64 12/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:17.2.7-819.g6699495c.el9.x86_64 13/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-06T03:27:10.513 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-06T03:27:10.514 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-06T03:27:10.514 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-06T03:27:10.514 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-06T03:27:10.514 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:17.2.7-819.g6699495c.el9.x86_64 23/26 2024-04-06T03:27:10.514 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-06T03:27:10.514 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:17.2.7-819.g6699495c.el9.x86_64 25/26 2024-04-06T03:27:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-06T03:27:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:11.360 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-04-06T03:27:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-06T03:27:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:11.363 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:27:11.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-test 2024-04-06T03:27:12.001 INFO:teuthology.orchestra.run.smithi136.stdout:(6/6): ceph-test-17.2.7-819.g6699495c.el9.x86_6 18 MB/s | 48 MB 00:02 2024-04-06T03:27:12.003 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:12.004 INFO:teuthology.orchestra.run.smithi136.stdout:Total 16 MB/s | 49 MB 00:03 2024-04-06T03:27:12.052 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:27:12.071 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:27:12.071 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:27:12.258 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:29 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:27:12.470 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:27:12.470 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:27:12.551 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:27:12.552 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:27:12.553 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.554 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.555 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:27:12.556 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:12.557 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.558 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.559 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.560 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:12.627 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:27:12.628 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-06T03:27:12.628 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:12.628 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:27:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:17.2.7-819.g6699495c.el9 ceph 48 M 2024-04-06T03:27:12.628 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-06T03:27:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-06T03:27:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:17.2.7-819.g6699495c.el9 ceph 163 k 2024-04-06T03:27:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-06T03:27:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-06T03:27:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-06T03:27:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:12.629 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:27:12.629 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:12.629 INFO:teuthology.orchestra.run.smithi094.stdout:Install 6 Packages 2024-04-06T03:27:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:12.630 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 49 M 2024-04-06T03:27:12.630 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 193 M 2024-04-06T03:27:12.630 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:27:12.988 INFO:teuthology.orchestra.run.smithi094.stdout:(1/6): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-04-06T03:27:13.037 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:27:13.088 INFO:teuthology.orchestra.run.smithi094.stdout:(2/6): libcephsqlite-17.2.7-819.g6699495c.el9.x 632 kB/s | 163 kB 00:00 2024-04-06T03:27:13.119 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-06T03:27:13.121 INFO:teuthology.orchestra.run.smithi094.stdout:(3/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.6 MB/s | 218 kB 00:00 2024-04-06T03:27:13.152 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-06T03:27:13.189 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-06T03:27:13.234 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-06T03:27:13.262 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:17.2.7-819.g6699495c.el9.x86_64 5/6 2024-04-06T03:27:13.263 INFO:teuthology.orchestra.run.smithi094.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 450 kB/s | 64 kB 00:00 2024-04-06T03:27:13.288 INFO:teuthology.orchestra.run.smithi094.stdout:(5/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-06T03:27:15.514 INFO:teuthology.orchestra.run.smithi094.stdout:(6/6): ceph-test-17.2.7-819.g6699495c.el9.x86_6 18 MB/s | 48 MB 00:02 2024-04-06T03:27:15.516 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:15.516 INFO:teuthology.orchestra.run.smithi094.stdout:Total 17 MB/s | 49 MB 00:02 2024-04-06T03:27:15.567 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:27:15.586 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:27:15.586 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:27:15.981 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:27:15.981 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:27:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:27:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-06T03:27:16.673 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-06T03:27:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-06T03:27:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-06T03:27:16.792 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:17.2.7-819.g6699495c.el9.x86_64 5/6 2024-04-06T03:27:17.701 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:17.2.7-819.g6699495c.el9.x86_64 5/6 2024-04-06T03:27:17.779 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:17.2.7-819.g6699495c.el9.x86_64 6/6 2024-04-06T03:27:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-test-2:17.2.7-819.g6699495c.el9.x86_64 6/6 2024-04-06T03:27:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:17.2.7-819.g6699495c.el9.x86_64 1/6 2024-04-06T03:27:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:17.2.7-819.g6699495c.el9.x86_64 2/6 2024-04-06T03:27:19.978 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2024-04-06T03:27:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2024-04-06T03:27:19.979 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2024-04-06T03:27:20.370 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: jq-1.6-16.el9.x86_64 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:20.371 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:27:20.673 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph 2024-04-06T03:27:21.272 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:44 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:27:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:17.2.7-819.g6699495c.el9.x86_64 5/6 2024-04-06T03:27:21.441 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:17.2.7-819.g6699495c.el9.x86_64 6/6 2024-04-06T03:27:21.562 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:27:21.562 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:27:21.562 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:27:21.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.562 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:27:21.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.563 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:27:21.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.563 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:27:21.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.563 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:27:21.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.563 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:27:21.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.564 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:27:21.564 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.564 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:27:21.564 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.564 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:27:21.564 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.564 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:27:21.564 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.565 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:27:21.565 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.565 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:27:21.565 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.565 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:27:21.565 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.565 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:27:21.565 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.566 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:27:21.566 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.566 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:27:21.566 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.566 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:27:21.566 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.566 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:27:21.566 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.567 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:27:21.567 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.567 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:27:21.567 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.567 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:27:21.567 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.567 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:27:21.567 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.567 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:27:21.568 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.568 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:27:21.568 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.568 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:27:21.568 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.568 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:27:21.568 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.568 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:27:21.568 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.569 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.570 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.571 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.572 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.573 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:21.633 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:27:21.635 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:17.2.7-819.g6699495c.el9 ceph 6.5 k 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:17.2.7-819.g6699495c.el9 ceph 2.1 M 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:17.2.7-819.g6699495c.el9 ceph 1.4 M 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 240 k 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:17.2.7-819.g6699495c.el9 ceph 3.9 M 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:17.2.7-819.g6699495c.el9 ceph 15 M 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-06T03:27:21.636 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-06T03:27:21.637 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout:Install 33 Packages 2024-04-06T03:27:21.638 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:21.639 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 27 M 2024-04-06T03:27:21.639 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 96 M 2024-04-06T03:27:21.639 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:27:22.062 INFO:teuthology.orchestra.run.smithi136.stdout:(1/33): ceph-17.2.7-819.g6699495c.el9.x86_64.rp 40 kB/s | 6.5 kB 00:00 2024-04-06T03:27:22.304 INFO:teuthology.orchestra.run.smithi136.stdout:(2/33): ceph-mgr-17.2.7-819.g6699495c.el9.x86_6 3.5 MB/s | 1.4 MB 00:00 2024-04-06T03:27:22.563 INFO:teuthology.orchestra.run.smithi136.stdout:(3/33): ceph-mon-17.2.7-819.g6699495c.el9.x86_6 7.8 MB/s | 3.9 MB 00:00 2024-04-06T03:27:22.613 INFO:teuthology.orchestra.run.smithi136.stdout:(4/33): ceph-mds-17.2.7-819.g6699495c.el9.x86_6 2.9 MB/s | 2.1 MB 00:00 2024-04-06T03:27:22.638 INFO:teuthology.orchestra.run.smithi136.stdout:(5/33): ceph-mgr-modules-core-17.2.7-819.g66994 3.1 MB/s | 240 kB 00:00 2024-04-06T03:27:22.739 INFO:teuthology.orchestra.run.smithi136.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-06T03:27:22.822 INFO:teuthology.orchestra.run.smithi136.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-06T03:27:22.898 INFO:teuthology.orchestra.run.smithi136.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-04-06T03:27:22.973 INFO:teuthology.orchestra.run.smithi136.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 1.6 MB/s | 125 kB 00:00 2024-04-06T03:27:23.007 INFO:teuthology.orchestra.run.smithi136.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 3.3 MB/s | 1.2 MB 00:00 2024-04-06T03:27:23.166 INFO:teuthology.orchestra.run.smithi136.stdout:(11/33): ceph-osd-17.2.7-819.g6699495c.el9.x86_ 18 MB/s | 15 MB 00:00 2024-04-06T03:27:23.216 INFO:teuthology.orchestra.run.smithi136.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 886 kB/s | 215 kB 00:00 2024-04-06T03:27:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-test-2:17.2.7-819.g6699495c.el9.x86_64 6/6 2024-04-06T03:27:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:17.2.7-819.g6699495c.el9.x86_64 1/6 2024-04-06T03:27:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:17.2.7-819.g6699495c.el9.x86_64 2/6 2024-04-06T03:27:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2024-04-06T03:27:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2024-04-06T03:27:23.648 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2024-04-06T03:27:23.816 INFO:teuthology.orchestra.run.smithi136.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 53 kB/s | 35 kB 00:00 2024-04-06T03:27:23.874 INFO:teuthology.orchestra.run.smithi136.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-04-06T03:27:23.958 INFO:teuthology.orchestra.run.smithi136.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 522 kB/s | 43 kB 00:00 2024-04-06T03:27:24.000 INFO:teuthology.orchestra.run.smithi136.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 173 kB/s | 172 kB 00:00 2024-04-06T03:27:24.050 INFO:teuthology.orchestra.run.smithi136.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout: jq-1.6-16.el9.x86_64 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-06T03:27:24.101 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:24.102 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:27:24.109 INFO:teuthology.orchestra.run.smithi136.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 323 kB/s | 11 kB 00:00 2024-04-06T03:27:24.142 INFO:teuthology.orchestra.run.smithi136.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 532 kB/s | 18 kB 00:00 2024-04-06T03:27:24.176 INFO:teuthology.orchestra.run.smithi136.stdout:(20/33): python3-jaraco-collections-3.0.0-8.el9 696 kB/s | 23 kB 00:00 2024-04-06T03:27:24.201 INFO:teuthology.orchestra.run.smithi136.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-04-06T03:27:24.218 INFO:teuthology.orchestra.run.smithi136.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 466 kB/s | 19 kB 00:00 2024-04-06T03:27:24.235 INFO:teuthology.orchestra.run.smithi136.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 585 kB/s | 20 kB 00:00 2024-04-06T03:27:24.252 INFO:teuthology.orchestra.run.smithi136.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-04-06T03:27:24.278 INFO:teuthology.orchestra.run.smithi136.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-04-06T03:27:24.320 INFO:teuthology.orchestra.run.smithi136.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 394 kB/s | 16 kB 00:00 2024-04-06T03:27:24.345 INFO:teuthology.orchestra.run.smithi136.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-06T03:27:24.370 INFO:teuthology.orchestra.run.smithi136.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-04-06T03:27:24.395 INFO:teuthology.orchestra.run.smithi136.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-04-06T03:27:24.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph 2024-04-06T03:27:24.437 INFO:teuthology.orchestra.run.smithi136.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-06T03:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout:(31/33): python3-toml-0.10.2-6.el9.noarch.rpm 64 kB/s | 42 kB 00:00 2024-04-06T03:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 487 kB/s | 20 kB 00:00 2024-04-06T03:27:24.513 INFO:teuthology.orchestra.run.smithi136.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-04-06T03:27:24.516 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:24.516 INFO:teuthology.orchestra.run.smithi136.stdout:Total 9.4 MB/s | 27 MB 00:02 2024-04-06T03:27:24.633 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:27:24.669 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:27:24.670 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:27:25.001 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:42 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:27:25.017 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:27:25.018 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:27:25.297 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:27:25.298 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:27:25.299 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:27:25.300 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.301 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.302 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.303 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:25.304 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.305 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:25.306 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.306 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.306 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.306 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.306 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:25.306 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.306 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.306 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.306 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:25.379 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:17.2.7-819.g6699495c.el9 ceph 6.5 k 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:17.2.7-819.g6699495c.el9 ceph 2.1 M 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:17.2.7-819.g6699495c.el9 ceph 1.4 M 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 240 k 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:17.2.7-819.g6699495c.el9 ceph 3.9 M 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:17.2.7-819.g6699495c.el9 ceph 15 M 2024-04-06T03:27:25.382 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-06T03:27:25.383 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:25.384 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:27:25.385 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:25.385 INFO:teuthology.orchestra.run.smithi094.stdout:Install 33 Packages 2024-04-06T03:27:25.385 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:25.385 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 27 M 2024-04-06T03:27:25.385 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 96 M 2024-04-06T03:27:25.385 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:27:25.909 INFO:teuthology.orchestra.run.smithi094.stdout:(1/33): ceph-17.2.7-819.g6699495c.el9.x86_64.rp 31 kB/s | 6.5 kB 00:00 2024-04-06T03:27:26.026 INFO:teuthology.orchestra.run.smithi094.stdout:(2/33): ceph-mgr-17.2.7-819.g6699495c.el9.x86_6 4.3 MB/s | 1.4 MB 00:00 2024-04-06T03:27:26.302 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:27:26.366 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-06T03:27:26.399 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-06T03:27:26.418 INFO:teuthology.orchestra.run.smithi094.stdout:(3/33): ceph-mds-17.2.7-819.g6699495c.el9.x86_6 2.9 MB/s | 2.1 MB 00:00 2024-04-06T03:27:26.431 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-06T03:27:26.510 INFO:teuthology.orchestra.run.smithi094.stdout:(4/33): ceph-mgr-modules-core-17.2.7-819.g66994 2.6 MB/s | 240 kB 00:00 2024-04-06T03:27:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-06T03:27:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-06T03:27:26.643 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-06T03:27:26.660 INFO:teuthology.orchestra.run.smithi094.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-06T03:27:26.677 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-06T03:27:26.707 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-06T03:27:26.739 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-06T03:27:26.869 INFO:teuthology.orchestra.run.smithi094.stdout:(6/33): ceph-osd-17.2.7-819.g6699495c.el9.x86_6 18 MB/s | 15 MB 00:00 2024-04-06T03:27:26.875 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-06T03:27:26.911 INFO:teuthology.orchestra.run.smithi094.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 4.9 MB/s | 1.2 MB 00:00 2024-04-06T03:27:26.916 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-06T03:27:26.953 INFO:teuthology.orchestra.run.smithi094.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-04-06T03:27:26.959 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-06T03:27:26.994 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-06T03:27:26.995 INFO:teuthology.orchestra.run.smithi094.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-04-06T03:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-06T03:27:27.053 INFO:teuthology.orchestra.run.smithi094.stdout:(10/33): ceph-mon-17.2.7-819.g6699495c.el9.x86_ 3.4 MB/s | 3.9 MB 00:01 2024-04-06T03:27:27.070 INFO:teuthology.orchestra.run.smithi094.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 530 kB/s | 106 kB 00:00 2024-04-06T03:27:27.095 INFO:teuthology.orchestra.run.smithi094.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-04-06T03:27:27.105 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-06T03:27:27.242 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-06T03:27:27.323 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-06T03:27:27.377 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-06T03:27:27.415 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-06T03:27:27.472 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-06T03:27:27.528 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-06T03:27:27.704 INFO:teuthology.orchestra.run.smithi094.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 55 kB/s | 35 kB 00:00 2024-04-06T03:27:27.719 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-06T03:27:27.769 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-06T03:27:27.796 INFO:teuthology.orchestra.run.smithi094.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 73 kB/s | 51 kB 00:00 2024-04-06T03:27:27.838 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-06T03:27:27.913 INFO:teuthology.orchestra.run.smithi094.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 200 kB/s | 172 kB 00:00 2024-04-06T03:27:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-06T03:27:27.970 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-06T03:27:28.084 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-06T03:27:28.097 INFO:teuthology.orchestra.run.smithi094.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 145 kB/s | 43 kB 00:00 2024-04-06T03:27:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-819.g6699495c.el9.n 28/33 2024-04-06T03:27:28.270 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 29/33 2024-04-06T03:27:28.295 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 29/33 2024-04-06T03:27:28.295 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:28.295 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-06T03:27:28.295 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-06T03:27:28.295 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-06T03:27:28.295 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:28.330 INFO:teuthology.orchestra.run.smithi094.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-04-06T03:27:28.356 INFO:teuthology.orchestra.run.smithi094.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 388 kB/s | 172 kB 00:00 2024-04-06T03:27:28.381 INFO:teuthology.orchestra.run.smithi094.stdout:(19/33): python3-toml-0.10.2-6.el9.noarch.rpm 62 kB/s | 42 kB 00:00 2024-04-06T03:27:28.406 INFO:teuthology.orchestra.run.smithi094.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 141 kB/s | 11 kB 00:00 2024-04-06T03:27:28.432 INFO:teuthology.orchestra.run.smithi094.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 237 kB/s | 18 kB 00:00 2024-04-06T03:27:28.457 INFO:teuthology.orchestra.run.smithi094.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 306 kB/s | 23 kB 00:00 2024-04-06T03:27:28.482 INFO:teuthology.orchestra.run.smithi094.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 257 kB/s | 19 kB 00:00 2024-04-06T03:27:28.507 INFO:teuthology.orchestra.run.smithi094.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-04-06T03:27:28.533 INFO:teuthology.orchestra.run.smithi094.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-04-06T03:27:28.558 INFO:teuthology.orchestra.run.smithi094.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-04-06T03:27:28.608 INFO:teuthology.orchestra.run.smithi094.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-06T03:27:28.626 INFO:teuthology.orchestra.run.smithi094.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 178 kB/s | 16 kB 00:00 2024-04-06T03:27:28.651 INFO:teuthology.orchestra.run.smithi094.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 975 kB/s | 90 kB 00:00 2024-04-06T03:27:28.676 INFO:teuthology.orchestra.run.smithi094.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 535 kB/s | 36 kB 00:00 2024-04-06T03:27:28.710 INFO:teuthology.orchestra.run.smithi094.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-06T03:27:28.743 INFO:teuthology.orchestra.run.smithi094.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-04-06T03:27:28.776 INFO:teuthology.orchestra.run.smithi094.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 199 kB/s | 20 kB 00:00 2024-04-06T03:27:28.782 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:28.782 INFO:teuthology.orchestra.run.smithi094.stdout:Total 8.0 MB/s | 27 MB 00:03 2024-04-06T03:27:28.902 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:27:28.938 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:27:28.939 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:27:29.284 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:27:29.285 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:27:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:17.2.7-819.g6699495c.el9.x86_64 30/33 2024-04-06T03:27:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:17.2.7-819.g6699495c.el9.x86_64 30/33 2024-04-06T03:27:29.633 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:29.634 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-06T03:27:29.634 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-06T03:27:29.634 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-06T03:27:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:29.997 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:17.2.7-819.g6699495c.el9.x86_64 31/33 2024-04-06T03:27:30.020 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:17.2.7-819.g6699495c.el9.x86_64 31/33 2024-04-06T03:27:30.021 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:30.021 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-06T03:27:30.021 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-06T03:27:30.021 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-06T03:27:30.021 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:17.2.7-819.g6699495c.el9.x86_64 32/33 2024-04-06T03:27:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:17.2.7-819.g6699495c.el9.x86_64 32/33 2024-04-06T03:27:30.247 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:30.247 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-06T03:27:30.247 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-06T03:27:30.247 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-06T03:27:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:17.2.7-819.g6699495c.el9.x86_64 33/33 2024-04-06T03:27:30.529 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:27:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-06T03:27:30.645 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-06T03:27:30.678 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-06T03:27:30.769 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-06T03:27:30.847 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-06T03:27:30.897 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-06T03:27:30.941 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-06T03:27:30.971 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-06T03:27:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-06T03:27:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-06T03:27:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-06T03:27:31.114 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-06T03:27:31.149 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-06T03:27:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-06T03:27:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-06T03:27:31.405 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-06T03:27:31.486 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-06T03:27:31.540 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-06T03:27:31.579 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-06T03:27:31.628 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-06T03:27:31.683 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-06T03:27:31.883 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-06T03:27:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-2:17.2.7-819.g6699495c.el9.x86_64 33/33 2024-04-06T03:27:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:17.2.7-819.g6699495c.el9.x86_64 1/33 2024-04-06T03:27:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:17.2.7-819.g6699495c.el9.x86_64 2/33 2024-04-06T03:27:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 3/33 2024-04-06T03:27:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:17.2.7-819.g6699495c.el9.x86_64 4/33 2024-04-06T03:27:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:17.2.7-819.g6699495c.el9.x86_64 5/33 2024-04-06T03:27:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-819.g6699495c.el9.n 6/33 2024-04-06T03:27:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-06T03:27:31.919 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-06T03:27:31.920 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-06T03:27:31.921 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-06T03:27:31.921 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-06T03:27:31.921 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-06T03:27:31.922 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-06T03:27:31.922 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-06T03:27:31.922 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-06T03:27:31.922 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-06T03:27:31.922 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-06T03:27:31.922 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-06T03:27:31.922 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-06T03:27:31.941 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-06T03:27:32.010 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-06T03:27:32.099 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-06T03:27:32.137 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-06T03:27:32.253 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-06T03:27:32.412 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-819.g6699495c.el9.n 28/33 2024-04-06T03:27:32.443 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 29/33 2024-04-06T03:27:32.468 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 29/33 2024-04-06T03:27:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:32.468 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-06T03:27:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-06T03:27:32.469 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-06T03:27:32.469 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:33.030 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-06T03:27:33.031 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-06T03:27:33.032 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:33.033 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:27:33.389 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-base 2024-04-06T03:27:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:17.2.7-819.g6699495c.el9.x86_64 30/33 2024-04-06T03:27:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:17.2.7-819.g6699495c.el9.x86_64 30/33 2024-04-06T03:27:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-06T03:27:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-06T03:27:33.804 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-06T03:27:33.804 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:34.002 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:57 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:27:34.170 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:17.2.7-819.g6699495c.el9.x86_64 31/33 2024-04-06T03:27:34.193 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:17.2.7-819.g6699495c.el9.x86_64 31/33 2024-04-06T03:27:34.193 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:34.194 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-06T03:27:34.194 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-06T03:27:34.194 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-06T03:27:34.194 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:27:34.292 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.293 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:27:34.294 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:27:34.295 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:27:34.296 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.297 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.298 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.299 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.300 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:34.301 INFO:teuthology.orchestra.run.smithi136.stdout:Package ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:27:34.359 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:27:34.360 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-06T03:27:34.360 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:27:34.395 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:17.2.7-819.g6699495c.el9.x86_64 32/33 2024-04-06T03:27:34.419 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:17.2.7-819.g6699495c.el9.x86_64 32/33 2024-04-06T03:27:34.419 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:34.419 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-06T03:27:34.419 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-06T03:27:34.420 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-06T03:27:34.420 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:34.432 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install cephadm 2024-04-06T03:27:34.484 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:17.2.7-819.g6699495c.el9.x86_64 33/33 2024-04-06T03:27:35.029 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:58 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:27:35.321 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:27:35.321 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:27:35.321 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:27:35.321 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.321 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:27:35.321 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.321 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:27:35.321 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.322 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.323 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.324 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:27:35.325 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.326 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.327 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.328 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.329 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:35.389 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:27:35.390 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:35.390 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-06T03:27:35.390 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:35.390 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:27:35.390 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 82 k 2024-04-06T03:27:35.390 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:35.390 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:27:35.390 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:35.390 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2024-04-06T03:27:35.391 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:35.391 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 82 k 2024-04-06T03:27:35.391 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 372 k 2024-04-06T03:27:35.391 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:27:35.741 INFO:teuthology.orchestra.run.smithi136.stdout:cephadm-17.2.7-819.g6699495c.el9.noarch.rpm 234 kB/s | 82 kB 00:00 2024-04-06T03:27:35.750 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:35.750 INFO:teuthology.orchestra.run.smithi136.stdout:Total 233 kB/s | 82 kB 00:00 2024-04-06T03:27:35.750 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:27:35.750 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:27:35.750 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:27:35.750 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:27:35.750 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:27:35.819 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:27:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-2:17.2.7-819.g6699495c.el9.x86_64 33/33 2024-04-06T03:27:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:17.2.7-819.g6699495c.el9.x86_64 1/33 2024-04-06T03:27:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:17.2.7-819.g6699495c.el9.x86_64 2/33 2024-04-06T03:27:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 3/33 2024-04-06T03:27:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:17.2.7-819.g6699495c.el9.x86_64 4/33 2024-04-06T03:27:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:17.2.7-819.g6699495c.el9.x86_64 5/33 2024-04-06T03:27:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-819.g6699495c.el9.n 6/33 2024-04-06T03:27:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-06T03:27:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-06T03:27:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-06T03:27:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-06T03:27:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-06T03:27:36.130 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-06T03:27:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-06T03:27:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-06T03:27:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-06T03:27:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-06T03:27:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-06T03:27:36.131 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-06T03:27:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-06T03:27:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-06T03:27:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-06T03:27:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-06T03:27:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-06T03:27:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-06T03:27:36.132 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-06T03:27:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-06T03:27:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-06T03:27:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-06T03:27:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-06T03:27:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-06T03:27:36.133 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-06T03:27:36.134 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-06T03:27:36.187 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/1 2024-04-06T03:27:36.277 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/1 2024-04-06T03:27:36.642 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/1 2024-04-06T03:27:36.939 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/1 2024-04-06T03:27:36.940 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:36.940 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:27:36.940 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:36.940 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:36.940 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:27:37.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-06T03:27:37.063 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-06T03:27:37.064 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:37.065 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:27:37.147 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-immutable-object-cache 2024-04-06T03:27:37.428 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-base 2024-04-06T03:27:37.753 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:00 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:27:38.027 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:55 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:27:38.041 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.042 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:27:38.043 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.044 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:27:38.045 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:27:38.046 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.047 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.048 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:38.049 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.050 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.109 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:27:38.110 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:38.110 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repo Size 2024-04-06T03:27:38.110 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:38.110 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:27:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-819.g6699495c.el9 ceph 138 k 2024-04-06T03:27:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:38.111 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:27:38.111 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:38.111 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2024-04-06T03:27:38.111 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:38.111 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 138 k 2024-04-06T03:27:38.111 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 379 k 2024-04-06T03:27:38.111 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:27:38.319 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-immutable-object-cache-17.2.7-819.g6699495 662 kB/s | 138 kB 00:00 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:27:38.320 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.321 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:27:38.322 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.323 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.324 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.325 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.326 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.327 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.328 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.328 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.328 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:38.328 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.328 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.328 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.328 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:38.329 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:38.329 INFO:teuthology.orchestra.run.smithi136.stdout:Total 658 kB/s | 138 kB 00:00 2024-04-06T03:27:38.329 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:27:38.329 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:27:38.329 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:27:38.329 INFO:teuthology.orchestra.run.smithi094.stdout:Package ceph-base-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:27:38.381 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:27:38.381 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:27:38.387 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:27:38.388 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-06T03:27:38.388 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:27:38.458 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install cephadm 2024-04-06T03:27:38.520 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:27:38.595 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-819.g6699495c.e 1/1 2024-04-06T03:27:38.620 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-819.g6699495c.e 1/1 2024-04-06T03:27:38.620 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:38.620 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-06T03:27:38.620 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:39.048 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:56 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:27:39.337 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.338 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:27:39.339 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.340 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.341 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:27:39.342 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.343 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.344 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:39.345 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.346 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.346 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.346 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.346 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:39.346 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.346 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.346 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.346 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:39.405 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:27:39.406 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-06T03:27:39.406 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:39.407 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:27:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 82 k 2024-04-06T03:27:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:39.407 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:27:39.407 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:39.407 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-04-06T03:27:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:39.408 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 82 k 2024-04-06T03:27:39.408 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 372 k 2024-04-06T03:27:39.408 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:27:39.596 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm-17.2.7-819.g6699495c.el9.noarch.rpm 434 kB/s | 82 kB 00:00 2024-04-06T03:27:39.597 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:39.597 INFO:teuthology.orchestra.run.smithi094.stdout:Total 430 kB/s | 82 kB 00:00 2024-04-06T03:27:39.597 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:27:39.602 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:27:39.602 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:27:39.604 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:27:39.604 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:27:39.633 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-819.g6699495c.e 1/1 2024-04-06T03:27:39.633 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:39.633 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:27:39.633 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:39.633 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:39.633 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:27:39.679 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:27:39.904 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr 2024-04-06T03:27:40.074 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/1 2024-04-06T03:27:40.164 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/1 2024-04-06T03:27:40.514 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:03 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:27:40.515 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/1 2024-04-06T03:27:40.801 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.802 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.803 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.804 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.805 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.806 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:27:40.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.810 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:40.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:40.810 INFO:teuthology.orchestra.run.smithi136.stdout:Package ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:27:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/1 2024-04-06T03:27:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:40.826 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:27:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:40.826 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:27:40.869 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:27:40.870 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-06T03:27:40.870 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:27:40.960 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr-dashboard 2024-04-06T03:27:41.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-immutable-object-cache 2024-04-06T03:27:41.553 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:04 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:27:41.623 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:58 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:27:41.842 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:27:41.842 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:27:41.842 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:27:41.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.842 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:27:41.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.842 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.843 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:27:41.844 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:27:41.845 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.846 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:27:41.847 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.848 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.849 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:41.850 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.850 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.850 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.850 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.850 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:41.850 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.850 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.850 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.851 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.851 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:41.851 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.851 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.851 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.851 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.910 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 1.7 M 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 24 k 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 15 k 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-06T03:27:41.911 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-06T03:27:41.912 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-06T03:27:41.912 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-04-06T03:27:41.912 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-06T03:27:41.912 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:41.912 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:27:41.912 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:41.912 INFO:teuthology.orchestra.run.smithi136.stdout:Install 7 Packages 2024-04-06T03:27:41.912 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:27:41.913 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.914 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.915 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.916 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.917 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.918 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.919 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.920 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.921 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:41.922 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 2.0 M 2024-04-06T03:27:41.922 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 8.1 M 2024-04-06T03:27:41.922 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:27:41.979 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:27:41.979 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:41.979 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-04-06T03:27:41.980 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:41.980 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:27:41.980 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-819.g6699495c.el9 ceph 138 k 2024-04-06T03:27:41.980 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:41.980 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:27:41.980 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:41.980 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-04-06T03:27:41.980 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:41.981 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 138 k 2024-04-06T03:27:41.981 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 379 k 2024-04-06T03:27:41.981 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:27:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-immutable-object-cache-17.2.7-819.g6699495 709 kB/s | 138 kB 00:00 2024-04-06T03:27:42.175 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:Total 705 kB/s | 138 kB 00:00 2024-04-06T03:27:42.176 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:27:42.183 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:27:42.183 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:27:42.236 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:27:42.236 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:27:42.264 INFO:teuthology.orchestra.run.smithi136.stdout:(1/7): ceph-grafana-dashboards-17.2.7-819.g6699 127 kB/s | 24 kB 00:00 2024-04-06T03:27:42.289 INFO:teuthology.orchestra.run.smithi136.stdout:(2/7): ceph-prometheus-alerts-17.2.7-819.g66994 67 kB/s | 15 kB 00:00 2024-04-06T03:27:42.388 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:27:42.451 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-819.g6699495c.e 1/1 2024-04-06T03:27:42.475 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-819.g6699495c.e 1/1 2024-04-06T03:27:42.475 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:27:42.476 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-06T03:27:42.476 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:42.582 INFO:teuthology.orchestra.run.smithi136.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 28 kB/s | 9.0 kB 00:00 2024-04-06T03:27:42.707 INFO:teuthology.orchestra.run.smithi136.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 98 kB/s | 41 kB 00:00 2024-04-06T03:27:42.766 INFO:teuthology.orchestra.run.smithi136.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 168 kB/s | 31 kB 00:00 2024-04-06T03:27:42.983 INFO:teuthology.orchestra.run.smithi136.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 685 kB/s | 188 kB 00:00 2024-04-06T03:27:43.241 INFO:teuthology.orchestra.run.smithi136.stdout:(7/7): ceph-mgr-dashboard-17.2.7-819.g6699495c. 1.5 MB/s | 1.7 MB 00:01 2024-04-06T03:27:43.245 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:43.245 INFO:teuthology.orchestra.run.smithi136.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2024-04-06T03:27:43.246 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:27:43.254 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:27:43.254 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:27:43.317 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:27:43.317 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:27:43.470 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-819.g6699495c.e 1/1 2024-04-06T03:27:43.470 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:43.470 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:27:43.470 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:27:43.470 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:43.470 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:27:43.621 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:27:43.676 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-06T03:27:43.710 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-06T03:27:43.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr 2024-04-06T03:27:43.765 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-06T03:27:43.805 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-06T03:27:43.834 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-819.g6699495c.el9.no 5/7 2024-04-06T03:27:44.063 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-819.g6699495c.el9.n 6/7 2024-04-06T03:27:44.095 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-819.g6699495c.el9.noarch 7/7 2024-04-06T03:27:44.366 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:01 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:27:44.602 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-819.g6699495c.el9.noarch 7/7 2024-04-06T03:27:44.602 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-819.g6699495c.el9.n 1/7 2024-04-06T03:27:44.602 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-819.g6699495c.el9.noarch 2/7 2024-04-06T03:27:44.602 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-819.g6699495c.el9.no 3/7 2024-04-06T03:27:44.602 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-06T03:27:44.602 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-06T03:27:44.602 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-06T03:27:44.658 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:27:44.658 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:27:44.658 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:27:44.658 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.658 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:27:44.658 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.658 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:27:44.658 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.659 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:27:44.660 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:27:44.661 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:27:44.662 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.663 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.664 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:44.666 INFO:teuthology.orchestra.run.smithi094.stdout:Package ceph-mgr-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:27:44.724 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:27:44.725 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-06T03:27:44.725 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:27:44.791 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-dashboard 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-06T03:27:44.916 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-06T03:27:44.917 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:44.917 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:27:45.177 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-06T03:27:45.376 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:02 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:27:45.664 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:27:45.665 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.666 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:27:45.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.666 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:27:45.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.666 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:27:45.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.666 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:27:45.666 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.666 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.667 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:27:45.668 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:27:45.669 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:45.670 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.671 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.672 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.673 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:45.674 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.674 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.674 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.674 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:45.734 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:27:45.735 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:45.735 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-06T03:27:45.735 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:45.735 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:27:45.735 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 1.7 M 2024-04-06T03:27:45.735 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-06T03:27:45.735 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 24 k 2024-04-06T03:27:45.735 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 15 k 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout:Install 7 Packages 2024-04-06T03:27:45.736 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:45.737 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 2.0 M 2024-04-06T03:27:45.737 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 8.1 M 2024-04-06T03:27:45.737 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:27:45.770 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:08 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:27:45.980 INFO:teuthology.orchestra.run.smithi094.stdout:(1/7): ceph-prometheus-alerts-17.2.7-819.g66994 76 kB/s | 15 kB 00:00 2024-04-06T03:27:46.005 INFO:teuthology.orchestra.run.smithi094.stdout:(2/7): ceph-grafana-dashboards-17.2.7-819.g6699 113 kB/s | 24 kB 00:00 2024-04-06T03:27:46.059 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.060 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:27:46.061 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.062 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.063 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.064 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.065 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.066 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.067 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.068 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:46.127 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================== 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================== 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 7.4 M 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-06T03:27:46.129 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================== 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout:Install 12 Packages 2024-04-06T03:27:46.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:46.131 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 38 M 2024-04-06T03:27:46.131 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 209 M 2024-04-06T03:27:46.131 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:27:46.181 INFO:teuthology.orchestra.run.smithi094.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 45 kB/s | 9.0 kB 00:00 2024-04-06T03:27:46.331 INFO:teuthology.orchestra.run.smithi094.stdout:(4/7): ceph-mgr-dashboard-17.2.7-819.g6699495c. 3.1 MB/s | 1.7 MB 00:00 2024-04-06T03:27:46.356 INFO:teuthology.orchestra.run.smithi094.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 117 kB/s | 41 kB 00:00 2024-04-06T03:27:46.381 INFO:teuthology.orchestra.run.smithi094.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 153 kB/s | 31 kB 00:00 2024-04-06T03:27:46.573 INFO:teuthology.orchestra.run.smithi094.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 778 kB/s | 188 kB 00:00 2024-04-06T03:27:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:Total 2.4 MB/s | 2.0 MB 00:00 2024-04-06T03:27:46.575 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:27:46.583 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:27:46.584 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:27:46.650 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:27:46.650 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:27:46.916 INFO:teuthology.orchestra.run.smithi136.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 379 kB/s | 195 kB 00:00 2024-04-06T03:27:46.952 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:27:46.999 INFO:teuthology.orchestra.run.smithi136.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-04-06T03:27:47.008 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-06T03:27:47.042 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-06T03:27:47.074 INFO:teuthology.orchestra.run.smithi136.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-04-06T03:27:47.087 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-06T03:27:47.129 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-06T03:27:47.150 INFO:teuthology.orchestra.run.smithi136.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 243 kB/s | 18 kB 00:00 2024-04-06T03:27:47.158 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-819.g6699495c.el9.no 5/7 2024-04-06T03:27:47.208 INFO:teuthology.orchestra.run.smithi136.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2024-04-06T03:27:47.233 INFO:teuthology.orchestra.run.smithi136.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 422 kB/s | 35 kB 00:00 2024-04-06T03:27:47.334 INFO:teuthology.orchestra.run.smithi136.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 38 MB/s | 4.7 MB 00:00 2024-04-06T03:27:47.386 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-819.g6699495c.el9.n 6/7 2024-04-06T03:27:47.419 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-819.g6699495c.el9.noarch 7/7 2024-04-06T03:27:47.459 INFO:teuthology.orchestra.run.smithi136.stdout:(8/12): ceph-mgr-diskprediction-local-17.2.7-81 7.0 MB/s | 7.4 MB 00:01 2024-04-06T03:27:47.484 INFO:teuthology.orchestra.run.smithi136.stdout:(9/12): python3-devel-3.9.17-2.el9.x86_64.rpm 979 kB/s | 245 kB 00:00 2024-04-06T03:27:47.576 INFO:teuthology.orchestra.run.smithi136.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-04-06T03:27:47.652 INFO:teuthology.orchestra.run.smithi136.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.3 MB/s | 250 kB 00:00 2024-04-06T03:27:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-04-06T03:27:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:Total 22 MB/s | 38 MB 00:01 2024-04-06T03:27:47.914 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-819.g6699495c.el9.noarch 7/7 2024-04-06T03:27:47.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-819.g6699495c.el9.n 1/7 2024-04-06T03:27:47.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-819.g6699495c.el9.noarch 2/7 2024-04-06T03:27:47.915 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-819.g6699495c.el9.no 3/7 2024-04-06T03:27:47.915 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-06T03:27:47.915 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-06T03:27:47.915 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-06T03:27:48.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-06T03:27:48.227 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:48.227 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:27:48.227 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:48.227 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:48.227 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:48.227 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-06T03:27:48.227 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-06T03:27:48.227 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-06T03:27:48.227 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-06T03:27:48.228 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:48.228 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:27:48.236 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:27:48.262 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:27:48.263 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:27:48.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-06T03:27:48.766 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:27:48.767 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:27:49.066 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:06 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:27:49.353 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.354 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.355 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:27:49.356 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:27:49.357 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:27:49.358 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.359 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.360 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.361 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:27:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:====================================================================================== 2024-04-06T03:27:49.423 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-06T03:27:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:====================================================================================== 2024-04-06T03:27:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:27:49.423 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 7.4 M 2024-04-06T03:27:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-06T03:27:49.423 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:====================================================================================== 2024-04-06T03:27:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:Install 12 Packages 2024-04-06T03:27:49.425 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:49.425 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 38 M 2024-04-06T03:27:49.425 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 209 M 2024-04-06T03:27:49.425 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:27:49.518 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:27:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-06T03:27:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-06T03:27:49.749 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-06T03:27:49.863 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-06T03:27:50.014 INFO:teuthology.orchestra.run.smithi094.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 768 kB/s | 195 kB 00:00 2024-04-06T03:27:50.064 INFO:teuthology.orchestra.run.smithi094.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-04-06T03:27:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-06T03:27:50.114 INFO:teuthology.orchestra.run.smithi094.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-04-06T03:27:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-06T03:27:50.164 INFO:teuthology.orchestra.run.smithi094.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 365 kB/s | 18 kB 00:00 2024-04-06T03:27:50.226 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-06T03:27:50.232 INFO:teuthology.orchestra.run.smithi094.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 528 kB/s | 35 kB 00:00 2024-04-06T03:27:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-04-06T03:27:50.383 INFO:teuthology.orchestra.run.smithi094.stdout:(7/12): python3-devel-3.9.17-2.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-04-06T03:27:50.533 INFO:teuthology.orchestra.run.smithi094.stdout:(8/12): ceph-mgr-diskprediction-local-17.2.7-81 9.6 MB/s | 7.4 MB 00:00 2024-04-06T03:27:50.608 INFO:teuthology.orchestra.run.smithi094.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2024-04-06T03:27:50.683 INFO:teuthology.orchestra.run.smithi094.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.6 MB/s | 250 kB 00:00 2024-04-06T03:27:50.784 INFO:teuthology.orchestra.run.smithi094.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-06T03:27:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-06T03:27:50.971 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-06T03:27:51.384 INFO:teuthology.orchestra.run.smithi094.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-06T03:27:51.386 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:27:51.387 INFO:teuthology.orchestra.run.smithi094.stdout:Total 19 MB/s | 38 MB 00:01 2024-04-06T03:27:51.736 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:27:51.763 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:27:51.763 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:27:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-06T03:27:52.266 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:27:52.267 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:27:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:27:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-06T03:27:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-06T03:27:53.235 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-06T03:27:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-06T03:27:53.259 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-819.g669949 12/12 2024-04-06T03:27:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-06T03:27:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-06T03:27:53.515 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-06T03:27:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-06T03:27:54.284 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-06T03:27:54.404 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-06T03:27:55.644 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-06T03:27:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-819.g669949 12/12 2024-04-06T03:27:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-819.g669949 1/12 2024-04-06T03:27:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-06T03:27:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-06T03:27:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-06T03:27:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-06T03:27:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-06T03:27:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-06T03:27:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-06T03:27:55.775 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 9/12 2024-04-06T03:27:55.775 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-06T03:27:55.775 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-06T03:27:56.184 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-06T03:27:56.184 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:56.184 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-06T03:27:56.185 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:56.186 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:27:56.587 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr-rook 2024-04-06T03:27:56.690 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-06T03:27:56.716 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-819.g669949 12/12 2024-04-06T03:27:57.194 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:20 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:27:57.481 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:27:57.481 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:27:57.481 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:27:57.481 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.481 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:27:57.481 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.481 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:27:57.482 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.483 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.484 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.485 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:27:57.486 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.486 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:27:57.486 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.486 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:27:57.486 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.487 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.488 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.489 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.490 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.491 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.491 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:27:57.549 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:27:57.550 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:57.550 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-06T03:27:57.550 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:57.550 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 47 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-06T03:27:57.551 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-06T03:27:57.552 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-06T03:27:57.552 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:57.552 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:27:57.552 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:27:57.552 INFO:teuthology.orchestra.run.smithi136.stdout:Install 13 Packages 2024-04-06T03:27:57.552 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:27:57.552 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 2.2 M 2024-04-06T03:27:57.553 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 26 M 2024-04-06T03:27:57.553 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:27:58.118 INFO:teuthology.orchestra.run.smithi136.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 215 kB/s | 19 kB 00:00 2024-04-06T03:27:58.143 INFO:teuthology.orchestra.run.smithi136.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 224 kB/s | 26 kB 00:00 2024-04-06T03:27:58.226 INFO:teuthology.orchestra.run.smithi136.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-04-06T03:27:58.252 INFO:teuthology.orchestra.run.smithi136.stdout:(4/13): ceph-mgr-rook-17.2.7-819.g6699495c.el9. 212 kB/s | 47 kB 00:00 2024-04-06T03:27:58.277 INFO:teuthology.orchestra.run.smithi136.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-06T03:27:58.302 INFO:teuthology.orchestra.run.smithi136.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-04-06T03:27:58.327 INFO:teuthology.orchestra.run.smithi136.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-04-06T03:27:58.753 INFO:teuthology.orchestra.run.smithi136.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 33 kB/s | 15 kB 00:00 2024-04-06T03:27:58.778 INFO:teuthology.orchestra.run.smithi136.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 64 kB/s | 32 kB 00:00 2024-04-06T03:27:58.970 INFO:teuthology.orchestra.run.smithi136.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 308 kB/s | 59 kB 00:00 2024-04-06T03:27:59.129 INFO:teuthology.orchestra.run.smithi136.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 253 kB/s | 203 kB 00:00 2024-04-06T03:27:59.162 INFO:teuthology.orchestra.run.smithi136.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 467 kB/s | 90 kB 00:00 2024-04-06T03:27:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-819.g669949 12/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-819.g669949 1/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 9/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-06T03:27:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-06T03:27:59.791 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-06T03:27:59.792 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-06T03:27:59.793 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-06T03:27:59.793 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-06T03:27:59.793 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:27:59.793 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:00.145 INFO:teuthology.orchestra.run.smithi136.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 751 kB/s | 1.0 MB 00:01 2024-04-06T03:28:00.149 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:00.149 INFO:teuthology.orchestra.run.smithi136.stdout:Total 866 kB/s | 2.2 MB 00:02 2024-04-06T03:28:00.200 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-rook 2024-04-06T03:28:00.217 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:28:00.233 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:28:00.233 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:28:00.328 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:28:00.329 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:28:00.806 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:17 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:00.837 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:28:01.033 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-06T03:28:01.097 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:01.097 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:01.097 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:01.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.097 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:01.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.098 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.099 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.100 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.101 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.102 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.103 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:01.104 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.105 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:01.106 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.106 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.106 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.106 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.106 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:01.106 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.106 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.106 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.106 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:01.121 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-06T03:28:01.165 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 47 k 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-06T03:28:01.167 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout:Install 13 Packages 2024-04-06T03:28:01.168 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:01.169 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 2.2 M 2024-04-06T03:28:01.169 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 26 M 2024-04-06T03:28:01.169 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:28:01.185 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-06T03:28:01.214 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-06T03:28:01.256 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-06T03:28:01.349 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-06T03:28:01.466 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-06T03:28:01.516 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-06T03:28:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:(1/13): ceph-mgr-rook-17.2.7-819.g6699495c.el9. 255 kB/s | 47 kB 00:00 2024-04-06T03:28:02.127 INFO:teuthology.orchestra.run.smithi094.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 36 kB/s | 19 kB 00:00 2024-04-06T03:28:02.219 INFO:teuthology.orchestra.run.smithi094.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 41 kB/s | 26 kB 00:00 2024-04-06T03:28:02.236 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-06T03:28:02.288 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-06T03:28:02.322 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-06T03:28:02.368 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-06T03:28:02.392 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:17.2.7-819.g6699495c.el9.noarch 13/13 2024-04-06T03:28:02.727 INFO:teuthology.orchestra.run.smithi094.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 236 kB/s | 222 kB 00:00 2024-04-06T03:28:02.994 INFO:teuthology.orchestra.run.smithi094.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 183 kB/s | 159 kB 00:00 2024-04-06T03:28:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-819.g6699495c.el9.noarch 13/13 2024-04-06T03:28:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:17.2.7-819.g6699495c.el9.noarch 1/13 2024-04-06T03:28:03.180 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-06T03:28:03.181 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-06T03:28:03.211 INFO:teuthology.orchestra.run.smithi094.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 281 kB/s | 279 kB 00:00 2024-04-06T03:28:03.228 INFO:teuthology.orchestra.run.smithi094.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 138 kB/s | 32 kB 00:00 2024-04-06T03:28:03.387 INFO:teuthology.orchestra.run.smithi094.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 81 kB/s | 54 kB 00:00 2024-04-06T03:28:03.454 INFO:teuthology.orchestra.run.smithi094.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2024-04-06T03:28:03.553 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-06T03:28:03.553 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:03.553 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:28:03.553 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:28:03.553 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:03.554 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:03.579 INFO:teuthology.orchestra.run.smithi094.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 578 kB/s | 203 kB 00:00 2024-04-06T03:28:03.613 INFO:teuthology.orchestra.run.smithi094.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 372 kB/s | 59 kB 00:00 2024-04-06T03:28:03.754 INFO:teuthology.orchestra.run.smithi094.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 512 kB/s | 90 kB 00:00 2024-04-06T03:28:03.815 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-mgr-cephadm 2024-04-06T03:28:03.988 INFO:teuthology.orchestra.run.smithi094.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.7 MB/s | 1.0 MB 00:00 2024-04-06T03:28:03.989 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:03.989 INFO:teuthology.orchestra.run.smithi094.stdout:Total 797 kB/s | 2.2 MB 00:02 2024-04-06T03:28:04.050 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:28:04.065 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:28:04.065 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:28:04.159 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:28:04.160 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:28:04.413 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:27 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:04.687 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:28:04.703 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:04.703 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:04.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:04.705 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:04.706 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.707 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:04.708 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.709 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.710 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.711 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.712 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:04.773 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 123 k 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-06T03:28:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-06T03:28:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-06T03:28:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:04.775 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:28:04.775 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:04.775 INFO:teuthology.orchestra.run.smithi136.stdout:Install 6 Packages 2024-04-06T03:28:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:04.775 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 7.0 M 2024-04-06T03:28:04.776 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 33 M 2024-04-06T03:28:04.776 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:28:04.860 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-06T03:28:04.914 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-06T03:28:04.970 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-06T03:28:05.007 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-06T03:28:05.048 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-06T03:28:05.057 INFO:teuthology.orchestra.run.smithi136.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 2.4 MB/s | 248 kB 00:00 2024-04-06T03:28:05.142 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-06T03:28:05.165 INFO:teuthology.orchestra.run.smithi136.stdout:(2/6): ceph-mgr-cephadm-17.2.7-819.g6699495c.el 587 kB/s | 123 kB 00:00 2024-04-06T03:28:05.241 INFO:teuthology.orchestra.run.smithi136.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-04-06T03:28:05.251 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-06T03:28:05.300 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-06T03:28:05.341 INFO:teuthology.orchestra.run.smithi136.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 329 kB/s | 58 kB 00:00 2024-04-06T03:28:05.366 INFO:teuthology.orchestra.run.smithi136.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.7 MB/s | 548 kB 00:00 2024-04-06T03:28:05.383 INFO:teuthology.orchestra.run.smithi136.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 359 kB/s | 51 kB 00:00 2024-04-06T03:28:05.383 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:05.384 INFO:teuthology.orchestra.run.smithi136.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-04-06T03:28:05.454 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:28:05.466 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:28:05.466 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:28:05.569 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:28:05.570 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:28:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:28:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-06T03:28:06.049 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-06T03:28:06.083 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-06T03:28:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-06T03:28:06.116 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-06T03:28:06.170 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-06T03:28:06.195 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:17.2.7-819.g6699495c.el9.noarch 13/13 2024-04-06T03:28:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-06T03:28:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-06T03:28:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-06T03:28:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-819.g6699495c.el9.noarch 6/6 2024-04-06T03:28:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-819.g6699495c.el9.noarch 13/13 2024-04-06T03:28:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:17.2.7-819.g6699495c.el9.noarch 1/13 2024-04-06T03:28:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-06T03:28:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-06T03:28:07.365 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-06T03:28:07.365 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:07.365 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:28:07.365 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:28:07.365 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-06T03:28:07.365 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-06T03:28:07.365 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:07.366 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:07.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-cephadm 2024-04-06T03:28:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-819.g6699495c.el9.noarch 6/6 2024-04-06T03:28:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/6 2024-04-06T03:28:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-06T03:28:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-06T03:28:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-06T03:28:07.641 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-06T03:28:07.984 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-06T03:28:07.984 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:07.984 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:28:07.984 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:28:07.984 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-06T03:28:07.984 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-06T03:28:07.984 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-06T03:28:07.984 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-06T03:28:07.985 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-06T03:28:07.985 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:07.985 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:08.240 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:25 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:08.264 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-fuse 2024-04-06T03:28:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:08.531 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:08.531 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:08.531 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.531 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:08.531 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:08.532 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.533 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.534 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:08.535 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.536 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:08.537 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.538 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.539 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.540 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:08.600 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:08.600 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:08.600 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:17.2.7-819.g6699495c.el9 ceph-noarch 123 k 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout:Install 6 Packages 2024-04-06T03:28:08.601 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 7.0 M 2024-04-06T03:28:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 33 M 2024-04-06T03:28:08.602 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:28:08.866 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:31 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:09.154 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:09.154 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:09.155 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:09.156 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.157 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:09.158 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:09.159 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.160 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.161 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.162 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.163 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:09.223 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:09.224 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:09.224 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-06T03:28:09.224 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:17.2.7-819.g6699495c.el9 ceph 813 k 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout:Install 2 Packages 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 893 k 2024-04-06T03:28:09.225 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 2.6 M 2024-04-06T03:28:09.226 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:28:10.018 INFO:teuthology.orchestra.run.smithi136.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 153 kB/s | 80 kB 00:00 2024-04-06T03:28:10.276 INFO:teuthology.orchestra.run.smithi094.stdout:(1/6): ceph-mgr-cephadm-17.2.7-819.g6699495c.el 203 kB/s | 123 kB 00:00 2024-04-06T03:28:10.735 INFO:teuthology.orchestra.run.smithi094.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.2 MB/s | 548 kB 00:00 2024-04-06T03:28:10.810 INFO:teuthology.orchestra.run.smithi094.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 770 kB/s | 58 kB 00:00 2024-04-06T03:28:10.877 INFO:teuthology.orchestra.run.smithi094.stdout:(4/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 205 kB/s | 248 kB 00:01 2024-04-06T03:28:10.902 INFO:teuthology.orchestra.run.smithi094.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 554 kB/s | 51 kB 00:00 2024-04-06T03:28:11.102 INFO:teuthology.orchestra.run.smithi136.stdout:(2/2): ceph-fuse-17.2.7-819.g6699495c.el9.x86_6 506 kB/s | 813 kB 00:01 2024-04-06T03:28:11.103 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:11.103 INFO:teuthology.orchestra.run.smithi136.stdout:Total 475 kB/s | 893 kB 00:01 2024-04-06T03:28:11.115 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:28:11.139 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:28:11.139 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:28:11.217 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:28:11.217 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:28:11.400 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:28:11.539 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-06T03:28:11.612 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:17.2.7-819.g6699495c.el9.x86_64 2/2 2024-04-06T03:28:12.307 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-fuse-2:17.2.7-819.g6699495c.el9.x86_64 2/2 2024-04-06T03:28:12.307 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:17.2.7-819.g6699495c.el9.x86_64 1/2 2024-04-06T03:28:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-06T03:28:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:12.650 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:28:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:17.2.7-819.g6699495c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-06T03:28:12.650 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:12.650 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:12.907 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install librados-devel 2024-04-06T03:28:13.513 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:36 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:13.806 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:13.806 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:13.806 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:13.806 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.806 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:13.806 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:13.807 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.808 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:13.809 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.810 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.811 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.812 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:13.813 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.814 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:13.815 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.815 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.815 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.815 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:13.875 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:13.876 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:13.876 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repo Size 2024-04-06T03:28:13.876 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:13.876 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:28:13.876 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:17.2.7-819.g6699495c.el9 ceph 126 k 2024-04-06T03:28:13.876 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:13.876 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:28:13.876 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:13.876 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2024-04-06T03:28:13.877 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:13.877 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 126 k 2024-04-06T03:28:13.877 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 448 k 2024-04-06T03:28:13.877 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:28:14.225 INFO:teuthology.orchestra.run.smithi136.stdout:librados-devel-17.2.7-819.g6699495c.el9.x86_64. 363 kB/s | 126 kB 00:00 2024-04-06T03:28:14.226 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:14.226 INFO:teuthology.orchestra.run.smithi136.stdout:Total 361 kB/s | 126 kB 00:00 2024-04-06T03:28:14.226 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:28:14.231 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:28:14.232 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:28:14.291 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:28:14.291 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:28:14.478 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:28:14.582 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:15.318 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:15.319 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:15.319 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:28:15.319 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:15.319 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:15.319 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:15.602 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install libcephfs2 2024-04-06T03:28:16.206 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:39 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.496 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.497 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.498 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:16.499 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.500 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.501 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.502 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.503 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.504 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:16.505 INFO:teuthology.orchestra.run.smithi136.stdout:Package libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:16.564 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:16.565 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-06T03:28:16.565 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:16.664 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install libcephfs-devel 2024-04-06T03:28:16.794 INFO:teuthology.orchestra.run.smithi094.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 857 kB/s | 6.0 MB 00:07 2024-04-06T03:28:16.796 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:16.797 INFO:teuthology.orchestra.run.smithi094.stdout:Total 870 kB/s | 7.0 MB 00:08 2024-04-06T03:28:16.870 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:28:16.882 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:28:16.882 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:28:16.984 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:28:16.985 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:28:17.257 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:40 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:28:17.461 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-06T03:28:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-06T03:28:17.549 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:17.549 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:17.549 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:17.550 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.550 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:17.550 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.550 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:17.550 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.550 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:17.550 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.550 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:17.550 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:17.551 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:17.552 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:17.552 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:17.552 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.552 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:17.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:17.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:17.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:17.553 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.553 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:17.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:17.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:17.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:17.554 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.554 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:17.555 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.555 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:17.555 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.555 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:17.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:17.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:17.556 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.556 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:17.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.557 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:17.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.557 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:17.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.557 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:17.557 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.558 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:17.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.558 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:17.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.558 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:17.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.558 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:17.558 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.559 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:17.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.559 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:17.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.559 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.560 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:17.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.560 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.560 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:17.561 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.561 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.561 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.561 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.561 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.561 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.561 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:17.562 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:17.563 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.564 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.564 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.564 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:17.617 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:17.618 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:17.618 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repo Size 2024-04-06T03:28:17.618 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:17.618 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:28:17.618 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:17.2.7-819.g6699495c.el9 ceph 25 k 2024-04-06T03:28:17.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:28:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2024-04-06T03:28:17.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 25 k 2024-04-06T03:28:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 110 k 2024-04-06T03:28:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:28:17.863 INFO:teuthology.orchestra.run.smithi136.stdout:libcephfs-devel-17.2.7-819.g6699495c.el9.x86_64 102 kB/s | 25 kB 00:00 2024-04-06T03:28:17.863 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:17.863 INFO:teuthology.orchestra.run.smithi136.stdout:Total 102 kB/s | 25 kB 00:00 2024-04-06T03:28:17.864 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:28:17.867 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:28:17.867 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:28:17.868 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:28:17.869 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:28:17.945 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:28:18.035 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-06T03:28:18.068 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-06T03:28:18.178 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-06T03:28:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-819.g6699495c.el9.noarch 6/6 2024-04-06T03:28:18.443 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:18.721 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:28:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:18.721 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:18.721 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:18.950 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install librados2 2024-04-06T03:28:19.017 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-819.g6699495c.el9.noarch 6/6 2024-04-06T03:28:19.017 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-819.g6699495c.el9.noarch 1/6 2024-04-06T03:28:19.017 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-06T03:28:19.017 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-06T03:28:19.017 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-06T03:28:19.017 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-06T03:28:19.322 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:17.2.7-819.g6699495c.el9.noarch 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:19.323 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:19.547 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:42 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:19.582 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-fuse 2024-04-06T03:28:19.834 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.835 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:19.836 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:19.837 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.838 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.839 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.840 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.841 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.842 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.843 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:19.844 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:19.904 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:19.905 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-06T03:28:19.906 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:20.017 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install librbd1 2024-04-06T03:28:20.186 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:37 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:20.473 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:20.473 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:20.473 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.474 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:20.475 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.476 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.477 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:20.478 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.479 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:20.480 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.481 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:20.482 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.482 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.482 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.482 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.543 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:20.543 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-06T03:28:20.543 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:20.544 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:28:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:17.2.7-819.g6699495c.el9 ceph 813 k 2024-04-06T03:28:20.544 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-06T03:28:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-06T03:28:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:20.544 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:28:20.544 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:20.544 INFO:teuthology.orchestra.run.smithi094.stdout:Install 2 Packages 2024-04-06T03:28:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:20.545 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 893 k 2024-04-06T03:28:20.545 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 2.6 M 2024-04-06T03:28:20.545 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:28:20.614 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:43 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:20.829 INFO:teuthology.orchestra.run.smithi094.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 80 kB 00:00 2024-04-06T03:28:20.902 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:20.902 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:20.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:20.902 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:20.902 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.902 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:20.903 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:20.904 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:20.905 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:20.906 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.907 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:20.908 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.909 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:20.910 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:20.971 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:20.972 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-06T03:28:20.972 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:21.071 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install python3-rados 2024-04-06T03:28:21.670 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:44 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:21.671 INFO:teuthology.orchestra.run.smithi094.stdout:(2/2): ceph-fuse-17.2.7-819.g6699495c.el9.x86_6 891 kB/s | 813 kB 00:00 2024-04-06T03:28:21.671 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:21.672 INFO:teuthology.orchestra.run.smithi094.stdout:Total 792 kB/s | 893 kB 00:01 2024-04-06T03:28:21.684 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:28:21.706 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:28:21.706 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:28:21.784 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:28:21.785 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:28:21.958 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:21.959 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:21.959 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:21.959 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.959 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:21.959 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.959 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:21.960 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.960 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:21.960 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.960 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:21.960 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.960 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:21.961 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.961 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:21.961 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.961 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:21.961 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.961 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:21.962 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.962 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:21.962 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.962 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:21.962 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.962 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:21.963 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.963 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.964 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.965 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:21.966 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:21.967 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.968 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:21.969 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.970 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:21.971 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:28:21.971 INFO:teuthology.orchestra.run.smithi136.stdout:Package python3-rados-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:22.027 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:22.028 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-06T03:28:22.028 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:22.102 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-06T03:28:22.130 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install python3-rgw 2024-04-06T03:28:22.162 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:17.2.7-819.g6699495c.el9.x86_64 2/2 2024-04-06T03:28:22.725 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:45 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:22.839 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-fuse-2:17.2.7-819.g6699495c.el9.x86_64 2/2 2024-04-06T03:28:22.839 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:17.2.7-819.g6699495c.el9.x86_64 1/2 2024-04-06T03:28:23.015 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:23.016 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:23.017 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:23.018 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:23.019 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.020 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.021 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:23.022 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.023 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.024 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.025 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:23.025 INFO:teuthology.orchestra.run.smithi136.stdout:Package python3-rgw-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:23.085 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:23.085 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-06T03:28:23.085 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:23.180 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-06T03:28:23.180 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:23.181 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:28:23.181 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:17.2.7-819.g6699495c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-06T03:28:23.181 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:23.181 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:23.187 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install python3-cephfs 2024-04-06T03:28:23.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install librados-devel 2024-04-06T03:28:23.803 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:46 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:24.070 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:41 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:24.103 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:24.103 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.104 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:24.105 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.106 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.107 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.108 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.109 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:24.110 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:24.111 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.112 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.112 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.112 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.112 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:24.112 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.112 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.112 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.112 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.112 INFO:teuthology.orchestra.run.smithi136.stdout:Package python3-cephfs-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:24.174 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:24.176 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-06T03:28:24.176 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:24.273 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install python3-rbd 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:24.363 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.364 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.365 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:24.366 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:24.367 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.368 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.369 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.370 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.371 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.371 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.371 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:24.371 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.371 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.371 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.371 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.371 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:24.372 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.372 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.372 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.372 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:24.432 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:17.2.7-819.g6699495c.el9 ceph 126 k 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-04-06T03:28:24.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:24.434 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 126 k 2024-04-06T03:28:24.434 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 448 k 2024-04-06T03:28:24.435 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:28:24.643 INFO:teuthology.orchestra.run.smithi094.stdout:librados-devel-17.2.7-819.g6699495c.el9.x86_64. 603 kB/s | 126 kB 00:00 2024-04-06T03:28:24.644 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:24.644 INFO:teuthology.orchestra.run.smithi094.stdout:Total 600 kB/s | 126 kB 00:00 2024-04-06T03:28:24.644 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:28:24.650 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:28:24.650 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:28:24.709 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:28:24.709 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:28:24.851 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:28:24.884 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:47 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:24.951 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.175 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:25.176 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:25.177 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:25.178 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:25.179 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:25.180 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.181 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.181 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.181 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:25.181 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.181 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.181 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.181 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.181 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:25.182 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.182 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.182 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.182 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.182 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.182 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.183 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:25.183 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.183 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.183 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.183 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.183 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:25.184 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.184 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.184 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.184 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.184 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:25.184 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.185 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.185 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.185 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.185 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:25.185 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.185 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.185 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.185 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.186 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:25.186 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.186 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.186 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.186 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.186 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:25.186 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.187 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.187 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.187 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:25.187 INFO:teuthology.orchestra.run.smithi136.stdout:Package python3-rbd-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:25.243 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:25.244 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-06T03:28:25.245 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:25.350 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install rbd-fuse 2024-04-06T03:28:25.373 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:25.683 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:25.683 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:25.684 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:28:25.684 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:25.684 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:25.684 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:25.952 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:48 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:25.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install libcephfs2 2024-04-06T03:28:26.248 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.249 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:26.250 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.251 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.252 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:26.253 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:26.254 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.255 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.256 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.257 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.257 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:26.257 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.257 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.257 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.257 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.317 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:26.318 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:26.318 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2024-04-06T03:28:26.318 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:26.318 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:28:26.318 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:17.2.7-819.g6699495c.el9 ceph 82 k 2024-04-06T03:28:26.319 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:26.319 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:28:26.319 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:26.319 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2024-04-06T03:28:26.319 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:26.319 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 82 k 2024-04-06T03:28:26.319 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 222 k 2024-04-06T03:28:26.319 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:28:26.561 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:43 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:26.633 INFO:teuthology.orchestra.run.smithi136.stdout:rbd-fuse-17.2.7-819.g6699495c.el9.x86_64.rpm 261 kB/s | 82 kB 00:00 2024-04-06T03:28:26.634 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:26.634 INFO:teuthology.orchestra.run.smithi136.stdout:Total 260 kB/s | 82 kB 00:00 2024-04-06T03:28:26.634 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:28:26.641 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:28:26.641 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:28:26.700 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:28:26.700 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:28:26.837 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:26.852 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:26.853 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:26.854 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.855 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:26.856 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:26.857 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.858 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.859 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:26.860 INFO:teuthology.orchestra.run.smithi094.stdout:Package libcephfs2-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:26.920 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:26.962 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:26.962 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-06T03:28:26.962 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:26.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install libcephfs-devel 2024-04-06T03:28:27.412 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-fuse-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:27.598 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:44 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:27.748 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:27.748 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:27.748 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:28:27.748 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:27.748 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:27.748 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:27.901 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.902 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.903 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:27.904 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.905 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:27.906 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:27.907 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.908 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.909 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.910 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:27.973 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:27.974 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-04-06T03:28:27.974 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:27.974 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:28:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:17.2.7-819.g6699495c.el9 ceph 25 k 2024-04-06T03:28:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:27.974 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:28:27.974 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:27.975 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-04-06T03:28:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:27.975 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 25 k 2024-04-06T03:28:27.975 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 110 k 2024-04-06T03:28:27.975 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:28:28.032 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install rbd-mirror 2024-04-06T03:28:28.148 INFO:teuthology.orchestra.run.smithi094.stdout:libcephfs-devel-17.2.7-819.g6699495c.el9.x86_64 144 kB/s | 25 kB 00:00 2024-04-06T03:28:28.148 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:28.149 INFO:teuthology.orchestra.run.smithi094.stdout:Total 143 kB/s | 25 kB 00:00 2024-04-06T03:28:28.149 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:28:28.153 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:28:28.153 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:28:28.154 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:28:28.154 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:28:28.238 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:28:28.340 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:28.647 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:51 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:28.700 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:28.939 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:28.939 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:28.939 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:28.940 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:28.941 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.942 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:28.943 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:28.944 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:28.945 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.946 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.947 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.948 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.948 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.948 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:28.948 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.948 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.948 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.948 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:28.977 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:28.977 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:28.977 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:28:28.977 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:28.977 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:28.977 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:29.007 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:29.008 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repo Size 2024-04-06T03:28:29.008 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:29.008 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:28:29.008 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:17.2.7-819.g6699495c.el9 ceph 2.9 M 2024-04-06T03:28:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:29.009 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:28:29.009 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:29.009 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2024-04-06T03:28:29.009 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:29.009 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 2.9 M 2024-04-06T03:28:29.009 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 12 M 2024-04-06T03:28:29.009 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:28:29.174 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install librados2 2024-04-06T03:28:29.770 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:46 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:29.774 INFO:teuthology.orchestra.run.smithi136.stdout:rbd-mirror-17.2.7-819.g6699495c.el9.x86_64.rpm 3.8 MB/s | 2.9 MB 00:00 2024-04-06T03:28:29.775 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:29.775 INFO:teuthology.orchestra.run.smithi136.stdout:Total 3.8 MB/s | 2.9 MB 00:00 2024-04-06T03:28:29.775 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:28:29.785 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:28:29.785 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:28:29.862 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:28:29.863 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:30.057 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.058 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:30.059 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:30.060 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.061 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:30.062 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.063 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.064 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:30.065 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:30.126 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:30.127 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-06T03:28:30.127 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:30.202 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install librbd1 2024-04-06T03:28:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:28:30.354 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:30.379 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:30.379 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:28:30.379 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-06T03:28:30.379 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-06T03:28:30.379 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-06T03:28:30.380 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:30.797 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:47 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:31.090 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.091 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:31.092 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:31.093 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:31.094 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.095 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.096 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.097 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.098 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:31.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.098 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:31.098 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:31.159 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:31.160 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-06T03:28:31.161 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:31.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-rados 2024-04-06T03:28:31.498 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:31.498 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:31.498 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:28:31.498 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:31.499 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:31.499 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:31.791 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install rbd-nbd 2024-04-06T03:28:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:48 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:32.118 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:32.119 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:32.119 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:32.119 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.119 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:32.119 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.119 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:32.119 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.119 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:32.119 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:32.120 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:32.121 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:32.122 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.123 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:32.123 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.125 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:32.126 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.126 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:32.126 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.127 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:32.127 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.127 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:32.127 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.127 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:32.128 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.128 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:32.128 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.128 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:32.128 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.128 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:32.128 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.129 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:32.129 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.129 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:32.129 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.129 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:32.129 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.130 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:32.130 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.130 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:32.130 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.130 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:32.130 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.131 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.131 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.131 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:32.131 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.131 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.131 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.131 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.132 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:32.132 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.132 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.132 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.132 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.132 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.133 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.133 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:32.133 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.133 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.133 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.133 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.134 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:32.134 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.134 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.134 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.134 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.134 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:32.135 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.135 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.135 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.135 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.135 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:32.135 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.136 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.136 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.136 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.136 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:32.136 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.136 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.137 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.137 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.137 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:32.137 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.137 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.137 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.138 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.138 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-rados-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:32.187 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:32.188 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-06T03:28:32.188 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:32.261 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-rgw 2024-04-06T03:28:32.407 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:55 ago on Sat 06 Apr 2024 03:26:37 AM UTC. 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr:Modular dependency problems: 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 1: conflicting requests 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 2: conflicting requests 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 3: conflicting requests 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 4: conflicting requests 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.696 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 5: conflicting requests 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 6: conflicting requests 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 7: conflicting requests 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 8: conflicting requests 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 9: conflicting requests 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 10: conflicting requests 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 11: conflicting requests 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 12: conflicting requests 2024-04-06T03:28:32.697 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 13: conflicting requests 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 14: conflicting requests 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 15: conflicting requests 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 16: conflicting requests 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 17: conflicting requests 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 18: conflicting requests 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 19: conflicting requests 2024-04-06T03:28:32.698 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 20: conflicting requests 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 21: conflicting requests 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 22: conflicting requests 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 23: conflicting requests 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 24: conflicting requests 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 25: conflicting requests 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 26: conflicting requests 2024-04-06T03:28:32.699 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 27: conflicting requests 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 28: conflicting requests 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 29: conflicting requests 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 30: conflicting requests 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 31: conflicting requests 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 32: conflicting requests 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 33: conflicting requests 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.700 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 34: conflicting requests 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 35: conflicting requests 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 36: conflicting requests 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 37: conflicting requests 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.701 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 39: conflicting requests 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.702 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.703 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.704 INFO:teuthology.orchestra.run.smithi136.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:32.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.704 INFO:teuthology.orchestra.run.smithi136.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:32.764 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-06T03:28:32.765 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2024-04-06T03:28:32.765 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:32.765 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-06T03:28:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:17.2.7-819.g6699495c.el9 ceph 165 k 2024-04-06T03:28:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:32.766 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-06T03:28:32.766 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-06T03:28:32.766 INFO:teuthology.orchestra.run.smithi136.stdout:Install 1 Package 2024-04-06T03:28:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:32.766 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 165 k 2024-04-06T03:28:32.766 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 477 k 2024-04-06T03:28:32.766 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-06T03:28:32.852 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:49 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:32.983 INFO:teuthology.orchestra.run.smithi136.stdout:rbd-nbd-17.2.7-819.g6699495c.el9.x86_64.rpm 763 kB/s | 165 kB 00:00 2024-04-06T03:28:32.983 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:32.984 INFO:teuthology.orchestra.run.smithi136.stdout:Total 758 kB/s | 165 kB 00:00 2024-04-06T03:28:32.984 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-06T03:28:32.992 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-06T03:28:32.992 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-06T03:28:33.055 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-06T03:28:33.056 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-06T03:28:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:33.139 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:33.139 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:33.139 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.139 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:33.140 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.141 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:33.142 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.142 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:33.142 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.142 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:33.142 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.142 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:33.142 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.142 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:33.142 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.143 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:33.143 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.143 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:33.143 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.143 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:33.143 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.143 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:33.143 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.143 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:33.144 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:33.145 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.146 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.147 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:33.149 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.149 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.149 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.150 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.151 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.151 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:33.151 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.151 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.151 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.151 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.151 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.152 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.153 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.153 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:33.153 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-rgw-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:33.207 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:33.207 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-06T03:28:33.208 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-06T03:28:33.208 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:33.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-cephfs 2024-04-06T03:28:33.314 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-nbd-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:33.875 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:50 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:34.069 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-06T03:28:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:34.069 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:28:34.069 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-06T03:28:34.160 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:34.160 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:34.160 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:34.160 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.160 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:34.160 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:34.161 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:34.162 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:34.163 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.164 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.165 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.166 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:34.167 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.168 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:34.169 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.169 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.169 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.169 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:34.169 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-cephfs-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:34.229 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:34.230 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-06T03:28:34.230 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:34.351 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-rbd 2024-04-06T03:28:34.364 DEBUG:teuthology.parallel:result is None 2024-04-06T03:28:34.944 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:51 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:35.231 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:35.231 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:35.232 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:35.232 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.232 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:35.232 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.232 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:35.232 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.232 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:35.232 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.232 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:35.233 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.234 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:35.235 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.236 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.237 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:35.238 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.238 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.238 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.238 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.238 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:35.238 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.238 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.238 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.238 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:35.239 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.240 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:35.241 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.241 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.241 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.241 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:35.241 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-rbd-2:17.2.7-819.g6699495c.el9.x86_64 is already installed. 2024-04-06T03:28:35.300 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:35.301 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-06T03:28:35.301 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:35.393 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install rbd-fuse 2024-04-06T03:28:35.992 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:52 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:36.283 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.284 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.285 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:36.286 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.287 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:36.288 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:36.289 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.290 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.291 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.292 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.293 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.293 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:36.351 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:36.352 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-04-06T03:28:36.352 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:36.352 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:28:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:17.2.7-819.g6699495c.el9 ceph 82 k 2024-04-06T03:28:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:36.352 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:28:36.352 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:36.352 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-04-06T03:28:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:36.353 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 82 k 2024-04-06T03:28:36.353 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 222 k 2024-04-06T03:28:36.353 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:28:36.577 INFO:teuthology.orchestra.run.smithi094.stdout:rbd-fuse-17.2.7-819.g6699495c.el9.x86_64.rpm 366 kB/s | 82 kB 00:00 2024-04-06T03:28:36.577 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:36.577 INFO:teuthology.orchestra.run.smithi094.stdout:Total 364 kB/s | 82 kB 00:00 2024-04-06T03:28:36.578 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:28:36.584 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:28:36.584 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:28:36.645 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:28:36.645 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:28:36.787 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:28:36.882 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:37.268 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-fuse-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:37.562 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:37.562 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:37.563 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:28:37.563 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:37.563 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:37.563 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:37.813 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install rbd-mirror 2024-04-06T03:28:38.427 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:55 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:38.727 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:38.728 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:38.729 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:38.730 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.731 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.732 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:38.733 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.734 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:38.735 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.736 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.736 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.736 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:38.797 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:17.2.7-819.g6699495c.el9 ceph 2.9 M 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-04-06T03:28:38.798 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:38.799 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 2.9 M 2024-04-06T03:28:38.799 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 12 M 2024-04-06T03:28:38.799 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:28:39.822 INFO:teuthology.orchestra.run.smithi094.stdout:rbd-mirror-17.2.7-819.g6699495c.el9.x86_64.rpm 2.9 MB/s | 2.9 MB 00:01 2024-04-06T03:28:39.823 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:39.823 INFO:teuthology.orchestra.run.smithi094.stdout:Total 2.9 MB/s | 2.9 MB 00:01 2024-04-06T03:28:39.823 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:28:39.832 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:28:39.832 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:28:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:28:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:28:40.321 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:28:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:40.406 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-06T03:28:40.406 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-06T03:28:40.406 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-06T03:28:40.406 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-06T03:28:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:41.502 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:28:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:41.502 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:41.502 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:41.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install rbd-nbd 2024-04-06T03:28:42.348 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:59 ago on Sat 06 Apr 2024 03:26:43 AM UTC. 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr:Modular dependency problems: 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 1: conflicting requests 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 2: conflicting requests 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 3: conflicting requests 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 4: conflicting requests 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 5: conflicting requests 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.640 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 6: conflicting requests 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 7: conflicting requests 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 8: conflicting requests 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 9: conflicting requests 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 10: conflicting requests 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 11: conflicting requests 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 12: conflicting requests 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.641 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 13: conflicting requests 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 14: conflicting requests 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 15: conflicting requests 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 16: conflicting requests 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 17: conflicting requests 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 18: conflicting requests 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 19: conflicting requests 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.642 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 20: conflicting requests 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 21: conflicting requests 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 22: conflicting requests 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 23: conflicting requests 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 24: conflicting requests 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 25: conflicting requests 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 26: conflicting requests 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.643 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 27: conflicting requests 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 28: conflicting requests 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 29: conflicting requests 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 30: conflicting requests 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 31: conflicting requests 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 32: conflicting requests 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 33: conflicting requests 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 34: conflicting requests 2024-04-06T03:28:42.644 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 35: conflicting requests 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 36: conflicting requests 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 37: conflicting requests 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 38: cannot install the best candidate for the job 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.645 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 39: conflicting requests 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 40: cannot install the best candidate for the job 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 41: cannot install the best candidate for the job 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.646 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 42: cannot install the best candidate for the job 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 43: cannot install the best candidate for the job 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 44: cannot install the best candidate for the job 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.647 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.648 INFO:teuthology.orchestra.run.smithi094.stderr: Problem 45: cannot install the best candidate for the job 2024-04-06T03:28:42.648 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.648 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.648 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.648 INFO:teuthology.orchestra.run.smithi094.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-06T03:28:42.709 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:17.2.7-819.g6699495c.el9 ceph 165 k 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-04-06T03:28:42.710 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:42.711 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 165 k 2024-04-06T03:28:42.711 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 477 k 2024-04-06T03:28:42.711 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-06T03:28:43.018 INFO:teuthology.orchestra.run.smithi094.stdout:rbd-nbd-17.2.7-819.g6699495c.el9.x86_64.rpm 540 kB/s | 165 kB 00:00 2024-04-06T03:28:43.019 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:28:43.019 INFO:teuthology.orchestra.run.smithi094.stdout:Total 537 kB/s | 165 kB 00:00 2024-04-06T03:28:43.019 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-06T03:28:43.026 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-06T03:28:43.027 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-06T03:28:43.089 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-06T03:28:43.089 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-06T03:28:43.240 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-06T03:28:43.349 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:43.762 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-nbd-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:17.2.7-819.g6699495c.el9.x86_64 1/1 2024-04-06T03:28:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:44.063 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-06T03:28:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:17.2.7-819.g6699495c.el9.x86_64 2024-04-06T03:28:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:28:44.063 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-06T03:28:44.295 DEBUG:teuthology.parallel:result is None 2024-04-06T03:28:44.295 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-06T03:28:44.295 INFO:teuthology.packaging:ref: None 2024-04-06T03:28:44.295 INFO:teuthology.packaging:tag: None 2024-04-06T03:28:44.295 INFO:teuthology.packaging:branch: quincy 2024-04-06T03:28:44.295 INFO:teuthology.packaging:sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:28:44.296 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-06T03:28:44.543 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-06T03:28:44.563 INFO:teuthology.orchestra.run.smithi094.stdout:17.2.7-819.g6699495c.el9 2024-04-06T03:28:44.564 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-819.g6699495c.el9 2024-04-06T03:28:44.564 INFO:teuthology.task.install:The correct ceph version 17.2.7-819.g6699495c is installed. 2024-04-06T03:28:44.565 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-06T03:28:44.566 INFO:teuthology.packaging:ref: None 2024-04-06T03:28:44.566 INFO:teuthology.packaging:tag: None 2024-04-06T03:28:44.566 INFO:teuthology.packaging:branch: quincy 2024-04-06T03:28:44.566 INFO:teuthology.packaging:sha1: 9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:28:44.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-06T03:28:44.791 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-06T03:28:44.816 INFO:teuthology.orchestra.run.smithi136.stdout:17.2.7-819.g6699495c.el9 2024-04-06T03:28:44.816 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-819.g6699495c.el9 2024-04-06T03:28:44.816 INFO:teuthology.task.install:The correct ceph version 17.2.7-819.g6699495c is installed. 2024-04-06T03:28:44.818 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-06T03:28:44.818 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:28:44.818 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-06T03:28:44.846 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:28:44.846 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-06T03:28:44.888 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-06T03:28:44.889 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:28:44.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-04-06T03:28:44.932 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-06T03:28:44.999 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:28:44.999 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-04-06T03:28:45.030 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-06T03:28:45.101 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-06T03:28:45.101 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:28:45.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-06T03:28:45.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-06T03:28:45.194 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:28:45.195 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-06T03:28:45.225 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-06T03:28:45.296 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-06T03:28:45.296 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:28:45.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-04-06T03:28:45.323 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-06T03:28:45.389 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:28:45.390 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-04-06T03:28:45.420 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-06T03:28:45.491 INFO:teuthology.run_tasks:Running task print... 2024-04-06T03:28:45.501 INFO:teuthology.task.print:**** done install task... 2024-04-06T03:28:45.501 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-06T03:28:45.630 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '9f192350ba5fe6d0a365b251a8c2af61f56a5fa6'} 2024-04-06T03:28:45.633 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-04-06T03:28:45.634 INFO:tasks.cephadm:Cluster fsid is c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:28:45.634 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-06T03:28:45.634 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-06T03:28:45.634 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi094': '172.21.15.94', 'mon.smithi136': '172.21.15.136'} 2024-04-06T03:28:45.634 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-06T03:28:45.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-04-06T03:28:45.662 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-04-06T03:28:45.693 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-04-06T03:28:45.694 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-06T03:28:45.914 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 6 03:28 /home/ubuntu/cephtest/cephadm 2024-04-06T03:28:45.914 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-06T03:28:46.091 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 6 03:28 /home/ubuntu/cephtest/cephadm 2024-04-06T03:28:46.091 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-06T03:28:46.107 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-06T03:28:46.134 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-04-06T03:28:46.134 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-06T03:28:46.149 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-06T03:28:47.035 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-06T03:28:47.062 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-06T03:30:41.628 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:30:41.629 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)", 2024-04-06T03:30:41.629 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "ec300eaa768b18bd6cc8d1b248efb446cbfceeec0acea282e973a2c7c0cc62b7", 2024-04-06T03:30:41.629 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-04-06T03:30:41.629 INFO:teuthology.orchestra.run.smithi094.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8" 2024-04-06T03:30:41.629 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-06T03:30:41.629 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:30:54.392 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-06T03:30:54.392 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)", 2024-04-06T03:30:54.392 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "ec300eaa768b18bd6cc8d1b248efb446cbfceeec0acea282e973a2c7c0cc62b7", 2024-04-06T03:30:54.392 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-04-06T03:30:54.392 INFO:teuthology.orchestra.run.smithi136.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8" 2024-04-06T03:30:54.392 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-06T03:30:54.392 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-06T03:30:54.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-04-06T03:30:54.448 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-04-06T03:30:54.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-04-06T03:30:54.519 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-04-06T03:30:54.547 INFO:tasks.cephadm:Writing seed config... 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-06T03:30:54.548 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-06T03:30:54.549 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-06T03:30:54.550 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-06T03:30:54.551 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-06T03:30:54.551 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-06T03:30:54.551 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-06T03:30:54.551 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:30:54.551 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-06T03:30:54.576 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c6509676-f3c5-11ee-b649-cb9ed24678a4 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-06T03:30:54.576 DEBUG:teuthology.orchestra.run.smithi094:mon.smithi094> sudo journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094.service 2024-04-06T03:30:54.618 INFO:tasks.cephadm:Bootstrapping... 2024-04-06T03:30:54.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.94 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-06T03:30:54.823 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-06T03:30:54.823 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'c6509676-f3c5-11ee-b649-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.94', '--skip-admin-label'] 2024-04-06T03:30:54.848 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-06T03:30:54.849 INFO:teuthology.orchestra.run.smithi094.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-06T03:30:54.849 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying podman|docker is present... 2024-04-06T03:30:54.873 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-06T03:30:54.873 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying lvm2 is present... 2024-04-06T03:30:54.873 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying time synchronization is in place... 2024-04-06T03:30:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-06T03:30:54.881 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-06T03:30:54.887 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-06T03:30:54.887 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-06T03:30:54.894 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-04-06T03:30:54.900 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-04-06T03:30:54.900 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-04-06T03:30:54.900 INFO:teuthology.orchestra.run.smithi094.stdout:Repeating the final host check... 2024-04-06T03:30:54.923 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-06T03:30:54.923 INFO:teuthology.orchestra.run.smithi094.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-06T03:30:54.924 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl is present 2024-04-06T03:30:54.924 INFO:teuthology.orchestra.run.smithi094.stdout:lvcreate is present 2024-04-06T03:30:54.930 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-06T03:30:54.930 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-06T03:30:54.936 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-06T03:30:54.936 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-06T03:30:54.942 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-04-06T03:30:54.949 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-04-06T03:30:54.949 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-04-06T03:30:54.949 INFO:teuthology.orchestra.run.smithi094.stdout:Host looks OK 2024-04-06T03:30:54.949 INFO:teuthology.orchestra.run.smithi094.stdout:Cluster fsid: c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:30:54.949 INFO:teuthology.orchestra.run.smithi094.stdout:Acquiring lock 140474072051872 on /run/cephadm/c6509676-f3c5-11ee-b649-cb9ed24678a4.lock 2024-04-06T03:30:54.949 INFO:teuthology.orchestra.run.smithi094.stdout:Lock 140474072051872 acquired on /run/cephadm/c6509676-f3c5-11ee-b649-cb9ed24678a4.lock 2024-04-06T03:30:54.950 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 3300 ... 2024-04-06T03:30:54.950 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 6789 ... 2024-04-06T03:30:54.950 INFO:teuthology.orchestra.run.smithi094.stdout:Base mon IP(s) is [172.21.15.94:3300, 172.21.15.94:6789], mon addrv is [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-04-06T03:30:54.953 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.94 metric 100 2024-04-06T03:30:54.953 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.94 metric 100 2024-04-06T03:30:54.956 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-06T03:30:54.956 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-06T03:30:54.956 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-04-06T03:30:54.959 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-06T03:30:54.959 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-06T03:30:54.959 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-06T03:30:54.959 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-06T03:30:54.959 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-04-06T03:30:54.959 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-06T03:30:54.960 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-04-06T03:30:54.960 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-04-06T03:30:54.960 INFO:teuthology.orchestra.run.smithi094.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-06T03:30:54.960 INFO:teuthology.orchestra.run.smithi094.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-06T03:30:54.961 INFO:teuthology.orchestra.run.smithi094.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-06T03:30:55.459 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout ec300eaa768b18bd6cc8d1b248efb446cbfceeec0acea282e973a2c7c0cc62b7 2024-04-06T03:30:55.459 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-06T03:30:55.459 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Getting image source signatures 2024-04-06T03:30:55.459 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:b91f190a3ce1b82447b54a8039ed3c8de0d076fc686a09d670cfcd1b7134f590 2024-04-06T03:30:55.459 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:77070339c81bcb5c53bbbf3f31a47f7187e5393cb0e096452852a31192d97570 2024-04-06T03:30:55.459 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying config sha256:ec300eaa768b18bd6cc8d1b248efb446cbfceeec0acea282e973a2c7c0cc62b7 2024-04-06T03:30:55.459 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-06T03:30:56.397 INFO:teuthology.orchestra.run.smithi094.stdout:ceph: stdout ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable) 2024-04-06T03:30:56.398 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph version: ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable) 2024-04-06T03:30:56.398 INFO:teuthology.orchestra.run.smithi094.stdout:Extracting ceph user uid/gid from container image... 2024-04-06T03:30:57.445 INFO:teuthology.orchestra.run.smithi094.stdout:stat: stdout 167 167 2024-04-06T03:30:57.445 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial keys... 2024-04-06T03:30:58.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQDxwRBmlbU+KxAARCCa5JMsQrzn+T14MN2NEg== 2024-04-06T03:30:59.289 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQDywRBmSFUeJxAAYaXKpGWUicnhZL6D2JzEeQ== 2024-04-06T03:31:00.253 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQDzwRBmBhCHJBAAkka5xwZRA1lWw8GYi2sPYA== 2024-04-06T03:31:00.253 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial monmap... 2024-04-06T03:31:08.496 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-06T03:31:08.496 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-04-06T03:31:08.496 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:31:08.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-06T03:31:08.497 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool for smithi094 [v2:172.21.15.94:3300,v1:172.21.15.94:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-06T03:31:08.497 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = octopus 2024-04-06T03:31:08.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: set fsid to c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:31:08.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-06T03:31:08.497 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:31:08.497 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mon... 2024-04-06T03:31:11.066 INFO:teuthology.orchestra.run.smithi094.stdout:create mon.smithi094 on 2024-04-06T03:31:11.277 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-06T03:31:11.422 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-06T03:31:11.560 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4.target → /etc/systemd/system/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4.target. 2024-04-06T03:31:11.561 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4.target → /etc/systemd/system/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4.target. 2024-04-06T03:31:11.718 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094 2024-04-06T03:31:11.718 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094.service: Unit ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094.service not loaded. 2024-04-06T03:31:11.878 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4.target.wants/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094.service → /etc/systemd/system/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@.service. 2024-04-06T03:31:12.394 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:12 smithi094 podman[29324]: 2024-04-06 03:31:12.33759288 +0000 UTC m=+0.348412560 container start 62dd28f6ae858b6dd3e2a1900432d10a1b2c75cc306508135c4d23dbcbc9c05e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=quincy-6699495, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-06T03:31:12.394 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:12 smithi094 bash[29324]: 62dd28f6ae858b6dd3e2a1900432d10a1b2c75cc306508135c4d23dbcbc9c05e 2024-04-06T03:31:12.407 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-06T03:31:12.407 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-06T03:31:12.415 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-06T03:31:12.416 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-06T03:31:12.416 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-06T03:31:12.416 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-04-06T03:31:12.416 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon to start... 2024-04-06T03:31:12.416 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon... 2024-04-06T03:31:12.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:12 smithi094 systemd[1]: Started Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:31:12.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:12 smithi094 ceph-mon[29343]: mkfs c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:31:12.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:12 smithi094 ceph-mon[29343]: mon.smithi094 is new leader, mons smithi094 in quorum (ranks 0) 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout cluster: 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout id: c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout services: 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi094 (age 0.518453s) 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:13.509 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout data: 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pgs: 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.840+0000 7f79d9f71700 1 Processor -- start 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.840+0000 7f79d9f71700 1 -- start start 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.840+0000 7f79d9f71700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d41054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.840+0000 7f79d9f71700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4105a10 con 0x7f79d4076bc0 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.840+0000 7f79d37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d41054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.840+0000 7f79d37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d41054d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53486/0 (socket says 172.21.15.94:53486) 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.840+0000 7f79d37fe700 1 -- 172.21.15.94:0/942936441 learned_addr learned my addr 172.21.15.94:0/942936441 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:13.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d37fe700 1 -- 172.21.15.94:0/942936441 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4105b50 con 0x7f79d4076bc0 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d37fe700 1 --2- 172.21.15.94:0/942936441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d41054d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f79c40098e0 tx=0x7f79c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a6c005647a1b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d27fc700 1 -- 172.21.15.94:0/942936441 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79c400b860 con 0x7f79d4076bc0 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d27fc700 1 -- 172.21.15.94:0/942936441 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c400b9c0 con 0x7f79d4076bc0 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d27fc700 1 -- 172.21.15.94:0/942936441 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79c400bc90 con 0x7f79d4076bc0 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d9f71700 1 -- 172.21.15.94:0/942936441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 msgr2=0x7f79d41054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d9f71700 1 --2- 172.21.15.94:0/942936441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d41054d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f79c40098e0 tx=0x7f79c4009250 comp rx=0 tx=0).stop 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d9f71700 1 -- 172.21.15.94:0/942936441 shutdown_connections 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d9f71700 1 --2- 172.21.15.94:0/942936441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d41054d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.841+0000 7f79d9f71700 1 -- 172.21.15.94:0/942936441 >> 172.21.15.94:0/942936441 conn(0x7f79d40fcf40 msgr2=0x7f79d40ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.842+0000 7f79d9f71700 1 -- 172.21.15.94:0/942936441 shutdown_connections 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.842+0000 7f79d9f71700 1 -- 172.21.15.94:0/942936441 wait complete. 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.842+0000 7f79d9f71700 1 Processor -- start 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.842+0000 7f79d9f71700 1 -- start start 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.842+0000 7f79d9f71700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d4110f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:13.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.842+0000 7f79d9f71700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4111450 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79d37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d4110f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79d37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d4110f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53500/0 (socket says 172.21.15.94:53500) 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79d37fe700 1 -- 172.21.15.94:0/3196258305 learned_addr learned my addr 172.21.15.94:0/3196258305 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79d37fe700 1 -- 172.21.15.94:0/3196258305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40094f0 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79d37fe700 1 --2- 172.21.15.94:0/3196258305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d4110f10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009290 tx=0x7f79c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79bbfff700 1 -- 172.21.15.94:0/3196258305 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79c401c5c0 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79bbfff700 1 -- 172.21.15.94:0/3196258305 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c401c740 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79d9f71700 1 -- 172.21.15.94:0/3196258305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d4111650 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79bbfff700 1 -- 172.21.15.94:0/3196258305 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79c401ca50 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79d9f71700 1 -- 172.21.15.94:0/3196258305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d4112080 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79bbfff700 1 -- 172.21.15.94:0/3196258305 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f79c401a450 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.843+0000 7f79bbfff700 1 -- 172.21.15.94:0/3196258305 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f79c402e790 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.844+0000 7f79d9f71700 1 -- 172.21.15.94:0/3196258305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d405f5f0 con 0x7f79d4076bc0 2024-04-06T03:31:13.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.846+0000 7f79bbfff700 1 -- 172.21.15.94:0/3196258305 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f79c401ce30 con 0x7f79d4076bc0 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.914+0000 7f79d9f71700 1 -- 172.21.15.94:0/3196258305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f79d4112bf0 con 0x7f79d4076bc0 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.914+0000 7f79bbfff700 1 -- 172.21.15.94:0/3196258305 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f79c401a840 con 0x7f79d4076bc0 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.915+0000 7f79d9f71700 1 -- 172.21.15.94:0/3196258305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 msgr2=0x7f79d4110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.915+0000 7f79d9f71700 1 --2- 172.21.15.94:0/3196258305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d4110f10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009290 tx=0x7f79c4016000 comp rx=0 tx=0).stop 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.915+0000 7f79d9f71700 1 -- 172.21.15.94:0/3196258305 shutdown_connections 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.915+0000 7f79d9f71700 1 --2- 172.21.15.94:0/3196258305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d4076bc0 0x7f79d4110f10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.915+0000 7f79d9f71700 1 -- 172.21.15.94:0/3196258305 >> 172.21.15.94:0/3196258305 conn(0x7f79d40fcf40 msgr2=0x7f79d40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.915+0000 7f79d9f71700 1 -- 172.21.15.94:0/3196258305 shutdown_connections 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:12.915+0000 7f79d9f71700 1 -- 172.21.15.94:0/3196258305 wait complete. 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:mon is available 2024-04-06T03:31:13.513 INFO:teuthology.orchestra.run.smithi094.stdout:Assimilating anything we can from ceph.conf... 2024-04-06T03:31:13.718 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:13 smithi094 ceph-mon[29343]: mon.smithi094 is new leader, mons smithi094 in quorum (ranks 0) 2024-04-06T03:31:13.718 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:13 smithi094 ceph-mon[29343]: monmap e1: 1 mons at {smithi094=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-06T03:31:13.718 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:13 smithi094 ceph-mon[29343]: fsmap 2024-04-06T03:31:13.718 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:13 smithi094 ceph-mon[29343]: osdmap e1: 0 total, 0 up, 0 in 2024-04-06T03:31:13.718 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:13 smithi094 ceph-mon[29343]: mgrmap e1: no daemons active 2024-04-06T03:31:13.718 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:13 smithi094 ceph-mon[29343]: from='client.? 172.21.15.94:0/3196258305' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-06T03:31:14.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:14 smithi094 ceph-mon[29343]: from='client.? 172.21.15.94:0/1508586932' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-06T03:31:14.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:14 smithi094 ceph-mon[29343]: from='client.? 172.21.15.94:0/1508586932' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-06T03:31:15.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-06T03:31:15.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.297+0000 7fecb7894700 1 Processor -- start 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.297+0000 7fecb7894700 1 -- start start 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.297+0000 7fecb7894700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb0077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.297+0000 7fecb7894700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb0079040 con 0x7fecb0078c30 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.298+0000 7fecb5630700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb0077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.298+0000 7fecb5630700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb0077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53504/0 (socket says 172.21.15.94:53504) 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.298+0000 7fecb5630700 1 -- 172.21.15.94:0/942119133 learned_addr learned my addr 172.21.15.94:0/942119133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.298+0000 7fecb5630700 1 -- 172.21.15.94:0/942119133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb0077770 con 0x7fecb0078c30 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.298+0000 7fecb5630700 1 --2- 172.21.15.94:0/942119133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb0077230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fecac0098e0 tx=0x7fecac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3820d9b908ec6b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.298+0000 7feca7fff700 1 -- 172.21.15.94:0/942119133 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecac00b860 con 0x7fecb0078c30 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.298+0000 7feca7fff700 1 -- 172.21.15.94:0/942119133 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac00b9c0 con 0x7fecb0078c30 2024-04-06T03:31:15.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.298+0000 7feca7fff700 1 -- 172.21.15.94:0/942119133 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecac00bc90 con 0x7fecb0078c30 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.299+0000 7fecb7894700 1 -- 172.21.15.94:0/942119133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 msgr2=0x7fecb0077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.299+0000 7fecb7894700 1 --2- 172.21.15.94:0/942119133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb0077230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fecac0098e0 tx=0x7fecac009250 comp rx=0 tx=0).stop 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.299+0000 7fecb7894700 1 -- 172.21.15.94:0/942119133 shutdown_connections 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.299+0000 7fecb7894700 1 --2- 172.21.15.94:0/942119133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb0077230 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.299+0000 7fecb7894700 1 -- 172.21.15.94:0/942119133 >> 172.21.15.94:0/942119133 conn(0x7fecb00fe070 msgr2=0x7fecb0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.299+0000 7fecb7894700 1 -- 172.21.15.94:0/942119133 shutdown_connections 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.299+0000 7fecb7894700 1 -- 172.21.15.94:0/942119133 wait complete. 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.300+0000 7fecb7894700 1 Processor -- start 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.300+0000 7fecb7894700 1 -- start start 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.300+0000 7fecb7894700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb01130a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.300+0000 7fecb7894700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb01135e0 con 0x7fecb0078c30 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.300+0000 7fecb5630700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb01130a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:15.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.300+0000 7fecb5630700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb01130a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53518/0 (socket says 172.21.15.94:53518) 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.300+0000 7fecb5630700 1 -- 172.21.15.94:0/1508586932 learned_addr learned my addr 172.21.15.94:0/1508586932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.300+0000 7fecb5630700 1 -- 172.21.15.94:0/1508586932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0094f0 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.300+0000 7fecb5630700 1 --2- 172.21.15.94:0/1508586932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb01130a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fecac009290 tx=0x7fecac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.301+0000 7feca5ffb700 1 -- 172.21.15.94:0/1508586932 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecac01c5c0 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.301+0000 7feca5ffb700 1 -- 172.21.15.94:0/1508586932 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac01c740 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.301+0000 7fecb7894700 1 -- 172.21.15.94:0/1508586932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb01137e0 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.301+0000 7feca5ffb700 1 -- 172.21.15.94:0/1508586932 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecac01ca50 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.301+0000 7fecb7894700 1 -- 172.21.15.94:0/1508586932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb0114350 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.301+0000 7feca5ffb700 1 -- 172.21.15.94:0/1508586932 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fecac01a450 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.301+0000 7feca5ffb700 1 -- 172.21.15.94:0/1508586932 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fecac02e790 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.302+0000 7fecb7894700 1 -- 172.21.15.94:0/1508586932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb005f5f0 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.304+0000 7feca5ffb700 1 -- 172.21.15.94:0/1508586932 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fecac01ce30 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.370+0000 7fecb7894700 1 -- 172.21.15.94:0/1508586932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fecb0077c70 con 0x7fecb0078c30 2024-04-06T03:31:15.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.374+0000 7feca5ffb700 1 -- 172.21.15.94:0/1508586932 <== mon.0 v2:172.21.15.94:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fecac0243e0 con 0x7fecb0078c30 2024-04-06T03:31:15.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.374+0000 7feca5ffb700 1 -- 172.21.15.94:0/1508586932 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fecb0077c70 con 0x7fecb0078c30 2024-04-06T03:31:15.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.375+0000 7fecb7894700 1 -- 172.21.15.94:0/1508586932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 msgr2=0x7fecb01130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:15.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.375+0000 7fecb7894700 1 --2- 172.21.15.94:0/1508586932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb01130a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fecac009290 tx=0x7fecac016000 comp rx=0 tx=0).stop 2024-04-06T03:31:15.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.375+0000 7fecb7894700 1 -- 172.21.15.94:0/1508586932 shutdown_connections 2024-04-06T03:31:15.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.375+0000 7fecb7894700 1 --2- 172.21.15.94:0/1508586932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fecb0078c30 0x7fecb01130a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:15.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.375+0000 7fecb7894700 1 -- 172.21.15.94:0/1508586932 >> 172.21.15.94:0/1508586932 conn(0x7fecb00fe070 msgr2=0x7fecb0109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:15.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.375+0000 7fecb7894700 1 -- 172.21.15.94:0/1508586932 shutdown_connections 2024-04-06T03:31:15.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:14.375+0000 7fecb7894700 1 -- 172.21.15.94:0/1508586932 wait complete. 2024-04-06T03:31:15.036 INFO:teuthology.orchestra.run.smithi094.stdout:Generating new minimal ceph.conf... 2024-04-06T03:31:15.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:15 smithi094 ceph-mon[29343]: from='client.? 172.21.15.94:0/3338612993' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:31:16.953 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3afb21c700 1 Processor -- start 2024-04-06T03:31:16.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3afb21c700 1 -- start start 2024-04-06T03:31:16.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3afb21c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af4077270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:16.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3afb21c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4079080 con 0x7f3af4078c70 2024-04-06T03:31:16.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3af8fb8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af4077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:16.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3af8fb8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af4077270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53524/0 (socket says 172.21.15.94:53524) 2024-04-06T03:31:16.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3af8fb8700 1 -- 172.21.15.94:0/2368373240 learned_addr learned my addr 172.21.15.94:0/2368373240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:16.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3af8fb8700 1 -- 172.21.15.94:0/2368373240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af40777b0 con 0x7f3af4078c70 2024-04-06T03:31:16.954 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3af8fb8700 1 --2- 172.21.15.94:0/2368373240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af4077270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80098e0 tx=0x7f3ae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eec42b63670c8c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3af37fe700 1 -- 172.21.15.94:0/2368373240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ae800b860 con 0x7f3af4078c70 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3af37fe700 1 -- 172.21.15.94:0/2368373240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3ae800be50 con 0x7f3af4078c70 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.667+0000 7f3af37fe700 1 -- 172.21.15.94:0/2368373240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ae8010380 con 0x7f3af4078c70 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.668+0000 7f3afb21c700 1 -- 172.21.15.94:0/2368373240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 msgr2=0x7f3af4077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.668+0000 7f3afb21c700 1 --2- 172.21.15.94:0/2368373240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af4077270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80098e0 tx=0x7f3ae8009250 comp rx=0 tx=0).stop 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.668+0000 7f3afb21c700 1 -- 172.21.15.94:0/2368373240 shutdown_connections 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.668+0000 7f3afb21c700 1 --2- 172.21.15.94:0/2368373240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af4077270 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.668+0000 7f3afb21c700 1 -- 172.21.15.94:0/2368373240 >> 172.21.15.94:0/2368373240 conn(0x7f3af40fe070 msgr2=0x7f3af4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.668+0000 7f3afb21c700 1 -- 172.21.15.94:0/2368373240 shutdown_connections 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.668+0000 7f3afb21c700 1 -- 172.21.15.94:0/2368373240 wait complete. 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.668+0000 7f3afb21c700 1 Processor -- start 2024-04-06T03:31:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.669+0000 7f3afb21c700 1 -- start start 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.669+0000 7f3afb21c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af41138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.669+0000 7f3afb21c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af4113e20 con 0x7f3af4078c70 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.669+0000 7f3af8fb8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af41138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.669+0000 7f3af8fb8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af41138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53540/0 (socket says 172.21.15.94:53540) 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.669+0000 7f3af8fb8700 1 -- 172.21.15.94:0/3338612993 learned_addr learned my addr 172.21.15.94:0/3338612993 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.669+0000 7f3af8fb8700 1 -- 172.21.15.94:0/3338612993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae80094f0 con 0x7f3af4078c70 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.669+0000 7f3af8fb8700 1 --2- 172.21.15.94:0/3338612993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af41138e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8006fd0 tx=0x7f3ae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.669+0000 7f3af17fa700 1 -- 172.21.15.94:0/3338612993 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ae80107c0 con 0x7f3af4078c70 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.670+0000 7f3af17fa700 1 -- 172.21.15.94:0/3338612993 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3ae8010db0 con 0x7f3af4078c70 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.670+0000 7f3af17fa700 1 -- 172.21.15.94:0/3338612993 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ae801ada0 con 0x7f3af4078c70 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.670+0000 7f3afb21c700 1 -- 172.21.15.94:0/3338612993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af4114020 con 0x7f3af4078c70 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.670+0000 7f3afb21c700 1 -- 172.21.15.94:0/3338612993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af4114b90 con 0x7f3af4078c70 2024-04-06T03:31:16.956 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.670+0000 7f3af17fa700 1 -- 172.21.15.94:0/3338612993 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3ae8019690 con 0x7f3af4078c70 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.670+0000 7f3af17fa700 1 -- 172.21.15.94:0/3338612993 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3ae80248c0 con 0x7f3af4078c70 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.671+0000 7f3afb21c700 1 -- 172.21.15.94:0/3338612993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af405f5f0 con 0x7f3af4078c70 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.673+0000 7f3af17fa700 1 -- 172.21.15.94:0/3338612993 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3ae8029060 con 0x7f3af4078c70 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.739+0000 7f3afb21c700 1 -- 172.21.15.94:0/3338612993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3af4108ad0 con 0x7f3af4078c70 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.739+0000 7f3af17fa700 1 -- 172.21.15.94:0/3338612993 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f3ae8020070 con 0x7f3af4078c70 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.740+0000 7f3afb21c700 1 -- 172.21.15.94:0/3338612993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 msgr2=0x7f3af41138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.740+0000 7f3afb21c700 1 --2- 172.21.15.94:0/3338612993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af41138e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8006fd0 tx=0x7f3ae8009250 comp rx=0 tx=0).stop 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.740+0000 7f3afb21c700 1 -- 172.21.15.94:0/3338612993 shutdown_connections 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.740+0000 7f3afb21c700 1 --2- 172.21.15.94:0/3338612993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3af4078c70 0x7f3af41138e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.740+0000 7f3afb21c700 1 -- 172.21.15.94:0/3338612993 >> 172.21.15.94:0/3338612993 conn(0x7f3af40fe070 msgr2=0x7f3af40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.740+0000 7f3afb21c700 1 -- 172.21.15.94:0/3338612993 shutdown_connections 2024-04-06T03:31:16.957 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:15.740+0000 7f3afb21c700 1 -- 172.21.15.94:0/3338612993 wait complete. 2024-04-06T03:31:16.958 INFO:teuthology.orchestra.run.smithi094.stdout:Restarting the monitor... 2024-04-06T03:31:17.223 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:16 smithi094 systemd[1]: Stopping Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:31:17.523 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:17 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[29339]: 2024-04-06T03:31:17.222+0000 7f2901749700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi094 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:31:17.523 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:17 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[29339]: 2024-04-06T03:31:17.222+0000 7f2901749700 -1 mon.smithi094@0(leader) e1 *** Got Signal Terminated *** 2024-04-06T03:31:17.523 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:17 smithi094 podman[29586]: 2024-04-06 03:31:17.243308563 +0000 UTC m=+0.116050882 container died 62dd28f6ae858b6dd3e2a1900432d10a1b2c75cc306508135c4d23dbcbc9c05e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-6699495, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-06T03:31:17.873 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:17 smithi094 podman[29586]: 2024-04-06 03:31:17.622256443 +0000 UTC m=+0.494998759 container cleanup 62dd28f6ae858b6dd3e2a1900432d10a1b2c75cc306508135c4d23dbcbc9c05e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, RELEASE=quincy-6699495, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:31:17.873 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:17 smithi094 bash[29586]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094 2024-04-06T03:31:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:17 smithi094 podman[29600]: 2024-04-06 03:31:17.873213473 +0000 UTC m=+0.632735650 container remove 62dd28f6ae858b6dd3e2a1900432d10a1b2c75cc306508135c4d23dbcbc9c05e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=quincy-6699495, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-06T03:31:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094.service: Deactivated successfully. 2024-04-06T03:31:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 systemd[1]: Stopped Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:31:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 systemd[1]: Starting Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:31:18.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 podman[29694]: 2024-04-06 03:31:18.147136962 +0000 UTC m=+0.017214994 image pull ec300eaa768b18bd6cc8d1b248efb446cbfceeec0acea282e973a2c7c0cc62b7 quay.ceph.io/ceph-ci/ceph:quincy 2024-04-06T03:31:18.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 podman[29694]: 2024-04-06 03:31:18.248890459 +0000 UTC m=+0.118968470 container create b36acaaa3d783580f1ff821eeee34f0fdfc5c5cdadf892a57a0130bd9739a6a6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=quincy-6699495, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-06T03:31:18.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 podman[29694]: 2024-04-06 03:31:18.418033073 +0000 UTC m=+0.288111084 container init b36acaaa3d783580f1ff821eeee34f0fdfc5c5cdadf892a57a0130bd9739a6a6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=quincy-6699495, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:31:18.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 podman[29694]: 2024-04-06 03:31:18.420879623 +0000 UTC m=+0.290957644 container start b36acaaa3d783580f1ff821eeee34f0fdfc5c5cdadf892a57a0130bd9739a6a6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=quincy-6699495, GIT_CLEAN=True, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: set uid:gid to 167:167 (ceph:ceph) 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable), process ceph-mon, pid 2 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: pidfile_write: ignore empty --pid-file 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: load: jerasure load: lrc 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: RocksDB version: 6.15.5 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Compile date Apr 5 2024 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: DB SUMMARY 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: DB Session ID: LETSRYGOBGZQ1E6IOU3Y 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: CURRENT file: CURRENT 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: IDENTITY file: IDENTITY 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi094/store.db dir, Total Num: 1, files: 000008.sst 2024-04-06T03:31:18.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi094/store.db: 000010.log size: 98150 ; 2024-04-06T03:31:18.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.error_if_exists: 0 2024-04-06T03:31:18.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.create_if_missing: 0 2024-04-06T03:31:18.497 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.paranoid_checks: 1 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.env: 0x556d5c975a80 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.fs: Posix File System 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.info_log: 0x556d5f0cfc00 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_file_opening_threads: 16 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.statistics: (nil) 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.use_fsync: 0 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_log_file_size: 0 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.keep_log_file_num: 1000 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.recycle_log_file_num: 0 2024-04-06T03:31:18.498 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.allow_fallocate: 1 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.allow_mmap_reads: 0 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.allow_mmap_writes: 0 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.use_direct_reads: 0 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.create_missing_column_families: 0 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.db_log_dir: 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi094/store.db 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-06T03:31:18.499 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.advise_random_on_open: 1 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.db_write_buffer_size: 0 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.write_buffer_manager: 0x556d5fe449f0 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.rate_limiter: (nil) 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.wal_recovery_mode: 2 2024-04-06T03:31:18.500 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.enable_thread_tracking: 0 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.enable_pipelined_write: 0 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.unordered_write: 0 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.row_cache: None 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.wal_filter: None 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.allow_ingest_behind: 0 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.preserve_deletes: 0 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.two_write_queues: 0 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.manual_wal_flush: 0 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.atomic_flush: 0 2024-04-06T03:31:18.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.log_readahead_size: 0 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.best_efforts_recovery: 0 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.allow_data_in_errors: 0 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.db_host_id: __hostname__ 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_background_jobs: 2 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_background_compactions: -1 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_subcompactions: 1 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-06T03:31:18.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_total_wal_size: 0 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_open_files: -1 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bytes_per_sync: 0 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_readahead_size: 0 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_background_flushes: -1 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Compression algorithms supported: 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: kZSTD supported: 0 2024-04-06T03:31:18.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: kXpressCompression supported: 0 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: kLZ4HCCompression supported: 1 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: kZlibCompression supported: 1 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: kSnappyCompression supported: 1 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: kLZ4Compression supported: 1 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: kBZip2Compression supported: 0 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi094/store.db/MANIFEST-000009 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.merge_operator: 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_filter: None 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_filter_factory: None 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.sst_partitioner_factory: None 2024-04-06T03:31:18.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556d5f124f58) 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_type: 0 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_block_index_type: 0 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_shortening: 1 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: hash_index_allow_collision: 1 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: checksum: 1 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: no_block_cache: 0 2024-04-06T03:31:18.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache: 0x556d5f157090 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_options: 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: capacity : 536870912 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: num_shard_bits : 4 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: strict_capacity_limit : 0 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_compressed: (nil) 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: persistent_cache: (nil) 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_size: 4096 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_size_deviation: 10 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_restart_interval: 16 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_block_restart_interval: 1 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: metadata_block_size: 4096 2024-04-06T03:31:18.506 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: partition_filters: 0 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: use_delta_encoding: 1 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: whole_key_filtering: 1 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: verify_compression: 0 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: format_version: 4 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: enable_index_compression: 1 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_align: 0 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.write_buffer_size: 33554432 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_write_buffer_number: 2 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compression: NoCompression 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bottommost_compression: Disabled 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.prefix_extractor: nullptr 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-06T03:31:18.507 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.num_levels: 7 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compression_opts.level: 32767 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compression_opts.strategy: 0 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-06T03:31:18.508 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compression_opts.enabled: false 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.target_file_size_base: 67108864 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-06T03:31:18.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-06T03:31:18.509 INFO:teuthology.orchestra.run.smithi094.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-06T03:31:18.510 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-06T03:31:18.510 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-06T03:31:18.510 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-06T03:31:18.510 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-06T03:31:18.510 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.arena_block_size: 4194304 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.disable_auto_compactions: 0 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-06T03:31:18.511 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.table_properties_collectors: 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.inplace_update_support: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.bloom_locality: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.max_successive_merges: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.paranoid_file_checks: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.force_consistency_checks: 1 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.report_bg_io_stats: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.ttl: 2592000 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-06T03:31:18.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.enable_blob_files: false 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.min_blob_size: 0 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.blob_file_size: 268435456 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi094/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712374278446796, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712374278447710, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93356, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91468, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84032, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1712374278, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "5caf4c69-916a-4e4a-a013-4b0eaf3b20b8", "db_session_id": "LETSRYGOBGZQ1E6IOU3Y"}} 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712374278448031, "job": 1, "event": "recovery_finished"} 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi094/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-06T03:31:18.513 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x556d5f0eb180 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: DB pointer 0x556d5f1c0000 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** DB Stats ** 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-06T03:31:18.514 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-06T03:31:18.515 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: L0 2/0 92.97 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.2 0.00 0.00 1 0.001 0 0 2024-04-06T03:31:18.515 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Sum 2/0 92.97 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.2 0.00 0.00 1 0.001 0 0 2024-04-06T03:31:18.515 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.2 0.00 0.00 1 0.001 0 0 2024-04-06T03:31:18.515 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:31:18.515 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-06T03:31:18.515 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-06T03:31:18.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-06T03:31:18.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.2 0.00 0.00 1 0.001 0 0 2024-04-06T03:31:18.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-06T03:31:18.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-06T03:31:18.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-06T03:31:18.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative compaction: 0.00 GB write, 28.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval compaction: 0.00 GB write, 28.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: L0 2/0 92.97 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.2 0.00 0.00 1 0.001 0 0 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Sum 2/0 92.97 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.2 0.00 0.00 1 0.001 0 0 2024-04-06T03:31:18.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.2 0.00 0.00 1 0.001 0 0 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative compaction: 0.00 GB write, 28.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:31:18.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: starting mon.smithi094 rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi094 fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mon.smithi094@-1(???) e1 preinit fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mon.smithi094@-1(???).mds e1 new map 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mon.smithi094@-1(???).mds e1 print_map 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: e1 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: legacy client fscid: -1 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: No filesystems configured 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mon.smithi094@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mon.smithi094@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mon.smithi094@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mon.smithi094@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-06T03:31:18.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mon.smithi094@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-06T03:31:18.521 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 bash[29694]: b36acaaa3d783580f1ff821eeee34f0fdfc5c5cdadf892a57a0130bd9739a6a6 2024-04-06T03:31:18.521 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 systemd[1]: Started Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:31:18.762 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mon.smithi094 is new leader, mons smithi094 in quorum (ranks 0) 2024-04-06T03:31:18.763 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: monmap e1: 1 mons at {smithi094=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-06T03:31:18.763 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: fsmap 2024-04-06T03:31:18.763 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: osdmap e1: 0 total, 0 up, 0 in 2024-04-06T03:31:18.763 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:18 smithi094 ceph-mon[29713]: mgrmap e1: no daemons active 2024-04-06T03:31:19.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.862+0000 7f40aa0c4700 1 Processor -- start 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.862+0000 7f40aa0c4700 1 -- start start 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.862+0000 7f40aa0c4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4101060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.862+0000 7f40aa0c4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a41015a0 con 0x7f40a4100c50 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.863+0000 7f40a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.863+0000 7f40a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47596/0 (socket says 172.21.15.94:47596) 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.863+0000 7f40a37fe700 1 -- 172.21.15.94:0/4272167003 learned_addr learned my addr 172.21.15.94:0/4272167003 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.863+0000 7f40a37fe700 1 -- 172.21.15.94:0/4272167003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a41016e0 con 0x7f40a4100c50 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.863+0000 7f40a37fe700 1 --2- 172.21.15.94:0/4272167003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4101060 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4094009e50 tx=0x7f4094009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3d05bf01973790 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.863+0000 7f40a27fc700 1 -- 172.21.15.94:0/4272167003 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40940092e0 con 0x7f40a4100c50 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.863+0000 7f40a27fc700 1 -- 172.21.15.94:0/4272167003 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4094012430 con 0x7f40a4100c50 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.863+0000 7f40a27fc700 1 -- 172.21.15.94:0/4272167003 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40940075c0 con 0x7f40a4100c50 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.864+0000 7f40aa0c4700 1 -- 172.21.15.94:0/4272167003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 msgr2=0x7f40a4101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.864+0000 7f40aa0c4700 1 --2- 172.21.15.94:0/4272167003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4101060 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4094009e50 tx=0x7f4094009840 comp rx=0 tx=0).stop 2024-04-06T03:31:19.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.864+0000 7f40aa0c4700 1 -- 172.21.15.94:0/4272167003 shutdown_connections 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.864+0000 7f40aa0c4700 1 --2- 172.21.15.94:0/4272167003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4101060 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.864+0000 7f40aa0c4700 1 -- 172.21.15.94:0/4272167003 >> 172.21.15.94:0/4272167003 conn(0x7f40a40fc670 msgr2=0x7f40a40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.864+0000 7f40aa0c4700 1 -- 172.21.15.94:0/4272167003 shutdown_connections 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.864+0000 7f40aa0c4700 1 -- 172.21.15.94:0/4272167003 wait complete. 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.864+0000 7f40aa0c4700 1 Processor -- start 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f40aa0c4700 1 -- start start 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f40aa0c4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4111680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f40aa0c4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a4111bc0 con 0x7f40a4100c50 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f40a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4111680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f40a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4111680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47604/0 (socket says 172.21.15.94:47604) 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f40a37fe700 1 -- 172.21.15.94:0/3065448431 learned_addr learned my addr 172.21.15.94:0/3065448431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f40a37fe700 1 -- 172.21.15.94:0/3065448431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009ae0 con 0x7f40a4100c50 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f40a37fe700 1 --2- 172.21.15.94:0/3065448431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4111680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4094009fd0 tx=0x7f4094009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f408bfff700 1 -- 172.21.15.94:0/3065448431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f40940092e0 con 0x7f40a4100c50 2024-04-06T03:31:19.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f408bfff700 1 -- 172.21.15.94:0/3065448431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4094007cc0 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f408bfff700 1 -- 172.21.15.94:0/3065448431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f409401c9a0 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.865+0000 7f40aa0c4700 1 -- 172.21.15.94:0/3065448431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a4111dc0 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.866+0000 7f40aa0c4700 1 -- 172.21.15.94:0/3065448431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a4112930 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.866+0000 7f408bfff700 1 -- 172.21.15.94:0/3065448431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4094019070 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.866+0000 7f408bfff700 1 -- 172.21.15.94:0/3065448431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4094025b10 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.866+0000 7f40aa0c4700 1 -- 172.21.15.94:0/3065448431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a405f5f0 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.869+0000 7f408bfff700 1 -- 172.21.15.94:0/3065448431 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f409402a060 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.934+0000 7f40aa0c4700 1 -- 172.21.15.94:0/3065448431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f40a4106800 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.936+0000 7f408bfff700 1 -- 172.21.15.94:0/3065448431 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4094020070 con 0x7f40a4100c50 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.937+0000 7f40aa0c4700 1 -- 172.21.15.94:0/3065448431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 msgr2=0x7f40a4111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.937+0000 7f40aa0c4700 1 --2- 172.21.15.94:0/3065448431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4111680 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4094009fd0 tx=0x7f4094009840 comp rx=0 tx=0).stop 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.937+0000 7f40aa0c4700 1 -- 172.21.15.94:0/3065448431 shutdown_connections 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.937+0000 7f40aa0c4700 1 --2- 172.21.15.94:0/3065448431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40a4100c50 0x7f40a4111680 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.937+0000 7f40aa0c4700 1 -- 172.21.15.94:0/3065448431 >> 172.21.15.94:0/3065448431 conn(0x7f40a40fc670 msgr2=0x7f40a40fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:19.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.937+0000 7f40aa0c4700 1 -- 172.21.15.94:0/3065448431 shutdown_connections 2024-04-06T03:31:19.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:18.937+0000 7f40aa0c4700 1 -- 172.21.15.94:0/3065448431 wait complete. 2024-04-06T03:31:19.485 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-06T03:31:19.485 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-06T03:31:19.485 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mgr... 2024-04-06T03:31:19.485 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 9283 ... 2024-04-06T03:31:19.648 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mgr.smithi094.purlzp 2024-04-06T03:31:19.649 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mgr.smithi094.purlzp.service: Unit ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mgr.smithi094.purlzp.service not loaded. 2024-04-06T03:31:19.787 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4.target.wants/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mgr.smithi094.purlzp.service → /etc/systemd/system/ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@.service. 2024-04-06T03:31:20.140 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:19 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3065448431' entity='client.admin' 2024-04-06T03:31:20.299 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-06T03:31:20.300 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-06T03:31:20.306 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-06T03:31:20.307 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-06T03:31:20.307 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-06T03:31:20.307 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-04-06T03:31:20.315 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-06T03:31:20.315 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-06T03:31:20.323 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-06T03:31:20.323 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-06T03:31:20.323 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-06T03:31:20.323 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-04-06T03:31:20.323 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr to start... 2024-04-06T03:31:20.323 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr... 2024-04-06T03:31:21.141 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:20 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1591025256' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-06T03:31:21.445 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:21.448 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-06T03:31:21.448 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "c6509676-f3c5-11ee-b649-cb9ed24678a4", 2024-04-06T03:31:21.448 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-06T03:31:21.448 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-06T03:31:21.448 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-06T03:31:21.448 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-06T03:31:21.448 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "smithi094" 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-06T03:31:21.449 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-06T03:31:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-06T03:31:21.450 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-06T03:31:21.457 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-06T03:31:21.457 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-06T03:31:21.457 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-06T03:31:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-06T03:31:12.411866+0000", 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-06T03:31:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.753+0000 7f83479d9700 1 Processor -- start 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.754+0000 7f83479d9700 1 -- start start 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.754+0000 7f83479d9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f83401021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.754+0000 7f83479d9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83401026e0 con 0x7f8340101d90 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.754+0000 7f8345775700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f83401021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.754+0000 7f8345775700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f83401021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47630/0 (socket says 172.21.15.94:47630) 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.754+0000 7f8345775700 1 -- 172.21.15.94:0/3124881973 learned_addr learned my addr 172.21.15.94:0/3124881973 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.755+0000 7f8345775700 1 -- 172.21.15.94:0/3124881973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340102820 con 0x7f8340101d90 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.755+0000 7f8345775700 1 --2- 172.21.15.94:0/3124881973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f83401021a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f833c0098e0 tx=0x7f833c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8839be8aed71d67 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.755+0000 7f8337fff700 1 -- 172.21.15.94:0/3124881973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f833c00b860 con 0x7f8340101d90 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.755+0000 7f8337fff700 1 -- 172.21.15.94:0/3124881973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f833c00be50 con 0x7f8340101d90 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.755+0000 7f8337fff700 1 -- 172.21.15.94:0/3124881973 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f833c010380 con 0x7f8340101d90 2024-04-06T03:31:21.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.756+0000 7f83479d9700 1 -- 172.21.15.94:0/3124881973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 msgr2=0x7f83401021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.756+0000 7f83479d9700 1 --2- 172.21.15.94:0/3124881973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f83401021a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f833c0098e0 tx=0x7f833c009250 comp rx=0 tx=0).stop 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.756+0000 7f83479d9700 1 -- 172.21.15.94:0/3124881973 shutdown_connections 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.756+0000 7f83479d9700 1 --2- 172.21.15.94:0/3124881973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f83401021a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.756+0000 7f83479d9700 1 -- 172.21.15.94:0/3124881973 >> 172.21.15.94:0/3124881973 conn(0x7f83400fc6e0 msgr2=0x7f83400feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.756+0000 7f83479d9700 1 -- 172.21.15.94:0/3124881973 shutdown_connections 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.756+0000 7f83479d9700 1 -- 172.21.15.94:0/3124881973 wait complete. 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.756+0000 7f83479d9700 1 Processor -- start 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.756+0000 7f83479d9700 1 -- start start 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f83479d9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f8340101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f83479d9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8340112170 con 0x7f8340101d90 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f8345775700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f8340101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f8345775700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f8340101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47636/0 (socket says 172.21.15.94:47636) 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f8345775700 1 -- 172.21.15.94:0/1591025256 learned_addr learned my addr 172.21.15.94:0/1591025256 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f8345775700 1 -- 172.21.15.94:0/1591025256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c0094f0 con 0x7f8340101d90 2024-04-06T03:31:21.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f8345775700 1 --2- 172.21.15.94:0/1591025256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f8340101590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f833c00b440 tx=0x7f833c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f8335ffb700 1 -- 172.21.15.94:0/1591025256 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f833c010760 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f83479d9700 1 -- 172.21.15.94:0/1591025256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8340116780 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.757+0000 7f8335ffb700 1 -- 172.21.15.94:0/1591025256 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f833c010d50 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.758+0000 7f83479d9700 1 -- 172.21.15.94:0/1591025256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8340116b50 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.758+0000 7f8335ffb700 1 -- 172.21.15.94:0/1591025256 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f833c01ae10 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.758+0000 7f8335ffb700 1 -- 172.21.15.94:0/1591025256 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f833c02e6c0 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.758+0000 7f8335ffb700 1 -- 172.21.15.94:0/1591025256 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f833c024a60 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.759+0000 7f83479d9700 1 -- 172.21.15.94:0/1591025256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834004bed0 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.761+0000 7f8335ffb700 1 -- 172.21.15.94:0/1591025256 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f833c029070 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.833+0000 7f83479d9700 1 -- 172.21.15.94:0/1591025256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8340103630 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.833+0000 7f8335ffb700 1 -- 172.21.15.94:0/1591025256 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f833c01a6a0 con 0x7f8340101d90 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.834+0000 7f83479d9700 1 -- 172.21.15.94:0/1591025256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 msgr2=0x7f8340101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.834+0000 7f83479d9700 1 --2- 172.21.15.94:0/1591025256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f8340101590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f833c00b440 tx=0x7f833c009250 comp rx=0 tx=0).stop 2024-04-06T03:31:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.834+0000 7f83479d9700 1 -- 172.21.15.94:0/1591025256 shutdown_connections 2024-04-06T03:31:21.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.834+0000 7f83479d9700 1 --2- 172.21.15.94:0/1591025256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8340101d90 0x7f8340101590 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:21.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.834+0000 7f83479d9700 1 -- 172.21.15.94:0/1591025256 >> 172.21.15.94:0/1591025256 conn(0x7f83400fc6e0 msgr2=0x7f83400fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:21.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.834+0000 7f83479d9700 1 -- 172.21.15.94:0/1591025256 shutdown_connections 2024-04-06T03:31:21.463 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:20.835+0000 7f83479d9700 1 -- 172.21.15.94:0/1591025256 wait complete. 2024-04-06T03:31:21.463 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (1/15)... 2024-04-06T03:31:24.118 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:23 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2390305193' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-06T03:31:24.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:24.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-06T03:31:24.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "c6509676-f3c5-11ee-b649-cb9ed24678a4", 2024-04-06T03:31:24.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-06T03:31:24.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-06T03:31:24.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-06T03:31:24.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-06T03:31:24.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:24.570 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "smithi094" 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-06T03:31:24.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-06T03:31:24.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-06T03:31:24.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-06T03:31:24.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-06T03:31:12.411866+0000", 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.823+0000 7f366de50700 1 Processor -- start 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.823+0000 7f366de50700 1 -- start start 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.823+0000 7f366de50700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.823+0000 7f366de50700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668105a00 con 0x7f3668076ce0 2024-04-06T03:31:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.823+0000 7f36677fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.823+0000 7f36677fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47646/0 (socket says 172.21.15.94:47646) 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.823+0000 7f36677fe700 1 -- 172.21.15.94:0/2155939433 learned_addr learned my addr 172.21.15.94:0/2155939433 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.824+0000 7f36677fe700 1 -- 172.21.15.94:0/2155939433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3668105b40 con 0x7f3668076ce0 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.824+0000 7f36677fe700 1 --2- 172.21.15.94:0/2155939433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681054c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f36580098e0 tx=0x7f3658009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f48a990f9cb573 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.824+0000 7f36667fc700 1 -- 172.21.15.94:0/2155939433 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f365800b860 con 0x7f3668076ce0 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.824+0000 7f36667fc700 1 -- 172.21.15.94:0/2155939433 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f365800be50 con 0x7f3668076ce0 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.824+0000 7f36667fc700 1 -- 172.21.15.94:0/2155939433 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3658010380 con 0x7f3668076ce0 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.825+0000 7f366de50700 1 -- 172.21.15.94:0/2155939433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 msgr2=0x7f36681054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.825+0000 7f366de50700 1 --2- 172.21.15.94:0/2155939433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681054c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f36580098e0 tx=0x7f3658009250 comp rx=0 tx=0).stop 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.825+0000 7f366de50700 1 -- 172.21.15.94:0/2155939433 shutdown_connections 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.825+0000 7f366de50700 1 --2- 172.21.15.94:0/2155939433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681054c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.825+0000 7f366de50700 1 -- 172.21.15.94:0/2155939433 >> 172.21.15.94:0/2155939433 conn(0x7f36680fcf30 msgr2=0x7f36680ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:24.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.825+0000 7f366de50700 1 -- 172.21.15.94:0/2155939433 shutdown_connections 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.825+0000 7f366de50700 1 -- 172.21.15.94:0/2155939433 wait complete. 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.825+0000 7f366de50700 1 Processor -- start 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.825+0000 7f366de50700 1 -- start start 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f366de50700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f366de50700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668111be0 con 0x7f3668076ce0 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f36677fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f36677fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47654/0 (socket says 172.21.15.94:47654) 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f36677fe700 1 -- 172.21.15.94:0/2390305193 learned_addr learned my addr 172.21.15.94:0/2390305193 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f36677fe700 1 -- 172.21.15.94:0/2390305193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36580094f0 con 0x7f3668076ce0 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f36677fe700 1 --2- 172.21.15.94:0/2390305193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681116a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3658003050 tx=0x7f3658009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f364ffff700 1 -- 172.21.15.94:0/2390305193 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3658010760 con 0x7f3668076ce0 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f366de50700 1 -- 172.21.15.94:0/2390305193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3668111de0 con 0x7f3668076ce0 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f364ffff700 1 -- 172.21.15.94:0/2390305193 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3658010d50 con 0x7f3668076ce0 2024-04-06T03:31:24.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.826+0000 7f364ffff700 1 -- 172.21.15.94:0/2390305193 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f365801ae20 con 0x7f3668076ce0 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.827+0000 7f366de50700 1 -- 172.21.15.94:0/2390305193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3668112950 con 0x7f3668076ce0 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.827+0000 7f364ffff700 1 -- 172.21.15.94:0/2390305193 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f365802e6c0 con 0x7f3668076ce0 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.827+0000 7f364ffff700 1 -- 172.21.15.94:0/2390305193 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3658024860 con 0x7f3668076ce0 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.827+0000 7f366de50700 1 -- 172.21.15.94:0/2390305193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3668112270 con 0x7f3668076ce0 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.829+0000 7f364ffff700 1 -- 172.21.15.94:0/2390305193 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3658029070 con 0x7f3668076ce0 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.901+0000 7f366de50700 1 -- 172.21.15.94:0/2390305193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f366805f5f0 con 0x7f3668076ce0 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.901+0000 7f364ffff700 1 -- 172.21.15.94:0/2390305193 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f365801a6a0 con 0x7f3668076ce0 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.902+0000 7f366de50700 1 -- 172.21.15.94:0/2390305193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 msgr2=0x7f36681116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.903+0000 7f366de50700 1 --2- 172.21.15.94:0/2390305193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681116a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3658003050 tx=0x7f3658009250 comp rx=0 tx=0).stop 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.903+0000 7f366de50700 1 -- 172.21.15.94:0/2390305193 shutdown_connections 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.903+0000 7f366de50700 1 --2- 172.21.15.94:0/2390305193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3668076ce0 0x7f36681116a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.903+0000 7f366de50700 1 -- 172.21.15.94:0/2390305193 >> 172.21.15.94:0/2390305193 conn(0x7f36680fcf30 msgr2=0x7f36680fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.903+0000 7f366de50700 1 -- 172.21.15.94:0/2390305193 shutdown_connections 2024-04-06T03:31:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:23.903+0000 7f366de50700 1 -- 172.21.15.94:0/2390305193 wait complete. 2024-04-06T03:31:24.586 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (2/15)... 2024-04-06T03:31:27.054 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: Activating manager daemon smithi094.purlzp 2024-04-06T03:31:27.054 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: mgrmap e2: smithi094.purlzp(active, starting, since 0.00299609s) 2024-04-06T03:31:27.054 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:31:27.054 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:31:27.055 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:31:27.055 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:31:27.055 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:31:27.055 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: Manager daemon smithi094.purlzp is now available 2024-04-06T03:31:27.055 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:31:27.055 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:31:27.055 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:27.055 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:27.055 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:26 smithi094 ceph-mon[29713]: from='mgr.14100 172.21.15.94:0/3928712353' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:27.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:27.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-06T03:31:27.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "c6509676-f3c5-11ee-b649-cb9ed24678a4", 2024-04-06T03:31:27.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-06T03:31:27.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-06T03:31:27.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-06T03:31:27.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-06T03:31:27.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "smithi094" 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:27.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-06T03:31:27.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:27.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-06T03:31:27.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-06T03:31:27.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-06T03:31:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-06T03:31:27.654 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-06T03:31:12.411866+0000", 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-06T03:31:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.967+0000 7fb709eeb700 1 Processor -- start 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.967+0000 7fb709eeb700 1 -- start start 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.967+0000 7fb709eeb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704102d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.967+0000 7fb709eeb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb704103260 con 0x7fb704102910 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.967+0000 7fb7037fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.967+0000 7fb7037fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704102d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47704/0 (socket says 172.21.15.94:47704) 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.967+0000 7fb7037fe700 1 -- 172.21.15.94:0/1379370319 learned_addr learned my addr 172.21.15.94:0/1379370319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.967+0000 7fb7037fe700 1 -- 172.21.15.94:0/1379370319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7041033a0 con 0x7fb704102910 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.967+0000 7fb7037fe700 1 --2- 172.21.15.94:0/1379370319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704102d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40098e0 tx=0x7fb6f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d59925491d2dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb7027fc700 1 -- 172.21.15.94:0/1379370319 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f400b860 con 0x7fb704102910 2024-04-06T03:31:27.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb7027fc700 1 -- 172.21.15.94:0/1379370319 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb6f400be50 con 0x7fb704102910 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb7027fc700 1 -- 172.21.15.94:0/1379370319 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f4010380 con 0x7fb704102910 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb709eeb700 1 -- 172.21.15.94:0/1379370319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 msgr2=0x7fb704102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb709eeb700 1 --2- 172.21.15.94:0/1379370319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704102d20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40098e0 tx=0x7fb6f4009250 comp rx=0 tx=0).stop 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb709eeb700 1 -- 172.21.15.94:0/1379370319 shutdown_connections 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb709eeb700 1 --2- 172.21.15.94:0/1379370319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704102d20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb709eeb700 1 -- 172.21.15.94:0/1379370319 >> 172.21.15.94:0/1379370319 conn(0x7fb7040fe070 msgr2=0x7fb7041004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb709eeb700 1 -- 172.21.15.94:0/1379370319 shutdown_connections 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.968+0000 7fb709eeb700 1 -- 172.21.15.94:0/1379370319 wait complete. 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.969+0000 7fb709eeb700 1 Processor -- start 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.969+0000 7fb709eeb700 1 -- start start 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.969+0000 7fb709eeb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704111700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.969+0000 7fb709eeb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb704111c40 con 0x7fb704102910 2024-04-06T03:31:27.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.969+0000 7fb7037fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704111700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.969+0000 7fb7037fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704111700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47710/0 (socket says 172.21.15.94:47710) 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.969+0000 7fb7037fe700 1 -- 172.21.15.94:0/3775721950 learned_addr learned my addr 172.21.15.94:0/3775721950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.969+0000 7fb7037fe700 1 -- 172.21.15.94:0/3775721950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40094f0 con 0x7fb704102910 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.969+0000 7fb7037fe700 1 --2- 172.21.15.94:0/3775721950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704111700 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b440 tx=0x7fb6f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.970+0000 7fb6ebfff700 1 -- 172.21.15.94:0/3775721950 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f4010760 con 0x7fb704102910 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.970+0000 7fb709eeb700 1 -- 172.21.15.94:0/3775721950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb704111e40 con 0x7fb704102910 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.970+0000 7fb6ebfff700 1 -- 172.21.15.94:0/3775721950 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb6f4010d50 con 0x7fb704102910 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.970+0000 7fb6ebfff700 1 -- 172.21.15.94:0/3775721950 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6f401ae20 con 0x7fb704102910 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.970+0000 7fb709eeb700 1 -- 172.21.15.94:0/3775721950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7041129b0 con 0x7fb704102910 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.970+0000 7fb6ebfff700 1 -- 172.21.15.94:0/3775721950 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 2) v1 ==== 43124+0+0 (secure 0 0 0) 0x7fb6f402e440 con 0x7fb704102910 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.971+0000 7fb6ebfff700 1 -- 172.21.15.94:0/3775721950 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb6f404b090 con 0x7fb704102910 2024-04-06T03:31:27.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.971+0000 7fb709eeb700 1 -- 172.21.15.94:0/3775721950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70405f5f0 con 0x7fb704102910 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:26.973+0000 7fb6ebfff700 1 -- 172.21.15.94:0/3775721950 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb6f4018940 con 0x7fb704102910 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:27.042+0000 7fb709eeb700 1 -- 172.21.15.94:0/3775721950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb704112bf0 con 0x7fb704102910 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:27.042+0000 7fb6ebfff700 1 -- 172.21.15.94:0/3775721950 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fb6f4049090 con 0x7fb704102910 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:27.043+0000 7fb709eeb700 1 -- 172.21.15.94:0/3775721950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 msgr2=0x7fb704111700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:27.043+0000 7fb709eeb700 1 --2- 172.21.15.94:0/3775721950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704111700 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400b440 tx=0x7fb6f4009250 comp rx=0 tx=0).stop 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:27.043+0000 7fb709eeb700 1 -- 172.21.15.94:0/3775721950 shutdown_connections 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:27.043+0000 7fb709eeb700 1 --2- 172.21.15.94:0/3775721950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb704102910 0x7fb704111700 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:27.043+0000 7fb709eeb700 1 -- 172.21.15.94:0/3775721950 >> 172.21.15.94:0/3775721950 conn(0x7fb7040fe070 msgr2=0x7fb7040fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:27.043+0000 7fb709eeb700 1 -- 172.21.15.94:0/3775721950 shutdown_connections 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:27.043+0000 7fb709eeb700 1 -- 172.21.15.94:0/3775721950 wait complete. 2024-04-06T03:31:27.659 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (3/15)... 2024-04-06T03:31:27.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:27 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3775721950' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-06T03:31:27.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:27 smithi094 ceph-mon[29713]: mgrmap e3: smithi094.purlzp(active, since 1.00735s) 2024-04-06T03:31:30.026 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:29 smithi094 ceph-mon[29713]: mgrmap e4: smithi094.purlzp(active, since 2s) 2024-04-06T03:31:30.944 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:30 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3769643652' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-06T03:31:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "c6509676-f3c5-11ee-b649-cb9ed24678a4", 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "smithi094" 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-06T03:31:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-06T03:31:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-06T03:31:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-06T03:31:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-06T03:31:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-06T03:31:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-06T03:31:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-06T03:31:30.958 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-06T03:31:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-06T03:31:12.411866+0000", 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.126+0000 7f62072f7700 1 Processor -- start 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.126+0000 7f62072f7700 1 -- start start 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.126+0000 7f62072f7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f6200101070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.126+0000 7f62072f7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62001015b0 con 0x7f6200100c60 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.126+0000 7f6205093700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f6200101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.126+0000 7f6205093700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f6200101070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42760/0 (socket says 172.21.15.94:42760) 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.126+0000 7f6205093700 1 -- 172.21.15.94:0/3883625434 learned_addr learned my addr 172.21.15.94:0/3883625434 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.126+0000 7f6205093700 1 -- 172.21.15.94:0/3883625434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62001016f0 con 0x7f6200100c60 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f6205093700 1 --2- 172.21.15.94:0/3883625434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f6200101070 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61f00098e0 tx=0x7f61f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b2cfb90ffd1601 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f61f7fff700 1 -- 172.21.15.94:0/3883625434 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61f000b860 con 0x7f6200100c60 2024-04-06T03:31:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f61f7fff700 1 -- 172.21.15.94:0/3883625434 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f61f000be50 con 0x7f6200100c60 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f61f7fff700 1 -- 172.21.15.94:0/3883625434 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61f0010380 con 0x7f6200100c60 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f62072f7700 1 -- 172.21.15.94:0/3883625434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 msgr2=0x7f6200101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f62072f7700 1 --2- 172.21.15.94:0/3883625434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f6200101070 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61f00098e0 tx=0x7f61f0009250 comp rx=0 tx=0).stop 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f62072f7700 1 -- 172.21.15.94:0/3883625434 shutdown_connections 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f62072f7700 1 --2- 172.21.15.94:0/3883625434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f6200101070 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f62072f7700 1 -- 172.21.15.94:0/3883625434 >> 172.21.15.94:0/3883625434 conn(0x7f62000fc6c0 msgr2=0x7f62000fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.127+0000 7f62072f7700 1 -- 172.21.15.94:0/3883625434 shutdown_connections 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.128+0000 7f62072f7700 1 -- 172.21.15.94:0/3883625434 wait complete. 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.128+0000 7f62072f7700 1 Processor -- start 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.128+0000 7f62072f7700 1 -- start start 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.128+0000 7f62072f7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f62001116d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.128+0000 7f62072f7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6200111c10 con 0x7f6200100c60 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.128+0000 7f6205093700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f62001116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.128+0000 7f6205093700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f62001116d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42772/0 (socket says 172.21.15.94:42772) 2024-04-06T03:31:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.128+0000 7f6205093700 1 -- 172.21.15.94:0/3769643652 learned_addr learned my addr 172.21.15.94:0/3769643652 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.128+0000 7f6205093700 1 -- 172.21.15.94:0/3769643652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00094f0 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.129+0000 7f6205093700 1 --2- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f62001116d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f61f0006b00 tx=0x7f61f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.129+0000 7f61f5ffb700 1 -- 172.21.15.94:0/3769643652 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61f0010760 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.129+0000 7f61f5ffb700 1 -- 172.21.15.94:0/3769643652 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f61f0010d50 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.129+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6200111e10 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.129+0000 7f61f5ffb700 1 -- 172.21.15.94:0/3769643652 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61f001ad40 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.129+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6200112980 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.130+0000 7f61f5ffb700 1 -- 172.21.15.94:0/3769643652 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f61f002e440 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.130+0000 7f61f5ffb700 1 --2- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f61ec036740 0x7f61ec038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.130+0000 7f61f5ffb700 1 -- 172.21.15.94:0/3769643652 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f61f004a550 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.130+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620005f5f0 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.130+0000 7f6204892700 1 --2- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f61ec036740 0x7f61ec038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.130+0000 7f6204892700 1 --2- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f61ec036740 0x7f61ec038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009990 tx=0x7f61fc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.135+0000 7f61f5ffb700 1 -- 172.21.15.94:0/3769643652 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f61f00243f0 con 0x7f6200100c60 2024-04-06T03:31:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.394+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6200112bf0 con 0x7f6200100c60 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.394+0000 7f61f5ffb700 1 -- 172.21.15.94:0/3769643652 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f61f0048030 con 0x7f6200100c60 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f61ec036740 msgr2=0x7f61ec038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 --2- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f61ec036740 0x7f61ec038bf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009990 tx=0x7f61fc006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 msgr2=0x7f62001116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 --2- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f62001116d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f61f0006b00 tx=0x7f61f0009250 comp rx=0 tx=0).stop 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 shutdown_connections 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 --2- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f61ec036740 0x7f61ec038bf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 --2- 172.21.15.94:0/3769643652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6200100c60 0x7f62001116d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 >> 172.21.15.94:0/3769643652 conn(0x7f62000fc6c0 msgr2=0x7f62000fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 shutdown_connections 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:30.397+0000 7f62072f7700 1 -- 172.21.15.94:0/3769643652 wait complete. 2024-04-06T03:31:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:mgr is available 2024-04-06T03:31:31.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:31 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1602849003' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-06T03:31:32.342 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:32.342 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-04-06T03:31:32.342 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.382+0000 7fcf7d4c8700 1 Processor -- start 2024-04-06T03:31:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.382+0000 7fcf7d4c8700 1 -- start start 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.382+0000 7fcf7d4c8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.382+0000 7fcf7d4c8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78102730 con 0x7fcf78101de0 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.383+0000 7fcf76ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.383+0000 7fcf76ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42788/0 (socket says 172.21.15.94:42788) 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.383+0000 7fcf76ffd700 1 -- 172.21.15.94:0/3774079135 learned_addr learned my addr 172.21.15.94:0/3774079135 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.383+0000 7fcf76ffd700 1 -- 172.21.15.94:0/3774079135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf78102870 con 0x7fcf78101de0 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.383+0000 7fcf76ffd700 1 --2- 172.21.15.94:0/3774079135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781021f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcf680098e0 tx=0x7fcf68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923766429d59f83f server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf75ffb700 1 -- 172.21.15.94:0/3774079135 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf6800b860 con 0x7fcf78101de0 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf75ffb700 1 -- 172.21.15.94:0/3774079135 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcf6800be50 con 0x7fcf78101de0 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf75ffb700 1 -- 172.21.15.94:0/3774079135 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf68010380 con 0x7fcf78101de0 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/3774079135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 msgr2=0x7fcf781021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf7d4c8700 1 --2- 172.21.15.94:0/3774079135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781021f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcf680098e0 tx=0x7fcf68009250 comp rx=0 tx=0).stop 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/3774079135 shutdown_connections 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf7d4c8700 1 --2- 172.21.15.94:0/3774079135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781021f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/3774079135 >> 172.21.15.94:0/3774079135 conn(0x7fcf780fc750 msgr2=0x7fcf780feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:32.344 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/3774079135 shutdown_connections 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.384+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/3774079135 wait complete. 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.385+0000 7fcf7d4c8700 1 Processor -- start 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.385+0000 7fcf7d4c8700 1 -- start start 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.385+0000 7fcf7d4c8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.385+0000 7fcf7d4c8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78114380 con 0x7fcf78101de0 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.385+0000 7fcf76ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.385+0000 7fcf76ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42792/0 (socket says 172.21.15.94:42792) 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.385+0000 7fcf76ffd700 1 -- 172.21.15.94:0/1602849003 learned_addr learned my addr 172.21.15.94:0/1602849003 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.385+0000 7fcf76ffd700 1 -- 172.21.15.94:0/1602849003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf680094f0 con 0x7fcf78101de0 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.386+0000 7fcf76ffd700 1 --2- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781015e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800b440 tx=0x7fcf68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.386+0000 7fcf5f7fe700 1 -- 172.21.15.94:0/1602849003 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf68010760 con 0x7fcf78101de0 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.386+0000 7fcf5f7fe700 1 -- 172.21.15.94:0/1602849003 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcf68010d50 con 0x7fcf78101de0 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.386+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf781144c0 con 0x7fcf78101de0 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.386+0000 7fcf5f7fe700 1 -- 172.21.15.94:0/1602849003 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf6801ae20 con 0x7fcf78101de0 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.386+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf78114950 con 0x7fcf78101de0 2024-04-06T03:31:32.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.387+0000 7fcf5f7fe700 1 -- 172.21.15.94:0/1602849003 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fcf6802e440 con 0x7fcf78101de0 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.387+0000 7fcf5f7fe700 1 --2- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7fcf60036790 0x7fcf60038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.387+0000 7fcf5f7fe700 1 -- 172.21.15.94:0/1602849003 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcf6804a540 con 0x7fcf78101de0 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.387+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7804bed0 con 0x7fcf78101de0 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.387+0000 7fcf767fc700 1 --2- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7fcf60036790 0x7fcf60038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.387+0000 7fcf767fc700 1 --2- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7fcf60036790 0x7fcf60038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009990 tx=0x7fcf6c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.392+0000 7fcf5f7fe700 1 -- 172.21.15.94:0/1602849003 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcf68018940 con 0x7fcf78101de0 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.580+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fcf78103680 con 0x7fcf78101de0 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.580+0000 7fcf5f7fe700 1 -- 172.21.15.94:0/1602849003 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fcf68048020 con 0x7fcf78101de0 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.582+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7fcf60036790 msgr2=0x7fcf60038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.582+0000 7fcf7d4c8700 1 --2- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7fcf60036790 0x7fcf60038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009990 tx=0x7fcf6c006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.582+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 msgr2=0x7fcf781015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.582+0000 7fcf7d4c8700 1 --2- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781015e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800b440 tx=0x7fcf68009250 comp rx=0 tx=0).stop 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.583+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 shutdown_connections 2024-04-06T03:31:32.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.583+0000 7fcf7d4c8700 1 --2- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7fcf60036790 0x7fcf60038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:32.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.583+0000 7fcf7d4c8700 1 --2- 172.21.15.94:0/1602849003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf78101de0 0x7fcf781015e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:32.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.583+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 >> 172.21.15.94:0/1602849003 conn(0x7fcf780fc750 msgr2=0x7fcf780fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:32.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.583+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 shutdown_connections 2024-04-06T03:31:32.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:31.583+0000 7fcf7d4c8700 1 -- 172.21.15.94:0/1602849003 wait complete. 2024-04-06T03:31:32.347 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling cephadm module... 2024-04-06T03:31:33.306 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:32 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3191573903' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.697+0000 7f843b7f9700 1 Processor -- start 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.697+0000 7f843b7f9700 1 -- start start 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.697+0000 7f843b7f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434101140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.697+0000 7f843b7f9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8434103b70 con 0x7f8434100d10 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.697+0000 7f8439595700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.697+0000 7f8439595700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42806/0 (socket says 172.21.15.94:42806) 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.697+0000 7f8439595700 1 -- 172.21.15.94:0/598195861 learned_addr learned my addr 172.21.15.94:0/598195861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.697+0000 7f8439595700 1 -- 172.21.15.94:0/598195861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8434101680 con 0x7f8434100d10 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f8439595700 1 --2- 172.21.15.94:0/598195861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434101140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84300098e0 tx=0x7f8430009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c77eb95583961a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f842bfff700 1 -- 172.21.15.94:0/598195861 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f843000b860 con 0x7f8434100d10 2024-04-06T03:31:33.631 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f842bfff700 1 -- 172.21.15.94:0/598195861 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f843000be50 con 0x7f8434100d10 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f842bfff700 1 -- 172.21.15.94:0/598195861 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8430010380 con 0x7f8434100d10 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f843b7f9700 1 -- 172.21.15.94:0/598195861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 msgr2=0x7f8434101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f843b7f9700 1 --2- 172.21.15.94:0/598195861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434101140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84300098e0 tx=0x7f8430009250 comp rx=0 tx=0).stop 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f843b7f9700 1 -- 172.21.15.94:0/598195861 shutdown_connections 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f843b7f9700 1 --2- 172.21.15.94:0/598195861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434101140 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f843b7f9700 1 -- 172.21.15.94:0/598195861 >> 172.21.15.94:0/598195861 conn(0x7f84340fc750 msgr2=0x7f84340feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f843b7f9700 1 -- 172.21.15.94:0/598195861 shutdown_connections 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.698+0000 7f843b7f9700 1 -- 172.21.15.94:0/598195861 wait complete. 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.699+0000 7f843b7f9700 1 Processor -- start 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.699+0000 7f843b7f9700 1 -- start start 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.699+0000 7f843b7f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.699+0000 7f843b7f9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8434113e40 con 0x7f8434100d10 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.699+0000 7f8439595700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.699+0000 7f8439595700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42816/0 (socket says 172.21.15.94:42816) 2024-04-06T03:31:33.632 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.699+0000 7f8439595700 1 -- 172.21.15.94:0/3191573903 learned_addr learned my addr 172.21.15.94:0/3191573903 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.699+0000 7f8439595700 1 -- 172.21.15.94:0/3191573903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300094f0 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.700+0000 7f8439595700 1 --2- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434113900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8430003050 tx=0x7f8430009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.700+0000 7f8429ffb700 1 -- 172.21.15.94:0/3191573903 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8430010760 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.700+0000 7f8429ffb700 1 -- 172.21.15.94:0/3191573903 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8430010d50 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.700+0000 7f8429ffb700 1 -- 172.21.15.94:0/3191573903 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f843001ae20 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.700+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8434114040 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.700+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8434114bb0 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.701+0000 7f8429ffb700 1 -- 172.21.15.94:0/3191573903 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f84300108c0 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.701+0000 7f8429ffb700 1 --2- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8420036790 0x7f8420038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.701+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843405f5f0 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.701+0000 7f8429ffb700 1 -- 172.21.15.94:0/3191573903 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f843004a4b0 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.701+0000 7f8438d94700 1 --2- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8420036790 0x7f8420038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.701+0000 7f8438d94700 1 --2- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8420036790 0x7f8420038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8424009990 tx=0x7f8424006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.706+0000 7f8429ffb700 1 -- 172.21.15.94:0/3191573903 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f843001a6a0 con 0x7f8434100d10 2024-04-06T03:31:33.633 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.938+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f843404bed0 con 0x7f8434100d10 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.991+0000 7f8429ffb700 1 -- 172.21.15.94:0/3191573903 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f843002ac40 con 0x7f8434100d10 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.992+0000 7f8429ffb700 1 -- 172.21.15.94:0/3191573903 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f8430018350 con 0x7f8434100d10 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8420036790 msgr2=0x7f8420038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 --2- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8420036790 0x7f8420038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8424009990 tx=0x7f8424006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 msgr2=0x7f8434113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 --2- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434113900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8430003050 tx=0x7f8430009250 comp rx=0 tx=0).stop 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 shutdown_connections 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 --2- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8420036790 0x7f8420038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 --2- 172.21.15.94:0/3191573903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8434100d10 0x7f8434113900 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 >> 172.21.15.94:0/3191573903 conn(0x7f84340fc750 msgr2=0x7f84340fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 shutdown_connections 2024-04-06T03:31:33.634 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:32.995+0000 7f843b7f9700 1 -- 172.21.15.94:0/3191573903 wait complete. 2024-04-06T03:31:34.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:33 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3191573903' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-06T03:31:34.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:33 smithi094 ceph-mon[29713]: mgrmap e5: smithi094.purlzp(active, since 6s) 2024-04-06T03:31:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-06T03:31:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-06T03:31:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-06T03:31:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "smithi094.purlzp", 2024-04-06T03:31:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-06T03:31:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-06T03:31:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.055+0000 7f8fb47c0700 1 Processor -- start 2024-04-06T03:31:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.055+0000 7f8fb47c0700 1 -- start start 2024-04-06T03:31:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.055+0000 7f8fb47c0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac105500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.055+0000 7f8fb47c0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0716a0 con 0x7f8fac071290 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.055+0000 7f8fb255c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.055+0000 7f8fb255c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac105500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42854/0 (socket says 172.21.15.94:42854) 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.055+0000 7f8fb255c700 1 -- 172.21.15.94:0/2026846162 learned_addr learned my addr 172.21.15.94:0/2026846162 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.056+0000 7f8fb255c700 1 -- 172.21.15.94:0/2026846162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac105a40 con 0x7f8fac071290 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.056+0000 7f8fb255c700 1 --2- 172.21.15.94:0/2026846162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac105500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80098e0 tx=0x7f8fa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c766a9d92cb7396 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.056+0000 7f8fb155a700 1 -- 172.21.15.94:0/2026846162 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8fa800b860 con 0x7f8fac071290 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.056+0000 7f8fb155a700 1 -- 172.21.15.94:0/2026846162 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8fa800be50 con 0x7f8fac071290 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.056+0000 7f8fb155a700 1 -- 172.21.15.94:0/2026846162 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8fa8010380 con 0x7f8fac071290 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 -- 172.21.15.94:0/2026846162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 msgr2=0x7f8fac105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 --2- 172.21.15.94:0/2026846162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac105500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80098e0 tx=0x7f8fa8009250 comp rx=0 tx=0).stop 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 -- 172.21.15.94:0/2026846162 shutdown_connections 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 --2- 172.21.15.94:0/2026846162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac105500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:34.872 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 -- 172.21.15.94:0/2026846162 >> 172.21.15.94:0/2026846162 conn(0x7f8fac0fcfb0 msgr2=0x7f8fac0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 -- 172.21.15.94:0/2026846162 shutdown_connections 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 -- 172.21.15.94:0/2026846162 wait complete. 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 Processor -- start 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 -- start start 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac111740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.057+0000 7f8fb47c0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac111c80 con 0x7f8fac071290 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fb255c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac111740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fb255c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac111740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42864/0 (socket says 172.21.15.94:42864) 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fb255c700 1 -- 172.21.15.94:0/1549888969 learned_addr learned my addr 172.21.15.94:0/1549888969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fb255c700 1 -- 172.21.15.94:0/1549888969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa80094f0 con 0x7f8fac071290 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fb255c700 1 --2- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac111740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80090b0 tx=0x7f8fa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fa2ffd700 1 -- 172.21.15.94:0/1549888969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8fa8010760 con 0x7f8fac071290 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fa2ffd700 1 -- 172.21.15.94:0/1549888969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8fa8010d50 con 0x7f8fac071290 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac111e80 con 0x7f8fac071290 2024-04-06T03:31:34.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fa2ffd700 1 -- 172.21.15.94:0/1549888969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8fa801ace0 con 0x7f8fac071290 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.058+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac1129f0 con 0x7f8fac071290 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.059+0000 7f8fa2ffd700 1 -- 172.21.15.94:0/1549888969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f8fa802d3f0 con 0x7f8fac071290 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.059+0000 7f8fa2ffd700 1 --2- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8f980367e0 0x7f8f98038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.059+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac05f5f0 con 0x7f8fac071290 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.059+0000 7f8fb1d5b700 1 -- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8f980367e0 msgr2=0x7f8f98038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3295357679 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.059+0000 7f8fa2ffd700 1 -- 172.21.15.94:0/1549888969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8fa804a0e0 con 0x7f8fac071290 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.059+0000 7f8fb1d5b700 1 --2- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8f980367e0 0x7f8f98038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.064+0000 7f8fa2ffd700 1 -- 172.21.15.94:0/1549888969 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8fa802b8e0 con 0x7f8fac071290 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.260+0000 7f8fb1d5b700 1 -- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8f980367e0 msgr2=0x7f8f98038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3295357679 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.260+0000 7f8fb1d5b700 1 --2- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8f980367e0 0x7f8f98038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.310+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8fac112d30 con 0x7f8fac071290 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.310+0000 7f8fa2ffd700 1 -- 172.21.15.94:0/1549888969 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f8fa802b2f0 con 0x7f8fac071290 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8f980367e0 msgr2=0x7f8f98038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-06T03:31:34.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 --2- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8f980367e0 0x7f8f98038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 msgr2=0x7f8fac111740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 --2- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac111740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80090b0 tx=0x7f8fa8009250 comp rx=0 tx=0).stop 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 shutdown_connections 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 --2- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f8f980367e0 0x7f8f98038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 --2- 172.21.15.94:0/1549888969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fac071290 0x7f8fac111740 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 >> 172.21.15.94:0/1549888969 conn(0x7f8fac0fcfb0 msgr2=0x7f8fac0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 shutdown_connections 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:34.312+0000 7f8fb47c0700 1 -- 172.21.15.94:0/1549888969 wait complete. 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-04-06T03:31:34.875 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 5... 2024-04-06T03:31:35.088 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:34 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1549888969' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-06T03:31:39.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: Active manager daemon smithi094.purlzp restarted 2024-04-06T03:31:39.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: Activating manager daemon smithi094.purlzp 2024-04-06T03:31:39.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: osdmap e2: 0 total, 0 up, 0 in 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: mgrmap e6: smithi094.purlzp(active, starting, since 0.00327228s) 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: Manager daemon smithi094.purlzp is now available 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:31:39.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:39 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:31:40.785 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:40 smithi094 ceph-mon[29713]: Found migration_current of "None". Setting to last migration. 2024-04-06T03:31:40.785 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:40 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:31:40.785 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:40 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:31:40.786 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:40 smithi094 ceph-mon[29713]: mgrmap e7: smithi094.purlzp(active, since 1.00552s) 2024-04-06T03:31:41.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-06T03:31:41.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.340+0000 7f3317ff1700 1 Processor -- start 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.340+0000 7f3317ff1700 1 -- start start 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.340+0000 7f3317ff1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310105560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.340+0000 7f3317ff1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3310071640 con 0x7f3310071230 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.341+0000 7f3315d8d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310105560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.341+0000 7f3315d8d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310105560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42866/0 (socket says 172.21.15.94:42866) 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.341+0000 7f3315d8d700 1 -- 172.21.15.94:0/190714663 learned_addr learned my addr 172.21.15.94:0/190714663 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.341+0000 7f3315d8d700 1 -- 172.21.15.94:0/190714663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3310105aa0 con 0x7f3310071230 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.341+0000 7f3315d8d700 1 --2- 172.21.15.94:0/190714663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310105560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33000098e0 tx=0x7f3300009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1f4eefa472135d server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.341+0000 7f3314d8b700 1 -- 172.21.15.94:0/190714663 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f330000b860 con 0x7f3310071230 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.341+0000 7f3314d8b700 1 -- 172.21.15.94:0/190714663 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f330000be50 con 0x7f3310071230 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.341+0000 7f3314d8b700 1 -- 172.21.15.94:0/190714663 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3300010380 con 0x7f3310071230 2024-04-06T03:31:41.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.342+0000 7f3317ff1700 1 -- 172.21.15.94:0/190714663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 msgr2=0x7f3310105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.342+0000 7f3317ff1700 1 --2- 172.21.15.94:0/190714663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310105560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f33000098e0 tx=0x7f3300009250 comp rx=0 tx=0).stop 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.342+0000 7f3317ff1700 1 -- 172.21.15.94:0/190714663 shutdown_connections 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.342+0000 7f3317ff1700 1 --2- 172.21.15.94:0/190714663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310105560 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.342+0000 7f3317ff1700 1 -- 172.21.15.94:0/190714663 >> 172.21.15.94:0/190714663 conn(0x7f33100fcfb0 msgr2=0x7f33100ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.342+0000 7f3317ff1700 1 -- 172.21.15.94:0/190714663 shutdown_connections 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.342+0000 7f3317ff1700 1 -- 172.21.15.94:0/190714663 wait complete. 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3317ff1700 1 Processor -- start 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3317ff1700 1 -- start start 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3317ff1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310111730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3317ff1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3310111c70 con 0x7f3310071230 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3315d8d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310111730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3315d8d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310111730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42872/0 (socket says 172.21.15.94:42872) 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3315d8d700 1 -- 172.21.15.94:0/1824932145 learned_addr learned my addr 172.21.15.94:0/1824932145 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3315d8d700 1 -- 172.21.15.94:0/1824932145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33000094f0 con 0x7f3310071230 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3315d8d700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310111730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3300011bd0 tx=0x7f3300009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:41.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33000107c0 con 0x7f3310071230 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f3317ff1700 1 -- 172.21.15.94:0/1824932145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3310111e70 con 0x7f3310071230 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3300010db0 con 0x7f3310071230 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.343+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f330001ad60 con 0x7f3310071230 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.344+0000 7f3317ff1700 1 -- 172.21.15.94:0/1824932145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33101129e0 con 0x7f3310071230 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.344+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f330002e440 con 0x7f3310071230 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.344+0000 7f33067fc700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 0x7f32fc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.344+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 --> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32fc039350 con 0x7f32fc036790 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.344+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f330004abe0 con 0x7f3310071230 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.344+0000 7f331558c700 1 -- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 msgr2=0x7f32fc038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3295357679 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.344+0000 7f331558c700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 0x7f32fc038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.545+0000 7f331558c700 1 -- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 msgr2=0x7f32fc038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3295357679 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.545+0000 7f331558c700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 0x7f32fc038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.946+0000 7f331558c700 1 -- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 msgr2=0x7f32fc038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3295357679 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:35.946+0000 7f331558c700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 0x7f32fc038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:36.747+0000 7f331558c700 1 -- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 msgr2=0x7f32fc038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3295357679 2024-04-06T03:31:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:36.747+0000 7f331558c700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 0x7f32fc038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:38.348+0000 7f331558c700 1 -- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 msgr2=0x7f32fc038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/3295357679 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:38.348+0000 7f331558c700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 0x7f32fc038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:39.467+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7f330002ed40 con 0x7f3310071230 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:39.467+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 msgr2=0x7f32fc038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:39.467+0000 7f33067fc700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 0x7f32fc038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.468+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f330002c9b0 con 0x7f3310071230 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.468+0000 7f33067fc700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f32fc039fc0 0x7f32fc03c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.468+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32fc039350 con 0x7f32fc039fc0 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.469+0000 7f331558c700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f32fc039fc0 0x7f32fc03c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.469+0000 7f331558c700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f32fc039fc0 0x7f32fc03c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f330c0039d0 tx=0x7f330c009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.470+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f32fc039350 con 0x7f32fc039fc0 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.475+0000 7f3317ff1700 1 -- 172.21.15.94:0/1824932145 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f3310112300 con 0x7f32fc039fc0 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f33067fc700 1 -- 172.21.15.94:0/1824932145 <== mgr.14120 v2:172.21.15.94:6800/682257512 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f3310112300 con 0x7f32fc039fc0 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 -- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f32fc039fc0 msgr2=0x7f32fc03c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f32fc039fc0 0x7f32fc03c3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f330c0039d0 tx=0x7f330c009350 comp rx=0 tx=0).stop 2024-04-06T03:31:41.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 -- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 msgr2=0x7f3310111730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310111730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3300011bd0 tx=0x7f3300009250 comp rx=0 tx=0).stop 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 -- 172.21.15.94:0/1824932145 shutdown_connections 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f32fc039fc0 0x7f32fc03c3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3310071230 0x7f3310111730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 --2- 172.21.15.94:0/1824932145 >> [v2:172.21.15.94:6800/3295357679,v1:172.21.15.94:6801/3295357679] conn(0x7f32fc036790 0x7f32fc038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 -- 172.21.15.94:0/1824932145 >> 172.21.15.94:0/1824932145 conn(0x7f33100fcfb0 msgr2=0x7f33100fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 -- 172.21.15.94:0/1824932145 shutdown_connections 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:40.476+0000 7f3317ff1700 1 -- 172.21.15.94:0/1824932145 wait complete. 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 5 is available 2024-04-06T03:31:41.075 INFO:teuthology.orchestra.run.smithi094.stdout:Setting orchestrator backend to cephadm... 2024-04-06T03:31:41.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:41 smithi094 ceph-mon[29713]: [06/Apr/2024:03:31:40] ENGINE Bus STARTING 2024-04-06T03:31:41.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:41 smithi094 ceph-mon[29713]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-06T03:31:41.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:41 smithi094 ceph-mon[29713]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-06T03:31:41.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:41 smithi094 ceph-mon[29713]: [06/Apr/2024:03:31:40] ENGINE Serving on https://172.21.15.94:7150 2024-04-06T03:31:41.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:41 smithi094 ceph-mon[29713]: [06/Apr/2024:03:31:40] ENGINE Bus STARTED 2024-04-06T03:31:41.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:41 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:41.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:41 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:31:42.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.514+0000 7fc6518b2700 1 Processor -- start 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.514+0000 7fc6518b2700 1 -- start start 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.514+0000 7fc6518b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c1021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.514+0000 7fc6518b2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c102700 con 0x7fc64c101db0 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.515+0000 7fc64affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c1021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.515+0000 7fc64affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c1021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39488/0 (socket says 172.21.15.94:39488) 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.515+0000 7fc64affd700 1 -- 172.21.15.94:0/1055371715 learned_addr learned my addr 172.21.15.94:0/1055371715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.515+0000 7fc64affd700 1 -- 172.21.15.94:0/1055371715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c102840 con 0x7fc64c101db0 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.515+0000 7fc64affd700 1 --2- 172.21.15.94:0/1055371715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c1021c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00bb80 tx=0x7fc63c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edc422f418ae88b server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.515+0000 7fc64a7fc700 1 -- 172.21.15.94:0/1055371715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63c00dd00 con 0x7fc64c101db0 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.515+0000 7fc64a7fc700 1 -- 172.21.15.94:0/1055371715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc63c00de60 con 0x7fc64c101db0 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.515+0000 7fc64a7fc700 1 -- 172.21.15.94:0/1055371715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63c0188a0 con 0x7fc64c101db0 2024-04-06T03:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.516+0000 7fc6518b2700 1 -- 172.21.15.94:0/1055371715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 msgr2=0x7fc64c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.516+0000 7fc6518b2700 1 --2- 172.21.15.94:0/1055371715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c1021c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00bb80 tx=0x7fc63c00b4f0 comp rx=0 tx=0).stop 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.516+0000 7fc6518b2700 1 -- 172.21.15.94:0/1055371715 shutdown_connections 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.516+0000 7fc6518b2700 1 --2- 172.21.15.94:0/1055371715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c1021c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.516+0000 7fc6518b2700 1 -- 172.21.15.94:0/1055371715 >> 172.21.15.94:0/1055371715 conn(0x7fc64c0fc700 msgr2=0x7fc64c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.516+0000 7fc6518b2700 1 -- 172.21.15.94:0/1055371715 shutdown_connections 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.516+0000 7fc6518b2700 1 -- 172.21.15.94:0/1055371715 wait complete. 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.516+0000 7fc6518b2700 1 Processor -- start 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.516+0000 7fc6518b2700 1 -- start start 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc6518b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c10f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc6518b2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c10f950 con 0x7fc64c101db0 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc64affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c10f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc64affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c10f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39494/0 (socket says 172.21.15.94:39494) 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc64affd700 1 -- 172.21.15.94:0/2656199197 learned_addr learned my addr 172.21.15.94:0/2656199197 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc64affd700 1 -- 172.21.15.94:0/2656199197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c00b790 con 0x7fc64c101db0 2024-04-06T03:31:42.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc64affd700 1 --2- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c10f410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00d650 tx=0x7fc63c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc6437fe700 1 -- 172.21.15.94:0/2656199197 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63c010b50 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc6437fe700 1 -- 172.21.15.94:0/2656199197 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc63c01d050 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc6437fe700 1 -- 172.21.15.94:0/2656199197 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63c022590 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c10fb50 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.517+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c1106c0 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.518+0000 7fc6437fe700 1 -- 172.21.15.94:0/2656199197 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7fc63c023470 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.518+0000 7fc6437fe700 1 --2- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fc6380366b0 0x7fc638038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.518+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c05f5f0 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.518+0000 7fc6437fe700 1 -- 172.21.15.94:0/2656199197 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc63c014070 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.519+0000 7fc643fff700 1 --2- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fc6380366b0 0x7fc638038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.519+0000 7fc643fff700 1 --2- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fc6380366b0 0x7fc638038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc634009990 tx=0x7fc634006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.523+0000 7fc6437fe700 1 -- 172.21.15.94:0/2656199197 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc63c00fdf0 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.539+0000 7fc6437fe700 1 -- 172.21.15.94:0/2656199197 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fc63c022ac0 con 0x7fc64c101db0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.759+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc64c111240 con 0x7fc6380366b0 2024-04-06T03:31:42.385 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.768+0000 7fc6437fe700 1 -- 172.21.15.94:0/2656199197 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc64c111240 con 0x7fc6380366b0 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fc6380366b0 msgr2=0x7fc638038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 --2- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fc6380366b0 0x7fc638038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc634009990 tx=0x7fc634006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 msgr2=0x7fc64c10f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 --2- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c10f410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00d650 tx=0x7fc63c00b4f0 comp rx=0 tx=0).stop 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 shutdown_connections 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 --2- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fc6380366b0 0x7fc638038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 --2- 172.21.15.94:0/2656199197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc64c101db0 0x7fc64c10f410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 >> 172.21.15.94:0/2656199197 conn(0x7fc64c0fc700 msgr2=0x7fc64c1061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 shutdown_connections 2024-04-06T03:31:42.386 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:41.770+0000 7fc6518b2700 1 -- 172.21.15.94:0/2656199197 wait complete. 2024-04-06T03:31:42.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:42 smithi094 ceph-mon[29713]: mgrmap e8: smithi094.purlzp(active, since 2s) 2024-04-06T03:31:42.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:42 smithi094 ceph-mon[29713]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:42.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:42 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:42.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:42 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:31:43.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:43 smithi094 ceph-mon[29713]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.809+0000 7f82217c9700 1 Processor -- start 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.809+0000 7f82217c9700 1 -- start start 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.809+0000 7f82217c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.809+0000 7f82217c9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c103270 con 0x7f821c102920 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.810+0000 7f821affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.810+0000 7f821affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39500/0 (socket says 172.21.15.94:39500) 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.810+0000 7f821affd700 1 -- 172.21.15.94:0/3651868886 learned_addr learned my addr 172.21.15.94:0/3651868886 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.810+0000 7f821affd700 1 -- 172.21.15.94:0/3651868886 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c1033b0 con 0x7f821c102920 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.810+0000 7f821affd700 1 --2- 172.21.15.94:0/3651868886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c102d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f820c0098e0 tx=0x7f820c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa89ffc909e2f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.810+0000 7f821a7fc700 1 -- 172.21.15.94:0/3651868886 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f820c00b860 con 0x7f821c102920 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.810+0000 7f821a7fc700 1 -- 172.21.15.94:0/3651868886 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f820c00be50 con 0x7f821c102920 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.810+0000 7f821a7fc700 1 -- 172.21.15.94:0/3651868886 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f820c010380 con 0x7f821c102920 2024-04-06T03:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.811+0000 7f82217c9700 1 -- 172.21.15.94:0/3651868886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 msgr2=0x7f821c102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.811+0000 7f82217c9700 1 --2- 172.21.15.94:0/3651868886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c102d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f820c0098e0 tx=0x7f820c009250 comp rx=0 tx=0).stop 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.811+0000 7f82217c9700 1 -- 172.21.15.94:0/3651868886 shutdown_connections 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.811+0000 7f82217c9700 1 --2- 172.21.15.94:0/3651868886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c102d30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.811+0000 7f82217c9700 1 -- 172.21.15.94:0/3651868886 >> 172.21.15.94:0/3651868886 conn(0x7f821c0fe0a0 msgr2=0x7f821c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.811+0000 7f82217c9700 1 -- 172.21.15.94:0/3651868886 shutdown_connections 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.811+0000 7f82217c9700 1 -- 172.21.15.94:0/3651868886 wait complete. 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.811+0000 7f82217c9700 1 Processor -- start 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.811+0000 7f82217c9700 1 -- start start 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f82217c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f82217c9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c079180 con 0x7f821c102920 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f821affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f821affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c078c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39504/0 (socket says 172.21.15.94:39504) 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f821affd700 1 -- 172.21.15.94:0/1950749503 learned_addr learned my addr 172.21.15.94:0/1950749503 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f821affd700 1 -- 172.21.15.94:0/1950749503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c0094f0 con 0x7f821c102920 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f821affd700 1 --2- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c078c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b440 tx=0x7f820c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f8213fff700 1 -- 172.21.15.94:0/1950749503 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f820c010760 con 0x7f821c102920 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f8213fff700 1 -- 172.21.15.94:0/1950749503 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f820c010d50 con 0x7f821c102920 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c0771f0 con 0x7f821c102920 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f8213fff700 1 -- 172.21.15.94:0/1950749503 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f820c01ae20 con 0x7f821c102920 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.812+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c0776e0 con 0x7f821c102920 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.813+0000 7f8213fff700 1 -- 172.21.15.94:0/1950749503 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f820c0108c0 con 0x7f821c102920 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.813+0000 7f8213fff700 1 --2- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f82080367e0 0x7f8208038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.813+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c04bed0 con 0x7f821c102920 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.813+0000 7f8213fff700 1 -- 172.21.15.94:0/1950749503 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f820c04a570 con 0x7f821c102920 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.813+0000 7f82127fc700 1 --2- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f82080367e0 0x7f8208038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.814+0000 7f82127fc700 1 --2- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f82080367e0 0x7f8208038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8204009990 tx=0x7f8204006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:42.818+0000 7f8213fff700 1 -- 172.21.15.94:0/1950749503 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f820c018b40 con 0x7f821c102920 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.004+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f821c077ea0 con 0x7f82080367e0 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.004+0000 7f8213fff700 1 -- 172.21.15.94:0/1950749503 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f821c077ea0 con 0x7f82080367e0 2024-04-06T03:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f82080367e0 msgr2=0x7f8208038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 --2- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f82080367e0 0x7f8208038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8204009990 tx=0x7f8204006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 msgr2=0x7f821c078c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 --2- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c078c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b440 tx=0x7f820c009250 comp rx=0 tx=0).stop 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 shutdown_connections 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 --2- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f82080367e0 0x7f8208038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 --2- 172.21.15.94:0/1950749503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f821c102920 0x7f821c078c40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 >> 172.21.15.94:0/1950749503 conn(0x7f821c0fe0a0 msgr2=0x7f821c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 shutdown_connections 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.006+0000 7f82217c9700 1 -- 172.21.15.94:0/1950749503 wait complete. 2024-04-06T03:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout:Generating ssh key... 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.989+0000 7f982139e700 1 Processor -- start 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.989+0000 7f982139e700 1 -- start start 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.989+0000 7f982139e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.989+0000 7f982139e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c103270 con 0x7f981c102920 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.989+0000 7f981affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.989+0000 7f981affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39506/0 (socket says 172.21.15.94:39506) 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.989+0000 7f981affd700 1 -- 172.21.15.94:0/550358199 learned_addr learned my addr 172.21.15.94:0/550358199 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f981affd700 1 -- 172.21.15.94:0/550358199 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c1033b0 con 0x7f981c102920 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f981affd700 1 --2- 172.21.15.94:0/550358199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c102d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f980c0098e0 tx=0x7f980c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03bba3fe1960cff server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f981a7fc700 1 -- 172.21.15.94:0/550358199 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f980c00b860 con 0x7f981c102920 2024-04-06T03:31:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f981a7fc700 1 -- 172.21.15.94:0/550358199 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f980c00be50 con 0x7f981c102920 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f981a7fc700 1 -- 172.21.15.94:0/550358199 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f980c010380 con 0x7f981c102920 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f982139e700 1 -- 172.21.15.94:0/550358199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 msgr2=0x7f981c102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f982139e700 1 --2- 172.21.15.94:0/550358199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c102d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f980c0098e0 tx=0x7f980c009250 comp rx=0 tx=0).stop 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f982139e700 1 -- 172.21.15.94:0/550358199 shutdown_connections 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f982139e700 1 --2- 172.21.15.94:0/550358199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c102d30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f982139e700 1 -- 172.21.15.94:0/550358199 >> 172.21.15.94:0/550358199 conn(0x7f981c0fe0a0 msgr2=0x7f981c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f982139e700 1 -- 172.21.15.94:0/550358199 shutdown_connections 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.990+0000 7f982139e700 1 -- 172.21.15.94:0/550358199 wait complete. 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.991+0000 7f982139e700 1 Processor -- start 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.991+0000 7f982139e700 1 -- start start 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.991+0000 7f982139e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c1116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.991+0000 7f982139e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c111be0 con 0x7f981c102920 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.991+0000 7f981affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c1116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.991+0000 7f981affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c1116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39522/0 (socket says 172.21.15.94:39522) 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.991+0000 7f981affd700 1 -- 172.21.15.94:0/1016751341 learned_addr learned my addr 172.21.15.94:0/1016751341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.992+0000 7f981affd700 1 -- 172.21.15.94:0/1016751341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c0094f0 con 0x7f981c102920 2024-04-06T03:31:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.992+0000 7f981affd700 1 --2- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c1116a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b440 tx=0x7f980c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.992+0000 7f9813fff700 1 -- 172.21.15.94:0/1016751341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f980c010760 con 0x7f981c102920 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.992+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c111de0 con 0x7f981c102920 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.992+0000 7f9813fff700 1 -- 172.21.15.94:0/1016751341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f980c010d50 con 0x7f981c102920 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.992+0000 7f9813fff700 1 -- 172.21.15.94:0/1016751341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f980c01ae20 con 0x7f981c102920 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.992+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c112950 con 0x7f981c102920 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.993+0000 7f9813fff700 1 -- 172.21.15.94:0/1016751341 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f980c02e440 con 0x7f981c102920 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.993+0000 7f9813fff700 1 --2- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9808036770 0x7f9808038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.993+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c05f5f0 con 0x7f981c102920 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.993+0000 7f9813fff700 1 -- 172.21.15.94:0/1016751341 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f980c04a600 con 0x7f981c102920 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.993+0000 7f98127fc700 1 --2- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9808036770 0x7f9808038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.993+0000 7f98127fc700 1 --2- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9808036770 0x7f9808038c20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9804009990 tx=0x7f9804006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:43.998+0000 7f9813fff700 1 -- 172.21.15.94:0/1016751341 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f980c018940 con 0x7f981c102920 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.182+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f981c0ff310 con 0x7f9808036770 2024-04-06T03:31:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.482+0000 7f9813fff700 1 -- 172.21.15.94:0/1016751341 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f981c0ff310 con 0x7f9808036770 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9808036770 msgr2=0x7f9808038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 --2- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9808036770 0x7f9808038c20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9804009990 tx=0x7f9804006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 msgr2=0x7f981c1116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 --2- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c1116a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b440 tx=0x7f980c009250 comp rx=0 tx=0).stop 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 shutdown_connections 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 --2- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9808036770 0x7f9808038c20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 --2- 172.21.15.94:0/1016751341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f981c102920 0x7f981c1116a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 >> 172.21.15.94:0/1016751341 conn(0x7f981c0fe0a0 msgr2=0x7f981c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 shutdown_connections 2024-04-06T03:31:45.090 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:44.484+0000 7f982139e700 1 -- 172.21.15.94:0/1016751341 wait complete. 2024-04-06T03:31:45.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:45 smithi094 ceph-mon[29713]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:45.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:45 smithi094 ceph-mon[29713]: Generating ssh key... 2024-04-06T03:31:45.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:45 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:45.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:45 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD1dFF4UvsgVnmcfvTvz4ywKMZjIeyVmI5zqMjklKuzW8tUdQhlimnWVe0V6SM3yAPj7QJRl9HWGOloaCzxzTTZQa8/wfwYaonYqFT6Q7yFeWVPSB+NBVorWxUJde0AY4PHv9DaArrsJuonjPPQfPp9Y9rUyGdjUe8ERR7A27Cdx1r/2NMisBc1J8Ha93Vk6UjaOmX4Q1DW8niKOmI7Y5NvCKuR/MBxxJKNEQIzDi0LuU+b2+aY+kE158Be83Egns4jE/EsdLAYhnIbL6a302Kc49yjuu8fzas5O2Tw8mnS7z8pt2DIMUvV4pB1TWEOHcAbQ0d+bsBvv3ys8wZhiOW4LIBe8tFB65MeRKTcR/N/1hztsTXyOdC8g8VOmhi60P17WjJs//BGdbmAPAFQpYdB9s/gtP8vU/Rfmzb4Ds6rd2aYVg0iOFw3H+0J+SpBC2NlcYwaZkBxTpfpRuzojNd4avO9OjfmqbMs0FF3QF3ZI5mWDCGu1LoxesrOpcF1sF0= ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:31:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.495+0000 7f5d0fd2b700 1 Processor -- start 2024-04-06T03:31:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.495+0000 7f5d0fd2b700 1 -- start start 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.495+0000 7f5d0fd2b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.495+0000 7f5d0fd2b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08103270 con 0x7f5d08102920 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.495+0000 7f5d0dac7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.495+0000 7f5d0dac7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39530/0 (socket says 172.21.15.94:39530) 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.495+0000 7f5d0dac7700 1 -- 172.21.15.94:0/3326615382 learned_addr learned my addr 172.21.15.94:0/3326615382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.495+0000 7f5d0dac7700 1 -- 172.21.15.94:0/3326615382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d081033b0 con 0x7f5d08102920 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0dac7700 1 --2- 172.21.15.94:0/3326615382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08102d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d040098e0 tx=0x7f5d04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8b120ae06f988e server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0cac5700 1 -- 172.21.15.94:0/3326615382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d0400b860 con 0x7f5d08102920 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0cac5700 1 -- 172.21.15.94:0/3326615382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5d0400be50 con 0x7f5d08102920 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0cac5700 1 -- 172.21.15.94:0/3326615382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d04010380 con 0x7f5d08102920 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/3326615382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 msgr2=0x7f5d08102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0fd2b700 1 --2- 172.21.15.94:0/3326615382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08102d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d040098e0 tx=0x7f5d04009250 comp rx=0 tx=0).stop 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/3326615382 shutdown_connections 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0fd2b700 1 --2- 172.21.15.94:0/3326615382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08102d30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:46.237 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/3326615382 >> 172.21.15.94:0/3326615382 conn(0x7f5d080fe0a0 msgr2=0x7f5d081004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/3326615382 shutdown_connections 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.496+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/3326615382 wait complete. 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.497+0000 7f5d0fd2b700 1 Processor -- start 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.497+0000 7f5d0fd2b700 1 -- start start 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.497+0000 7f5d0fd2b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08078c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.497+0000 7f5d0fd2b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08079180 con 0x7f5d08102920 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.497+0000 7f5d0dac7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.497+0000 7f5d0dac7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08078c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39532/0 (socket says 172.21.15.94:39532) 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.497+0000 7f5d0dac7700 1 -- 172.21.15.94:0/2198707359 learned_addr learned my addr 172.21.15.94:0/2198707359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.497+0000 7f5d0dac7700 1 -- 172.21.15.94:0/2198707359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d040094f0 con 0x7f5d08102920 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.497+0000 7f5d0dac7700 1 --2- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08078c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400b440 tx=0x7f5d04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.498+0000 7f5cfe7fc700 1 -- 172.21.15.94:0/2198707359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d04010760 con 0x7f5d08102920 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.498+0000 7f5cfe7fc700 1 -- 172.21.15.94:0/2198707359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5d04010d50 con 0x7f5d08102920 2024-04-06T03:31:46.238 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.498+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d080771f0 con 0x7f5d08102920 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.498+0000 7f5cfe7fc700 1 -- 172.21.15.94:0/2198707359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d0401ae20 con 0x7f5d08102920 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.498+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d080776e0 con 0x7f5d08102920 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.498+0000 7f5cfe7fc700 1 -- 172.21.15.94:0/2198707359 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f5d040108c0 con 0x7f5d08102920 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.499+0000 7f5cfe7fc700 1 --2- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f5cf40367e0 0x7f5cf4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.499+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0804bed0 con 0x7f5d08102920 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.499+0000 7f5cfe7fc700 1 -- 172.21.15.94:0/2198707359 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5d0404a5b0 con 0x7f5d08102920 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.499+0000 7f5d0d2c6700 1 --2- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f5cf40367e0 0x7f5cf4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.499+0000 7f5d0d2c6700 1 --2- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f5cf40367e0 0x7f5cf4038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009990 tx=0x7f5cf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.503+0000 7f5cfe7fc700 1 -- 172.21.15.94:0/2198707359 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5d04079190 con 0x7f5d08102920 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.690+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d08077df0 con 0x7f5cf40367e0 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.691+0000 7f5cfe7fc700 1 -- 172.21.15.94:0/2198707359 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f5d08077df0 con 0x7f5cf40367e0 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f5cf40367e0 msgr2=0x7f5cf4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 --2- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f5cf40367e0 0x7f5cf4038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009990 tx=0x7f5cf8006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 msgr2=0x7f5d08078c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 --2- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08078c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400b440 tx=0x7f5d04009250 comp rx=0 tx=0).stop 2024-04-06T03:31:46.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 shutdown_connections 2024-04-06T03:31:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 --2- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f5cf40367e0 0x7f5cf4038c90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 --2- 172.21.15.94:0/2198707359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d08102920 0x7f5d08078c40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 >> 172.21.15.94:0/2198707359 conn(0x7f5d080fe0a0 msgr2=0x7f5d080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 shutdown_connections 2024-04-06T03:31:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:45.693+0000 7f5d0fd2b700 1 -- 172.21.15.94:0/2198707359 wait complete. 2024-04-06T03:31:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-06T03:31:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:Adding key to root@localhost authorized_keys... 2024-04-06T03:31:46.240 INFO:teuthology.orchestra.run.smithi094.stdout:Adding host smithi094... 2024-04-06T03:31:46.907 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:46 smithi094 ceph-mon[29713]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:47.740 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:47 smithi094 ceph-mon[29713]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:48.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:48 smithi094 ceph-mon[29713]: Deploying cephadm binary to smithi094 2024-04-06T03:31:49.510 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Added host 'smithi094' with addr '172.21.15.94' 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.708+0000 7f8a4c1da700 1 Processor -- start 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.708+0000 7f8a4c1da700 1 -- start start 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.708+0000 7f8a4c1da700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a44102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.708+0000 7f8a4c1da700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44103270 con 0x7f8a44102920 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.709+0000 7f8a49f76700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a44102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.709+0000 7f8a49f76700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a44102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39548/0 (socket says 172.21.15.94:39548) 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.709+0000 7f8a49f76700 1 -- 172.21.15.94:0/4169065228 learned_addr learned my addr 172.21.15.94:0/4169065228 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.709+0000 7f8a49f76700 1 -- 172.21.15.94:0/4169065228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a441033b0 con 0x7f8a44102920 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.709+0000 7f8a49f76700 1 --2- 172.21.15.94:0/4169065228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a44102d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8a400098e0 tx=0x7f8a40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b7ce9813dad120 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.709+0000 7f8a48f74700 1 -- 172.21.15.94:0/4169065228 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a4000b860 con 0x7f8a44102920 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.709+0000 7f8a48f74700 1 -- 172.21.15.94:0/4169065228 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8a4000be50 con 0x7f8a44102920 2024-04-06T03:31:49.511 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.709+0000 7f8a48f74700 1 -- 172.21.15.94:0/4169065228 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a40010380 con 0x7f8a44102920 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.710+0000 7f8a4c1da700 1 -- 172.21.15.94:0/4169065228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 msgr2=0x7f8a44102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.710+0000 7f8a4c1da700 1 --2- 172.21.15.94:0/4169065228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a44102d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8a400098e0 tx=0x7f8a40009250 comp rx=0 tx=0).stop 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.710+0000 7f8a4c1da700 1 -- 172.21.15.94:0/4169065228 shutdown_connections 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.710+0000 7f8a4c1da700 1 --2- 172.21.15.94:0/4169065228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a44102d30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.710+0000 7f8a4c1da700 1 -- 172.21.15.94:0/4169065228 >> 172.21.15.94:0/4169065228 conn(0x7f8a440fe0a0 msgr2=0x7f8a441004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.710+0000 7f8a4c1da700 1 -- 172.21.15.94:0/4169065228 shutdown_connections 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.710+0000 7f8a4c1da700 1 -- 172.21.15.94:0/4169065228 wait complete. 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a4c1da700 1 Processor -- start 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a4c1da700 1 -- start start 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a4c1da700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a441116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a4c1da700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a44111c00 con 0x7f8a44102920 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a49f76700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a441116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a49f76700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a441116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39552/0 (socket says 172.21.15.94:39552) 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a49f76700 1 -- 172.21.15.94:0/3692573551 learned_addr learned my addr 172.21.15.94:0/3692573551 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:49.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a49f76700 1 -- 172.21.15.94:0/3692573551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a400094f0 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a49f76700 1 --2- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a441116c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000b440 tx=0x7f8a40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a3a7fc700 1 -- 172.21.15.94:0/3692573551 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a40010760 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a3a7fc700 1 -- 172.21.15.94:0/3692573551 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8a40010d50 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a44111e00 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.711+0000 7f8a3a7fc700 1 -- 172.21.15.94:0/3692573551 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8a4001ae20 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.712+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a44112970 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.712+0000 7f8a3a7fc700 1 -- 172.21.15.94:0/3692573551 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f8a4002e440 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.712+0000 7f8a3a7fc700 1 --2- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f8a30036770 0x7f8a30038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.712+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4405f5f0 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.712+0000 7f8a3a7fc700 1 -- 172.21.15.94:0/3692573551 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8a4004a8c0 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.713+0000 7f8a49775700 1 --2- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f8a30036770 0x7f8a30038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.713+0000 7f8a49775700 1 --2- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f8a30036770 0x7f8a30038c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009990 tx=0x7f8a34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.718+0000 7f8a3a7fc700 1 -- 172.21.15.94:0/3692573551 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8a40018940 con 0x7f8a44102920 2024-04-06T03:31:49.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:46.905+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a440ff350 con 0x7f8a30036770 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.937+0000 7f8a3a7fc700 1 -- 172.21.15.94:0/3692573551 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f8a440ff350 con 0x7f8a30036770 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f8a30036770 msgr2=0x7f8a30038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 --2- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f8a30036770 0x7f8a30038c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009990 tx=0x7f8a34006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 msgr2=0x7f8a441116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 --2- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a441116c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000b440 tx=0x7f8a40009250 comp rx=0 tx=0).stop 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 shutdown_connections 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 --2- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f8a30036770 0x7f8a30038c20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 --2- 172.21.15.94:0/3692573551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a44102920 0x7f8a441116c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 >> 172.21.15.94:0/3692573551 conn(0x7f8a440fe0a0 msgr2=0x7f8a440fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 shutdown_connections 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:48.940+0000 7f8a4c1da700 1 -- 172.21.15.94:0/3692573551 wait complete. 2024-04-06T03:31:49.514 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying mon service with default placement... 2024-04-06T03:31:50.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:49 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:50.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:49 smithi094 ceph-mon[29713]: Added host smithi094 2024-04-06T03:31:50.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:49 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:31:50.753 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-06T03:31:50.753 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.927+0000 7f1d6133c700 1 Processor -- start 2024-04-06T03:31:50.753 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.927+0000 7f1d6133c700 1 -- start start 2024-04-06T03:31:50.753 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.927+0000 7f1d6133c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c101c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:50.753 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.927+0000 7f1d6133c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c102180 con 0x7f1d5c101830 2024-04-06T03:31:50.753 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.928+0000 7f1d5affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:50.753 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.928+0000 7f1d5affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57094/0 (socket says 172.21.15.94:57094) 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.928+0000 7f1d5affd700 1 -- 172.21.15.94:0/826609432 learned_addr learned my addr 172.21.15.94:0/826609432 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.928+0000 7f1d5affd700 1 -- 172.21.15.94:0/826609432 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c1022c0 con 0x7f1d5c101830 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.928+0000 7f1d5affd700 1 --2- 172.21.15.94:0/826609432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c101c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009e50 tx=0x7f1d4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fcc542a2045c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.928+0000 7f1d59ffb700 1 -- 172.21.15.94:0/826609432 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d4c0092e0 con 0x7f1d5c101830 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.928+0000 7f1d59ffb700 1 -- 172.21.15.94:0/826609432 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1d4c012430 con 0x7f1d5c101830 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.928+0000 7f1d59ffb700 1 -- 172.21.15.94:0/826609432 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d4c0075c0 con 0x7f1d5c101830 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.929+0000 7f1d6133c700 1 -- 172.21.15.94:0/826609432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 msgr2=0x7f1d5c101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.929+0000 7f1d6133c700 1 --2- 172.21.15.94:0/826609432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c101c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009e50 tx=0x7f1d4c009840 comp rx=0 tx=0).stop 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.929+0000 7f1d6133c700 1 -- 172.21.15.94:0/826609432 shutdown_connections 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.929+0000 7f1d6133c700 1 --2- 172.21.15.94:0/826609432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c101c40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.929+0000 7f1d6133c700 1 -- 172.21.15.94:0/826609432 >> 172.21.15.94:0/826609432 conn(0x7f1d5c0fcfb0 msgr2=0x7f1d5c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.929+0000 7f1d6133c700 1 -- 172.21.15.94:0/826609432 shutdown_connections 2024-04-06T03:31:50.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.929+0000 7f1d6133c700 1 -- 172.21.15.94:0/826609432 wait complete. 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.930+0000 7f1d6133c700 1 Processor -- start 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.930+0000 7f1d6133c700 1 -- start start 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.930+0000 7f1d6133c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.930+0000 7f1d6133c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c111ca0 con 0x7f1d5c101830 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.930+0000 7f1d5affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.930+0000 7f1d5affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c111760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57108/0 (socket says 172.21.15.94:57108) 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.930+0000 7f1d5affd700 1 -- 172.21.15.94:0/2948375278 learned_addr learned my addr 172.21.15.94:0/2948375278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.930+0000 7f1d5affd700 1 -- 172.21.15.94:0/2948375278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c009ae0 con 0x7f1d5c101830 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.930+0000 7f1d5affd700 1 --2- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c111760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c012da0 tx=0x7f1d4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.931+0000 7f1d537fe700 1 -- 172.21.15.94:0/2948375278 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d4c0092e0 con 0x7f1d5c101830 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.931+0000 7f1d537fe700 1 -- 172.21.15.94:0/2948375278 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1d4c007cc0 con 0x7f1d5c101830 2024-04-06T03:31:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.931+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c111ea0 con 0x7f1d5c101830 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.931+0000 7f1d537fe700 1 -- 172.21.15.94:0/2948375278 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d4c01c910 con 0x7f1d5c101830 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.931+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c112a10 con 0x7f1d5c101830 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.931+0000 7f1d537fe700 1 -- 172.21.15.94:0/2948375278 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f1d4c019070 con 0x7f1d5c101830 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.932+0000 7f1d537fe700 1 --2- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1d480367e0 0x7f1d48038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.932+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c05f5f0 con 0x7f1d5c101830 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.932+0000 7f1d5a7fc700 1 --2- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1d480367e0 0x7f1d48038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.932+0000 7f1d537fe700 1 -- 172.21.15.94:0/2948375278 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1d4c049dc0 con 0x7f1d5c101830 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.932+0000 7f1d5a7fc700 1 --2- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1d480367e0 0x7f1d48038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009990 tx=0x7f1d44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:49.936+0000 7f1d537fe700 1 -- 172.21.15.94:0/2948375278 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1d4c0196e0 con 0x7f1d5c101830 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.136+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d5c1135d0 con 0x7f1d480367e0 2024-04-06T03:31:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.140+0000 7f1d537fe700 1 -- 172.21.15.94:0/2948375278 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1d5c1135d0 con 0x7f1d480367e0 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.142+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1d480367e0 msgr2=0x7f1d48038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.142+0000 7f1d6133c700 1 --2- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1d480367e0 0x7f1d48038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009990 tx=0x7f1d44006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.142+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 msgr2=0x7f1d5c111760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.142+0000 7f1d6133c700 1 --2- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c111760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c012da0 tx=0x7f1d4c009840 comp rx=0 tx=0).stop 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.143+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 shutdown_connections 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.143+0000 7f1d6133c700 1 --2- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1d480367e0 0x7f1d48038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.143+0000 7f1d6133c700 1 --2- 172.21.15.94:0/2948375278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c101830 0x7f1d5c111760 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.143+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 >> 172.21.15.94:0/2948375278 conn(0x7f1d5c0fcfb0 msgr2=0x7f1d5c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.143+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 shutdown_connections 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:50.143+0000 7f1d6133c700 1 -- 172.21.15.94:0/2948375278 wait complete. 2024-04-06T03:31:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying mgr service with default placement... 2024-04-06T03:31:51.345 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:51 smithi094 ceph-mon[29713]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:51.345 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:51 smithi094 ceph-mon[29713]: Saving service mon spec with placement count:5 2024-04-06T03:31:51.345 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:51 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:52.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-06T03:31:52.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.163+0000 7fea09e16700 1 Processor -- start 2024-04-06T03:31:52.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.163+0000 7fea09e16700 1 -- start start 2024-04-06T03:31:52.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.163+0000 7fea09e16700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea04102d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.163+0000 7fea09e16700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04103270 con 0x7fea04102920 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.163+0000 7fea037fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea04102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.163+0000 7fea037fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea04102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57114/0 (socket says 172.21.15.94:57114) 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.163+0000 7fea037fe700 1 -- 172.21.15.94:0/2163663566 learned_addr learned my addr 172.21.15.94:0/2163663566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.163+0000 7fea037fe700 1 -- 172.21.15.94:0/2163663566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea041033b0 con 0x7fea04102920 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.163+0000 7fea037fe700 1 --2- 172.21.15.94:0/2163663566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea04102d30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80098e0 tx=0x7fe9f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcd06fdbbd978b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea027fc700 1 -- 172.21.15.94:0/2163663566 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9f800b860 con 0x7fea04102920 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea027fc700 1 -- 172.21.15.94:0/2163663566 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe9f800be50 con 0x7fea04102920 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea027fc700 1 -- 172.21.15.94:0/2163663566 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9f8010380 con 0x7fea04102920 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea09e16700 1 -- 172.21.15.94:0/2163663566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 msgr2=0x7fea04102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea09e16700 1 --2- 172.21.15.94:0/2163663566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea04102d30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80098e0 tx=0x7fe9f8009250 comp rx=0 tx=0).stop 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea09e16700 1 -- 172.21.15.94:0/2163663566 shutdown_connections 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea09e16700 1 --2- 172.21.15.94:0/2163663566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea04102d30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:52.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea09e16700 1 -- 172.21.15.94:0/2163663566 >> 172.21.15.94:0/2163663566 conn(0x7fea040fe0a0 msgr2=0x7fea041004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea09e16700 1 -- 172.21.15.94:0/2163663566 shutdown_connections 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.164+0000 7fea09e16700 1 -- 172.21.15.94:0/2163663566 wait complete. 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.165+0000 7fea09e16700 1 Processor -- start 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.165+0000 7fea09e16700 1 -- start start 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.165+0000 7fea09e16700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea041116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.165+0000 7fea09e16700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea04111be0 con 0x7fea04102920 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.165+0000 7fea037fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea041116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.165+0000 7fea037fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea041116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57124/0 (socket says 172.21.15.94:57124) 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.165+0000 7fea037fe700 1 -- 172.21.15.94:0/3382296859 learned_addr learned my addr 172.21.15.94:0/3382296859 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.165+0000 7fea037fe700 1 -- 172.21.15.94:0/3382296859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80094f0 con 0x7fea04102920 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.166+0000 7fea037fe700 1 --2- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea041116a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800b440 tx=0x7fe9f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.166+0000 7fe9ebfff700 1 -- 172.21.15.94:0/3382296859 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9f8010760 con 0x7fea04102920 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.166+0000 7fe9ebfff700 1 -- 172.21.15.94:0/3382296859 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe9f8010d50 con 0x7fea04102920 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.166+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea04111de0 con 0x7fea04102920 2024-04-06T03:31:52.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.166+0000 7fe9ebfff700 1 -- 172.21.15.94:0/3382296859 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9f801ae20 con 0x7fea04102920 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.166+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea04112950 con 0x7fea04102920 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.167+0000 7fe9ebfff700 1 -- 172.21.15.94:0/3382296859 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fe9f802e440 con 0x7fea04102920 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.167+0000 7fe9ebfff700 1 --2- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe9ec0367e0 0x7fe9ec038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.167+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0405f5f0 con 0x7fea04102920 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.167+0000 7fe9ebfff700 1 -- 172.21.15.94:0/3382296859 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe9f804a6f0 con 0x7fea04102920 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.167+0000 7fea02ffd700 1 --2- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe9ec0367e0 0x7fe9ec038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.167+0000 7fea02ffd700 1 --2- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe9ec0367e0 0x7fe9ec038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009990 tx=0x7fe9f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.171+0000 7fe9ebfff700 1 -- 172.21.15.94:0/3382296859 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe9f8018350 con 0x7fea04102920 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.372+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fea040ff310 con 0x7fe9ec0367e0 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.376+0000 7fe9ebfff700 1 -- 172.21.15.94:0/3382296859 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fea040ff310 con 0x7fe9ec0367e0 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe9ec0367e0 msgr2=0x7fe9ec038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 --2- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe9ec0367e0 0x7fe9ec038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009990 tx=0x7fe9f4006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 msgr2=0x7fea041116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:52.030 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 --2- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea041116a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800b440 tx=0x7fe9f8009250 comp rx=0 tx=0).stop 2024-04-06T03:31:52.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 shutdown_connections 2024-04-06T03:31:52.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 --2- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe9ec0367e0 0x7fe9ec038c90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:52.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 --2- 172.21.15.94:0/3382296859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea04102920 0x7fea041116a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:52.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 >> 172.21.15.94:0/3382296859 conn(0x7fea040fe0a0 msgr2=0x7fea040fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:52.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 shutdown_connections 2024-04-06T03:31:52.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:51.378+0000 7fea09e16700 1 -- 172.21.15.94:0/3382296859 wait complete. 2024-04-06T03:31:52.031 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying crash service with default placement... 2024-04-06T03:31:52.648 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:52 smithi094 ceph-mon[29713]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:52.648 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:52 smithi094 ceph-mon[29713]: Saving service mgr spec with placement count:2 2024-04-06T03:31:52.648 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:52 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:53.847 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:53 smithi094 ceph-mon[29713]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:53.847 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:53 smithi094 ceph-mon[29713]: Saving service crash spec with placement * 2024-04-06T03:31:53.847 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:53 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:53.847 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:53 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:53.854 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-06T03:31:53.854 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.450+0000 7f39d421d700 1 Processor -- start 2024-04-06T03:31:53.854 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.450+0000 7f39d421d700 1 -- start start 2024-04-06T03:31:53.854 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.450+0000 7f39d421d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc101010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:53.854 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.450+0000 7f39d421d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc101550 con 0x7f39cc100c00 2024-04-06T03:31:53.854 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.451+0000 7f39d1fb9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.451+0000 7f39d1fb9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc101010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57138/0 (socket says 172.21.15.94:57138) 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.451+0000 7f39d1fb9700 1 -- 172.21.15.94:0/4047220953 learned_addr learned my addr 172.21.15.94:0/4047220953 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.451+0000 7f39d1fb9700 1 -- 172.21.15.94:0/4047220953 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc101690 con 0x7f39cc100c00 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.451+0000 7f39d1fb9700 1 --2- 172.21.15.94:0/4047220953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc101010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009e50 tx=0x7f39c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fb9aab04e3f57f server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.451+0000 7f39d0fb7700 1 -- 172.21.15.94:0/4047220953 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39c40092e0 con 0x7f39cc100c00 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.451+0000 7f39d0fb7700 1 -- 172.21.15.94:0/4047220953 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f39c4012430 con 0x7f39cc100c00 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.451+0000 7f39d0fb7700 1 -- 172.21.15.94:0/4047220953 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39c40075c0 con 0x7f39cc100c00 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.452+0000 7f39d421d700 1 -- 172.21.15.94:0/4047220953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 msgr2=0x7f39cc101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.452+0000 7f39d421d700 1 --2- 172.21.15.94:0/4047220953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc101010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009e50 tx=0x7f39c4009840 comp rx=0 tx=0).stop 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.452+0000 7f39d421d700 1 -- 172.21.15.94:0/4047220953 shutdown_connections 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.452+0000 7f39d421d700 1 --2- 172.21.15.94:0/4047220953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc101010 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.452+0000 7f39d421d700 1 -- 172.21.15.94:0/4047220953 >> 172.21.15.94:0/4047220953 conn(0x7f39cc0fc6e0 msgr2=0x7f39cc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.452+0000 7f39d421d700 1 -- 172.21.15.94:0/4047220953 shutdown_connections 2024-04-06T03:31:53.855 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.452+0000 7f39d421d700 1 -- 172.21.15.94:0/4047220953 wait complete. 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.452+0000 7f39d421d700 1 Processor -- start 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.452+0000 7f39d421d700 1 -- start start 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39d421d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc1116b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39d421d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc111bf0 con 0x7f39cc100c00 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39d1fb9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc1116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39d1fb9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc1116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57154/0 (socket says 172.21.15.94:57154) 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39d1fb9700 1 -- 172.21.15.94:0/2354435095 learned_addr learned my addr 172.21.15.94:0/2354435095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39d1fb9700 1 -- 172.21.15.94:0/2354435095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4009ae0 con 0x7f39cc100c00 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39d1fb9700 1 --2- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc1116b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009ab0 tx=0x7f39c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39ca7fc700 1 -- 172.21.15.94:0/2354435095 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39c40092e0 con 0x7f39cc100c00 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39ca7fc700 1 -- 172.21.15.94:0/2354435095 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f39c4007d10 con 0x7f39cc100c00 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39cc111df0 con 0x7f39cc100c00 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39ca7fc700 1 -- 172.21.15.94:0/2354435095 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39c401c990 con 0x7f39cc100c00 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.453+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39cc112960 con 0x7f39cc100c00 2024-04-06T03:31:53.856 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.454+0000 7f39ca7fc700 1 -- 172.21.15.94:0/2354435095 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f39c4019070 con 0x7f39cc100c00 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.454+0000 7f39ca7fc700 1 --2- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f39b4036790 0x7f39b4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.454+0000 7f39ca7fc700 1 -- 172.21.15.94:0/2354435095 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f39c4049ea0 con 0x7f39cc100c00 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.454+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39cc05f5f0 con 0x7f39cc100c00 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.454+0000 7f39d17b8700 1 --2- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f39b4036790 0x7f39b4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.455+0000 7f39d17b8700 1 --2- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f39b4036790 0x7f39b4038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009990 tx=0x7f39c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.459+0000 7f39ca7fc700 1 -- 172.21.15.94:0/2354435095 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f39c4017080 con 0x7f39cc100c00 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.658+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f39cc0fdac0 con 0x7f39b4036790 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.663+0000 7f39ca7fc700 1 -- 172.21.15.94:0/2354435095 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f39cc0fdac0 con 0x7f39b4036790 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f39b4036790 msgr2=0x7f39b4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 --2- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f39b4036790 0x7f39b4038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009990 tx=0x7f39c0006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 msgr2=0x7f39cc1116b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 --2- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc1116b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009ab0 tx=0x7f39c4009840 comp rx=0 tx=0).stop 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 shutdown_connections 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 --2- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f39b4036790 0x7f39b4038c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:53.857 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 --2- 172.21.15.94:0/2354435095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc100c00 0x7f39cc1116b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:53.858 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 >> 172.21.15.94:0/2354435095 conn(0x7f39cc0fc6e0 msgr2=0x7f39cc0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:53.858 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 shutdown_connections 2024-04-06T03:31:53.858 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:52.665+0000 7f39d421d700 1 -- 172.21.15.94:0/2354435095 wait complete. 2024-04-06T03:31:53.858 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying prometheus service with default placement... 2024-04-06T03:31:55.053 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:54 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:55.054 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:54 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:55.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-06T03:31:55.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.273+0000 7f89d86e5700 1 Processor -- start 2024-04-06T03:31:55.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.273+0000 7f89d86e5700 1 -- start start 2024-04-06T03:31:55.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.273+0000 7f89d86e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:55.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.273+0000 7f89d86e5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d0105960 con 0x7f89d0076c90 2024-04-06T03:31:55.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.273+0000 7f89d6481700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:55.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.273+0000 7f89d6481700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57168/0 (socket says 172.21.15.94:57168) 2024-04-06T03:31:55.614 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.273+0000 7f89d6481700 1 -- 172.21.15.94:0/922773816 learned_addr learned my addr 172.21.15.94:0/922773816 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.274+0000 7f89d6481700 1 -- 172.21.15.94:0/922773816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d0105aa0 con 0x7f89d0076c90 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.274+0000 7f89d6481700 1 --2- 172.21.15.94:0/922773816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0105420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0098e0 tx=0x7f89cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edada6bb1087a8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.274+0000 7f89d547f700 1 -- 172.21.15.94:0/922773816 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89cc00b860 con 0x7f89d0076c90 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.274+0000 7f89d547f700 1 -- 172.21.15.94:0/922773816 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89cc00be90 con 0x7f89d0076c90 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.274+0000 7f89d547f700 1 -- 172.21.15.94:0/922773816 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89cc0103c0 con 0x7f89d0076c90 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.274+0000 7f89d86e5700 1 -- 172.21.15.94:0/922773816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 msgr2=0x7f89d0105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.274+0000 7f89d86e5700 1 --2- 172.21.15.94:0/922773816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0105420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0098e0 tx=0x7f89cc009250 comp rx=0 tx=0).stop 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d86e5700 1 -- 172.21.15.94:0/922773816 shutdown_connections 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d86e5700 1 --2- 172.21.15.94:0/922773816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0105420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d86e5700 1 -- 172.21.15.94:0/922773816 >> 172.21.15.94:0/922773816 conn(0x7f89d00fcf30 msgr2=0x7f89d00ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d86e5700 1 -- 172.21.15.94:0/922773816 shutdown_connections 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d86e5700 1 -- 172.21.15.94:0/922773816 wait complete. 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d86e5700 1 Processor -- start 2024-04-06T03:31:55.615 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d86e5700 1 -- start start 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d86e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0111710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d86e5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d0111c50 con 0x7f89d0076c90 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d6481700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0111710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d6481700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0111710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57176/0 (socket says 172.21.15.94:57176) 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.275+0000 7f89d6481700 1 -- 172.21.15.94:0/91846101 learned_addr learned my addr 172.21.15.94:0/91846101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.276+0000 7f89d6481700 1 -- 172.21.15.94:0/91846101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0094f0 con 0x7f89d0076c90 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.276+0000 7f89d6481700 1 --2- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0111710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0105b0 tx=0x7f89cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.276+0000 7f89c6ffd700 1 -- 172.21.15.94:0/91846101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89cc010840 con 0x7f89d0076c90 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.276+0000 7f89c6ffd700 1 -- 172.21.15.94:0/91846101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89cc010e70 con 0x7f89d0076c90 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.276+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d0111e50 con 0x7f89d0076c90 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.276+0000 7f89c6ffd700 1 -- 172.21.15.94:0/91846101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89cc01ae30 con 0x7f89d0076c90 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.276+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d01129c0 con 0x7f89d0076c90 2024-04-06T03:31:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.277+0000 7f89c6ffd700 1 -- 172.21.15.94:0/91846101 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f89cc02d3f0 con 0x7f89d0076c90 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.277+0000 7f89c6ffd700 1 --2- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f89bc0367e0 0x7f89bc038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.277+0000 7f89c6ffd700 1 -- 172.21.15.94:0/91846101 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f89cc04a1a0 con 0x7f89d0076c90 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.277+0000 7f89d5c80700 1 --2- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f89bc0367e0 0x7f89bc038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.277+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d005f5f0 con 0x7f89d0076c90 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.277+0000 7f89d5c80700 1 --2- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f89bc0367e0 0x7f89bc038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009990 tx=0x7f89c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.282+0000 7f89c6ffd700 1 -- 172.21.15.94:0/91846101 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f89cc02b8e0 con 0x7f89d0076c90 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.506+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f89d00fe270 con 0x7f89bc0367e0 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.510+0000 7f89c6ffd700 1 -- 172.21.15.94:0/91846101 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f89d00fe270 con 0x7f89bc0367e0 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.512+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f89bc0367e0 msgr2=0x7f89bc038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:55.617 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.512+0000 7f89d86e5700 1 --2- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f89bc0367e0 0x7f89bc038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009990 tx=0x7f89c0006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:55.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.512+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 msgr2=0x7f89d0111710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:55.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.512+0000 7f89d86e5700 1 --2- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0111710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0105b0 tx=0x7f89cc009250 comp rx=0 tx=0).stop 2024-04-06T03:31:55.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.512+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 shutdown_connections 2024-04-06T03:31:55.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.512+0000 7f89d86e5700 1 --2- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f89bc0367e0 0x7f89bc038c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:55.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.512+0000 7f89d86e5700 1 --2- 172.21.15.94:0/91846101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89d0076c90 0x7f89d0111710 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:55.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.512+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 >> 172.21.15.94:0/91846101 conn(0x7f89d00fcf30 msgr2=0x7f89d00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:55.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.513+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 shutdown_connections 2024-04-06T03:31:55.618 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:54.513+0000 7f89d86e5700 1 -- 172.21.15.94:0/91846101 wait complete. 2024-04-06T03:31:55.619 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying grafana service with default placement... 2024-04-06T03:31:56.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:55 smithi094 ceph-mon[29713]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:55 smithi094 ceph-mon[29713]: Saving service prometheus spec with placement count:1 2024-04-06T03:31:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:55 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:56.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-06T03:31:56.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.092+0000 7fbb2e384700 1 Processor -- start 2024-04-06T03:31:56.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.092+0000 7fbb2e384700 1 -- start start 2024-04-06T03:31:56.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.092+0000 7fbb2e384700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb28103200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:56.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.092+0000 7fbb2e384700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28103740 con 0x7fbb28102df0 2024-04-06T03:31:56.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.092+0000 7fbb27fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb28103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:56.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.092+0000 7fbb27fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb28103200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57182/0 (socket says 172.21.15.94:57182) 2024-04-06T03:31:56.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.092+0000 7fbb27fff700 1 -- 172.21.15.94:0/1087605403 learned_addr learned my addr 172.21.15.94:0/1087605403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:56.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.093+0000 7fbb27fff700 1 -- 172.21.15.94:0/1087605403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb28101480 con 0x7fbb28102df0 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.093+0000 7fbb27fff700 1 --2- 172.21.15.94:0/1087605403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb28103200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb180098e0 tx=0x7fbb18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f71f04be4e6bbe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.093+0000 7fbb26ffd700 1 -- 172.21.15.94:0/1087605403 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb1800b860 con 0x7fbb28102df0 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.093+0000 7fbb26ffd700 1 -- 172.21.15.94:0/1087605403 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb1800be90 con 0x7fbb28102df0 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.093+0000 7fbb26ffd700 1 -- 172.21.15.94:0/1087605403 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb180103c0 con 0x7fbb28102df0 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.094+0000 7fbb2e384700 1 -- 172.21.15.94:0/1087605403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 msgr2=0x7fbb28103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.094+0000 7fbb2e384700 1 --2- 172.21.15.94:0/1087605403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb28103200 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb180098e0 tx=0x7fbb18009250 comp rx=0 tx=0).stop 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.094+0000 7fbb2e384700 1 -- 172.21.15.94:0/1087605403 shutdown_connections 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.094+0000 7fbb2e384700 1 --2- 172.21.15.94:0/1087605403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb28103200 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.094+0000 7fbb2e384700 1 -- 172.21.15.94:0/1087605403 >> 172.21.15.94:0/1087605403 conn(0x7fbb280fcf30 msgr2=0x7fbb280ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.094+0000 7fbb2e384700 1 -- 172.21.15.94:0/1087605403 shutdown_connections 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.094+0000 7fbb2e384700 1 -- 172.21.15.94:0/1087605403 wait complete. 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.094+0000 7fbb2e384700 1 Processor -- start 2024-04-06T03:31:56.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.095+0000 7fbb2e384700 1 -- start start 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.095+0000 7fbb2e384700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb281138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.095+0000 7fbb2e384700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28113e10 con 0x7fbb28102df0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.095+0000 7fbb27fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb281138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.095+0000 7fbb27fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb281138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57188/0 (socket says 172.21.15.94:57188) 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.095+0000 7fbb27fff700 1 -- 172.21.15.94:0/904610369 learned_addr learned my addr 172.21.15.94:0/904610369 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.095+0000 7fbb27fff700 1 -- 172.21.15.94:0/904610369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180094f0 con 0x7fbb28102df0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.095+0000 7fbb27fff700 1 --2- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb281138d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb18011650 tx=0x7fbb18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.095+0000 7fbb24ff9700 1 -- 172.21.15.94:0/904610369 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb18010890 con 0x7fbb28102df0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.096+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb28114010 con 0x7fbb28102df0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.096+0000 7fbb24ff9700 1 -- 172.21.15.94:0/904610369 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb18010ec0 con 0x7fbb28102df0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.096+0000 7fbb24ff9700 1 -- 172.21.15.94:0/904610369 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb180193c0 con 0x7fbb28102df0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.096+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb28114b80 con 0x7fbb28102df0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.096+0000 7fbb24ff9700 1 -- 172.21.15.94:0/904610369 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbb18019600 con 0x7fbb28102df0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.097+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb281144a0 con 0x7fbb28102df0 2024-04-06T03:31:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.097+0000 7fbb24ff9700 1 --2- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fbb10036770 0x7fbb10038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.097+0000 7fbb24ff9700 1 -- 172.21.15.94:0/904610369 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbb1804a5c0 con 0x7fbb28102df0 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.097+0000 7fbb277fe700 1 --2- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fbb10036770 0x7fbb10038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.097+0000 7fbb277fe700 1 --2- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fbb10036770 0x7fbb10038c20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009990 tx=0x7fbb1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.102+0000 7fbb24ff9700 1 -- 172.21.15.94:0/904610369 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbb1802bb30 con 0x7fbb28102df0 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.315+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb28114f30 con 0x7fbb10036770 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.320+0000 7fbb24ff9700 1 -- 172.21.15.94:0/904610369 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fbb28114f30 con 0x7fbb10036770 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fbb10036770 msgr2=0x7fbb10038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 --2- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fbb10036770 0x7fbb10038c20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009990 tx=0x7fbb1c006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 msgr2=0x7fbb281138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 --2- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb281138d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb18011650 tx=0x7fbb18009250 comp rx=0 tx=0).stop 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 shutdown_connections 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 --2- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fbb10036770 0x7fbb10038c20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 --2- 172.21.15.94:0/904610369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb28102df0 0x7fbb281138d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 >> 172.21.15.94:0/904610369 conn(0x7fbb280fcf30 msgr2=0x7fbb28105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:56.915 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 shutdown_connections 2024-04-06T03:31:56.916 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:56.322+0000 7fbb2e384700 1 -- 172.21.15.94:0/904610369 wait complete. 2024-04-06T03:31:56.916 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying node-exporter service with default placement... 2024-04-06T03:31:57.549 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:57 smithi094 ceph-mon[29713]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:57.549 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:57 smithi094 ceph-mon[29713]: Saving service grafana spec with placement count:1 2024-04-06T03:31:57.549 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:57 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:57.549 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:57 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:58.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:58 smithi094 ceph-mon[29713]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:31:58.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:58 smithi094 ceph-mon[29713]: Saving service node-exporter spec with placement * 2024-04-06T03:31:58.648 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:31:58 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:31:58.650 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-06T03:31:58.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.309+0000 7f1088bd5700 1 Processor -- start 2024-04-06T03:31:58.656 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.309+0000 7f1088bd5700 1 -- start start 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.309+0000 7f1088bd5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084102d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.309+0000 7f1088bd5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1084103260 con 0x7f1084102910 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.310+0000 7f108259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.310+0000 7f108259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084102d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57190/0 (socket says 172.21.15.94:57190) 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.310+0000 7f108259c700 1 -- 172.21.15.94:0/2685734804 learned_addr learned my addr 172.21.15.94:0/2685734804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.310+0000 7f108259c700 1 -- 172.21.15.94:0/2685734804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10841033a0 con 0x7f1084102910 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.310+0000 7f108259c700 1 --2- 172.21.15.94:0/2685734804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084102d20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10740098e0 tx=0x7f1074009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b55f528c380f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.310+0000 7f1081d9b700 1 -- 172.21.15.94:0/2685734804 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f107400b860 con 0x7f1084102910 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.310+0000 7f1081d9b700 1 -- 172.21.15.94:0/2685734804 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f107400be90 con 0x7f1084102910 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.310+0000 7f1081d9b700 1 -- 172.21.15.94:0/2685734804 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10740103c0 con 0x7f1084102910 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.311+0000 7f1088bd5700 1 -- 172.21.15.94:0/2685734804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 msgr2=0x7f1084102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.311+0000 7f1088bd5700 1 --2- 172.21.15.94:0/2685734804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084102d20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10740098e0 tx=0x7f1074009250 comp rx=0 tx=0).stop 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.311+0000 7f1088bd5700 1 -- 172.21.15.94:0/2685734804 shutdown_connections 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.311+0000 7f1088bd5700 1 --2- 172.21.15.94:0/2685734804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084102d20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.311+0000 7f1088bd5700 1 -- 172.21.15.94:0/2685734804 >> 172.21.15.94:0/2685734804 conn(0x7f10840fe070 msgr2=0x7f10841004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.311+0000 7f1088bd5700 1 -- 172.21.15.94:0/2685734804 shutdown_connections 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.311+0000 7f1088bd5700 1 -- 172.21.15.94:0/2685734804 wait complete. 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.311+0000 7f1088bd5700 1 Processor -- start 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.311+0000 7f1088bd5700 1 -- start start 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f1088bd5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084111720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f1088bd5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1084111c60 con 0x7f1084102910 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f108259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084111720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f108259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084111720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57192/0 (socket says 172.21.15.94:57192) 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f108259c700 1 -- 172.21.15.94:0/1851113388 learned_addr learned my addr 172.21.15.94:0/1851113388 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f108259c700 1 -- 172.21.15.94:0/1851113388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10740094f0 con 0x7f1084102910 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f108259c700 1 --2- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084111720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10740090b0 tx=0x7f1074009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f107b7fe700 1 -- 172.21.15.94:0/1851113388 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1074010840 con 0x7f1084102910 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f107b7fe700 1 -- 172.21.15.94:0/1851113388 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1074010e70 con 0x7f1084102910 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1084111e60 con 0x7f1084102910 2024-04-06T03:31:58.658 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f107b7fe700 1 -- 172.21.15.94:0/1851113388 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1074024e40 con 0x7f1084102910 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.312+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10841129d0 con 0x7f1084102910 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.313+0000 7f107b7fe700 1 -- 172.21.15.94:0/1851113388 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f107401a530 con 0x7f1084102910 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.313+0000 7f107b7fe700 1 --2- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1070036740 0x7f1070038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.313+0000 7f107b7fe700 1 -- 172.21.15.94:0/1851113388 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f107404a860 con 0x7f1084102910 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.313+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108405f5f0 con 0x7f1084102910 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.313+0000 7f1079d9b700 1 --2- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1070036740 0x7f1070038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.314+0000 7f1079d9b700 1 --2- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1070036740 0x7f1070038bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f106c009990 tx=0x7f106c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.318+0000 7f107b7fe700 1 -- 172.21.15.94:0/1851113388 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1074018940 con 0x7f1084102910 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.541+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f10840ff360 con 0x7f1070036740 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.545+0000 7f107b7fe700 1 -- 172.21.15.94:0/1851113388 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f10840ff360 con 0x7f1070036740 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1070036740 msgr2=0x7f1070038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 --2- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1070036740 0x7f1070038bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f106c009990 tx=0x7f106c006e40 comp rx=0 tx=0).stop 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 msgr2=0x7f1084111720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 --2- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084111720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10740090b0 tx=0x7f1074009250 comp rx=0 tx=0).stop 2024-04-06T03:31:58.659 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 shutdown_connections 2024-04-06T03:31:58.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 --2- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f1070036740 0x7f1070038bf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:58.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 --2- 172.21.15.94:0/1851113388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1084102910 0x7f1084111720 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:31:58.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 >> 172.21.15.94:0/1851113388 conn(0x7f10840fe070 msgr2=0x7f10840fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:31:58.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 shutdown_connections 2024-04-06T03:31:58.660 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:57.547+0000 7f1088bd5700 1 -- 172.21.15.94:0/1851113388 wait complete. 2024-04-06T03:31:58.660 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying alertmanager service with default placement... 2024-04-06T03:32:00.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-06T03:32:00.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.084+0000 7f3e40d90700 1 Processor -- start 2024-04-06T03:32:00.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.084+0000 7f3e40d90700 1 -- start start 2024-04-06T03:32:00.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.084+0000 7f3e40d90700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:00.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.085+0000 7f3e40d90700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c103210 con 0x7f3e3c1028c0 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.085+0000 7f3e3a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.085+0000 7f3e3a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41600/0 (socket says 172.21.15.94:41600) 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.085+0000 7f3e3a59c700 1 -- 172.21.15.94:0/2413919518 learned_addr learned my addr 172.21.15.94:0/2413919518 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.085+0000 7f3e3a59c700 1 -- 172.21.15.94:0/2413919518 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c103350 con 0x7f3e3c1028c0 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.085+0000 7f3e3a59c700 1 --2- 172.21.15.94:0/2413919518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c102cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0098e0 tx=0x7f3e2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648b584a5e7712c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.085+0000 7f3e3959a700 1 -- 172.21.15.94:0/2413919518 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e2c00b860 con 0x7f3e3c1028c0 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.085+0000 7f3e3959a700 1 -- 172.21.15.94:0/2413919518 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e2c00be90 con 0x7f3e3c1028c0 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.085+0000 7f3e3959a700 1 -- 172.21.15.94:0/2413919518 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e2c0103c0 con 0x7f3e3c1028c0 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.086+0000 7f3e40d90700 1 -- 172.21.15.94:0/2413919518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 msgr2=0x7f3e3c102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.086+0000 7f3e40d90700 1 --2- 172.21.15.94:0/2413919518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c102cd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0098e0 tx=0x7f3e2c009250 comp rx=0 tx=0).stop 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.086+0000 7f3e40d90700 1 -- 172.21.15.94:0/2413919518 shutdown_connections 2024-04-06T03:32:00.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.086+0000 7f3e40d90700 1 --2- 172.21.15.94:0/2413919518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c102cd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.086+0000 7f3e40d90700 1 -- 172.21.15.94:0/2413919518 >> 172.21.15.94:0/2413919518 conn(0x7f3e3c0fe080 msgr2=0x7f3e3c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.086+0000 7f3e40d90700 1 -- 172.21.15.94:0/2413919518 shutdown_connections 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.086+0000 7f3e40d90700 1 -- 172.21.15.94:0/2413919518 wait complete. 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.086+0000 7f3e40d90700 1 Processor -- start 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e40d90700 1 -- start start 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e40d90700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c1116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e40d90700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c111c00 con 0x7f3e3c1028c0 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e3a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c1116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e3a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c1116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41608/0 (socket says 172.21.15.94:41608) 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e3a59c700 1 -- 172.21.15.94:0/3998683395 learned_addr learned my addr 172.21.15.94:0/3998683395 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e3a59c700 1 -- 172.21.15.94:0/3998683395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0094f0 con 0x7f3e3c1028c0 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e3a59c700 1 --2- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c1116c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c011680 tx=0x7f3e2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e2affd700 1 -- 172.21.15.94:0/3998683395 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e2c010890 con 0x7f3e3c1028c0 2024-04-06T03:32:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e2affd700 1 -- 172.21.15.94:0/3998683395 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e2c010ec0 con 0x7f3e3c1028c0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c111e00 con 0x7f3e3c1028c0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e2affd700 1 -- 172.21.15.94:0/3998683395 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e2c0243f0 con 0x7f3e3c1028c0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.087+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c112970 con 0x7f3e3c1028c0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.088+0000 7f3e2affd700 1 -- 172.21.15.94:0/3998683395 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f3e2c02d3f0 con 0x7f3e3c1028c0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.088+0000 7f3e2affd700 1 --2- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f3e24036790 0x7f3e24038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.088+0000 7f3e2affd700 1 -- 172.21.15.94:0/3998683395 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f3e2c04a280 con 0x7f3e3c1028c0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.088+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c05f5f0 con 0x7f3e3c1028c0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.088+0000 7f3e39d9b700 1 --2- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f3e24036790 0x7f3e24038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.089+0000 7f3e39d9b700 1 --2- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f3e24036790 0x7f3e24038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009990 tx=0x7f3e30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.093+0000 7f3e2affd700 1 -- 172.21.15.94:0/3998683395 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3e2c02b8e0 con 0x7f3e3c1028c0 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.292+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e3c0ff340 con 0x7f3e24036790 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.296+0000 7f3e2affd700 1 -- 172.21.15.94:0/3998683395 <== mgr.14120 v2:172.21.15.94:6800/682257512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f3e3c0ff340 con 0x7f3e24036790 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f3e24036790 msgr2=0x7f3e24038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 --2- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f3e24036790 0x7f3e24038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009990 tx=0x7f3e30006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 msgr2=0x7f3e3c1116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 --2- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c1116c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c011680 tx=0x7f3e2c009250 comp rx=0 tx=0).stop 2024-04-06T03:32:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 shutdown_connections 2024-04-06T03:32:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 --2- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f3e24036790 0x7f3e24038c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 --2- 172.21.15.94:0/3998683395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e3c1028c0 0x7f3e3c1116c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 >> 172.21.15.94:0/3998683395 conn(0x7f3e3c0fe080 msgr2=0x7f3e3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 shutdown_connections 2024-04-06T03:32:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:31:59.298+0000 7f3e40d90700 1 -- 172.21.15.94:0/3998683395 wait complete. 2024-04-06T03:32:00.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:00 smithi094 ceph-mon[29713]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:32:00.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:00 smithi094 ceph-mon[29713]: Saving service alertmanager spec with placement count:1 2024-04-06T03:32:00.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:00 smithi094 ceph-mon[29713]: from='mgr.14120 172.21.15.94:0/1372463263' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:00.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:00 smithi094 ceph-mon[29713]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.522+0000 7f0443de6700 1 Processor -- start 2024-04-06T03:32:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.522+0000 7f0443de6700 1 -- start start 2024-04-06T03:32:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.522+0000 7f0443de6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.522+0000 7f0443de6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c1020f0 con 0x7f043c1017a0 2024-04-06T03:32:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.522+0000 7f0441b82700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:01.309 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.522+0000 7f0441b82700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c101bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41610/0 (socket says 172.21.15.94:41610) 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.522+0000 7f0441b82700 1 -- 172.21.15.94:0/417342661 learned_addr learned my addr 172.21.15.94:0/417342661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0441b82700 1 -- 172.21.15.94:0/417342661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c102230 con 0x7f043c1017a0 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0441b82700 1 --2- 172.21.15.94:0/417342661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c101bb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f043800bb80 tx=0x7f043800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83432f8752eaf78 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0440b80700 1 -- 172.21.15.94:0/417342661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f043800dd00 con 0x7f043c1017a0 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0440b80700 1 -- 172.21.15.94:0/417342661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f043800de60 con 0x7f043c1017a0 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0440b80700 1 -- 172.21.15.94:0/417342661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0438018920 con 0x7f043c1017a0 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0443de6700 1 -- 172.21.15.94:0/417342661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 msgr2=0x7f043c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0443de6700 1 --2- 172.21.15.94:0/417342661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c101bb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f043800bb80 tx=0x7f043800b4f0 comp rx=0 tx=0).stop 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0443de6700 1 -- 172.21.15.94:0/417342661 shutdown_connections 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0443de6700 1 --2- 172.21.15.94:0/417342661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c101bb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.523+0000 7f0443de6700 1 -- 172.21.15.94:0/417342661 >> 172.21.15.94:0/417342661 conn(0x7f043c0fcf40 msgr2=0x7f043c0ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.524+0000 7f0443de6700 1 -- 172.21.15.94:0/417342661 shutdown_connections 2024-04-06T03:32:01.310 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.524+0000 7f0443de6700 1 -- 172.21.15.94:0/417342661 wait complete. 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.524+0000 7f0443de6700 1 Processor -- start 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.524+0000 7f0443de6700 1 -- start start 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.524+0000 7f0443de6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c10f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.524+0000 7f0443de6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c10f9f0 con 0x7f043c1017a0 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.524+0000 7f0441b82700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c10f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.524+0000 7f0441b82700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c10f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41620/0 (socket says 172.21.15.94:41620) 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.524+0000 7f0441b82700 1 -- 172.21.15.94:0/112981094 learned_addr learned my addr 172.21.15.94:0/112981094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.525+0000 7f0441b82700 1 -- 172.21.15.94:0/112981094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043800b790 con 0x7f043c1017a0 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.525+0000 7f0441b82700 1 --2- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c10f4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0438006b00 tx=0x7f043800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.525+0000 7f04327fc700 1 -- 172.21.15.94:0/112981094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0438010bd0 con 0x7f043c1017a0 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.525+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c10fbf0 con 0x7f043c1017a0 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.525+0000 7f04327fc700 1 -- 172.21.15.94:0/112981094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f043801d050 con 0x7f043c1017a0 2024-04-06T03:32:01.311 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.525+0000 7f04327fc700 1 -- 172.21.15.94:0/112981094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0438022590 con 0x7f043c1017a0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.525+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c110710 con 0x7f043c1017a0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.526+0000 7f04327fc700 1 -- 172.21.15.94:0/112981094 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f0438022810 con 0x7f043c1017a0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.526+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c05f5f0 con 0x7f043c1017a0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.526+0000 7f04327fc700 1 --2- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f0428036790 0x7f0428038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.526+0000 7f04327fc700 1 -- 172.21.15.94:0/112981094 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0438014070 con 0x7f043c1017a0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.526+0000 7f0441381700 1 --2- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f0428036790 0x7f0428038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.526+0000 7f0441381700 1 --2- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f0428036790 0x7f0428038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f042c009990 tx=0x7f042c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.530+0000 7f04327fc700 1 -- 172.21.15.94:0/112981094 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f043800e360 con 0x7f043c1017a0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.714+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f043c04bed0 con 0x7f043c1017a0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.717+0000 7f04327fc700 1 -- 172.21.15.94:0/112981094 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f04380202d0 con 0x7f043c1017a0 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.719+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f0428036790 msgr2=0x7f0428038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.719+0000 7f0443de6700 1 --2- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f0428036790 0x7f0428038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f042c009990 tx=0x7f042c006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.719+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 msgr2=0x7f043c10f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:01.312 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.719+0000 7f0443de6700 1 --2- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c10f4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0438006b00 tx=0x7f043800b4f0 comp rx=0 tx=0).stop 2024-04-06T03:32:01.313 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.719+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 shutdown_connections 2024-04-06T03:32:01.313 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.720+0000 7f0443de6700 1 --2- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f0428036790 0x7f0428038c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:01.313 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.720+0000 7f0443de6700 1 --2- 172.21.15.94:0/112981094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f043c1017a0 0x7f043c10f4b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:01.313 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.720+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 >> 172.21.15.94:0/112981094 conn(0x7f043c0fcf40 msgr2=0x7f043c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:01.313 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.720+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 shutdown_connections 2024-04-06T03:32:01.313 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:00.720+0000 7f0443de6700 1 -- 172.21.15.94:0/112981094 wait complete. 2024-04-06T03:32:01.908 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:01 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/112981094' entity='client.admin' 2024-04-06T03:32:02.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.703+0000 7fe0659d2700 1 Processor -- start 2024-04-06T03:32:02.458 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.703+0000 7fe0659d2700 1 -- start start 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.703+0000 7fe0659d2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe0601010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.703+0000 7fe0659d2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe060101630 con 0x7fe060100ce0 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.703+0000 7fe05effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe0601010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.703+0000 7fe05effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe0601010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41630/0 (socket says 172.21.15.94:41630) 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.703+0000 7fe05effd700 1 -- 172.21.15.94:0/804747950 learned_addr learned my addr 172.21.15.94:0/804747950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.703+0000 7fe05effd700 1 -- 172.21.15.94:0/804747950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe060101770 con 0x7fe060100ce0 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.704+0000 7fe05effd700 1 --2- 172.21.15.94:0/804747950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe0601010f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0500098e0 tx=0x7fe050009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62ca1097f5237e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.704+0000 7fe05dffb700 1 -- 172.21.15.94:0/804747950 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe05000b860 con 0x7fe060100ce0 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.704+0000 7fe05dffb700 1 -- 172.21.15.94:0/804747950 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe05000be90 con 0x7fe060100ce0 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.704+0000 7fe05dffb700 1 -- 172.21.15.94:0/804747950 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0500103c0 con 0x7fe060100ce0 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.704+0000 7fe0659d2700 1 -- 172.21.15.94:0/804747950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 msgr2=0x7fe0601010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.704+0000 7fe0659d2700 1 --2- 172.21.15.94:0/804747950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe0601010f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0500098e0 tx=0x7fe050009250 comp rx=0 tx=0).stop 2024-04-06T03:32:02.459 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.705+0000 7fe0659d2700 1 -- 172.21.15.94:0/804747950 shutdown_connections 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.705+0000 7fe0659d2700 1 --2- 172.21.15.94:0/804747950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe0601010f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.705+0000 7fe0659d2700 1 -- 172.21.15.94:0/804747950 >> 172.21.15.94:0/804747950 conn(0x7fe0600fc700 msgr2=0x7fe0600feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.705+0000 7fe0659d2700 1 -- 172.21.15.94:0/804747950 shutdown_connections 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.705+0000 7fe0659d2700 1 -- 172.21.15.94:0/804747950 wait complete. 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.705+0000 7fe0659d2700 1 Processor -- start 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.705+0000 7fe0659d2700 1 -- start start 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.705+0000 7fe0659d2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe060103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.705+0000 7fe0659d2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe060103c50 con 0x7fe060100ce0 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe05effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe060103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe05effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe060103710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41640/0 (socket says 172.21.15.94:41640) 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe05effd700 1 -- 172.21.15.94:0/2721969625 learned_addr learned my addr 172.21.15.94:0/2721969625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:02.460 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe05effd700 1 -- 172.21.15.94:0/2721969625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0500094f0 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe05effd700 1 --2- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe060103710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe050011680 tx=0x7fe050009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe0477fe700 1 -- 172.21.15.94:0/2721969625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe050010890 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe0477fe700 1 -- 172.21.15.94:0/2721969625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe050010ec0 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe060101cc0 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe0477fe700 1 -- 172.21.15.94:0/2721969625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0500243f0 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.706+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0601021b0 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.707+0000 7fe0477fe700 1 -- 172.21.15.94:0/2721969625 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fe05002d3f0 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.707+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06004bed0 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.707+0000 7fe0477fe700 1 --2- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe048036790 0x7fe048038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.707+0000 7fe0477fe700 1 -- 172.21.15.94:0/2721969625 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe05004a400 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.707+0000 7fe05e7fc700 1 --2- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe048036790 0x7fe048038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.708+0000 7fe05e7fc700 1 --2- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe048036790 0x7fe048038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe054009990 tx=0x7fe054006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.712+0000 7fe0477fe700 1 -- 172.21.15.94:0/2721969625 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe0500193c0 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.907+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe060106830 con 0x7fe060100ce0 2024-04-06T03:32:02.461 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.911+0000 7fe0477fe700 1 -- 172.21.15.94:0/2721969625 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe05002c350 con 0x7fe060100ce0 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe048036790 msgr2=0x7fe048038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 --2- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe048036790 0x7fe048038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe054009990 tx=0x7fe054006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 msgr2=0x7fe060103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 --2- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe060103710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe050011680 tx=0x7fe050009250 comp rx=0 tx=0).stop 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 shutdown_connections 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 --2- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7fe048036790 0x7fe048038c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 --2- 172.21.15.94:0/2721969625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe060100ce0 0x7fe060103710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 >> 172.21.15.94:0/2721969625 conn(0x7fe0600fc700 msgr2=0x7fe0600fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 shutdown_connections 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:01.914+0000 7fe0659d2700 1 -- 172.21.15.94:0/2721969625 wait complete. 2024-04-06T03:32:02.462 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling the dashboard module... 2024-04-06T03:32:03.105 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:02 smithi094 ceph-mon[29713]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:03.105 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:02 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2721969625' entity='client.admin' 2024-04-06T03:32:04.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:03 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2057017983' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-06T03:32:04.550 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.859+0000 7f9f78521700 1 Processor -- start 2024-04-06T03:32:04.550 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.859+0000 7f9f78521700 1 -- start start 2024-04-06T03:32:04.550 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.859+0000 7f9f78521700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70102160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.859+0000 7f9f78521700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701026a0 con 0x7f9f70101d50 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.859+0000 7f9f762bd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.859+0000 7f9f762bd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70102160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41652/0 (socket says 172.21.15.94:41652) 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.859+0000 7f9f762bd700 1 -- 172.21.15.94:0/377277772 learned_addr learned my addr 172.21.15.94:0/377277772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.860+0000 7f9f762bd700 1 -- 172.21.15.94:0/377277772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f701027e0 con 0x7f9f70101d50 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.860+0000 7f9f762bd700 1 --2- 172.21.15.94:0/377277772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70102160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0098e0 tx=0x7f9f6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c0ce43b36bc4b server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.860+0000 7f9f752bb700 1 -- 172.21.15.94:0/377277772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f6c00b860 con 0x7f9f70101d50 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.860+0000 7f9f752bb700 1 -- 172.21.15.94:0/377277772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f6c00be90 con 0x7f9f70101d50 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.860+0000 7f9f752bb700 1 -- 172.21.15.94:0/377277772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f6c0103c0 con 0x7f9f70101d50 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 -- 172.21.15.94:0/377277772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 msgr2=0x7f9f70102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 --2- 172.21.15.94:0/377277772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70102160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c0098e0 tx=0x7f9f6c009250 comp rx=0 tx=0).stop 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 -- 172.21.15.94:0/377277772 shutdown_connections 2024-04-06T03:32:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 --2- 172.21.15.94:0/377277772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70102160 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 -- 172.21.15.94:0/377277772 >> 172.21.15.94:0/377277772 conn(0x7f9f700fc6c0 msgr2=0x7f9f700fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 -- 172.21.15.94:0/377277772 shutdown_connections 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 -- 172.21.15.94:0/377277772 wait complete. 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 Processor -- start 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 -- start start 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.861+0000 7f9f78521700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701144b0 con 0x7f9f70101d50 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f762bd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f762bd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70101550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41664/0 (socket says 172.21.15.94:41664) 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f762bd700 1 -- 172.21.15.94:0/2057017983 learned_addr learned my addr 172.21.15.94:0/2057017983 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f762bd700 1 -- 172.21.15.94:0/2057017983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c0094f0 con 0x7f9f70101d50 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f762bd700 1 --2- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70101550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c011c10 tx=0x7f9f6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f66ffd700 1 -- 172.21.15.94:0/2057017983 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f6c010890 con 0x7f9f70101d50 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f66ffd700 1 -- 172.21.15.94:0/2057017983 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f6c010ec0 con 0x7f9f70101d50 2024-04-06T03:32:04.552 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f701145f0 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f66ffd700 1 -- 172.21.15.94:0/2057017983 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f6c0243f0 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.862+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f70114a80 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.863+0000 7f9f66ffd700 1 -- 172.21.15.94:0/2057017983 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f9f6c02d3f0 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.863+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7004bed0 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.863+0000 7f9f66ffd700 1 --2- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9f5c036790 0x7f9f5c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.863+0000 7f9f66ffd700 1 -- 172.21.15.94:0/2057017983 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9f6c04a430 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.863+0000 7f9f75abc700 1 --2- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9f5c036790 0x7f9f5c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.864+0000 7f9f75abc700 1 --2- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9f5c036790 0x7f9f5c038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009990 tx=0x7f9f60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:02.868+0000 7f9f66ffd700 1 -- 172.21.15.94:0/2057017983 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9f6c018e60 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.104+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f9f70103560 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.913+0000 7f9f66ffd700 1 -- 172.21.15.94:0/2057017983 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f9f6c02d6a0 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.915+0000 7f9f66ffd700 1 -- 172.21.15.94:0/2057017983 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f9f6c02cbc0 con 0x7f9f70101d50 2024-04-06T03:32:04.553 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9f5c036790 msgr2=0x7f9f5c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 --2- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9f5c036790 0x7f9f5c038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009990 tx=0x7f9f60006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 msgr2=0x7f9f70101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 --2- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70101550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c011c10 tx=0x7f9f6c009250 comp rx=0 tx=0).stop 2024-04-06T03:32:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 shutdown_connections 2024-04-06T03:32:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 --2- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f9f5c036790 0x7f9f5c038c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 --2- 172.21.15.94:0/2057017983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f70101d50 0x7f9f70101550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 >> 172.21.15.94:0/2057017983 conn(0x7f9f700fc6c0 msgr2=0x7f9f700fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 shutdown_connections 2024-04-06T03:32:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:03.918+0000 7f9f78521700 1 -- 172.21.15.94:0/2057017983 wait complete. 2024-04-06T03:32:05.156 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:04 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2057017983' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-06T03:32:05.156 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:04 smithi094 ceph-mon[29713]: mgrmap e9: smithi094.purlzp(active, since 24s) 2024-04-06T03:32:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-06T03:32:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-06T03:32:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-06T03:32:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "smithi094.purlzp", 2024-04-06T03:32:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-06T03:32:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-06T03:32:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.950+0000 7f26b540a700 1 Processor -- start 2024-04-06T03:32:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.950+0000 7f26b540a700 1 -- start start 2024-04-06T03:32:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.950+0000 7f26b540a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0105560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.950+0000 7f26b540a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b0071640 con 0x7f26b0071230 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.950+0000 7f26aeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0105560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.950+0000 7f26aeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0105560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41688/0 (socket says 172.21.15.94:41688) 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.950+0000 7f26aeffd700 1 -- 172.21.15.94:0/2214386333 learned_addr learned my addr 172.21.15.94:0/2214386333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.950+0000 7f26aeffd700 1 -- 172.21.15.94:0/2214386333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b0105aa0 con 0x7f26b0071230 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.951+0000 7f26aeffd700 1 --2- 172.21.15.94:0/2214386333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0105560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098e0 tx=0x7f26a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2fc57c43e057 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.951+0000 7f26adffb700 1 -- 172.21.15.94:0/2214386333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26a000b860 con 0x7f26b0071230 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.951+0000 7f26adffb700 1 -- 172.21.15.94:0/2214386333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26a000be90 con 0x7f26b0071230 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.951+0000 7f26adffb700 1 -- 172.21.15.94:0/2214386333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26a00103c0 con 0x7f26b0071230 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.951+0000 7f26b540a700 1 -- 172.21.15.94:0/2214386333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 msgr2=0x7f26b0105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.951+0000 7f26b540a700 1 --2- 172.21.15.94:0/2214386333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0105560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098e0 tx=0x7f26a0009250 comp rx=0 tx=0).stop 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.951+0000 7f26b540a700 1 -- 172.21.15.94:0/2214386333 shutdown_connections 2024-04-06T03:32:05.978 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.951+0000 7f26b540a700 1 --2- 172.21.15.94:0/2214386333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0105560 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.951+0000 7f26b540a700 1 -- 172.21.15.94:0/2214386333 >> 172.21.15.94:0/2214386333 conn(0x7f26b00fcfb0 msgr2=0x7f26b00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.952+0000 7f26b540a700 1 -- 172.21.15.94:0/2214386333 shutdown_connections 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.952+0000 7f26b540a700 1 -- 172.21.15.94:0/2214386333 wait complete. 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.952+0000 7f26b540a700 1 Processor -- start 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.952+0000 7f26b540a700 1 -- start start 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.952+0000 7f26b540a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.952+0000 7f26b540a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b01033f0 con 0x7f26b0071230 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.952+0000 7f26aeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.952+0000 7f26aeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41696/0 (socket says 172.21.15.94:41696) 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.953+0000 7f26aeffd700 1 -- 172.21.15.94:0/594129927 learned_addr learned my addr 172.21.15.94:0/594129927 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.953+0000 7f26aeffd700 1 -- 172.21.15.94:0/594129927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a00094f0 con 0x7f26b0071230 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.953+0000 7f26aeffd700 1 --2- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0102eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f26a00105b0 tx=0x7f26a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.953+0000 7f26977fe700 1 -- 172.21.15.94:0/594129927 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26a0010840 con 0x7f26b0071230 2024-04-06T03:32:05.979 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.953+0000 7f26977fe700 1 -- 172.21.15.94:0/594129927 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26a0010e70 con 0x7f26b0071230 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.953+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b0101460 con 0x7f26b0071230 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.953+0000 7f26977fe700 1 -- 172.21.15.94:0/594129927 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26a001ae30 con 0x7f26b0071230 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.953+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b0101950 con 0x7f26b0071230 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.954+0000 7f26977fe700 1 -- 172.21.15.94:0/594129927 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f26a002d3f0 con 0x7f26b0071230 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.954+0000 7f26977fe700 1 --2- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f26980367e0 0x7f2698038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.954+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b004bed0 con 0x7f26b0071230 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.954+0000 7f26977fe700 1 -- 172.21.15.94:0/594129927 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f26a004a230 con 0x7f26b0071230 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.954+0000 7f26ae7fc700 1 -- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f26980367e0 msgr2=0x7f2698038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/682257512 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.954+0000 7f26ae7fc700 1 --2- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f26980367e0 0x7f2698038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:04.959+0000 7f26977fe700 1 -- 172.21.15.94:0/594129927 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f26a002b8e0 con 0x7f26b0071230 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.155+0000 7f26ae7fc700 1 -- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f26980367e0 msgr2=0x7f2698038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/682257512 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.155+0000 7f26ae7fc700 1 --2- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f26980367e0 0x7f2698038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.203+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f26b0101ff0 con 0x7f26b0071230 2024-04-06T03:32:05.980 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.203+0000 7f26977fe700 1 -- 172.21.15.94:0/594129927 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f26a002b2f0 con 0x7f26b0071230 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f26980367e0 msgr2=0x7f2698038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 --2- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f26980367e0 0x7f2698038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 msgr2=0x7f26b0102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 --2- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0102eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f26a00105b0 tx=0x7f26a0009250 comp rx=0 tx=0).stop 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 shutdown_connections 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 --2- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7f26980367e0 0x7f2698038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 --2- 172.21.15.94:0/594129927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26b0071230 0x7f26b0102eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 >> 172.21.15.94:0/594129927 conn(0x7f26b00fcfb0 msgr2=0x7f26b00fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 shutdown_connections 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:05.206+0000 7f26b540a700 1 -- 172.21.15.94:0/594129927 wait complete. 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-04-06T03:32:05.981 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 9... 2024-04-06T03:32:06.039 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:05 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/594129927' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: Active manager daemon smithi094.purlzp restarted 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: Activating manager daemon smithi094.purlzp 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: osdmap e3: 0 total, 0 up, 0 in 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: mgrmap e10: smithi094.purlzp(active, starting, since 0.00312951s) 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: Manager daemon smithi094.purlzp is now available 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:32:10.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:10 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:32:11.828 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:11 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:32:11.828 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:11 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:32:11.829 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:11 smithi094 ceph-mon[29713]: mgrmap e11: smithi094.purlzp(active, since 1.00557s) 2024-04-06T03:32:12.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-06T03:32:12.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.410+0000 7ff5ab61c700 1 Processor -- start 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.410+0000 7ff5ab61c700 1 -- start start 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.411+0000 7ff5ab61c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a41010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.411+0000 7ff5ab61c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4101610 con 0x7ff5a4100cc0 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.411+0000 7ff5a93b8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a41010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.411+0000 7ff5a93b8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a41010d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41708/0 (socket says 172.21.15.94:41708) 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.411+0000 7ff5a93b8700 1 -- 172.21.15.94:0/518594785 learned_addr learned my addr 172.21.15.94:0/518594785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.411+0000 7ff5a93b8700 1 -- 172.21.15.94:0/518594785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4101750 con 0x7ff5a4100cc0 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.411+0000 7ff5a93b8700 1 --2- 172.21.15.94:0/518594785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a41010d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00098e0 tx=0x7ff5a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79c8c8d0f2f80e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.411+0000 7ff59bfff700 1 -- 172.21.15.94:0/518594785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5a000b860 con 0x7ff5a4100cc0 2024-04-06T03:32:12.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.411+0000 7ff59bfff700 1 -- 172.21.15.94:0/518594785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5a000be90 con 0x7ff5a4100cc0 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.412+0000 7ff59bfff700 1 -- 172.21.15.94:0/518594785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5a00103c0 con 0x7ff5a4100cc0 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.412+0000 7ff5ab61c700 1 -- 172.21.15.94:0/518594785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 msgr2=0x7ff5a41010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.412+0000 7ff5ab61c700 1 --2- 172.21.15.94:0/518594785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a41010d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00098e0 tx=0x7ff5a0009250 comp rx=0 tx=0).stop 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.412+0000 7ff5ab61c700 1 -- 172.21.15.94:0/518594785 shutdown_connections 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.412+0000 7ff5ab61c700 1 --2- 172.21.15.94:0/518594785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a41010d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.412+0000 7ff5ab61c700 1 -- 172.21.15.94:0/518594785 >> 172.21.15.94:0/518594785 conn(0x7ff5a40fc6e0 msgr2=0x7ff5a40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.412+0000 7ff5ab61c700 1 -- 172.21.15.94:0/518594785 shutdown_connections 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.412+0000 7ff5ab61c700 1 -- 172.21.15.94:0/518594785 wait complete. 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff5ab61c700 1 Processor -- start 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff5ab61c700 1 -- start start 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff5ab61c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a4111690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff5ab61c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4111bd0 con 0x7ff5a4100cc0 2024-04-06T03:32:12.099 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff5a93b8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a4111690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff5a93b8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a4111690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41716/0 (socket says 172.21.15.94:41716) 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff5a93b8700 1 -- 172.21.15.94:0/2038604709 learned_addr learned my addr 172.21.15.94:0/2038604709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff5a93b8700 1 -- 172.21.15.94:0/2038604709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a00094f0 con 0x7ff5a4100cc0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff5a93b8700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a4111690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0011680 tx=0x7ff5a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.413+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5a0010890 con 0x7ff5a4100cc0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.414+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5a0010ec0 con 0x7ff5a4100cc0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.414+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a4111dd0 con 0x7ff5a4100cc0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.414+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5a00243f0 con 0x7ff5a4100cc0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.414+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a4112940 con 0x7ff5a4100cc0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.414+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7ff5a002d3f0 con 0x7ff5a4100cc0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.415+0000 7ff599ffb700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 0x7ff590038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.415+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 --> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5a405f5f0 con 0x7ff5900367e0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.415+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff5a004a2f0 con 0x7ff5a4100cc0 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.415+0000 7ff5a8bb7700 1 -- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 msgr2=0x7ff590038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/682257512 2024-04-06T03:32:12.100 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.415+0000 7ff5a8bb7700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 0x7ff590038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.616+0000 7ff5a8bb7700 1 -- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 msgr2=0x7ff590038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/682257512 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:06.616+0000 7ff5a8bb7700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 0x7ff590038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:07.017+0000 7ff5a8bb7700 1 -- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 msgr2=0x7ff590038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/682257512 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:07.017+0000 7ff5a8bb7700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 0x7ff590038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:07.818+0000 7ff5a8bb7700 1 -- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 msgr2=0x7ff590038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/682257512 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:07.818+0000 7ff5a8bb7700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 0x7ff590038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:09.419+0000 7ff5a8bb7700 1 -- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 msgr2=0x7ff590038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/682257512 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:09.419+0000 7ff5a8bb7700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 0x7ff590038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:10.458+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7ff5a001a8c0 con 0x7ff5a4100cc0 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:10.458+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 msgr2=0x7ff590038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:10.458+0000 7ff599ffb700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 0x7ff590038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.459+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7ff5a0029d00 con 0x7ff5a4100cc0 2024-04-06T03:32:12.101 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.459+0000 7ff599ffb700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff590039ff0 0x7ff59003c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.459+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 --> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5a405f5f0 con 0x7ff590039ff0 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.459+0000 7ff5a8bb7700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff590039ff0 0x7ff59003c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.460+0000 7ff5a8bb7700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff590039ff0 0x7ff59003c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5940039d0 tx=0x7ff5940092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.461+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 <== mgr.14162 v2:172.21.15.94:6800/2299971604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7ff5a405f5f0 con 0x7ff590039ff0 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 --> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff5a4112260 con 0x7ff590039ff0 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff599ffb700 1 -- 172.21.15.94:0/2038604709 <== mgr.14162 v2:172.21.15.94:6800/2299971604 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7ff5a4112260 con 0x7ff590039ff0 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff590039ff0 msgr2=0x7ff59003c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff590039ff0 0x7ff59003c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5940039d0 tx=0x7ff5940092f0 comp rx=0 tx=0).stop 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 msgr2=0x7ff5a4111690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a4111690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0011680 tx=0x7ff5a0009250 comp rx=0 tx=0).stop 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 shutdown_connections 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff590039ff0 0x7ff59003c3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a4100cc0 0x7ff5a4111690 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:12.102 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 --2- 172.21.15.94:0/2038604709 >> [v2:172.21.15.94:6800/682257512,v1:172.21.15.94:6801/682257512] conn(0x7ff5900367e0 0x7ff590038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:12.103 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 >> 172.21.15.94:0/2038604709 conn(0x7ff5a40fc6e0 msgr2=0x7ff5a40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:12.103 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 shutdown_connections 2024-04-06T03:32:12.103 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:11.467+0000 7ff5ab61c700 1 -- 172.21.15.94:0/2038604709 wait complete. 2024-04-06T03:32:12.103 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 9 is available 2024-04-06T03:32:12.103 INFO:teuthology.orchestra.run.smithi094.stdout:Generating a dashboard self-signed certificate... 2024-04-06T03:32:13.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:12 smithi094 ceph-mon[29713]: [06/Apr/2024:03:32:11] ENGINE Bus STARTING 2024-04-06T03:32:13.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:12 smithi094 ceph-mon[29713]: [06/Apr/2024:03:32:11] ENGINE Serving on https://172.21.15.94:7150 2024-04-06T03:32:13.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:12 smithi094 ceph-mon[29713]: [06/Apr/2024:03:32:11] ENGINE Bus STARTED 2024-04-06T03:32:13.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:12 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:13.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:12 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:13.487 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.562+0000 7ff791322700 1 Processor -- start 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.562+0000 7ff791322700 1 -- start start 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.562+0000 7ff791322700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.562+0000 7ff791322700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c071640 con 0x7ff78c071230 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.562+0000 7ff78affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.562+0000 7ff78affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54462/0 (socket says 172.21.15.94:54462) 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.562+0000 7ff78affd700 1 -- 172.21.15.94:0/1547617301 learned_addr learned my addr 172.21.15.94:0/1547617301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.563+0000 7ff78affd700 1 -- 172.21.15.94:0/1547617301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c103350 con 0x7ff78c071230 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.563+0000 7ff78affd700 1 --2- 172.21.15.94:0/1547617301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c107960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff7800098e0 tx=0x7ff780009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ec8a0d1f06e6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.563+0000 7ff789ffb700 1 -- 172.21.15.94:0/1547617301 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff78000b860 con 0x7ff78c071230 2024-04-06T03:32:13.488 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.563+0000 7ff789ffb700 1 -- 172.21.15.94:0/1547617301 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff78000be90 con 0x7ff78c071230 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.563+0000 7ff789ffb700 1 -- 172.21.15.94:0/1547617301 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7800103c0 con 0x7ff78c071230 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.564+0000 7ff791322700 1 -- 172.21.15.94:0/1547617301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 msgr2=0x7ff78c107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.564+0000 7ff791322700 1 --2- 172.21.15.94:0/1547617301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c107960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff7800098e0 tx=0x7ff780009250 comp rx=0 tx=0).stop 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.564+0000 7ff791322700 1 -- 172.21.15.94:0/1547617301 shutdown_connections 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.564+0000 7ff791322700 1 --2- 172.21.15.94:0/1547617301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c107960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.564+0000 7ff791322700 1 -- 172.21.15.94:0/1547617301 >> 172.21.15.94:0/1547617301 conn(0x7ff78c0fcfb0 msgr2=0x7ff78c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.564+0000 7ff791322700 1 -- 172.21.15.94:0/1547617301 shutdown_connections 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.564+0000 7ff791322700 1 -- 172.21.15.94:0/1547617301 wait complete. 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.565+0000 7ff791322700 1 Processor -- start 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.565+0000 7ff791322700 1 -- start start 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.565+0000 7ff791322700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c1138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.565+0000 7ff791322700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c113e20 con 0x7ff78c071230 2024-04-06T03:32:13.489 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.565+0000 7ff78affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c1138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.565+0000 7ff78affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c1138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54464/0 (socket says 172.21.15.94:54464) 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.565+0000 7ff78affd700 1 -- 172.21.15.94:0/1767552534 learned_addr learned my addr 172.21.15.94:0/1767552534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.565+0000 7ff78affd700 1 -- 172.21.15.94:0/1767552534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800094f0 con 0x7ff78c071230 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.565+0000 7ff78affd700 1 --2- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c1138e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff7800105b0 tx=0x7ff780009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.566+0000 7ff7737fe700 1 -- 172.21.15.94:0/1767552534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff780010840 con 0x7ff78c071230 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.566+0000 7ff7737fe700 1 -- 172.21.15.94:0/1767552534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff780010e70 con 0x7ff78c071230 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.566+0000 7ff7737fe700 1 -- 172.21.15.94:0/1767552534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff78001ae30 con 0x7ff78c071230 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.566+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c114080 con 0x7ff78c071230 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.566+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c114b90 con 0x7ff78c071230 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.567+0000 7ff7737fe700 1 -- 172.21.15.94:0/1767552534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7ff78002d3f0 con 0x7ff78c071230 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.567+0000 7ff7737fe700 1 --2- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff774036710 0x7ff774038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.567+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c04bed0 con 0x7ff78c071230 2024-04-06T03:32:13.490 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.567+0000 7ff7737fe700 1 -- 172.21.15.94:0/1767552534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff78004a1b0 con 0x7ff78c071230 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.567+0000 7ff78a7fc700 1 --2- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff774036710 0x7ff774038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.567+0000 7ff78a7fc700 1 --2- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff774036710 0x7ff774038bc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009990 tx=0x7ff77c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.572+0000 7ff7737fe700 1 -- 172.21.15.94:0/1767552534 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff78002b4d0 con 0x7ff78c071230 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.808+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 --> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ff78c0fe330 con 0x7ff774036710 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.885+0000 7ff7737fe700 1 -- 172.21.15.94:0/1767552534 <== mgr.14162 v2:172.21.15.94:6800/2299971604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ff78c0fe330 con 0x7ff774036710 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff774036710 msgr2=0x7ff774038bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 --2- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff774036710 0x7ff774038bc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009990 tx=0x7ff77c006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 msgr2=0x7ff78c1138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 --2- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c1138e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff7800105b0 tx=0x7ff780009250 comp rx=0 tx=0).stop 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 shutdown_connections 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 --2- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff774036710 0x7ff774038bc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 --2- 172.21.15.94:0/1767552534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff78c071230 0x7ff78c1138e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:13.491 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 >> 172.21.15.94:0/1767552534 conn(0x7ff78c0fcfb0 msgr2=0x7ff78c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:13.492 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 shutdown_connections 2024-04-06T03:32:13.492 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:12.887+0000 7ff791322700 1 -- 172.21.15.94:0/1767552534 wait complete. 2024-04-06T03:32:13.492 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial admin user... 2024-04-06T03:32:14.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:13 smithi094 ceph-mon[29713]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:32:14.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:13 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:14.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:13 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:14.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:13 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:14.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:13 smithi094 ceph-mon[29713]: mgrmap e12: smithi094.purlzp(active, since 2s) 2024-04-06T03:32:15.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$SCBNNrFM2tEVrli9.ZB0J.fRr67ijX/cBC1RcRQt5/tMKPc/9.myS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712374334, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.891+0000 7f312ad9f700 1 Processor -- start 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.891+0000 7f312ad9f700 1 -- start start 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.891+0000 7f312ad9f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.891+0000 7f312ad9f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3124103730 con 0x7f3124102de0 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.892+0000 7f3128b3b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.892+0000 7f3128b3b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54472/0 (socket says 172.21.15.94:54472) 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.892+0000 7f3128b3b700 1 -- 172.21.15.94:0/3689427217 learned_addr learned my addr 172.21.15.94:0/3689427217 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.892+0000 7f3128b3b700 1 -- 172.21.15.94:0/3689427217 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3124101470 con 0x7f3124102de0 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.892+0000 7f3128b3b700 1 --2- 172.21.15.94:0/3689427217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241031f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31180098e0 tx=0x7f3118009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956af9116e62585e server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:15.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.892+0000 7f31237fe700 1 -- 172.21.15.94:0/3689427217 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f311800b860 con 0x7f3124102de0 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.893+0000 7f31237fe700 1 -- 172.21.15.94:0/3689427217 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f311800be90 con 0x7f3124102de0 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.893+0000 7f31237fe700 1 -- 172.21.15.94:0/3689427217 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31180103c0 con 0x7f3124102de0 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.893+0000 7f312ad9f700 1 -- 172.21.15.94:0/3689427217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 msgr2=0x7f31241031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.893+0000 7f312ad9f700 1 --2- 172.21.15.94:0/3689427217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241031f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31180098e0 tx=0x7f3118009250 comp rx=0 tx=0).stop 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.893+0000 7f312ad9f700 1 -- 172.21.15.94:0/3689427217 shutdown_connections 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.893+0000 7f312ad9f700 1 --2- 172.21.15.94:0/3689427217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241031f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.893+0000 7f312ad9f700 1 -- 172.21.15.94:0/3689427217 >> 172.21.15.94:0/3689427217 conn(0x7f31240fcf40 msgr2=0x7f31240ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.893+0000 7f312ad9f700 1 -- 172.21.15.94:0/3689427217 shutdown_connections 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.893+0000 7f312ad9f700 1 -- 172.21.15.94:0/3689427217 wait complete. 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.894+0000 7f312ad9f700 1 Processor -- start 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.894+0000 7f312ad9f700 1 -- start start 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.894+0000 7f312ad9f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.894+0000 7f312ad9f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3124111c20 con 0x7f3124102de0 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.894+0000 7f3128b3b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:15.027 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.895+0000 7f3128b3b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54482/0 (socket says 172.21.15.94:54482) 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.895+0000 7f3128b3b700 1 -- 172.21.15.94:0/2134574012 learned_addr learned my addr 172.21.15.94:0/2134574012 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.895+0000 7f3128b3b700 1 -- 172.21.15.94:0/2134574012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180094f0 con 0x7f3124102de0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.895+0000 7f3128b3b700 1 --2- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241116e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31180090b0 tx=0x7f3118009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.895+0000 7f31217fa700 1 -- 172.21.15.94:0/2134574012 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3118010840 con 0x7f3124102de0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.895+0000 7f31217fa700 1 -- 172.21.15.94:0/2134574012 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3118010e70 con 0x7f3124102de0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.895+0000 7f31217fa700 1 -- 172.21.15.94:0/2134574012 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3118024e40 con 0x7f3124102de0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.895+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3124111e20 con 0x7f3124102de0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.895+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3124112990 con 0x7f3124102de0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.896+0000 7f31217fa700 1 -- 172.21.15.94:0/2134574012 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f311801a510 con 0x7f3124102de0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.896+0000 7f31217fa700 1 --2- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f3114036790 0x7f3114038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.896+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31241122b0 con 0x7f3124102de0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.897+0000 7f31217fa700 1 -- 172.21.15.94:0/2134574012 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f311804a9a0 con 0x7f3124102de0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.897+0000 7f3123fff700 1 --2- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f3114036790 0x7f3114038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.897+0000 7f3123fff700 1 --2- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f3114036790 0x7f3114038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3110009990 tx=0x7f3110006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:15.028 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:13.902+0000 7f31217fa700 1 -- 172.21.15.94:0/2134574012 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f311802e440 con 0x7f3124102de0 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.107+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 --> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f312405f5f0 con 0x7f3114036790 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.331+0000 7f31217fa700 1 -- 172.21.15.94:0/2134574012 <== mgr.14162 v2:172.21.15.94:6800/2299971604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f312405f5f0 con 0x7f3114036790 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f3114036790 msgr2=0x7f3114038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 --2- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f3114036790 0x7f3114038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3110009990 tx=0x7f3110006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 msgr2=0x7f31241116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 --2- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241116e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31180090b0 tx=0x7f3118009250 comp rx=0 tx=0).stop 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 shutdown_connections 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 --2- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f3114036790 0x7f3114038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 --2- 172.21.15.94:0/2134574012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3124102de0 0x7f31241116e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 >> 172.21.15.94:0/2134574012 conn(0x7f31240fcf40 msgr2=0x7f31240fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 shutdown_connections 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:14.333+0000 7f312ad9f700 1 -- 172.21.15.94:0/2134574012 wait complete. 2024-04-06T03:32:15.029 INFO:teuthology.orchestra.run.smithi094.stdout:Fetching dashboard port number... 2024-04-06T03:32:15.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:15 smithi094 ceph-mon[29713]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:32:15.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:15 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:16.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:16 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2848799647' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-06T03:32:16.696 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 8443 2024-04-06T03:32:16.696 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.911+0000 7fb3ab0b0700 1 Processor -- start 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.911+0000 7fb3ab0b0700 1 -- start start 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.911+0000 7fb3ab0b0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4105550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.911+0000 7fb3ab0b0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a4076f50 con 0x7fb3a4076b40 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.911+0000 7fb3a8e4c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.911+0000 7fb3a8e4c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4105550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54484/0 (socket says 172.21.15.94:54484) 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.911+0000 7fb3a8e4c700 1 -- 172.21.15.94:0/297933718 learned_addr learned my addr 172.21.15.94:0/297933718 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.911+0000 7fb3a8e4c700 1 -- 172.21.15.94:0/297933718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a4105a90 con 0x7fb3a4076b40 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3a8e4c700 1 --2- 172.21.15.94:0/297933718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4105550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb3980098e0 tx=0x7fb398009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cc477f6ba60210 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3a37fe700 1 -- 172.21.15.94:0/297933718 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb39800b860 con 0x7fb3a4076b40 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3a37fe700 1 -- 172.21.15.94:0/297933718 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb39800be90 con 0x7fb3a4076b40 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3a37fe700 1 -- 172.21.15.94:0/297933718 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3980103c0 con 0x7fb3a4076b40 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/297933718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 msgr2=0x7fb3a4105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:16.697 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3ab0b0700 1 --2- 172.21.15.94:0/297933718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4105550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb3980098e0 tx=0x7fb398009250 comp rx=0 tx=0).stop 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/297933718 shutdown_connections 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3ab0b0700 1 --2- 172.21.15.94:0/297933718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4105550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/297933718 >> 172.21.15.94:0/297933718 conn(0x7fb3a40fcf40 msgr2=0x7fb3a40ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/297933718 shutdown_connections 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.912+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/297933718 wait complete. 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.913+0000 7fb3ab0b0700 1 Processor -- start 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.913+0000 7fb3ab0b0700 1 -- start start 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.913+0000 7fb3ab0b0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4111700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.913+0000 7fb3ab0b0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a4111c40 con 0x7fb3a4076b40 2024-04-06T03:32:16.698 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.913+0000 7fb3a8e4c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4111700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.913+0000 7fb3a8e4c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4111700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54498/0 (socket says 172.21.15.94:54498) 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.913+0000 7fb3a8e4c700 1 -- 172.21.15.94:0/2848799647 learned_addr learned my addr 172.21.15.94:0/2848799647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.914+0000 7fb3a8e4c700 1 -- 172.21.15.94:0/2848799647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3980094f0 con 0x7fb3a4076b40 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.914+0000 7fb3a8e4c700 1 --2- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4111700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb398006d40 tx=0x7fb398009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.914+0000 7fb3a17fa700 1 -- 172.21.15.94:0/2848799647 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb398010840 con 0x7fb3a4076b40 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.914+0000 7fb3a17fa700 1 -- 172.21.15.94:0/2848799647 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb398010e70 con 0x7fb3a4076b40 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.914+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a4111e40 con 0x7fb3a4076b40 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.914+0000 7fb3a17fa700 1 -- 172.21.15.94:0/2848799647 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb398024ea0 con 0x7fb3a4076b40 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.914+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a41129b0 con 0x7fb3a4076b40 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.915+0000 7fb3a17fa700 1 -- 172.21.15.94:0/2848799647 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fb398024780 con 0x7fb3a4076b40 2024-04-06T03:32:16.699 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.915+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a405f5f0 con 0x7fb3a4076b40 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.915+0000 7fb3a17fa700 1 --2- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb394036790 0x7fb394038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.915+0000 7fb3a17fa700 1 -- 172.21.15.94:0/2848799647 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb39804a800 con 0x7fb3a4076b40 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.915+0000 7fb3a3fff700 1 --2- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb394036790 0x7fb394038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.915+0000 7fb3a3fff700 1 --2- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb394036790 0x7fb394038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb390009990 tx=0x7fb390006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:15.920+0000 7fb3a17fa700 1 -- 172.21.15.94:0/2848799647 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb39802e440 con 0x7fb3a4076b40 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.104+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb3a404bed0 con 0x7fb3a4076b40 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.105+0000 7fb3a17fa700 1 -- 172.21.15.94:0/2848799647 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fb39802c340 con 0x7fb3a4076b40 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb394036790 msgr2=0x7fb394038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 --2- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb394036790 0x7fb394038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb390009990 tx=0x7fb390006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 msgr2=0x7fb3a4111700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:16.700 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 --2- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4111700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb398006d40 tx=0x7fb398009250 comp rx=0 tx=0).stop 2024-04-06T03:32:16.701 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 shutdown_connections 2024-04-06T03:32:16.701 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 --2- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb394036790 0x7fb394038c40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:16.701 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 --2- 172.21.15.94:0/2848799647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3a4076b40 0x7fb3a4111700 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:16.701 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 >> 172.21.15.94:0/2848799647 conn(0x7fb3a40fcf40 msgr2=0x7fb3a40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:16.701 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 shutdown_connections 2024-04-06T03:32:16.701 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:16.107+0000 7fb3ab0b0700 1 -- 172.21.15.94:0/2848799647 wait complete. 2024-04-06T03:32:16.703 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-06T03:32:16.703 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-06T03:32:16.710 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-06T03:32:16.710 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-06T03:32:16.710 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-06T03:32:16.711 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-06T03:32:16.712 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph Dashboard is now available at: 2024-04-06T03:32:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: URL: https://smithi094.front.sepia.ceph.com:8443/ 2024-04-06T03:32:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: User: admin 2024-04-06T03:32:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: Password: 0nhvcr3yfn 2024-04-06T03:32:16.713 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:16.713 INFO:teuthology.orchestra.run.smithi094.stdout:Saving cluster configuration to /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config directory 2024-04-06T03:32:16.713 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling autotune for osd_memory_target 2024-04-06T03:32:17.903 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.129+0000 7fc6344fe700 1 Processor -- start 2024-04-06T03:32:17.903 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.129+0000 7fc6344fe700 1 -- start start 2024-04-06T03:32:17.903 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc6344fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c105500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc6344fe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c105a40 con 0x7fc62c0712a0 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc63229a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc63229a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c105500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54514/0 (socket says 172.21.15.94:54514) 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc63229a700 1 -- 172.21.15.94:0/74038007 learned_addr learned my addr 172.21.15.94:0/74038007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc63229a700 1 -- 172.21.15.94:0/74038007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c105b80 con 0x7fc62c0712a0 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc63229a700 1 --2- 172.21.15.94:0/74038007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c105500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098e0 tx=0x7fc628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b002bc7e48a295 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc631298700 1 -- 172.21.15.94:0/74038007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc62800b860 con 0x7fc62c0712a0 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc631298700 1 -- 172.21.15.94:0/74038007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc62800be90 con 0x7fc62c0712a0 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.130+0000 7fc631298700 1 -- 172.21.15.94:0/74038007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6280103c0 con 0x7fc62c0712a0 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.131+0000 7fc6344fe700 1 -- 172.21.15.94:0/74038007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 msgr2=0x7fc62c105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.131+0000 7fc6344fe700 1 --2- 172.21.15.94:0/74038007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c105500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098e0 tx=0x7fc628009250 comp rx=0 tx=0).stop 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.131+0000 7fc6344fe700 1 -- 172.21.15.94:0/74038007 shutdown_connections 2024-04-06T03:32:17.904 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.131+0000 7fc6344fe700 1 --2- 172.21.15.94:0/74038007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c105500 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.131+0000 7fc6344fe700 1 -- 172.21.15.94:0/74038007 >> 172.21.15.94:0/74038007 conn(0x7fc62c0fcfb0 msgr2=0x7fc62c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.131+0000 7fc6344fe700 1 -- 172.21.15.94:0/74038007 shutdown_connections 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.131+0000 7fc6344fe700 1 -- 172.21.15.94:0/74038007 wait complete. 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc6344fe700 1 Processor -- start 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc6344fe700 1 -- start start 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc6344fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c111770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc6344fe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c111cb0 con 0x7fc62c0712a0 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc63229a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c111770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc63229a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c111770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54526/0 (socket says 172.21.15.94:54526) 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc63229a700 1 -- 172.21.15.94:0/2134924141 learned_addr learned my addr 172.21.15.94:0/2134924141 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc63229a700 1 -- 172.21.15.94:0/2134924141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6280094f0 con 0x7fc62c0712a0 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc63229a700 1 --2- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c111770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6280094c0 tx=0x7fc628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.132+0000 7fc622ffd700 1 -- 172.21.15.94:0/2134924141 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6280118f0 con 0x7fc62c0712a0 2024-04-06T03:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.133+0000 7fc622ffd700 1 -- 172.21.15.94:0/2134924141 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc628010d60 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.133+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c111eb0 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.133+0000 7fc622ffd700 1 -- 172.21.15.94:0/2134924141 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc62801a430 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.133+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c112a20 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.133+0000 7fc622ffd700 1 -- 172.21.15.94:0/2134924141 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fc628024440 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.134+0000 7fc622ffd700 1 --2- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc618036830 0x7fc618038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.134+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c05f5f0 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.134+0000 7fc631a99700 1 --2- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc618036830 0x7fc618038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.134+0000 7fc622ffd700 1 -- 172.21.15.94:0/2134924141 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc62804a660 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.134+0000 7fc631a99700 1 --2- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc618036830 0x7fc618038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009990 tx=0x7fc61c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.138+0000 7fc622ffd700 1 -- 172.21.15.94:0/2134924141 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc628024760 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.324+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fc62c04bed0 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.324+0000 7fc622ffd700 1 -- 172.21.15.94:0/2134924141 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc628010ed0 con 0x7fc62c0712a0 2024-04-06T03:32:17.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc618036830 msgr2=0x7fc618038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:17.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 --2- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc618036830 0x7fc618038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009990 tx=0x7fc61c006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:17.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 msgr2=0x7fc62c111770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:17.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 --2- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c111770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6280094c0 tx=0x7fc628009250 comp rx=0 tx=0).stop 2024-04-06T03:32:17.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 shutdown_connections 2024-04-06T03:32:17.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 --2- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc618036830 0x7fc618038ce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:17.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 --2- 172.21.15.94:0/2134924141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc62c0712a0 0x7fc62c111770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:17.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 >> 172.21.15.94:0/2134924141 conn(0x7fc62c0fcfb0 msgr2=0x7fc62c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:17.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 shutdown_connections 2024-04-06T03:32:17.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:17.326+0000 7fc6344fe700 1 -- 172.21.15.94:0/2134924141 wait complete. 2024-04-06T03:32:18.604 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:18 smithi094 ceph-mon[29713]: mgrmap e13: smithi094.purlzp(active, since 6s) 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.308+0000 7f1d840fa700 1 Processor -- start 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.308+0000 7f1d840fa700 1 -- start start 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.308+0000 7f1d840fa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c101090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.308+0000 7f1d840fa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c1015d0 con 0x7f1d7c100c80 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.309+0000 7f1d81e96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.309+0000 7f1d81e96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51300/0 (socket says 172.21.15.94:51300) 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.309+0000 7f1d81e96700 1 -- 172.21.15.94:0/1115031922 learned_addr learned my addr 172.21.15.94:0/1115031922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.309+0000 7f1d81e96700 1 -- 172.21.15.94:0/1115031922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c101710 con 0x7f1d7c100c80 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.309+0000 7f1d81e96700 1 --2- 172.21.15.94:0/1115031922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c101090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d780098e0 tx=0x7f1d78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe89a53603583ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.309+0000 7f1d80e94700 1 -- 172.21.15.94:0/1115031922 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d7800b860 con 0x7f1d7c100c80 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.309+0000 7f1d80e94700 1 -- 172.21.15.94:0/1115031922 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d7800be90 con 0x7f1d7c100c80 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.309+0000 7f1d80e94700 1 -- 172.21.15.94:0/1115031922 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d780103c0 con 0x7f1d7c100c80 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.310+0000 7f1d840fa700 1 -- 172.21.15.94:0/1115031922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 msgr2=0x7f1d7c101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.310+0000 7f1d840fa700 1 --2- 172.21.15.94:0/1115031922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c101090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d780098e0 tx=0x7f1d78009250 comp rx=0 tx=0).stop 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.310+0000 7f1d840fa700 1 -- 172.21.15.94:0/1115031922 shutdown_connections 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.310+0000 7f1d840fa700 1 --2- 172.21.15.94:0/1115031922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c101090 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.310+0000 7f1d840fa700 1 -- 172.21.15.94:0/1115031922 >> 172.21.15.94:0/1115031922 conn(0x7f1d7c0fc700 msgr2=0x7f1d7c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.310+0000 7f1d840fa700 1 -- 172.21.15.94:0/1115031922 shutdown_connections 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.310+0000 7f1d840fa700 1 -- 172.21.15.94:0/1115031922 wait complete. 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.310+0000 7f1d840fa700 1 Processor -- start 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.310+0000 7f1d840fa700 1 -- start start 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d840fa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c1036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d840fa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c103bf0 con 0x7f1d7c100c80 2024-04-06T03:32:19.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d81e96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c1036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d81e96700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c1036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51308/0 (socket says 172.21.15.94:51308) 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d81e96700 1 -- 172.21.15.94:0/315519524 learned_addr learned my addr 172.21.15.94:0/315519524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d81e96700 1 -- 172.21.15.94:0/315519524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d780094f0 con 0x7f1d7c100c80 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d81e96700 1 --2- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c1036b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d78011680 tx=0x7f1d78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d727fc700 1 -- 172.21.15.94:0/315519524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d78010890 con 0x7f1d7c100c80 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d727fc700 1 -- 172.21.15.94:0/315519524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d78010ec0 con 0x7f1d7c100c80 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c101c60 con 0x7f1d7c100c80 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d727fc700 1 -- 172.21.15.94:0/315519524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d780243f0 con 0x7f1d7c100c80 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.311+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c102150 con 0x7f1d7c100c80 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.312+0000 7f1d727fc700 1 -- 172.21.15.94:0/315519524 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f1d7801a8c0 con 0x7f1d7c100c80 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.312+0000 7f1d727fc700 1 --2- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1d680368d0 0x7f1d68038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.312+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c04bed0 con 0x7f1d7c100c80 2024-04-06T03:32:19.138 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.312+0000 7f1d727fc700 1 -- 172.21.15.94:0/315519524 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1d7804a3f0 con 0x7f1d7c100c80 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.312+0000 7f1d81695700 1 --2- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1d680368d0 0x7f1d68038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.313+0000 7f1d81695700 1 --2- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1d680368d0 0x7f1d68038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009990 tx=0x7f1d6c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.317+0000 7f1d727fc700 1 -- 172.21.15.94:0/315519524 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1d7802b8c0 con 0x7f1d7c100c80 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.604+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f1d7c1025d0 con 0x7f1d7c100c80 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.606+0000 7f1d727fc700 1 -- 172.21.15.94:0/315519524 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f1d7802b2f0 con 0x7f1d7c100c80 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1d680368d0 msgr2=0x7f1d68038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 --2- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1d680368d0 0x7f1d68038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009990 tx=0x7f1d6c006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 msgr2=0x7f1d7c1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 --2- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c1036b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d78011680 tx=0x7f1d78009250 comp rx=0 tx=0).stop 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 shutdown_connections 2024-04-06T03:32:19.139 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 --2- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1d680368d0 0x7f1d68038d80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 --2- 172.21.15.94:0/315519524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c100c80 0x7f1d7c1036b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 >> 172.21.15.94:0/315519524 conn(0x7f1d7c0fc700 msgr2=0x7f1d7c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 shutdown_connections 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-06T03:32:18.608+0000 7f1d840fa700 1 -- 172.21.15.94:0/315519524 wait complete. 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout:Or, if you are only running a single cluster on this host: 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-06T03:32:19.140 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:19.141 INFO:teuthology.orchestra.run.smithi094.stdout: ceph telemetry on 2024-04-06T03:32:19.141 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:19.141 INFO:teuthology.orchestra.run.smithi094.stdout:For more information see: 2024-04-06T03:32:19.141 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:19.141 INFO:teuthology.orchestra.run.smithi094.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-04-06T03:32:19.141 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:19.141 INFO:teuthology.orchestra.run.smithi094.stdout:Bootstrap complete. 2024-04-06T03:32:19.172 INFO:tasks.cephadm:Fetching config... 2024-04-06T03:32:19.172 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:32:19.172 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-06T03:32:19.228 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-06T03:32:19.229 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:32:19.229 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-06T03:32:19.284 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-06T03:32:19.284 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:32:19.284 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/keyring of=/dev/stdout 2024-04-06T03:32:19.351 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-06T03:32:19.351 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:32:19.351 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-06T03:32:19.407 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-06T03:32:19.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD1dFF4UvsgVnmcfvTvz4ywKMZjIeyVmI5zqMjklKuzW8tUdQhlimnWVe0V6SM3yAPj7QJRl9HWGOloaCzxzTTZQa8/wfwYaonYqFT6Q7yFeWVPSB+NBVorWxUJde0AY4PHv9DaArrsJuonjPPQfPp9Y9rUyGdjUe8ERR7A27Cdx1r/2NMisBc1J8Ha93Vk6UjaOmX4Q1DW8niKOmI7Y5NvCKuR/MBxxJKNEQIzDi0LuU+b2+aY+kE158Be83Egns4jE/EsdLAYhnIbL6a302Kc49yjuu8fzas5O2Tw8mnS7z8pt2DIMUvV4pB1TWEOHcAbQ0d+bsBvv3ys8wZhiOW4LIBe8tFB65MeRKTcR/N/1hztsTXyOdC8g8VOmhi60P17WjJs//BGdbmAPAFQpYdB9s/gtP8vU/Rfmzb4Ds6rd2aYVg0iOFw3H+0J+SpBC2NlcYwaZkBxTpfpRuzojNd4avO9OjfmqbMs0FF3QF3ZI5mWDCGu1LoxesrOpcF1sF0= ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-06T03:32:19.487 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD1dFF4UvsgVnmcfvTvz4ywKMZjIeyVmI5zqMjklKuzW8tUdQhlimnWVe0V6SM3yAPj7QJRl9HWGOloaCzxzTTZQa8/wfwYaonYqFT6Q7yFeWVPSB+NBVorWxUJde0AY4PHv9DaArrsJuonjPPQfPp9Y9rUyGdjUe8ERR7A27Cdx1r/2NMisBc1J8Ha93Vk6UjaOmX4Q1DW8niKOmI7Y5NvCKuR/MBxxJKNEQIzDi0LuU+b2+aY+kE158Be83Egns4jE/EsdLAYhnIbL6a302Kc49yjuu8fzas5O2Tw8mnS7z8pt2DIMUvV4pB1TWEOHcAbQ0d+bsBvv3ys8wZhiOW4LIBe8tFB65MeRKTcR/N/1hztsTXyOdC8g8VOmhi60P17WjJs//BGdbmAPAFQpYdB9s/gtP8vU/Rfmzb4Ds6rd2aYVg0iOFw3H+0J+SpBC2NlcYwaZkBxTpfpRuzojNd4avO9OjfmqbMs0FF3QF3ZI5mWDCGu1LoxesrOpcF1sF0= ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:32:19.501 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD1dFF4UvsgVnmcfvTvz4ywKMZjIeyVmI5zqMjklKuzW8tUdQhlimnWVe0V6SM3yAPj7QJRl9HWGOloaCzxzTTZQa8/wfwYaonYqFT6Q7yFeWVPSB+NBVorWxUJde0AY4PHv9DaArrsJuonjPPQfPp9Y9rUyGdjUe8ERR7A27Cdx1r/2NMisBc1J8Ha93Vk6UjaOmX4Q1DW8niKOmI7Y5NvCKuR/MBxxJKNEQIzDi0LuU+b2+aY+kE158Be83Egns4jE/EsdLAYhnIbL6a302Kc49yjuu8fzas5O2Tw8mnS7z8pt2DIMUvV4pB1TWEOHcAbQ0d+bsBvv3ys8wZhiOW4LIBe8tFB65MeRKTcR/N/1hztsTXyOdC8g8VOmhi60P17WjJs//BGdbmAPAFQpYdB9s/gtP8vU/Rfmzb4Ds6rd2aYVg0iOFw3H+0J+SpBC2NlcYwaZkBxTpfpRuzojNd4avO9OjfmqbMs0FF3QF3ZI5mWDCGu1LoxesrOpcF1sF0= ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-06T03:32:19.548 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD1dFF4UvsgVnmcfvTvz4ywKMZjIeyVmI5zqMjklKuzW8tUdQhlimnWVe0V6SM3yAPj7QJRl9HWGOloaCzxzTTZQa8/wfwYaonYqFT6Q7yFeWVPSB+NBVorWxUJde0AY4PHv9DaArrsJuonjPPQfPp9Y9rUyGdjUe8ERR7A27Cdx1r/2NMisBc1J8Ha93Vk6UjaOmX4Q1DW8niKOmI7Y5NvCKuR/MBxxJKNEQIzDi0LuU+b2+aY+kE158Be83Egns4jE/EsdLAYhnIbL6a302Kc49yjuu8fzas5O2Tw8mnS7z8pt2DIMUvV4pB1TWEOHcAbQ0d+bsBvv3ys8wZhiOW4LIBe8tFB65MeRKTcR/N/1hztsTXyOdC8g8VOmhi60P17WjJs//BGdbmAPAFQpYdB9s/gtP8vU/Rfmzb4Ds6rd2aYVg0iOFw3H+0J+SpBC2NlcYwaZkBxTpfpRuzojNd4avO9OjfmqbMs0FF3QF3ZI5mWDCGu1LoxesrOpcF1sF0= ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:32:19.563 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-06T03:32:19.815 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:32:19.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:19 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/315519524' entity='client.admin' 2024-04-06T03:32:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.125+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3616416237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 msgr2=0x7fc4bc0fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.125+0000 7fc4c1ff1700 1 --2- 172.21.15.94:0/3616416237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 0x7fc4bc0fbd70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009c60 tx=0x7fc4ac004d10 comp rx=0 tx=0).stop 2024-04-06T03:32:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.125+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3616416237 shutdown_connections 2024-04-06T03:32:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.125+0000 7fc4c1ff1700 1 --2- 172.21.15.94:0/3616416237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 0x7fc4bc0fbd70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.125+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3616416237 >> 172.21.15.94:0/3616416237 conn(0x7fc4bc0f70c0 msgr2=0x7fc4bc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.125+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3616416237 shutdown_connections 2024-04-06T03:32:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.125+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3616416237 wait complete. 2024-04-06T03:32:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.125+0000 7fc4c1ff1700 1 Processor -- start 2024-04-06T03:32:22.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.126+0000 7fc4c1ff1700 1 -- start start 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.126+0000 7fc4c1ff1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 0x7fc4bc066480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.126+0000 7fc4c1ff1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc10fd90 con 0x7fc4bc0fb960 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.126+0000 7fc4bb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 0x7fc4bc066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.126+0000 7fc4bb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 0x7fc4bc066480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51334/0 (socket says 172.21.15.94:51334) 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.126+0000 7fc4bb7fe700 1 -- 172.21.15.94:0/3994425871 learned_addr learned my addr 172.21.15.94:0/3994425871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.126+0000 7fc4bb7fe700 1 -- 172.21.15.94:0/3994425871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac009940 con 0x7fc4bc0fb960 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.126+0000 7fc4bb7fe700 1 --2- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 0x7fc4bc066480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009500 tx=0x7fc4ac004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.126+0000 7fc4a3fff700 1 -- 172.21.15.94:0/3994425871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4ac007c80 con 0x7fc4bc0fb960 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.127+0000 7fc4a3fff700 1 -- 172.21.15.94:0/3994425871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4ac01d910 con 0x7fc4bc0fb960 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.127+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc10fed0 con 0x7fc4bc0fb960 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.127+0000 7fc4a3fff700 1 -- 172.21.15.94:0/3994425871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4ac0129d0 con 0x7fc4bc0fb960 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.127+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc1103c0 con 0x7fc4bc0fb960 2024-04-06T03:32:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.127+0000 7fc4a3fff700 1 -- 172.21.15.94:0/3994425871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fc4ac019040 con 0x7fc4bc0fb960 2024-04-06T03:32:22.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.128+0000 7fc4a3fff700 1 --2- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4a4036880 0x7fc4a4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:22.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.127+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc105920 con 0x7fc4bc0fb960 2024-04-06T03:32:22.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.128+0000 7fc4a3fff700 1 -- 172.21.15.94:0/3994425871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc4ac04a340 con 0x7fc4bc0fb960 2024-04-06T03:32:22.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.128+0000 7fc4baffd700 1 --2- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4a4036880 0x7fc4a4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:22.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.128+0000 7fc4baffd700 1 --2- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4a4036880 0x7fc4a4038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009990 tx=0x7fc4b0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:22.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.132+0000 7fc4a3fff700 1 -- 172.21.15.94:0/3994425871 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc4ac024310 con 0x7fc4bc0fb960 2024-04-06T03:32:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.324+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fc4bc05f5f0 con 0x7fc4bc0fb960 2024-04-06T03:32:22.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.328+0000 7fc4a3fff700 1 -- 172.21.15.94:0/3994425871 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc4ac017020 con 0x7fc4bc0fb960 2024-04-06T03:32:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4a4036880 msgr2=0x7fc4a4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 --2- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4a4036880 0x7fc4a4038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009990 tx=0x7fc4b0006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 msgr2=0x7fc4bc066480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 --2- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 0x7fc4bc066480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009500 tx=0x7fc4ac004d10 comp rx=0 tx=0).stop 2024-04-06T03:32:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 shutdown_connections 2024-04-06T03:32:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 --2- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4a4036880 0x7fc4a4038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 --2- 172.21.15.94:0/3994425871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc0fb960 0x7fc4bc066480 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 >> 172.21.15.94:0/3994425871 conn(0x7fc4bc0f70c0 msgr2=0x7fc4bc102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 shutdown_connections 2024-04-06T03:32:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:22.330+0000 7fc4c1ff1700 1 -- 172.21.15.94:0/3994425871 wait complete. 2024-04-06T03:32:23.148 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-06T03:32:23.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-06T03:32:23.396 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:32:23.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:23 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3994425871' entity='client.admin' 2024-04-06T03:32:24.994 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:24 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:24.995 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:24 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:24.995 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:24 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:32:24.995 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:24 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:24.995 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:24 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:32:24.995 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:24 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-06T03:32:24.995 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:24 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:32:24.995 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:24 smithi094 ceph-mon[29713]: Deploying daemon crash.smithi094 on smithi094 2024-04-06T03:32:26.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.284+0000 7fc43d04e700 1 -- 172.21.15.94:0/4078236743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 msgr2=0x7fc4380fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:26.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.284+0000 7fc43d04e700 1 --2- 172.21.15.94:0/4078236743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 0x7fc4380fe4c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc4280048c0 tx=0x7fc428009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:26.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.285+0000 7fc43d04e700 1 -- 172.21.15.94:0/4078236743 shutdown_connections 2024-04-06T03:32:26.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.285+0000 7fc43d04e700 1 --2- 172.21.15.94:0/4078236743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 0x7fc4380fe4c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:26.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.285+0000 7fc43d04e700 1 -- 172.21.15.94:0/4078236743 >> 172.21.15.94:0/4078236743 conn(0x7fc4380f6850 msgr2=0x7fc4380f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:26.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.285+0000 7fc43d04e700 1 -- 172.21.15.94:0/4078236743 shutdown_connections 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.285+0000 7fc43d04e700 1 -- 172.21.15.94:0/4078236743 wait complete. 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.285+0000 7fc43d04e700 1 Processor -- start 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc43d04e700 1 -- start start 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc43d04e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 0x7fc4380fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc43d04e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380fdd00 con 0x7fc4380fe0b0 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc436d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 0x7fc4380fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc436d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 0x7fc4380fd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51354/0 (socket says 172.21.15.94:51354) 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc436d9d700 1 -- 172.21.15.94:0/2442323159 learned_addr learned my addr 172.21.15.94:0/2442323159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc436d9d700 1 -- 172.21.15.94:0/2442323159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428005040 con 0x7fc4380fe0b0 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc436d9d700 1 --2- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 0x7fc4380fd7c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4280096b0 tx=0x7fc428009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc42f7fe700 1 -- 172.21.15.94:0/2442323159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc42800bed0 con 0x7fc4380fe0b0 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.286+0000 7fc42f7fe700 1 -- 172.21.15.94:0/2442323159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc42800b5c0 con 0x7fc4380fe0b0 2024-04-06T03:32:26.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.287+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4380fbdd0 con 0x7fc4380fe0b0 2024-04-06T03:32:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.287+0000 7fc42f7fe700 1 -- 172.21.15.94:0/2442323159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc42801c5b0 con 0x7fc4380fe0b0 2024-04-06T03:32:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.287+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4380fc2c0 con 0x7fc4380fe0b0 2024-04-06T03:32:26.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.288+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43805f5f0 con 0x7fc4380fe0b0 2024-04-06T03:32:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.288+0000 7fc42f7fe700 1 -- 172.21.15.94:0/2442323159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fc42801c710 con 0x7fc4380fe0b0 2024-04-06T03:32:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.288+0000 7fc42f7fe700 1 --2- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4240368d0 0x7fc424038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.288+0000 7fc42f7fe700 1 -- 172.21.15.94:0/2442323159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc4280499a0 con 0x7fc4380fe0b0 2024-04-06T03:32:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.288+0000 7fc43659c700 1 --2- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4240368d0 0x7fc424038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:26.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.288+0000 7fc43659c700 1 --2- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4240368d0 0x7fc424038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc420009990 tx=0x7fc420006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:26.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.293+0000 7fc42f7fe700 1 -- 172.21.15.94:0/2442323159 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc428025400 con 0x7fc4380fe0b0 2024-04-06T03:32:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.495+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 --> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fc438106db0 con 0x7fc4240368d0 2024-04-06T03:32:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.499+0000 7fc42f7fe700 1 -- 172.21.15.94:0/2442323159 <== mgr.14162 v2:172.21.15.94:6800/2299971604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc438106db0 con 0x7fc4240368d0 2024-04-06T03:32:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4240368d0 msgr2=0x7fc424038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 --2- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4240368d0 0x7fc424038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc420009990 tx=0x7fc420006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 msgr2=0x7fc4380fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 --2- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 0x7fc4380fd7c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4280096b0 tx=0x7fc428009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 shutdown_connections 2024-04-06T03:32:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 --2- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fc4240368d0 0x7fc424038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 --2- 172.21.15.94:0/2442323159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4380fe0b0 0x7fc4380fd7c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 >> 172.21.15.94:0/2442323159 conn(0x7fc4380f6850 msgr2=0x7fc4380f7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 shutdown_connections 2024-04-06T03:32:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:26.501+0000 7fc43d04e700 1 -- 172.21.15.94:0/2442323159 wait complete. 2024-04-06T03:32:27.181 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-04-06T03:32:27.181 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:32:27.181 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-04-06T03:32:27.199 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:32:27.199 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:32:27.257 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-04-06T03:32:27.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch host add smithi136 2024-04-06T03:32:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:32:27.769 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:27 smithi094 ceph-mon[29713]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:32:27.769 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:27 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:27.769 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:27 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:27.770 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:27 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:27.770 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:27 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:27.770 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:27 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:28.758 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:28 smithi094 ceph-mon[29713]: Deploying daemon node-exporter.smithi094 on smithi094 2024-04-06T03:32:29.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.085+0000 7fb44696d700 1 -- 172.21.15.94:0/859930806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 msgr2=0x7fb4400fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:29.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.085+0000 7fb44696d700 1 --2- 172.21.15.94:0/859930806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 0x7fb4400fce50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb4300048c0 tx=0x7fb430009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:29.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.085+0000 7fb44696d700 1 -- 172.21.15.94:0/859930806 shutdown_connections 2024-04-06T03:32:29.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.085+0000 7fb44696d700 1 --2- 172.21.15.94:0/859930806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 0x7fb4400fce50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.085+0000 7fb44696d700 1 -- 172.21.15.94:0/859930806 >> 172.21.15.94:0/859930806 conn(0x7fb4400f81c0 msgr2=0x7fb4400fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.085+0000 7fb44696d700 1 -- 172.21.15.94:0/859930806 shutdown_connections 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.085+0000 7fb44696d700 1 -- 172.21.15.94:0/859930806 wait complete. 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.086+0000 7fb44696d700 1 Processor -- start 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.086+0000 7fb44696d700 1 -- start start 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.086+0000 7fb44696d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 0x7fb44010d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.086+0000 7fb44696d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44010d600 con 0x7fb4400fca40 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.086+0000 7fb43ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 0x7fb44010d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.087+0000 7fb43ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 0x7fb44010d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42578/0 (socket says 172.21.15.94:42578) 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.087+0000 7fb43ffff700 1 -- 172.21.15.94:0/3148745003 learned_addr learned my addr 172.21.15.94:0/3148745003 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.087+0000 7fb43ffff700 1 -- 172.21.15.94:0/3148745003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb430005040 con 0x7fb4400fca40 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.087+0000 7fb43ffff700 1 --2- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 0x7fb44010d0c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb430009680 tx=0x7fb430009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.087+0000 7fb43cff9700 1 -- 172.21.15.94:0/3148745003 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb43000b5c0 con 0x7fb4400fca40 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.087+0000 7fb43cff9700 1 -- 172.21.15.94:0/3148745003 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb43000be70 con 0x7fb4400fca40 2024-04-06T03:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.087+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44010d800 con 0x7fb4400fca40 2024-04-06T03:32:29.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.087+0000 7fb43cff9700 1 -- 172.21.15.94:0/3148745003 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4300135a0 con 0x7fb4400fca40 2024-04-06T03:32:29.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.087+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44010e370 con 0x7fb4400fca40 2024-04-06T03:32:29.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.088+0000 7fb43cff9700 1 -- 172.21.15.94:0/3148745003 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb430013860 con 0x7fb4400fca40 2024-04-06T03:32:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.088+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb440102f50 con 0x7fb4400fca40 2024-04-06T03:32:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.088+0000 7fb43cff9700 1 --2- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb42c036880 0x7fb42c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.088+0000 7fb43cff9700 1 -- 172.21.15.94:0/3148745003 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb430049370 con 0x7fb4400fca40 2024-04-06T03:32:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.088+0000 7fb43f7fe700 1 --2- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb42c036880 0x7fb42c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.089+0000 7fb43f7fe700 1 --2- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb42c036880 0x7fb42c038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb428009990 tx=0x7fb428006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:29.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.093+0000 7fb43cff9700 1 -- 172.21.15.94:0/3148745003 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb430011960 con 0x7fb4400fca40 2024-04-06T03:32:29.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:29.281+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 --> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}) v1 -- 0x7fb44010e5b0 con 0x7fb42c036880 2024-04-06T03:32:30.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:30 smithi094 ceph-mon[29713]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:32:31.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.415+0000 7fb43cff9700 1 -- 172.21.15.94:0/3148745003 <== mgr.14162 v2:172.21.15.94:6800/2299971604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb44010e5b0 con 0x7fb42c036880 2024-04-06T03:32:31.418 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-04-06T03:32:31.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb42c036880 msgr2=0x7fb42c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:31.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 --2- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb42c036880 0x7fb42c038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb428009990 tx=0x7fb428006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:31.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 msgr2=0x7fb44010d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 --2- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 0x7fb44010d0c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb430009680 tx=0x7fb430009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 shutdown_connections 2024-04-06T03:32:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 --2- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fb42c036880 0x7fb42c038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 --2- 172.21.15.94:0/3148745003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4400fca40 0x7fb44010d0c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 >> 172.21.15.94:0/3148745003 conn(0x7fb4400f81c0 msgr2=0x7fb440103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 shutdown_connections 2024-04-06T03:32:31.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:31.417+0000 7fb44696d700 1 -- 172.21.15.94:0/3148745003 wait complete. 2024-04-06T03:32:31.633 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:31 smithi094 ceph-mon[29713]: Deploying cephadm binary to smithi136 2024-04-06T03:32:31.633 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:31 smithi094 ceph-mon[29713]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:31.633 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:31 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:31.633 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:31 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:32.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-06T03:32:32.531 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:32:32.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:32 smithi094 ceph-mon[29713]: Added host smithi136 2024-04-06T03:32:32.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:32 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:32.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:32 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:32.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:32 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:32.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:32 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:32.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:32 smithi094 ceph-mon[29713]: Deploying daemon alertmanager.smithi094 on smithi094 2024-04-06T03:32:34.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:33 smithi094 ceph-mon[29713]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:34.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.727+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/4079221608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 msgr2=0x7fa8f40ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.727+0000 7fa8f8a0a700 1 --2- 172.21.15.94:0/4079221608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 0x7fa8f40ffa30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40048c0 tx=0x7fa8e4009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.727+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/4079221608 shutdown_connections 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.727+0000 7fa8f8a0a700 1 --2- 172.21.15.94:0/4079221608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 0x7fa8f40ffa30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.727+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/4079221608 >> 172.21.15.94:0/4079221608 conn(0x7fa8f40f70d0 msgr2=0x7fa8f40f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.728+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/4079221608 shutdown_connections 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.728+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/4079221608 wait complete. 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.728+0000 7fa8f8a0a700 1 Processor -- start 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.728+0000 7fa8f8a0a700 1 -- start start 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.728+0000 7fa8f8a0a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 0x7fa8f410d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.728+0000 7fa8f8a0a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f410d5c0 con 0x7fa8f40fd650 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.728+0000 7fa8f259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 0x7fa8f410d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.728+0000 7fa8f259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 0x7fa8f410d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42596/0 (socket says 172.21.15.94:42596) 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.728+0000 7fa8f259c700 1 -- 172.21.15.94:0/2423530392 learned_addr learned my addr 172.21.15.94:0/2423530392 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:34.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.729+0000 7fa8f259c700 1 -- 172.21.15.94:0/2423530392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e4005040 con 0x7fa8f40fd650 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.729+0000 7fa8f259c700 1 --2- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 0x7fa8f410d080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009680 tx=0x7fa8e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.729+0000 7fa8e2ffd700 1 -- 172.21.15.94:0/2423530392 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8e400bed0 con 0x7fa8f40fd650 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.729+0000 7fa8e2ffd700 1 -- 172.21.15.94:0/2423530392 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8e400b5c0 con 0x7fa8f40fd650 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.729+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f410d7c0 con 0x7fa8f40fd650 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.729+0000 7fa8e2ffd700 1 -- 172.21.15.94:0/2423530392 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8e401c5b0 con 0x7fa8f40fd650 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.729+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f410e330 con 0x7fa8f40fd650 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.730+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f4102f60 con 0x7fa8f40fd650 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.730+0000 7fa8e2ffd700 1 -- 172.21.15.94:0/2423530392 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fa8e401c710 con 0x7fa8f40fd650 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.730+0000 7fa8e2ffd700 1 --2- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa8dc036880 0x7fa8dc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.730+0000 7fa8e2ffd700 1 -- 172.21.15.94:0/2423530392 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa8e4049a50 con 0x7fa8f40fd650 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.730+0000 7fa8f1d9b700 1 --2- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa8dc036880 0x7fa8dc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.731+0000 7fa8f1d9b700 1 --2- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa8dc036880 0x7fa8dc038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009990 tx=0x7fa8e8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:34.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.735+0000 7fa8e2ffd700 1 -- 172.21.15.94:0/2423530392 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa8e4025400 con 0x7fa8f40fd650 2024-04-06T03:32:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.968+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 --> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8f405f690 con 0x7fa8dc036880 2024-04-06T03:32:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.969+0000 7fa8e2ffd700 1 -- 172.21.15.94:0/2423530392 <== mgr.14162 v2:172.21.15.94:6800/2299971604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fa8f405f690 con 0x7fa8dc036880 2024-04-06T03:32:34.971 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:32:34.971 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-04-06T03:32:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.971+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa8dc036880 msgr2=0x7fa8dc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.971+0000 7fa8f8a0a700 1 --2- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa8dc036880 0x7fa8dc038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009990 tx=0x7fa8e8006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.971+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 msgr2=0x7fa8f410d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.971+0000 7fa8f8a0a700 1 --2- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 0x7fa8f410d080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009680 tx=0x7fa8e4009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.971+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 shutdown_connections 2024-04-06T03:32:34.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.971+0000 7fa8f8a0a700 1 --2- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa8dc036880 0x7fa8dc038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:34.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.971+0000 7fa8f8a0a700 1 --2- 172.21.15.94:0/2423530392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8f40fd650 0x7fa8f410d080 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:34.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.971+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 >> 172.21.15.94:0/2423530392 conn(0x7fa8f40f70d0 msgr2=0x7fa8f4103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:34.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.972+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 shutdown_connections 2024-04-06T03:32:34.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:34.972+0000 7fa8f8a0a700 1 -- 172.21.15.94:0/2423530392 wait complete. 2024-04-06T03:32:36.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:35 smithi094 ceph-mon[29713]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:36.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:35 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:36.735 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-06T03:32:36.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd crush tunables default 2024-04-06T03:32:37.161 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:36 smithi094 ceph-mon[29713]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-06T03:32:37.775 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:32:38.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:37 smithi094 ceph-mon[29713]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:40.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:39 smithi094 ceph-mon[29713]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:41.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.803+0000 7f98d616c700 1 -- 172.21.15.94:0/4012794065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 msgr2=0x7f98d00fc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:41.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.803+0000 7f98d616c700 1 --2- 172.21.15.94:0/4012794065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 0x7f98d00fc370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f98c00048c0 tx=0x7f98c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:41.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.803+0000 7f98d616c700 1 -- 172.21.15.94:0/4012794065 shutdown_connections 2024-04-06T03:32:41.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.803+0000 7f98d616c700 1 --2- 172.21.15.94:0/4012794065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 0x7f98d00fc370 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:41.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.803+0000 7f98d616c700 1 -- 172.21.15.94:0/4012794065 >> 172.21.15.94:0/4012794065 conn(0x7f98d00f6850 msgr2=0x7f98d00f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:41.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.803+0000 7f98d616c700 1 -- 172.21.15.94:0/4012794065 shutdown_connections 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.803+0000 7f98d616c700 1 -- 172.21.15.94:0/4012794065 wait complete. 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.803+0000 7f98d616c700 1 Processor -- start 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.804+0000 7f98d616c700 1 -- start start 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.804+0000 7f98d616c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 0x7f98d010d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.804+0000 7f98d616c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d010d630 con 0x7f98d00fbf60 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.804+0000 7f98cf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 0x7f98d010d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.804+0000 7f98cf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 0x7f98d010d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36010/0 (socket says 172.21.15.94:36010) 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.804+0000 7f98cf7fe700 1 -- 172.21.15.94:0/209175327 learned_addr learned my addr 172.21.15.94:0/209175327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.804+0000 7f98cf7fe700 1 -- 172.21.15.94:0/209175327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0005040 con 0x7f98d00fbf60 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.804+0000 7f98cf7fe700 1 --2- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 0x7f98d010d0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009680 tx=0x7f98c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.805+0000 7f98b7fff700 1 -- 172.21.15.94:0/209175327 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98c000bed0 con 0x7f98d00fbf60 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.805+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d010d830 con 0x7f98d00fbf60 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.805+0000 7f98b7fff700 1 -- 172.21.15.94:0/209175327 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98c000b5c0 con 0x7f98d00fbf60 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.805+0000 7f98b7fff700 1 -- 172.21.15.94:0/209175327 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98c001c5b0 con 0x7f98d00fbf60 2024-04-06T03:32:41.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.805+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d010e3a0 con 0x7f98d00fbf60 2024-04-06T03:32:41.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.805+0000 7f98b7fff700 1 -- 172.21.15.94:0/209175327 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f98c001c710 con 0x7f98d00fbf60 2024-04-06T03:32:41.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.806+0000 7f98b7fff700 1 --2- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f98b8036880 0x7f98b8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:41.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.806+0000 7f98b7fff700 1 -- 172.21.15.94:0/209175327 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f98c004aa20 con 0x7f98d00fbf60 2024-04-06T03:32:41.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.806+0000 7f98ceffd700 1 --2- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f98b8036880 0x7f98b8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:41.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.806+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d0101e70 con 0x7f98d00fbf60 2024-04-06T03:32:41.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.806+0000 7f98ceffd700 1 --2- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f98b8036880 0x7f98b8038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009990 tx=0x7f98c4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:41.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:41.811+0000 7f98b7fff700 1 -- 172.21.15.94:0/209175327 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f98c0025400 con 0x7f98d00fbf60 2024-04-06T03:32:42.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.014+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f98d0101740 con 0x7f98d00fbf60 2024-04-06T03:32:42.016 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:41 smithi094 ceph-mon[29713]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:42.016 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:41 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:42.016 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:41 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:42.017 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:41 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:42.017 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:41 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:42.017 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:41 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:42.017 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:41 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:42.017 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:41 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-06T03:32:42.017 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:41 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:42.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.924+0000 7f98b7fff700 1 -- 172.21.15.94:0/209175327 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f98c0011370 con 0x7f98d00fbf60 2024-04-06T03:32:42.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f98b8036880 msgr2=0x7f98b8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:42.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 --2- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f98b8036880 0x7f98b8038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009990 tx=0x7f98c4006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:42.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 msgr2=0x7f98d010d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:42.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 --2- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 0x7f98d010d0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009680 tx=0x7f98c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:42.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 shutdown_connections 2024-04-06T03:32:42.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 --2- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f98b8036880 0x7f98b8038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:42.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 --2- 172.21.15.94:0/209175327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98d00fbf60 0x7f98d010d0f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:42.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 >> 172.21.15.94:0/209175327 conn(0x7f98d00f6850 msgr2=0x7f98d00fabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:42.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 shutdown_connections 2024-04-06T03:32:42.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:32:42.927+0000 7f98d616c700 1 -- 172.21.15.94:0/209175327 wait complete. 2024-04-06T03:32:42.966 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-04-06T03:32:43.218 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:42 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-06T03:32:43.218 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:42 smithi094 ceph-mon[29713]: Deploying daemon grafana.smithi094 on smithi094 2024-04-06T03:32:43.218 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:42 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/209175327' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-06T03:32:43.578 INFO:tasks.cephadm:Adding mon.smithi094 on smithi094 2024-04-06T03:32:43.578 INFO:tasks.cephadm:Adding mon.smithi136 on smithi136 2024-04-06T03:32:43.579 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch apply mon '2;smithi094:172.21.15.94=smithi094;smithi136:172.21.15.136=smithi136' 2024-04-06T03:32:43.833 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:43.878 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:44.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:43 smithi094 ceph-mon[29713]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:44.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:43 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/209175327' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-06T03:32:44.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:43 smithi094 ceph-mon[29713]: osdmap e4: 0 total, 0 up, 0 in 2024-04-06T03:32:45.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.398+0000 7f2c0b35a700 1 -- 172.21.15.136:0/2449727755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 msgr2=0x7f2c040fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:45.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.398+0000 7f2c0b35a700 1 --2- 172.21.15.136:0/2449727755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 0x7f2c040fd460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40048c0 tx=0x7f2bf4009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.398+0000 7f2c0b35a700 1 -- 172.21.15.136:0/2449727755 shutdown_connections 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.398+0000 7f2c0b35a700 1 --2- 172.21.15.136:0/2449727755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 0x7f2c040fd460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.398+0000 7f2c0b35a700 1 -- 172.21.15.136:0/2449727755 >> 172.21.15.136:0/2449727755 conn(0x7f2c040f70d0 msgr2=0x7f2c040f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.398+0000 7f2c0b35a700 1 -- 172.21.15.136:0/2449727755 shutdown_connections 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.398+0000 7f2c0b35a700 1 -- 172.21.15.136:0/2449727755 wait complete. 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.399+0000 7f2c0b35a700 1 Processor -- start 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.399+0000 7f2c0b35a700 1 -- start start 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.399+0000 7f2c0b35a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 0x7f2c04065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.399+0000 7f2c0b35a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0410fde0 con 0x7f2c040fd050 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.399+0000 7f2c090f6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 0x7f2c04065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:45.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.399+0000 7f2c090f6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 0x7f2c04065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:40636/0 (socket says 172.21.15.136:40636) 2024-04-06T03:32:45.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.399+0000 7f2c090f6700 1 -- 172.21.15.136:0/157321404 learned_addr learned my addr 172.21.15.136:0/157321404 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:32:45.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.400+0000 7f2c090f6700 1 -- 172.21.15.136:0/157321404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf4005040 con 0x7f2c040fd050 2024-04-06T03:32:45.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.400+0000 7f2c090f6700 1 --2- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 0x7f2c04065bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40096b0 tx=0x7f2bf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:45.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.400+0000 7f2bf9ffb700 1 -- 172.21.15.136:0/157321404 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bf400bed0 con 0x7f2c040fd050 2024-04-06T03:32:45.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.400+0000 7f2bf9ffb700 1 -- 172.21.15.136:0/157321404 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bf400b5c0 con 0x7f2c040fd050 2024-04-06T03:32:45.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.400+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0410ff20 con 0x7f2c040fd050 2024-04-06T03:32:45.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.400+0000 7f2bf9ffb700 1 -- 172.21.15.136:0/157321404 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bf401c5b0 con 0x7f2c040fd050 2024-04-06T03:32:45.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.400+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c04110410 con 0x7f2c040fd050 2024-04-06T03:32:45.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.401+0000 7f2bf9ffb700 1 -- 172.21.15.136:0/157321404 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f2bf4013420 con 0x7f2c040fd050 2024-04-06T03:32:45.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.401+0000 7f2bf9ffb700 1 --2- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f2bf0036880 0x7f2bf0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:45.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.401+0000 7f2bf9ffb700 1 -- 172.21.15.136:0/157321404 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2bf4049ab0 con 0x7f2c040fd050 2024-04-06T03:32:45.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.401+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c04105920 con 0x7f2c040fd050 2024-04-06T03:32:45.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.401+0000 7f2c088f5700 1 --2- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f2bf0036880 0x7f2bf0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:45.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.402+0000 7f2c088f5700 1 --2- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f2bf0036880 0x7f2bf0038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009990 tx=0x7f2c00006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:45.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.406+0000 7f2bf9ffb700 1 -- 172.21.15.136:0/157321404 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2bf4011550 con 0x7f2c040fd050 2024-04-06T03:32:45.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.598+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 --> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi094:172.21.15.94=smithi094;smithi136:172.21.15.136=smithi136", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c0405e400 con 0x7f2bf0036880 2024-04-06T03:32:45.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.604+0000 7f2bf9ffb700 1 -- 172.21.15.136:0/157321404 <== mgr.14162 v2:172.21.15.94:6800/2299971604 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2c0405e400 con 0x7f2bf0036880 2024-04-06T03:32:45.605 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mon update... 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.605+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f2bf0036880 msgr2=0x7f2bf0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.605+0000 7f2c0b35a700 1 --2- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f2bf0036880 0x7f2bf0038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009990 tx=0x7f2c00006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.605+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 msgr2=0x7f2c04065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.605+0000 7f2c0b35a700 1 --2- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 0x7f2c04065bd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40096b0 tx=0x7f2bf4009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.606+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 shutdown_connections 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.606+0000 7f2c0b35a700 1 --2- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f2bf0036880 0x7f2bf0038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.606+0000 7f2c0b35a700 1 --2- 172.21.15.136:0/157321404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c040fd050 0x7f2c04065bd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.606+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 >> 172.21.15.136:0/157321404 conn(0x7f2c040f70d0 msgr2=0x7f2c04102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.606+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 shutdown_connections 2024-04-06T03:32:45.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:45.606+0000 7f2c0b35a700 1 -- 172.21.15.136:0/157321404 wait complete. 2024-04-06T03:32:46.196 DEBUG:teuthology.orchestra.run.smithi136:mon.smithi136> sudo journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi136.service 2024-04-06T03:32:46.198 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:32:46.199 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:32:46.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:45 smithi094 ceph-mon[29713]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:46.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:45 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:46.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:45 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:46.474 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:46.521 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:47.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:46 smithi094 ceph-mon[29713]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi094:172.21.15.94=smithi094;smithi136:172.21.15.136=smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:32:47.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:46 smithi094 ceph-mon[29713]: Saving service mon spec with placement smithi094:172.21.15.94=smithi094;smithi136:172.21.15.136=smithi136;count:2 2024-04-06T03:32:47.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.858+0000 7f70e1e84700 1 -- 172.21.15.136:0/4210026725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 msgr2=0x7f70dc0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:47.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.858+0000 7f70e1e84700 1 --2- 172.21.15.136:0/4210026725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 0x7f70dc0fce70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0048c0 tx=0x7f70cc009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.858+0000 7f70e1e84700 1 -- 172.21.15.136:0/4210026725 shutdown_connections 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.858+0000 7f70e1e84700 1 --2- 172.21.15.136:0/4210026725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 0x7f70dc0fce70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.858+0000 7f70e1e84700 1 -- 172.21.15.136:0/4210026725 >> 172.21.15.136:0/4210026725 conn(0x7f70dc0f81c0 msgr2=0x7f70dc0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.858+0000 7f70e1e84700 1 -- 172.21.15.136:0/4210026725 shutdown_connections 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.858+0000 7f70e1e84700 1 -- 172.21.15.136:0/4210026725 wait complete. 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.859+0000 7f70e1e84700 1 Processor -- start 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.859+0000 7f70e1e84700 1 -- start start 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.859+0000 7f70e1e84700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 0x7f70dc071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.859+0000 7f70e1e84700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0722a0 con 0x7f70dc0fca60 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.859+0000 7f70db7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 0x7f70dc071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.859+0000 7f70db7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 0x7f70dc071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:49184/0 (socket says 172.21.15.136:49184) 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.859+0000 7f70db7fe700 1 -- 172.21.15.136:0/1560167458 learned_addr learned my addr 172.21.15.136:0/1560167458 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:32:47.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.859+0000 7f70db7fe700 1 -- 172.21.15.136:0/1560167458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc005040 con 0x7f70dc0fca60 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.860+0000 7f70db7fe700 1 --2- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 0x7f70dc071d60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0096b0 tx=0x7f70cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.860+0000 7f70c3fff700 1 -- 172.21.15.136:0/1560167458 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70cc00bed0 con 0x7f70dc0fca60 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.860+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc070370 con 0x7f70dc0fca60 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.860+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc070860 con 0x7f70dc0fca60 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.860+0000 7f70c3fff700 1 -- 172.21.15.136:0/1560167458 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70cc00b5c0 con 0x7f70dc0fca60 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.860+0000 7f70c3fff700 1 -- 172.21.15.136:0/1560167458 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70cc0255c0 con 0x7f70dc0fca60 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.861+0000 7f70c3fff700 1 -- 172.21.15.136:0/1560167458 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f70cc01c3f0 con 0x7f70dc0fca60 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.861+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc05f5f0 con 0x7f70dc0fca60 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.861+0000 7f70c3fff700 1 --2- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f70c4036880 0x7f70c4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.861+0000 7f70c3fff700 1 -- 172.21.15.136:0/1560167458 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f70cc049a80 con 0x7f70dc0fca60 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.861+0000 7f70daffd700 1 --2- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f70c4036880 0x7f70c4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:47.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.862+0000 7f70daffd700 1 --2- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f70c4036880 0x7f70c4038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009990 tx=0x7f70d0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:47.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:47.866+0000 7f70c3fff700 1 -- 172.21.15.136:0/1560167458 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f70cc01a4d0 con 0x7f70dc0fca60 2024-04-06T03:32:48.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.126+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f70dc106440 con 0x7f70dc0fca60 2024-04-06T03:32:48.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.127+0000 7f70c3fff700 1 -- 172.21.15.136:0/1560167458 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f70cc01a2f0 con 0x7f70dc0fca60 2024-04-06T03:32:48.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:32:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:32:48.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f70c4036880 msgr2=0x7f70c4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:48.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 --2- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f70c4036880 0x7f70c4038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009990 tx=0x7f70d0006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:48.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 msgr2=0x7f70dc071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:48.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 --2- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 0x7f70dc071d60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0096b0 tx=0x7f70cc009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:48.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 shutdown_connections 2024-04-06T03:32:48.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 --2- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f70c4036880 0x7f70c4038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:48.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 --2- 172.21.15.136:0/1560167458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70dc0fca60 0x7f70dc071d60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:48.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 >> 172.21.15.136:0/1560167458 conn(0x7f70dc0f81c0 msgr2=0x7f70dc103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:48.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 shutdown_connections 2024-04-06T03:32:48.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:48.129+0000 7f70e1e84700 1 -- 172.21.15.136:0/1560167458 wait complete. 2024-04-06T03:32:48.131 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:32:48.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:47 smithi094 ceph-mon[29713]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:49.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:48 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/1560167458' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:32:49.649 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:32:49.649 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:32:49.890 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:49.937 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:50.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:49 smithi094 ceph-mon[29713]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.245+0000 7f4f34a49700 1 -- 172.21.15.136:0/3680034632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 msgr2=0x7f4f300fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.245+0000 7f4f34a49700 1 --2- 172.21.15.136:0/3680034632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 0x7f4f300fd450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f200048c0 tx=0x7f4f20009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.245+0000 7f4f34a49700 1 -- 172.21.15.136:0/3680034632 shutdown_connections 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.245+0000 7f4f34a49700 1 --2- 172.21.15.136:0/3680034632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 0x7f4f300fd450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.245+0000 7f4f34a49700 1 -- 172.21.15.136:0/3680034632 >> 172.21.15.136:0/3680034632 conn(0x7f4f300f70c0 msgr2=0x7f4f300f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.245+0000 7f4f34a49700 1 -- 172.21.15.136:0/3680034632 shutdown_connections 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.245+0000 7f4f34a49700 1 -- 172.21.15.136:0/3680034632 wait complete. 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f34a49700 1 Processor -- start 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f34a49700 1 -- start start 2024-04-06T03:32:51.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f34a49700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 0x7f4f300664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f34a49700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3010fd40 con 0x7f4f300fd040 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f2e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 0x7f4f300664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f2e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 0x7f4f300664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:49204/0 (socket says 172.21.15.136:49204) 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f2e59c700 1 -- 172.21.15.136:0/1672501604 learned_addr learned my addr 172.21.15.136:0/1672501604 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f2e59c700 1 -- 172.21.15.136:0/1672501604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f20005040 con 0x7f4f300fd040 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f2e59c700 1 --2- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 0x7f4f300664c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f200096b0 tx=0x7f4f20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f1effd700 1 -- 172.21.15.136:0/1672501604 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f2000bed0 con 0x7f4f300fd040 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.246+0000 7f4f1effd700 1 -- 172.21.15.136:0/1672501604 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f2000b5c0 con 0x7f4f300fd040 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.247+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3010fee0 con 0x7f4f300fd040 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.247+0000 7f4f1effd700 1 -- 172.21.15.136:0/1672501604 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f2001c5b0 con 0x7f4f300fd040 2024-04-06T03:32:51.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.247+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f30110370 con 0x7f4f300fd040 2024-04-06T03:32:51.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.247+0000 7f4f1effd700 1 -- 172.21.15.136:0/1672501604 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f4f20013420 con 0x7f4f300fd040 2024-04-06T03:32:51.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.247+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f30105910 con 0x7f4f300fd040 2024-04-06T03:32:51.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.248+0000 7f4f1effd700 1 --2- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f4f18036880 0x7f4f18038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:51.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.248+0000 7f4f1effd700 1 -- 172.21.15.136:0/1672501604 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4f20049a40 con 0x7f4f300fd040 2024-04-06T03:32:51.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.248+0000 7f4f2dd9b700 1 --2- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f4f18036880 0x7f4f18038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:51.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.248+0000 7f4f2dd9b700 1 --2- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f4f18036880 0x7f4f18038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009990 tx=0x7f4f24006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:51.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.253+0000 7f4f1effd700 1 -- 172.21.15.136:0/1672501604 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4f20013a80 con 0x7f4f300fd040 2024-04-06T03:32:51.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.517+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4f3002b2a0 con 0x7f4f300fd040 2024-04-06T03:32:51.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.518+0000 7f4f1effd700 1 -- 172.21.15.136:0/1672501604 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4f20011370 con 0x7f4f300fd040 2024-04-06T03:32:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:32:51.519 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:32:51.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f4f18036880 msgr2=0x7f4f18038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:51.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 --2- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f4f18036880 0x7f4f18038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009990 tx=0x7f4f24006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:51.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 msgr2=0x7f4f300664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:51.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 --2- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 0x7f4f300664c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f200096b0 tx=0x7f4f20009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:51.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 shutdown_connections 2024-04-06T03:32:51.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 --2- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f4f18036880 0x7f4f18038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:51.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 --2- 172.21.15.136:0/1672501604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f300fd040 0x7f4f300664c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:51.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 >> 172.21.15.136:0/1672501604 conn(0x7f4f300f70c0 msgr2=0x7f4f30102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:51.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 shutdown_connections 2024-04-06T03:32:51.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:51.520+0000 7f4f34a49700 1 -- 172.21.15.136:0/1672501604 wait complete. 2024-04-06T03:32:51.522 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:32:52.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:51 smithi094 ceph-mon[29713]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:52.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:51 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/1672501604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:32:53.103 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:32:53.103 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:32:53.354 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:53.400 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:54.311 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:53 smithi094 ceph-mon[29713]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:54.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.815+0000 7f9bd38e5700 1 -- 172.21.15.136:0/158689949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 msgr2=0x7f9bcc0fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:54.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.815+0000 7f9bd38e5700 1 --2- 172.21.15.136:0/158689949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 0x7f9bcc0fd450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009c60 tx=0x7f9bc4004d10 comp rx=0 tx=0).stop 2024-04-06T03:32:54.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.815+0000 7f9bd38e5700 1 -- 172.21.15.136:0/158689949 shutdown_connections 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.815+0000 7f9bd38e5700 1 --2- 172.21.15.136:0/158689949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 0x7f9bcc0fd450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.815+0000 7f9bd38e5700 1 -- 172.21.15.136:0/158689949 >> 172.21.15.136:0/158689949 conn(0x7f9bcc0f70c0 msgr2=0x7f9bcc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.815+0000 7f9bd38e5700 1 -- 172.21.15.136:0/158689949 shutdown_connections 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.815+0000 7f9bd38e5700 1 -- 172.21.15.136:0/158689949 wait complete. 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.816+0000 7f9bd38e5700 1 Processor -- start 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.816+0000 7f9bd38e5700 1 -- start start 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.816+0000 7f9bd38e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 0x7f9bcc0f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.816+0000 7f9bd38e5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0f9590 con 0x7f9bcc0fd040 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.816+0000 7f9bd1681700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 0x7f9bcc0f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.816+0000 7f9bd1681700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 0x7f9bcc0f9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:49212/0 (socket says 172.21.15.136:49212) 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.816+0000 7f9bd1681700 1 -- 172.21.15.136:0/473282215 learned_addr learned my addr 172.21.15.136:0/473282215 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.816+0000 7f9bd1681700 1 -- 172.21.15.136:0/473282215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4009940 con 0x7f9bcc0fd040 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.817+0000 7f9bd1681700 1 --2- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 0x7f9bcc0f9050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009500 tx=0x7f9bc4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:54.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.817+0000 7f9bc1ffb700 1 -- 172.21.15.136:0/473282215 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc4007c80 con 0x7f9bcc0fd040 2024-04-06T03:32:54.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.817+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc0f7660 con 0x7f9bcc0fd040 2024-04-06T03:32:54.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.817+0000 7f9bc1ffb700 1 -- 172.21.15.136:0/473282215 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc401d910 con 0x7f9bcc0fd040 2024-04-06T03:32:54.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.817+0000 7f9bc1ffb700 1 -- 172.21.15.136:0/473282215 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bc40129d0 con 0x7f9bcc0fd040 2024-04-06T03:32:54.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.817+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc0f7b50 con 0x7f9bcc0fd040 2024-04-06T03:32:54.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.818+0000 7f9bc1ffb700 1 -- 172.21.15.136:0/473282215 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f9bc4019040 con 0x7f9bcc0fd040 2024-04-06T03:32:54.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.818+0000 7f9bc1ffb700 1 --2- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f9bb8036880 0x7f9bb8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:54.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.818+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc05f5f0 con 0x7f9bcc0fd040 2024-04-06T03:32:54.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.818+0000 7f9bc1ffb700 1 -- 172.21.15.136:0/473282215 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9bc404a3f0 con 0x7f9bcc0fd040 2024-04-06T03:32:54.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.818+0000 7f9bd0e80700 1 --2- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f9bb8036880 0x7f9bb8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:54.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.819+0000 7f9bd0e80700 1 --2- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f9bb8036880 0x7f9bb8038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009990 tx=0x7f9bbc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:54.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:54.824+0000 7f9bc1ffb700 1 -- 172.21.15.136:0/473282215 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9bc4017020 con 0x7f9bcc0fd040 2024-04-06T03:32:55.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.083+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9bcc1017a0 con 0x7f9bcc0fd040 2024-04-06T03:32:55.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.083+0000 7f9bc1ffb700 1 -- 172.21.15.136:0/473282215 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9bc4021070 con 0x7f9bcc0fd040 2024-04-06T03:32:55.085 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:32:55.086 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f9bb8036880 msgr2=0x7f9bb8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 --2- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f9bb8036880 0x7f9bb8038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009990 tx=0x7f9bbc006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 msgr2=0x7f9bcc0f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 --2- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 0x7f9bcc0f9050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009500 tx=0x7f9bc4004d10 comp rx=0 tx=0).stop 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 shutdown_connections 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 --2- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f9bb8036880 0x7f9bb8038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 --2- 172.21.15.136:0/473282215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9bcc0fd040 0x7f9bcc0f9050 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 >> 172.21.15.136:0/473282215 conn(0x7f9bcc0f70c0 msgr2=0x7f9bcc065900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 shutdown_connections 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:55.085+0000 7f9bd38e5700 1 -- 172.21.15.136:0/473282215 wait complete. 2024-04-06T03:32:55.088 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:32:56.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:56.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: Deploying daemon prometheus.smithi094 on smithi094 2024-04-06T03:32:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/473282215' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:32:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:55 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:32:56.663 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:32:56.663 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:32:56.903 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:56.949 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:32:58.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:57 smithi094 ceph-mon[29713]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:32:58.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.275+0000 7f148dcda700 1 -- 172.21.15.136:0/785646625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 msgr2=0x7f14880fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:58.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.275+0000 7f148dcda700 1 --2- 172.21.15.136:0/785646625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 0x7f14880fe4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f14780048c0 tx=0x7f1478009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:58.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.275+0000 7f148dcda700 1 -- 172.21.15.136:0/785646625 shutdown_connections 2024-04-06T03:32:58.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.275+0000 7f148dcda700 1 --2- 172.21.15.136:0/785646625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 0x7f14880fe4b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:58.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.275+0000 7f148dcda700 1 -- 172.21.15.136:0/785646625 >> 172.21.15.136:0/785646625 conn(0x7f14880f6840 msgr2=0x7f14880f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:58.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.275+0000 7f148dcda700 1 -- 172.21.15.136:0/785646625 shutdown_connections 2024-04-06T03:32:58.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.275+0000 7f148dcda700 1 -- 172.21.15.136:0/785646625 wait complete. 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.275+0000 7f148dcda700 1 Processor -- start 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.275+0000 7f148dcda700 1 -- start start 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f148dcda700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 0x7f148810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f148dcda700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148810d5f0 con 0x7f14880fe0a0 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f14877fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 0x7f148810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f14877fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 0x7f148810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:40264/0 (socket says 172.21.15.136:40264) 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f14877fe700 1 -- 172.21.15.136:0/123416744 learned_addr learned my addr 172.21.15.136:0/123416744 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f14877fe700 1 -- 172.21.15.136:0/123416744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1478005040 con 0x7f14880fe0a0 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f14877fe700 1 --2- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 0x7f148810d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f14780096b0 tx=0x7f1478009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f146ffff700 1 -- 172.21.15.136:0/123416744 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f147800bed0 con 0x7f14880fe0a0 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f146ffff700 1 -- 172.21.15.136:0/123416744 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f147800b5c0 con 0x7f14880fe0a0 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148810d7f0 con 0x7f14880fe0a0 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f146ffff700 1 -- 172.21.15.136:0/123416744 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f147801c5b0 con 0x7f14880fe0a0 2024-04-06T03:32:58.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.276+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148810e360 con 0x7f14880fe0a0 2024-04-06T03:32:58.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.277+0000 7f146ffff700 1 -- 172.21.15.136:0/123416744 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f1478013420 con 0x7f14880fe0a0 2024-04-06T03:32:58.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.277+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1488102f70 con 0x7f14880fe0a0 2024-04-06T03:32:58.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.277+0000 7f146ffff700 1 --2- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1470036880 0x7f1470038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:32:58.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.277+0000 7f146ffff700 1 -- 172.21.15.136:0/123416744 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1478049a80 con 0x7f14880fe0a0 2024-04-06T03:32:58.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.278+0000 7f1486ffd700 1 --2- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1470036880 0x7f1470038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:32:58.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.278+0000 7f1486ffd700 1 --2- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1470036880 0x7f1470038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f147c009990 tx=0x7f147c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:32:58.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.283+0000 7f146ffff700 1 -- 172.21.15.136:0/123416744 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1478013a80 con 0x7f14880fe0a0 2024-04-06T03:32:58.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.539+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f148810ed70 con 0x7f14880fe0a0 2024-04-06T03:32:58.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.539+0000 7f146ffff700 1 -- 172.21.15.136:0/123416744 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1478011370 con 0x7f14880fe0a0 2024-04-06T03:32:58.541 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:32:58.541 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1470036880 msgr2=0x7f1470038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 --2- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1470036880 0x7f1470038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f147c009990 tx=0x7f147c006e40 comp rx=0 tx=0).stop 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 msgr2=0x7f148810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 --2- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 0x7f148810d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f14780096b0 tx=0x7f1478009e30 comp rx=0 tx=0).stop 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 shutdown_connections 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 --2- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f1470036880 0x7f1470038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 --2- 172.21.15.136:0/123416744 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14880fe0a0 0x7f148810d0b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 >> 172.21.15.136:0/123416744 conn(0x7f14880f6840 msgr2=0x7f1488103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 shutdown_connections 2024-04-06T03:32:58.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:32:58.541+0000 7f148dcda700 1 -- 172.21.15.136:0/123416744 wait complete. 2024-04-06T03:32:58.544 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:32:59.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:58 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/123416744' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:00.109 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:00.109 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:00.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:32:59 smithi094 ceph-mon[29713]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:00.357 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:00.403 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:01.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.628+0000 7f940206f700 1 -- 172.21.15.136:0/84149343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 msgr2=0x7f93fc0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:01.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.628+0000 7f940206f700 1 --2- 172.21.15.136:0/84149343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 0x7f93fc0ff6b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009c60 tx=0x7f93ec004d10 comp rx=0 tx=0).stop 2024-04-06T03:33:01.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.628+0000 7f940206f700 1 -- 172.21.15.136:0/84149343 shutdown_connections 2024-04-06T03:33:01.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.628+0000 7f940206f700 1 --2- 172.21.15.136:0/84149343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 0x7f93fc0ff6b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:01.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.628+0000 7f940206f700 1 -- 172.21.15.136:0/84149343 >> 172.21.15.136:0/84149343 conn(0x7f93fc0f70c0 msgr2=0x7f93fc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:01.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.628+0000 7f940206f700 1 -- 172.21.15.136:0/84149343 shutdown_connections 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.628+0000 7f940206f700 1 -- 172.21.15.136:0/84149343 wait complete. 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.629+0000 7f940206f700 1 Processor -- start 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.629+0000 7f940206f700 1 -- start start 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.629+0000 7f940206f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 0x7f93fc10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.629+0000 7f940206f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc10d5f0 con 0x7f93fc0669e0 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.629+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 0x7f93fc10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.629+0000 7f93fb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 0x7f93fc10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:40276/0 (socket says 172.21.15.136:40276) 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.629+0000 7f93fb7fe700 1 -- 172.21.15.136:0/1400135191 learned_addr learned my addr 172.21.15.136:0/1400135191 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.629+0000 7f93fb7fe700 1 -- 172.21.15.136:0/1400135191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec009940 con 0x7f93fc0669e0 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.630+0000 7f93fb7fe700 1 --2- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 0x7f93fc10d0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0097b0 tx=0x7f93ec004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.630+0000 7f93e3fff700 1 -- 172.21.15.136:0/1400135191 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93ec012410 con 0x7f93fc0669e0 2024-04-06T03:33:01.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.630+0000 7f93e3fff700 1 -- 172.21.15.136:0/1400135191 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93ec012a40 con 0x7f93fc0669e0 2024-04-06T03:33:01.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.630+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc10d7f0 con 0x7f93fc0669e0 2024-04-06T03:33:01.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.630+0000 7f93e3fff700 1 -- 172.21.15.136:0/1400135191 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93ec01b980 con 0x7f93fc0669e0 2024-04-06T03:33:01.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.630+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc10e360 con 0x7f93fc0669e0 2024-04-06T03:33:01.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.631+0000 7f93e3fff700 1 -- 172.21.15.136:0/1400135191 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f93ec019040 con 0x7f93fc0669e0 2024-04-06T03:33:01.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.631+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc102f60 con 0x7f93fc0669e0 2024-04-06T03:33:01.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.631+0000 7f93e3fff700 1 --2- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f93e4036880 0x7f93e4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:01.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.631+0000 7f93e3fff700 1 -- 172.21.15.136:0/1400135191 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f93ec049d30 con 0x7f93fc0669e0 2024-04-06T03:33:01.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.631+0000 7f93faffd700 1 --2- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f93e4036880 0x7f93e4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:01.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.631+0000 7f93faffd700 1 --2- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f93e4036880 0x7f93e4038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009990 tx=0x7f93f0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:01.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.636+0000 7f93e3fff700 1 -- 172.21.15.136:0/1400135191 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f93ec012bb0 con 0x7f93fc0669e0 2024-04-06T03:33:01.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.894+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f93fc10ed70 con 0x7f93fc0669e0 2024-04-06T03:33:01.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.894+0000 7f93e3fff700 1 -- 172.21.15.136:0/1400135191 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f93ec017090 con 0x7f93fc0669e0 2024-04-06T03:33:01.896 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:01.896 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:33:01.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.896+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f93e4036880 msgr2=0x7f93e4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:01.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.896+0000 7f940206f700 1 --2- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f93e4036880 0x7f93e4038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009990 tx=0x7f93f0006e40 comp rx=0 tx=0).stop 2024-04-06T03:33:01.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.897+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 msgr2=0x7f93fc10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:01.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.897+0000 7f940206f700 1 --2- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 0x7f93fc10d0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f93ec0097b0 tx=0x7f93ec004d10 comp rx=0 tx=0).stop 2024-04-06T03:33:01.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.897+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 shutdown_connections 2024-04-06T03:33:01.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.897+0000 7f940206f700 1 --2- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f93e4036880 0x7f93e4038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:01.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.897+0000 7f940206f700 1 --2- 172.21.15.136:0/1400135191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93fc0669e0 0x7f93fc10d0b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:01.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.897+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 >> 172.21.15.136:0/1400135191 conn(0x7f93fc0f70c0 msgr2=0x7f93fc103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:01.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.897+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 shutdown_connections 2024-04-06T03:33:01.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:01.897+0000 7f940206f700 1 -- 172.21.15.136:0/1400135191 wait complete. 2024-04-06T03:33:01.899 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:33:02.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:01 smithi094 ceph-mon[29713]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:02.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:01 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/1400135191' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:03.444 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:03.445 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:03.694 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:03.740 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:04.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:03 smithi094 ceph-mon[29713]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:05.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.119+0000 7fa2a8221700 1 -- 172.21.15.136:0/2774149027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 msgr2=0x7fa2a00ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:05.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.119+0000 7fa2a8221700 1 --2- 172.21.15.136:0/2774149027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 0x7fa2a00ff700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0048c0 tx=0x7fa29c009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:05.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.119+0000 7fa2a8221700 1 -- 172.21.15.136:0/2774149027 shutdown_connections 2024-04-06T03:33:05.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.119+0000 7fa2a8221700 1 --2- 172.21.15.136:0/2774149027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 0x7fa2a00ff700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:05.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.119+0000 7fa2a8221700 1 -- 172.21.15.136:0/2774149027 >> 172.21.15.136:0/2774149027 conn(0x7fa2a00f7090 msgr2=0x7fa2a00f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:05.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.119+0000 7fa2a8221700 1 -- 172.21.15.136:0/2774149027 shutdown_connections 2024-04-06T03:33:05.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.119+0000 7fa2a8221700 1 -- 172.21.15.136:0/2774149027 wait complete. 2024-04-06T03:33:05.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.120+0000 7fa2a8221700 1 Processor -- start 2024-04-06T03:33:05.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.120+0000 7fa2a8221700 1 -- start start 2024-04-06T03:33:05.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.120+0000 7fa2a8221700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 0x7fa2a010d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:05.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.120+0000 7fa2a8221700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a010d600 con 0x7fa2a0066940 2024-04-06T03:33:05.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.120+0000 7fa2a5fbd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 0x7fa2a010d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:05.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.120+0000 7fa2a5fbd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 0x7fa2a010d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:40280/0 (socket says 172.21.15.136:40280) 2024-04-06T03:33:05.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.120+0000 7fa2a5fbd700 1 -- 172.21.15.136:0/159205158 learned_addr learned my addr 172.21.15.136:0/159205158 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.121+0000 7fa2a5fbd700 1 -- 172.21.15.136:0/159205158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c005040 con 0x7fa2a0066940 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.121+0000 7fa2a5fbd700 1 --2- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 0x7fa2a010d0c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009680 tx=0x7fa29c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.121+0000 7fa2967fc700 1 -- 172.21.15.136:0/159205158 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa29c00bed0 con 0x7fa2a0066940 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.121+0000 7fa2967fc700 1 -- 172.21.15.136:0/159205158 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa29c00b5c0 con 0x7fa2a0066940 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.121+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a010d800 con 0x7fa2a0066940 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.121+0000 7fa2967fc700 1 -- 172.21.15.136:0/159205158 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa29c01c5b0 con 0x7fa2a0066940 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.121+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a010e370 con 0x7fa2a0066940 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.122+0000 7fa2967fc700 1 -- 172.21.15.136:0/159205158 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fa29c013420 con 0x7fa2a0066940 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.122+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a0102f50 con 0x7fa2a0066940 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.122+0000 7fa2967fc700 1 --2- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa28c036880 0x7fa28c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.122+0000 7fa2967fc700 1 -- 172.21.15.136:0/159205158 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa29c0499f0 con 0x7fa2a0066940 2024-04-06T03:33:05.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.122+0000 7fa2a57bc700 1 --2- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa28c036880 0x7fa28c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:05.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.123+0000 7fa2a57bc700 1 --2- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa28c036880 0x7fa28c038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa290009990 tx=0x7fa290006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:05.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.127+0000 7fa2967fc700 1 -- 172.21.15.136:0/159205158 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa29c013a80 con 0x7fa2a0066940 2024-04-06T03:33:05.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.382+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa2a010ed80 con 0x7fa2a0066940 2024-04-06T03:33:05.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.383+0000 7fa2967fc700 1 -- 172.21.15.136:0/159205158 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa29c011370 con 0x7fa2a0066940 2024-04-06T03:33:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:05.384 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:33:05.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa28c036880 msgr2=0x7fa28c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 --2- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa28c036880 0x7fa28c038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa290009990 tx=0x7fa290006e40 comp rx=0 tx=0).stop 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 msgr2=0x7fa2a010d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 --2- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 0x7fa2a010d0c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009680 tx=0x7fa29c009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 shutdown_connections 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 --2- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7fa28c036880 0x7fa28c038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 --2- 172.21.15.136:0/159205158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2a0066940 0x7fa2a010d0c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 >> 172.21.15.136:0/159205158 conn(0x7fa2a00f7090 msgr2=0x7fa2a0103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 shutdown_connections 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:05.385+0000 7fa2a8221700 1 -- 172.21.15.136:0/159205158 wait complete. 2024-04-06T03:33:05.387 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:33:06.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:05 smithi094 ceph-mon[29713]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:06.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:05 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/159205158' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:06.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:05 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:06.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:05 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:06.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:05 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:06.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:05 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-06T03:33:06.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:05 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:06.954 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:06.954 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:07.202 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:07.249 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:07.763 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:07 smithi094 ceph-mon[29713]: from='mgr.14162 172.21.15.94:0/1373445654' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-06T03:33:07.763 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:07 smithi094 ceph-mon[29713]: mgrmap e14: smithi094.purlzp(active, since 55s) 2024-04-06T03:33:08.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.600+0000 7ff2021b3700 1 -- 172.21.15.136:0/1183938294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 msgr2=0x7ff1fc0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:08.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.600+0000 7ff2021b3700 1 --2- 172.21.15.136:0/1183938294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 0x7ff1fc0fce70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0048c0 tx=0x7ff1ec009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:08.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.600+0000 7ff2021b3700 1 -- 172.21.15.136:0/1183938294 shutdown_connections 2024-04-06T03:33:08.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.600+0000 7ff2021b3700 1 --2- 172.21.15.136:0/1183938294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 0x7ff1fc0fce70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:08.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.600+0000 7ff2021b3700 1 -- 172.21.15.136:0/1183938294 >> 172.21.15.136:0/1183938294 conn(0x7ff1fc0f81c0 msgr2=0x7ff1fc0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:08.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.600+0000 7ff2021b3700 1 -- 172.21.15.136:0/1183938294 shutdown_connections 2024-04-06T03:33:08.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.600+0000 7ff2021b3700 1 -- 172.21.15.136:0/1183938294 wait complete. 2024-04-06T03:33:08.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.600+0000 7ff2021b3700 1 Processor -- start 2024-04-06T03:33:08.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.601+0000 7ff2021b3700 1 -- start start 2024-04-06T03:33:08.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.601+0000 7ff2021b3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 0x7ff1fc10d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:08.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.601+0000 7ff2021b3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc10d5d0 con 0x7ff1fc0fca60 2024-04-06T03:33:08.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.601+0000 7ff1fb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 0x7ff1fc10d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:08.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.601+0000 7ff1fb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 0x7ff1fc10d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:43694/0 (socket says 172.21.15.136:43694) 2024-04-06T03:33:08.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.601+0000 7ff1fb7fe700 1 -- 172.21.15.136:0/3357109746 learned_addr learned my addr 172.21.15.136:0/3357109746 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:08.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.601+0000 7ff1fb7fe700 1 -- 172.21.15.136:0/3357109746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec005040 con 0x7ff1fc0fca60 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.601+0000 7ff1fb7fe700 1 --2- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 0x7ff1fc10d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0096b0 tx=0x7ff1ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.602+0000 7ff1e3fff700 1 -- 172.21.15.136:0/3357109746 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ec00bed0 con 0x7ff1fc0fca60 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.602+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1fc10d7d0 con 0x7ff1fc0fca60 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.602+0000 7ff1e3fff700 1 -- 172.21.15.136:0/3357109746 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1ec00b5c0 con 0x7ff1fc0fca60 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.602+0000 7ff1e3fff700 1 -- 172.21.15.136:0/3357109746 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ec01c5b0 con 0x7ff1fc0fca60 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.602+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1fc10e340 con 0x7ff1fc0fca60 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.602+0000 7ff1e3fff700 1 -- 172.21.15.136:0/3357109746 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7ff1ec01c820 con 0x7ff1fc0fca60 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.602+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1fc102f70 con 0x7ff1fc0fca60 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.603+0000 7ff1e3fff700 1 --2- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff1e40368d0 0x7ff1e4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.603+0000 7ff1e3fff700 1 -- 172.21.15.136:0/3357109746 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff1ec049b70 con 0x7ff1fc0fca60 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.603+0000 7ff1faffd700 1 -- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff1e40368d0 msgr2=0x7ff1e4038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2299971604 2024-04-06T03:33:08.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.603+0000 7ff1faffd700 1 --2- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff1e40368d0 0x7ff1e4038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-06T03:33:08.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.607+0000 7ff1e3fff700 1 -- 172.21.15.136:0/3357109746 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff1ec025400 con 0x7ff1fc0fca60 2024-04-06T03:33:08.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.804+0000 7ff1faffd700 1 -- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff1e40368d0 msgr2=0x7ff1e4038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2299971604 2024-04-06T03:33:08.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.804+0000 7ff1faffd700 1 --2- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff1e40368d0 0x7ff1e4038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-06T03:33:08.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.867+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff1fc10ed50 con 0x7ff1fc0fca60 2024-04-06T03:33:08.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.868+0000 7ff1e3fff700 1 -- 172.21.15.136:0/3357109746 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff1ec011370 con 0x7ff1fc0fca60 2024-04-06T03:33:08.869 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:08.869 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:33:08.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.869+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff1e40368d0 msgr2=0x7ff1e4038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-06T03:33:08.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.870+0000 7ff2021b3700 1 --2- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff1e40368d0 0x7ff1e4038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:08.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.870+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 msgr2=0x7ff1fc10d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:08.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.870+0000 7ff2021b3700 1 --2- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 0x7ff1fc10d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0096b0 tx=0x7ff1ec009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:08.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.870+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 shutdown_connections 2024-04-06T03:33:08.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.870+0000 7ff2021b3700 1 --2- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7ff1e40368d0 0x7ff1e4038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:08.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.870+0000 7ff2021b3700 1 --2- 172.21.15.136:0/3357109746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1fc0fca60 0x7ff1fc10d090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:08.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.870+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 >> 172.21.15.136:0/3357109746 conn(0x7ff1fc0f81c0 msgr2=0x7ff1fc103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:08.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.870+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 shutdown_connections 2024-04-06T03:33:08.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:08.870+0000 7ff2021b3700 1 -- 172.21.15.136:0/3357109746 wait complete. 2024-04-06T03:33:08.872 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:33:09.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:08 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/3357109746' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:10.484 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:10.484 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:10.734 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:10.780 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:12.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.111+0000 7f0f5aa21700 1 -- 172.21.15.136:0/3777711144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 msgr2=0x7f0f540fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:12.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.111+0000 7f0f5aa21700 1 --2- 172.21.15.136:0/3777711144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 0x7f0f540fbd70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0f440048c0 tx=0x7f0f44009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:12.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.111+0000 7f0f5aa21700 1 -- 172.21.15.136:0/3777711144 shutdown_connections 2024-04-06T03:33:12.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.111+0000 7f0f5aa21700 1 --2- 172.21.15.136:0/3777711144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 0x7f0f540fbd70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:12.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.111+0000 7f0f5aa21700 1 -- 172.21.15.136:0/3777711144 >> 172.21.15.136:0/3777711144 conn(0x7f0f540f70c0 msgr2=0x7f0f540f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:12.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.111+0000 7f0f5aa21700 1 -- 172.21.15.136:0/3777711144 shutdown_connections 2024-04-06T03:33:12.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.111+0000 7f0f5aa21700 1 -- 172.21.15.136:0/3777711144 wait complete. 2024-04-06T03:33:12.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.111+0000 7f0f5aa21700 1 Processor -- start 2024-04-06T03:33:12.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.111+0000 7f0f5aa21700 1 -- start start 2024-04-06T03:33:12.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f5aa21700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 0x7f0f54066880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:12.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f5aa21700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5410fd90 con 0x7f0f540fb960 2024-04-06T03:33:12.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f53fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 0x7f0f54066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:12.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f53fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 0x7f0f54066880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:43712/0 (socket says 172.21.15.136:43712) 2024-04-06T03:33:12.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f53fff700 1 -- 172.21.15.136:0/117303499 learned_addr learned my addr 172.21.15.136:0/117303499 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:12.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f53fff700 1 -- 172.21.15.136:0/117303499 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44005040 con 0x7f0f540fb960 2024-04-06T03:33:12.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f53fff700 1 --2- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 0x7f0f54066880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009680 tx=0x7f0f44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:12.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f50ff9700 1 -- 172.21.15.136:0/117303499 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f4400b5c0 con 0x7f0f540fb960 2024-04-06T03:33:12.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5410fed0 con 0x7f0f540fb960 2024-04-06T03:33:12.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f50ff9700 1 -- 172.21.15.136:0/117303499 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f4400be70 con 0x7f0f540fb960 2024-04-06T03:33:12.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f50ff9700 1 -- 172.21.15.136:0/117303499 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f440135a0 con 0x7f0f540fb960 2024-04-06T03:33:12.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.112+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f541103c0 con 0x7f0f540fb960 2024-04-06T03:33:12.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.113+0000 7f0f50ff9700 1 -- 172.21.15.136:0/117303499 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f0f4401c430 con 0x7f0f540fb960 2024-04-06T03:33:12.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.113+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f54105920 con 0x7f0f540fb960 2024-04-06T03:33:12.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.114+0000 7f0f50ff9700 1 --2- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f0f3c0368d0 0x7f0f3c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:12.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.114+0000 7f0f50ff9700 1 -- 172.21.15.136:0/117303499 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0f440494c0 con 0x7f0f540fb960 2024-04-06T03:33:12.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.114+0000 7f0f537fe700 1 -- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f0f3c0368d0 msgr2=0x7f0f3c038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2299971604 2024-04-06T03:33:12.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.114+0000 7f0f537fe700 1 --2- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f0f3c0368d0 0x7f0f3c038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-06T03:33:12.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.118+0000 7f0f50ff9700 1 -- 172.21.15.136:0/117303499 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0f44004d30 con 0x7f0f540fb960 2024-04-06T03:33:12.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.315+0000 7f0f537fe700 1 -- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f0f3c0368d0 msgr2=0x7f0f3c038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2299971604 2024-04-06T03:33:12.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.315+0000 7f0f537fe700 1 --2- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f0f3c0368d0 0x7f0f3c038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-06T03:33:12.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.379+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0f5405f5f0 con 0x7f0f540fb960 2024-04-06T03:33:12.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.379+0000 7f0f50ff9700 1 -- 172.21.15.136:0/117303499 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f0f44021070 con 0x7f0f540fb960 2024-04-06T03:33:12.381 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:12.381 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.381+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f0f3c0368d0 msgr2=0x7f0f3c038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.381+0000 7f0f5aa21700 1 --2- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f0f3c0368d0 0x7f0f3c038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.381+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 msgr2=0x7f0f54066880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.381+0000 7f0f5aa21700 1 --2- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 0x7f0f54066880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009680 tx=0x7f0f44009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.381+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 shutdown_connections 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.381+0000 7f0f5aa21700 1 --2- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:6800/2299971604,v1:172.21.15.94:6801/2299971604] conn(0x7f0f3c0368d0 0x7f0f3c038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.381+0000 7f0f5aa21700 1 --2- 172.21.15.136:0/117303499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f540fb960 0x7f0f54066880 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.381+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 >> 172.21.15.136:0/117303499 conn(0x7f0f540f70c0 msgr2=0x7f0f54102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.382+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 shutdown_connections 2024-04-06T03:33:12.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:12.382+0000 7f0f5aa21700 1 -- 172.21.15.136:0/117303499 wait complete. 2024-04-06T03:33:12.384 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:33:12.750 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:12 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/117303499' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:13.793 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: Active manager daemon smithi094.purlzp restarted 2024-04-06T03:33:13.793 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: Activating manager daemon smithi094.purlzp 2024-04-06T03:33:13.793 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: osdmap e5: 0 total, 0 up, 0 in 2024-04-06T03:33:13.793 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: mgrmap e15: smithi094.purlzp(active, starting, since 0.00293856s) 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: Manager daemon smithi094.purlzp is now available 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:33:13.794 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:13.910 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:13.911 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:14.153 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:14.204 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:14.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:14 smithi094 ceph-mon[29713]: [06/Apr/2024:03:33:13] ENGINE Bus STARTING 2024-04-06T03:33:14.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:14 smithi094 ceph-mon[29713]: [06/Apr/2024:03:33:13] ENGINE Serving on https://172.21.15.94:7150 2024-04-06T03:33:14.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:14 smithi094 ceph-mon[29713]: [06/Apr/2024:03:33:13] ENGINE Bus STARTED 2024-04-06T03:33:14.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:14 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:14.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:14 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:14.746 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:14 smithi094 ceph-mon[29713]: mgrmap e16: smithi094.purlzp(active, since 1.00531s) 2024-04-06T03:33:15.902 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:15 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.218+0000 7f5823fcc700 1 -- 172.21.15.136:0/2819936247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 msgr2=0x7f581c0ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.218+0000 7f5823fcc700 1 --2- 172.21.15.136:0/2819936247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 0x7f581c0ffa90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f580c0048c0 tx=0x7f580c009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.219+0000 7f5823fcc700 1 -- 172.21.15.136:0/2819936247 shutdown_connections 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.219+0000 7f5823fcc700 1 --2- 172.21.15.136:0/2819936247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 0x7f581c0ffa90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.219+0000 7f5823fcc700 1 -- 172.21.15.136:0/2819936247 >> 172.21.15.136:0/2819936247 conn(0x7f581c0f70d0 msgr2=0x7f581c0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.219+0000 7f5823fcc700 1 -- 172.21.15.136:0/2819936247 shutdown_connections 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.219+0000 7f5823fcc700 1 -- 172.21.15.136:0/2819936247 wait complete. 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.219+0000 7f5823fcc700 1 Processor -- start 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.219+0000 7f5823fcc700 1 -- start start 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.219+0000 7f5823fcc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 0x7f581c10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.219+0000 7f5823fcc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c10d5f0 con 0x7f581c0fd6b0 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.220+0000 7f5821d68700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 0x7f581c10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.220+0000 7f5821d68700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 0x7f581c10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:58082/0 (socket says 172.21.15.136:58082) 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.220+0000 7f5821d68700 1 -- 172.21.15.136:0/2613641876 learned_addr learned my addr 172.21.15.136:0/2613641876 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.220+0000 7f5821d68700 1 -- 172.21.15.136:0/2613641876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c005040 con 0x7f581c0fd6b0 2024-04-06T03:33:16.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.220+0000 7f5821d68700 1 --2- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 0x7f581c10d0b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f580c009680 tx=0x7f580c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.220+0000 7f58127fc700 1 -- 172.21.15.136:0/2613641876 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f580c00bed0 con 0x7f581c0fd6b0 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.220+0000 7f58127fc700 1 -- 172.21.15.136:0/2613641876 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f580c00b5c0 con 0x7f581c0fd6b0 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.220+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c10d7f0 con 0x7f581c0fd6b0 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.220+0000 7f58127fc700 1 -- 172.21.15.136:0/2613641876 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f580c01c5b0 con 0x7f581c0fd6b0 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.221+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c10e360 con 0x7f581c0fd6b0 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.221+0000 7f58127fc700 1 -- 172.21.15.136:0/2613641876 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f580c013420 con 0x7f581c0fd6b0 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.221+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c102f60 con 0x7f581c0fd6b0 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.221+0000 7f58127fc700 1 --2- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5808036970 0x7f5808038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.222+0000 7f58127fc700 1 -- 172.21.15.136:0/2613641876 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f580c049c00 con 0x7f581c0fd6b0 2024-04-06T03:33:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.222+0000 7f5821567700 1 --2- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5808036970 0x7f5808038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.222+0000 7f5821567700 1 --2- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5808036970 0x7f5808038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5818009990 tx=0x7f5818006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.226+0000 7f58127fc700 1 -- 172.21.15.136:0/2613641876 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f580c013a80 con 0x7f581c0fd6b0 2024-04-06T03:33:16.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.490+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f581c10ed70 con 0x7f581c0fd6b0 2024-04-06T03:33:16.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.490+0000 7f58127fc700 1 -- 172.21.15.136:0/2613641876 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f580c049e60 con 0x7f581c0fd6b0 2024-04-06T03:33:16.492 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:16.492 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:33:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5808036970 msgr2=0x7f5808038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 --2- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5808036970 0x7f5808038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5818009990 tx=0x7f5818006e40 comp rx=0 tx=0).stop 2024-04-06T03:33:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 msgr2=0x7f581c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 --2- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 0x7f581c10d0b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f580c009680 tx=0x7f580c009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 shutdown_connections 2024-04-06T03:33:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 --2- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5808036970 0x7f5808038e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 --2- 172.21.15.136:0/2613641876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f581c0fd6b0 0x7f581c10d0b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 >> 172.21.15.136:0/2613641876 conn(0x7f581c0f70d0 msgr2=0x7f581c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 shutdown_connections 2024-04-06T03:33:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:16.493+0000 7f5823fcc700 1 -- 172.21.15.136:0/2613641876 wait complete. 2024-04-06T03:33:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:33:16.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:16 smithi094 ceph-mon[29713]: mgrmap e17: smithi094.purlzp(active, since 2s) 2024-04-06T03:33:16.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:16 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/2613641876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:18.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:33:18.505 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:18.506 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:19.094 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:19.171 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:20.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:20.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:20.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi136", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:33:20.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi136", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-06T03:33:20.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:20.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.729+0000 7fbeb04d3700 1 -- 172.21.15.136:0/1931427393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 msgr2=0x7fbea8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:20.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.729+0000 7fbeb04d3700 1 --2- 172.21.15.136:0/1931427393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 0x7fbea8101df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbea40048c0 tx=0x7fbea4009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:20.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.729+0000 7fbeb04d3700 1 -- 172.21.15.136:0/1931427393 shutdown_connections 2024-04-06T03:33:20.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.729+0000 7fbeb04d3700 1 --2- 172.21.15.136:0/1931427393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 0x7fbea8101df0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:20.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.729+0000 7fbeb04d3700 1 -- 172.21.15.136:0/1931427393 >> 172.21.15.136:0/1931427393 conn(0x7fbea80f7540 msgr2=0x7fbea80f9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:20.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.729+0000 7fbeb04d3700 1 -- 172.21.15.136:0/1931427393 shutdown_connections 2024-04-06T03:33:20.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.729+0000 7fbeb04d3700 1 -- 172.21.15.136:0/1931427393 wait complete. 2024-04-06T03:33:20.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.729+0000 7fbeb04d3700 1 Processor -- start 2024-04-06T03:33:20.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.730+0000 7fbeb04d3700 1 -- start start 2024-04-06T03:33:20.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.730+0000 7fbeb04d3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 0x7fbea81120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:20.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.730+0000 7fbeb04d3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea8112620 con 0x7fbea8101a20 2024-04-06T03:33:20.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.730+0000 7fbeae26f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 0x7fbea81120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:20.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.730+0000 7fbeae26f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 0x7fbea81120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:58096/0 (socket says 172.21.15.136:58096) 2024-04-06T03:33:20.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.730+0000 7fbeae26f700 1 -- 172.21.15.136:0/2207666546 learned_addr learned my addr 172.21.15.136:0/2207666546 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:20.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.730+0000 7fbeae26f700 1 -- 172.21.15.136:0/2207666546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea4005040 con 0x7fbea8101a20 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.730+0000 7fbeae26f700 1 --2- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 0x7fbea81120e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbea400b820 tx=0x7fbea4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.731+0000 7fbe9effd700 1 -- 172.21.15.136:0/2207666546 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbea40093c0 con 0x7fbea8101a20 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.731+0000 7fbe9effd700 1 -- 172.21.15.136:0/2207666546 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbea4013420 con 0x7fbea8101a20 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.731+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea8112820 con 0x7fbea8101a20 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.731+0000 7fbe9effd700 1 -- 172.21.15.136:0/2207666546 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbea401c5b0 con 0x7fbea8101a20 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.731+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea8112c90 con 0x7fbea8101a20 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.732+0000 7fbe9effd700 1 -- 172.21.15.136:0/2207666546 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fbea401c7f0 con 0x7fbea8101a20 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.732+0000 7fbe9effd700 1 --2- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fbe94036970 0x7fbe94038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.732+0000 7fbe9effd700 1 -- 172.21.15.136:0/2207666546 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fbea4023c30 con 0x7fbea8101a20 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.732+0000 7fbeada6e700 1 --2- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fbe94036970 0x7fbe94038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.732+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea80fd120 con 0x7fbea8101a20 2024-04-06T03:33:20.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.733+0000 7fbeada6e700 1 --2- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fbe94036970 0x7fbe94038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009990 tx=0x7fbe98006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:20.737+0000 7fbe9effd700 1 -- 172.21.15.136:0/2207666546 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbea4023300 con 0x7fbea8101a20 2024-04-06T03:33:21.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.014+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbea8105d10 con 0x7fbea8101a20 2024-04-06T03:33:21.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.015+0000 7fbe9effd700 1 -- 172.21.15.136:0/2207666546 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fbea4012e40 con 0x7fbea8101a20 2024-04-06T03:33:21.016 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:21.016 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fbe94036970 msgr2=0x7fbe94038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 --2- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fbe94036970 0x7fbe94038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009990 tx=0x7fbe98006e40 comp rx=0 tx=0).stop 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 msgr2=0x7fbea81120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 --2- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 0x7fbea81120e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbea400b820 tx=0x7fbea4009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 shutdown_connections 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 --2- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fbe94036970 0x7fbe94038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 --2- 172.21.15.136:0/2207666546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea8101a20 0x7fbea81120e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 >> 172.21.15.136:0/2207666546 conn(0x7fbea80f7540 msgr2=0x7fbea80f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 shutdown_connections 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:21.017+0000 7fbeb04d3700 1 -- 172.21.15.136:0/2207666546 wait complete. 2024-04-06T03:33:21.019 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:33:21.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:21 smithi094 ceph-mon[29713]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:33:21.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:21 smithi094 ceph-mon[29713]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:33:21.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:21 smithi094 ceph-mon[29713]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:33:21.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:21 smithi094 ceph-mon[29713]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:33:21.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:21 smithi094 ceph-mon[29713]: Deploying daemon crash.smithi136 on smithi136 2024-04-06T03:33:21.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:21 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/2207666546' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:22.944 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:22.944 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:23.202 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:33:25.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3241689619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 msgr2=0x7f9ee80fdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:25.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 --2- 172.21.15.136:0/3241689619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 0x7f9ee80fdeb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40048c0 tx=0x7f9ee4009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:25.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3241689619 shutdown_connections 2024-04-06T03:33:25.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 --2- 172.21.15.136:0/3241689619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 0x7f9ee80fdeb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:25.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3241689619 >> 172.21.15.136:0/3241689619 conn(0x7f9ee8071800 msgr2=0x7f9ee8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:25.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3241689619 shutdown_connections 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3241689619 wait complete. 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 Processor -- start 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 -- start start 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.196+0000 7f9eefc3f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 0x7f9ee81131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9eefc3f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee8113700 con 0x7f9ee80fdae0 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9eed9db700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 0x7f9ee81131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9eed9db700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 0x7f9ee81131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:58114/0 (socket says 172.21.15.136:58114) 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9eed9db700 1 -- 172.21.15.136:0/3193646579 learned_addr learned my addr 172.21.15.136:0/3193646579 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9eed9db700 1 -- 172.21.15.136:0/3193646579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee4005040 con 0x7f9ee80fdae0 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9eed9db700 1 --2- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 0x7f9ee81131c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400b820 tx=0x7f9ee4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9ede7fc700 1 -- 172.21.15.136:0/3193646579 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ee40093c0 con 0x7f9ee80fdae0 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee8113900 con 0x7f9ee80fdae0 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9ede7fc700 1 -- 172.21.15.136:0/3193646579 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ee4013420 con 0x7f9ee80fdae0 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.197+0000 7f9ede7fc700 1 -- 172.21.15.136:0/3193646579 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ee401c5b0 con 0x7f9ee80fdae0 2024-04-06T03:33:25.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.198+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee8113d70 con 0x7f9ee80fdae0 2024-04-06T03:33:25.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.198+0000 7f9ede7fc700 1 -- 172.21.15.136:0/3193646579 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f9ee4009520 con 0x7f9ee80fdae0 2024-04-06T03:33:25.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.198+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee80658f0 con 0x7f9ee80fdae0 2024-04-06T03:33:25.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.199+0000 7f9ede7fc700 1 --2- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ed4036970 0x7f9ed4038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:25.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.199+0000 7f9ede7fc700 1 -- 172.21.15.136:0/3193646579 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9ee4049b50 con 0x7f9ee80fdae0 2024-04-06T03:33:25.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.199+0000 7f9eed1da700 1 --2- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ed4036970 0x7f9ed4038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:25.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.199+0000 7f9eed1da700 1 --2- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ed4036970 0x7f9ed4038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009990 tx=0x7f9ed8006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:25.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.204+0000 7f9ede7fc700 1 -- 172.21.15.136:0/3193646579 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9ee4013a90 con 0x7f9ee80fdae0 2024-04-06T03:33:25.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.490+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9ee805f5f0 con 0x7f9ee80fdae0 2024-04-06T03:33:25.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.491+0000 7f9ede7fc700 1 -- 172.21.15.136:0/3193646579 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9ee4011370 con 0x7f9ee80fdae0 2024-04-06T03:33:25.493 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:25.493 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:33:25.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ed4036970 msgr2=0x7f9ed4038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:25.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 --2- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ed4036970 0x7f9ed4038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009990 tx=0x7f9ed8006e40 comp rx=0 tx=0).stop 2024-04-06T03:33:25.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 msgr2=0x7f9ee81131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:25.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 --2- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 0x7f9ee81131c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400b820 tx=0x7f9ee4009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:25.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 shutdown_connections 2024-04-06T03:33:25.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 --2- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ed4036970 0x7f9ed4038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:25.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 --2- 172.21.15.136:0/3193646579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee80fdae0 0x7f9ee81131c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:25.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 >> 172.21.15.136:0/3193646579 conn(0x7f9ee8071800 msgr2=0x7f9ee8103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:25.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 shutdown_connections 2024-04-06T03:33:25.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:25.494+0000 7f9eefc3f700 1 -- 172.21.15.136:0/3193646579 wait complete. 2024-04-06T03:33:25.496 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:33:25.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:25.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:25.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:25.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:25.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:25 smithi094 ceph-mon[29713]: Deploying daemon node-exporter.smithi136 on smithi136 2024-04-06T03:33:26.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:26 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/3193646579' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:27.207 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:27.207 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:27.465 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:33:29.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:29.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.671+0000 7fb75f3e5700 1 -- 172.21.15.136:0/1818709650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 msgr2=0x7fb758101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:29.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.671+0000 7fb75f3e5700 1 --2- 172.21.15.136:0/1818709650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 0x7fb758101d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb7540048c0 tx=0x7fb754009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:29.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.671+0000 7fb75f3e5700 1 -- 172.21.15.136:0/1818709650 shutdown_connections 2024-04-06T03:33:29.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.671+0000 7fb75f3e5700 1 --2- 172.21.15.136:0/1818709650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 0x7fb758101d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:29.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.671+0000 7fb75f3e5700 1 -- 172.21.15.136:0/1818709650 >> 172.21.15.136:0/1818709650 conn(0x7fb7580f7510 msgr2=0x7fb7580f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:29.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.671+0000 7fb75f3e5700 1 -- 172.21.15.136:0/1818709650 shutdown_connections 2024-04-06T03:33:29.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.671+0000 7fb75f3e5700 1 -- 172.21.15.136:0/1818709650 wait complete. 2024-04-06T03:33:29.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.672+0000 7fb75f3e5700 1 Processor -- start 2024-04-06T03:33:29.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.672+0000 7fb75f3e5700 1 -- start start 2024-04-06T03:33:29.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.672+0000 7fb75f3e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 0x7fb758112010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:29.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.672+0000 7fb75f3e5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb758112550 con 0x7fb7581019b0 2024-04-06T03:33:29.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.673+0000 7fb75d181700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 0x7fb758112010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:29.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.673+0000 7fb75d181700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 0x7fb758112010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:45806/0 (socket says 172.21.15.136:45806) 2024-04-06T03:33:29.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.673+0000 7fb75d181700 1 -- 172.21.15.136:0/3063625665 learned_addr learned my addr 172.21.15.136:0/3063625665 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:29.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.673+0000 7fb75d181700 1 -- 172.21.15.136:0/3063625665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb754005040 con 0x7fb7581019b0 2024-04-06T03:33:29.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.673+0000 7fb75d181700 1 --2- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 0x7fb758112010 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb75400b7f0 tx=0x7fb754009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:29.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.673+0000 7fb74dffb700 1 -- 172.21.15.136:0/3063625665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb754009370 con 0x7fb7581019b0 2024-04-06T03:33:29.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.673+0000 7fb74dffb700 1 -- 172.21.15.136:0/3063625665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb754013420 con 0x7fb7581019b0 2024-04-06T03:33:29.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.673+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb758112750 con 0x7fb7581019b0 2024-04-06T03:33:29.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.674+0000 7fb74dffb700 1 -- 172.21.15.136:0/3063625665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb75401c590 con 0x7fb7581019b0 2024-04-06T03:33:29.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.674+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb758112c40 con 0x7fb7581019b0 2024-04-06T03:33:29.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.675+0000 7fb74dffb700 1 -- 172.21.15.136:0/3063625665 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fb7540094d0 con 0x7fb7581019b0 2024-04-06T03:33:29.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.675+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7580fd0b0 con 0x7fb7581019b0 2024-04-06T03:33:29.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.675+0000 7fb74dffb700 1 --2- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb744036970 0x7fb744038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:29.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.675+0000 7fb74dffb700 1 -- 172.21.15.136:0/3063625665 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fb754049490 con 0x7fb7581019b0 2024-04-06T03:33:29.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.675+0000 7fb75c980700 1 --2- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb744036970 0x7fb744038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:29.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.676+0000 7fb75c980700 1 --2- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb744036970 0x7fb744038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb748009990 tx=0x7fb748006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:29.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.680+0000 7fb74dffb700 1 -- 172.21.15.136:0/3063625665 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb754013590 con 0x7fb7581019b0 2024-04-06T03:33:29.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.946+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb75805f5f0 con 0x7fb7581019b0 2024-04-06T03:33:29.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.947+0000 7fb74dffb700 1 -- 172.21.15.136:0/3063625665 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb754021030 con 0x7fb7581019b0 2024-04-06T03:33:29.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:29.949 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:33:29.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb744036970 msgr2=0x7fb744038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:29.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 --2- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb744036970 0x7fb744038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb748009990 tx=0x7fb748006e40 comp rx=0 tx=0).stop 2024-04-06T03:33:29.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 msgr2=0x7fb758112010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:29.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 --2- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 0x7fb758112010 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb75400b7f0 tx=0x7fb754009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:29.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 shutdown_connections 2024-04-06T03:33:29.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 --2- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb744036970 0x7fb744038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:29.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 --2- 172.21.15.136:0/3063625665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7581019b0 0x7fb758112010 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:29.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 >> 172.21.15.136:0/3063625665 conn(0x7fb7580f7510 msgr2=0x7fb7580f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:29.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 shutdown_connections 2024-04-06T03:33:29.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:29.949+0000 7fb75f3e5700 1 -- 172.21.15.136:0/3063625665 wait complete. 2024-04-06T03:33:29.952 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:33:30.318 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:30 smithi094 ceph-mon[29713]: Deploying daemon mgr.smithi136.jagvwt on smithi136 2024-04-06T03:33:30.318 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:30 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/3063625665' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:31.773 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:31.773 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:32.207 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:33:33.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:33.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:33.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:33.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:33.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:33:33.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:33.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:33 smithi094 ceph-mon[29713]: Deploying daemon mon.smithi136 on smithi136 2024-04-06T03:33:33.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:33.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.872+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/420195055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 msgr2=0x7f1ff40fdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:33.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.872+0000 7f1ff9c4b700 1 --2- 172.21.15.136:0/420195055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 0x7f1ff40fdeb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40048c0 tx=0x7f1fe4009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:33.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.872+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/420195055 shutdown_connections 2024-04-06T03:33:33.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.872+0000 7f1ff9c4b700 1 --2- 172.21.15.136:0/420195055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 0x7f1ff40fdeb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:33.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.872+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/420195055 >> 172.21.15.136:0/420195055 conn(0x7f1ff4071800 msgr2=0x7f1ff4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:33.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.872+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/420195055 shutdown_connections 2024-04-06T03:33:33.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.872+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/420195055 wait complete. 2024-04-06T03:33:33.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.873+0000 7f1ff9c4b700 1 Processor -- start 2024-04-06T03:33:33.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.873+0000 7f1ff9c4b700 1 -- start start 2024-04-06T03:33:33.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.873+0000 7f1ff9c4b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 0x7f1ff41153d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:33.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.873+0000 7f1ff9c4b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4115910 con 0x7f1ff40fdae0 2024-04-06T03:33:33.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 0x7f1ff41153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:33.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 0x7f1ff41153d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.136:45836/0 (socket says 172.21.15.136:45836) 2024-04-06T03:33:33.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1ff37fe700 1 -- 172.21.15.136:0/2507967090 learned_addr learned my addr 172.21.15.136:0/2507967090 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:33.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1ff37fe700 1 -- 172.21.15.136:0/2507967090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe4005040 con 0x7f1ff40fdae0 2024-04-06T03:33:33.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1ff37fe700 1 --2- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 0x7f1ff41153d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400b820 tx=0x7f1fe4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:33.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1fdbfff700 1 -- 172.21.15.136:0/2507967090 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fe40093c0 con 0x7f1ff40fdae0 2024-04-06T03:33:33.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff4115b10 con 0x7f1ff40fdae0 2024-04-06T03:33:33.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1fdbfff700 1 -- 172.21.15.136:0/2507967090 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fe4013420 con 0x7f1ff40fdae0 2024-04-06T03:33:33.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1fdbfff700 1 -- 172.21.15.136:0/2507967090 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fe401c5b0 con 0x7f1ff40fdae0 2024-04-06T03:33:33.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.874+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff4115f80 con 0x7f1ff40fdae0 2024-04-06T03:33:33.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.875+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff40658f0 con 0x7f1ff40fdae0 2024-04-06T03:33:33.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.875+0000 7f1fdbfff700 1 -- 172.21.15.136:0/2507967090 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f1fe4009520 con 0x7f1ff40fdae0 2024-04-06T03:33:33.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.875+0000 7f1fdbfff700 1 --2- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1fdc036970 0x7f1fdc038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:33.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.876+0000 7f1fdbfff700 1 -- 172.21.15.136:0/2507967090 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1fe4049b10 con 0x7f1ff40fdae0 2024-04-06T03:33:33.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.876+0000 7f1ff2ffd700 1 --2- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1fdc036970 0x7f1fdc038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:33.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.876+0000 7f1ff2ffd700 1 --2- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1fdc036970 0x7f1fdc038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009990 tx=0x7f1fe8006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:33.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:33.880+0000 7f1fdbfff700 1 -- 172.21.15.136:0/2507967090 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1fe4013a90 con 0x7f1ff40fdae0 2024-04-06T03:33:34.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.157+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1ff405f5f0 con 0x7f1ff40fdae0 2024-04-06T03:33:34.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.158+0000 7f1fdbfff700 1 -- 172.21.15.136:0/2507967090 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1fe4011370 con 0x7f1ff40fdae0 2024-04-06T03:33:34.159 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:34.159 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:31:02.176975Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-06T03:33:34.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1fdc036970 msgr2=0x7f1fdc038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:34.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 --2- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1fdc036970 0x7f1fdc038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009990 tx=0x7f1fe8006e40 comp rx=0 tx=0).stop 2024-04-06T03:33:34.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 msgr2=0x7f1ff41153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:34.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 --2- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 0x7f1ff41153d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400b820 tx=0x7f1fe4009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:34.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 shutdown_connections 2024-04-06T03:33:34.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 --2- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1fdc036970 0x7f1fdc038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:34.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 --2- 172.21.15.136:0/2507967090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ff40fdae0 0x7f1ff41153d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:34.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 >> 172.21.15.136:0/2507967090 conn(0x7f1ff4071800 msgr2=0x7f1ff41036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:34.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 shutdown_connections 2024-04-06T03:33:34.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:34.160+0000 7f1ff9c4b700 1 -- 172.21.15.136:0/2507967090 wait complete. 2024-04-06T03:33:34.162 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-06T03:33:34.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:34 smithi094 ceph-mon[29713]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:34.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:34 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/2507967090' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:36.322 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-06T03:33:36.323 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mon dump -f json 2024-04-06T03:33:36.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:36 smithi094 ceph-mon[29713]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:36.703 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi136/config 2024-04-06T03:33:38.436 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:38 smithi136 systemd[1]: Started Ceph mon.smithi136 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:33:38.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:38 smithi094 ceph-mon[29713]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:43.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:33:43.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: mon.smithi094 calling monitor election 2024-04-06T03:33:43.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1311144469' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: mon.smithi136 calling monitor election 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: mon.smithi094 is new leader, mons smithi094,smithi136 in quorum (ranks 0,1) 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: monmap e2: 2 mons at {smithi094=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],smithi136=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: fsmap 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: osdmap e5: 0 total, 0 up, 0 in 2024-04-06T03:33:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: mgrmap e17: smithi094.purlzp(active, since 30s) 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: overall HEALTH_OK 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1311144469' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1311144469' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1311144469' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:43.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:33:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: mon.smithi094 calling monitor election 2024-04-06T03:33:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1311144469' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:33:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: mon.smithi136 calling monitor election 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: mon.smithi094 is new leader, mons smithi094,smithi136 in quorum (ranks 0,1) 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: monmap e2: 2 mons at {smithi094=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],smithi136=[v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: fsmap 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: osdmap e5: 0 total, 0 up, 0 in 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: mgrmap e17: smithi094.purlzp(active, since 30s) 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: overall HEALTH_OK 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1311144469' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1311144469' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:33:43.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1311144469' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:33:43.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:43.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:43.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:43.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:44.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:44 smithi094 ceph-mon[29713]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:44.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:44 smithi094 ceph-mon[29713]: mgrmap e18: smithi094.purlzp(active, since 30s), standbys: smithi136.jagvwt 2024-04-06T03:33:44.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:44 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi136.jagvwt", "id": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:33:44.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:44 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:33:44.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:44 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:44.702 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:44 smithi136 ceph-mon[36436]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:44.702 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:44 smithi136 ceph-mon[36436]: mgrmap e18: smithi094.purlzp(active, since 30s), standbys: smithi136.jagvwt 2024-04-06T03:33:44.702 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:44 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi136.jagvwt", "id": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:33:44.702 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:44 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:33:44.702 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:44 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:33:44.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.840+0000 7f90a5eda700 1 -- 172.21.15.136:0/3774922387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90880061f0 msgr2=0x7f9088008650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:44.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.840+0000 7f90a5eda700 1 --2- 172.21.15.136:0/3774922387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90880061f0 0x7f9088008650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f90a00fdc70 tx=0x7f9094002040 comp rx=0 tx=0).stop 2024-04-06T03:33:44.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.840+0000 7f90a5eda700 1 -- 172.21.15.136:0/3774922387 shutdown_connections 2024-04-06T03:33:44.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.840+0000 7f90a5eda700 1 --2- 172.21.15.136:0/3774922387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90880061f0 0x7f9088008650 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:44.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.840+0000 7f90a5eda700 1 -- 172.21.15.136:0/3774922387 >> 172.21.15.136:0/3774922387 conn(0x7f90a00f86f0 msgr2=0x7f90a00fab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:44.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.840+0000 7f90a5eda700 1 -- 172.21.15.136:0/3774922387 shutdown_connections 2024-04-06T03:33:44.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.840+0000 7f90a5eda700 1 -- 172.21.15.136:0/3774922387 wait complete. 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f90a5eda700 1 Processor -- start 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f90a5eda700 1 -- start start 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f90a5eda700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90880061f0 0x7f90a0113330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f90a5eda700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 0x7f90a010d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f90a5eda700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a0113e80 con 0x7f90880061f0 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f90a5eda700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a010d7d0 con 0x7f90a0113870 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f909effd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 0x7f90a010d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f909effd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 0x7f90a010d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:38050/0 (socket says 172.21.15.136:38050) 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f909effd700 1 -- 172.21.15.136:0/431948418 learned_addr learned my addr 172.21.15.136:0/431948418 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f909f7fe700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90880061f0 0x7f90a0113330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f909effd700 1 -- 172.21.15.136:0/431948418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 msgr2=0x7f90a010d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-06T03:33:44.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f909effd700 1 -- 172.21.15.136:0/431948418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 msgr2=0x7f90a010d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f909effd700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 0x7f90a010d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.841+0000 7f909effd700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 0x7f90a010d290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.842+0000 7f909f7fe700 1 -- 172.21.15.136:0/431948418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 msgr2=0x7f90a010d290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.842+0000 7f909f7fe700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 0x7f90a010d290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.842+0000 7f909f7fe700 1 -- 172.21.15.136:0/431948418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094005040 con 0x7f90880061f0 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.842+0000 7f909f7fe700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90880061f0 0x7f90a0113330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f90a00fdc70 tx=0x7f9094002040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.842+0000 7f9087fff700 1 -- 172.21.15.136:0/431948418 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9094004db0 con 0x7f90880061f0 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.842+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a010da50 con 0x7f90880061f0 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.842+0000 7f9087fff700 1 -- 172.21.15.136:0/431948418 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9094018e30 con 0x7f90880061f0 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.842+0000 7f9087fff700 1 -- 172.21.15.136:0/431948418 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9094022be0 con 0x7f90880061f0 2024-04-06T03:33:44.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.842+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a010dfb0 con 0x7f90880061f0 2024-04-06T03:33:44.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.843+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a0101ec0 con 0x7f90880061f0 2024-04-06T03:33:44.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.844+0000 7f9087fff700 1 -- 172.21.15.136:0/431948418 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f9094009260 con 0x7f90880061f0 2024-04-06T03:33:44.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.844+0000 7f9087fff700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f908c069320 0x7f908c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:44.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.844+0000 7f9087fff700 1 -- 172.21.15.136:0/431948418 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9094088ad0 con 0x7f90880061f0 2024-04-06T03:33:44.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.844+0000 7f909effd700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f908c069320 0x7f908c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:44.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.845+0000 7f909effd700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f908c069320 0x7f908c06b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9090006fd0 tx=0x7f9090008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:44.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:44.849+0000 7f9087fff700 1 -- 172.21.15.136:0/431948418 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9094059970 con 0x7f90880061f0 2024-04-06T03:33:45.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.131+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f90a01048c0 con 0x7f90880061f0 2024-04-06T03:33:45.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.132+0000 7f9087fff700 1 -- 172.21.15.136:0/431948418 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7f9094088280 con 0x7f90880061f0 2024-04-06T03:33:45.134 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-06T03:33:45.134 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","modified":"2024-04-06T03:33:38.265304Z","created":"2024-04-06T03:31:02.176975Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi094","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi136","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-06T03:33:45.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f908c069320 msgr2=0x7f908c06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:45.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f908c069320 0x7f908c06b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9090006fd0 tx=0x7f9090008040 comp rx=0 tx=0).stop 2024-04-06T03:33:45.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90880061f0 msgr2=0x7f90a0113330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:45.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90880061f0 0x7f90a0113330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f90a00fdc70 tx=0x7f9094002040 comp rx=0 tx=0).stop 2024-04-06T03:33:45.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 shutdown_connections 2024-04-06T03:33:45.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f908c069320 0x7f908c06b7d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:45.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f90880061f0 0x7f90a0113330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:45.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 --2- 172.21.15.136:0/431948418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90a0113870 0x7f90a010d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:45.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 >> 172.21.15.136:0/431948418 conn(0x7f90a00f86f0 msgr2=0x7f90a01060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:45.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 shutdown_connections 2024-04-06T03:33:45.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:33:45.135+0000 7f90a5eda700 1 -- 172.21.15.136:0/431948418 wait complete. 2024-04-06T03:33:45.137 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-04-06T03:33:45.573 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:45 smithi136 ceph-mon[36436]: from='client.? 172.21.15.136:0/431948418' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:45.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:45 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/431948418' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-06T03:33:45.950 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-06T03:33:45.950 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-04-06T03:33:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:33:46.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:46.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:46.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:33:46.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.600 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:33:46.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:33:46.601 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:33:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:46.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:33:46.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:33:46.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:47.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:47 smithi094 ceph-mon[29713]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-06T03:33:47.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:47 smithi094 ceph-mon[29713]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-06T03:33:47.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:47 smithi094 ceph-mon[29713]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:33:47.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:47 smithi094 ceph-mon[29713]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:33:47.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:47 smithi094 ceph-mon[29713]: Reconfiguring mon.smithi094 (unknown last config time)... 2024-04-06T03:33:47.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:47 smithi094 ceph-mon[29713]: Reconfiguring daemon mon.smithi094 on smithi094 2024-04-06T03:33:47.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:47 smithi136 ceph-mon[36436]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-06T03:33:47.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:47 smithi136 ceph-mon[36436]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-06T03:33:47.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:47 smithi136 ceph-mon[36436]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:33:47.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:47 smithi136 ceph-mon[36436]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:33:47.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:47 smithi136 ceph-mon[36436]: Reconfiguring mon.smithi094 (unknown last config time)... 2024-04-06T03:33:47.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:47 smithi136 ceph-mon[36436]: Reconfiguring daemon mon.smithi094 on smithi094 2024-04-06T03:33:48.495 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:48 smithi094 ceph-mon[29713]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:48.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:48 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:48.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:48 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:48.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:48 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:48.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:48 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.purlzp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:33:48.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:48 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:33:48.496 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:48 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.694+0000 7fae731e9700 1 -- 172.21.15.94:0/2913909654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c1026b0 msgr2=0x7fae6c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.694+0000 7fae731e9700 1 --2- 172.21.15.94:0/2913909654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c1026b0 0x7fae6c0fbd20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fae58004940 tx=0x7fae58009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.694+0000 7fae731e9700 1 -- 172.21.15.94:0/2913909654 shutdown_connections 2024-04-06T03:33:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.694+0000 7fae731e9700 1 --2- 172.21.15.94:0/2913909654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c1026b0 0x7fae6c0fbd20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:48.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.694+0000 7fae731e9700 1 --2- 172.21.15.94:0/2913909654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6c101d10 0x7fae6c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:48.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.694+0000 7fae731e9700 1 -- 172.21.15.94:0/2913909654 >> 172.21.15.94:0/2913909654 conn(0x7fae6c0f7860 msgr2=0x7fae6c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:48.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.694+0000 7fae731e9700 1 -- 172.21.15.94:0/2913909654 shutdown_connections 2024-04-06T03:33:48.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.694+0000 7fae731e9700 1 -- 172.21.15.94:0/2913909654 wait complete. 2024-04-06T03:33:48.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.695+0000 7fae731e9700 1 Processor -- start 2024-04-06T03:33:48.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.695+0000 7fae731e9700 1 -- start start 2024-04-06T03:33:48.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.695+0000 7fae731e9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c101d10 0x7fae6c115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.695+0000 7fae731e9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6c1026b0 0x7fae6c1160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.695+0000 7fae731e9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c1167c0 con 0x7fae6c101d10 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.695+0000 7fae731e9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c10fd10 con 0x7fae6c1026b0 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.695+0000 7fae70f85700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c101d10 0x7fae6c115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.695+0000 7fae70f85700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c101d10 0x7fae6c115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45150/0 (socket says 172.21.15.94:45150) 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.695+0000 7fae70f85700 1 -- 172.21.15.94:0/79506340 learned_addr learned my addr 172.21.15.94:0/79506340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae6bfff700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6c1026b0 0x7fae6c1160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae70f85700 1 -- 172.21.15.94:0/79506340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6c1026b0 msgr2=0x7fae6c1160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae70f85700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6c1026b0 0x7fae6c1160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae70f85700 1 -- 172.21.15.94:0/79506340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58005040 con 0x7fae6c101d10 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae70f85700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c101d10 0x7fae6c115ba0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae6000b790 tx=0x7fae60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae697fa700 1 -- 172.21.15.94:0/79506340 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae6000f770 con 0x7fae6c101d10 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae697fa700 1 -- 172.21.15.94:0/79506340 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae6000fda0 con 0x7fae6c101d10 2024-04-06T03:33:48.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae697fa700 1 -- 172.21.15.94:0/79506340 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae6000d310 con 0x7fae6c101d10 2024-04-06T03:33:48.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6c10ff90 con 0x7fae6c101d10 2024-04-06T03:33:48.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.696+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6c110550 con 0x7fae6c101d10 2024-04-06T03:33:48.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.697+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6c106230 con 0x7fae6c101d10 2024-04-06T03:33:48.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.698+0000 7fae697fa700 1 -- 172.21.15.94:0/79506340 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fae60010420 con 0x7fae6c101d10 2024-04-06T03:33:48.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.698+0000 7fae697fa700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fae5c069210 0x7fae5c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:48.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.698+0000 7fae697fa700 1 -- 172.21.15.94:0/79506340 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fae60085f90 con 0x7fae6c101d10 2024-04-06T03:33:48.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.698+0000 7fae6bfff700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fae5c069210 0x7fae5c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:48.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.699+0000 7fae6bfff700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fae5c069210 0x7fae5c06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae6c111440 tx=0x7fae58009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:48.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.703+0000 7fae697fa700 1 -- 172.21.15.94:0/79506340 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fae60056ed0 con 0x7fae6c101d10 2024-04-06T03:33:48.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:48 smithi136 ceph-mon[36436]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:48.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:48 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:48.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:48 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:48.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:48 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:48.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:48 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.purlzp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:33:48.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:48 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:33:48.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:48 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:48.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.895+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fae6c108e60 con 0x7fae6c101d10 2024-04-06T03:33:48.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.895+0000 7fae697fa700 1 -- 172.21.15.94:0/79506340 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7fae60056a60 con 0x7fae6c101d10 2024-04-06T03:33:48.897 INFO:teuthology.orchestra.run.smithi094.stdout:# minimal ceph.conf for c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:33:48.897 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-04-06T03:33:48.897 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:33:48.897 INFO:teuthology.orchestra.run.smithi094.stdout: mon_host = [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fae5c069210 msgr2=0x7fae5c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fae5c069210 0x7fae5c06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fae6c111440 tx=0x7fae58009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c101d10 msgr2=0x7fae6c115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c101d10 0x7fae6c115ba0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae6000b790 tx=0x7fae60009250 comp rx=0 tx=0).stop 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 shutdown_connections 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fae5c069210 0x7fae5c06b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae6c101d10 0x7fae6c115ba0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 --2- 172.21.15.94:0/79506340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6c1026b0 0x7fae6c1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 >> 172.21.15.94:0/79506340 conn(0x7fae6c0f7860 msgr2=0x7fae6c0f8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 shutdown_connections 2024-04-06T03:33:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:48.897+0000 7fae731e9700 1 -- 172.21.15.94:0/79506340 wait complete. 2024-04-06T03:33:49.527 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-06T03:33:49.528 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:33:49.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-04-06T03:33:49.557 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:33:49.557 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:33:49.624 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:33:49.625 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-04-06T03:33:49.654 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:49 smithi136 ceph-mon[36436]: Reconfiguring mgr.smithi094.purlzp (unknown last config time)... 2024-04-06T03:33:49.654 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:49 smithi136 ceph-mon[36436]: Reconfiguring daemon mgr.smithi094.purlzp on smithi094 2024-04-06T03:33:49.654 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:49 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/79506340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:49.661 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:33:49.661 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:33:49.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:49 smithi094 ceph-mon[29713]: Reconfiguring mgr.smithi094.purlzp (unknown last config time)... 2024-04-06T03:33:49.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:49 smithi094 ceph-mon[29713]: Reconfiguring daemon mgr.smithi094.purlzp on smithi094 2024-04-06T03:33:49.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:49 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/79506340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:49.730 INFO:tasks.cephadm:Deploying OSDs... 2024-04-06T03:33:49.731 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:33:49.731 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-04-06T03:33:49.746 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-06T03:33:49.746 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-04-06T03:33:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-06T03:33:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-06T03:33:49.803 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-06T03:33:49.803 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:33:49.803 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-06T03:33:49.803 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-06 03:33:04.831231276 +0000 2024-04-06T03:33:49.804 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-06 03:25:13.461233422 +0000 2024-04-06T03:33:49.804 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-06 03:25:13.461233422 +0000 2024-04-06T03:33:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-06 03:25:13.461233422 +0000 2024-04-06T03:33:49.804 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-06T03:33:49.870 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-06T03:33:49.870 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-06T03:33:49.870 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.7458e-05 s, 5.9 MB/s 2024-04-06T03:33:49.871 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-06T03:33:49.926 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-04-06T03:33:49.981 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-06T03:33:49.981 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-06T03:33:49.981 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-06T03:33:49.981 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:33:49.982 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-06T03:33:49.982 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-06 03:33:04.832231260 +0000 2024-04-06T03:33:49.982 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-06 03:25:13.791227816 +0000 2024-04-06T03:33:49.982 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-06 03:25:13.791227816 +0000 2024-04-06T03:33:49.982 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-06 03:25:13.791227816 +0000 2024-04-06T03:33:49.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-06T03:33:50.049 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-06T03:33:50.049 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-06T03:33:50.049 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.2545e-05 s, 5.5 MB/s 2024-04-06T03:33:50.050 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-06T03:33:50.106 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-04-06T03:33:50.162 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-06T03:33:50.162 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-06T03:33:50.162 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-06T03:33:50.162 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:33:50.162 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-06T03:33:50.162 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-06 03:33:04.832231260 +0000 2024-04-06T03:33:50.162 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-06 03:25:14.121222210 +0000 2024-04-06T03:33:50.162 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-06 03:25:14.121222210 +0000 2024-04-06T03:33:50.162 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-06 03:25:14.121222210 +0000 2024-04-06T03:33:50.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-06T03:33:50.228 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-06T03:33:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-06T03:33:50.229 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000106874 s, 4.8 MB/s 2024-04-06T03:33:50.230 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-06T03:33:50.286 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-04-06T03:33:50.341 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-06T03:33:50.342 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-06T03:33:50.342 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-04-06T03:33:50.342 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:33:50.342 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-06T03:33:50.342 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-06 03:33:04.832231260 +0000 2024-04-06T03:33:50.342 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-06 03:25:14.454216553 +0000 2024-04-06T03:33:50.342 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-06 03:25:14.454216553 +0000 2024-04-06T03:33:50.342 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-06 03:25:14.454216553 +0000 2024-04-06T03:33:50.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-06T03:33:50.409 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-06T03:33:50.409 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-06T03:33:50.409 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000123725 s, 4.1 MB/s 2024-04-06T03:33:50.411 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-06T03:33:50.466 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:33:50.467 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-04-06T03:33:50.483 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-06T03:33:50.483 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-04-06T03:33:50.538 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-06T03:33:50.538 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-06T03:33:50.539 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-06T03:33:50.539 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:33:50.539 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-06T03:33:50.539 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-06 03:33:36.925040322 +0000 2024-04-06T03:33:50.539 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-06 03:25:06.032687085 +0000 2024-04-06T03:33:50.539 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-06 03:25:06.032687085 +0000 2024-04-06T03:33:50.539 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-06 03:25:06.032687085 +0000 2024-04-06T03:33:50.539 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-06T03:33:50.604 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-06T03:33:50.604 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-06T03:33:50.604 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000235505 s, 2.2 MB/s 2024-04-06T03:33:50.605 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-06T03:33:50.660 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-04-06T03:33:50.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:50 smithi094 ceph-mon[29713]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-06T03:33:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-06T03:33:50.714 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-06T03:33:50.714 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:33:50.715 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-06T03:33:50.715 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-06 03:33:36.926040300 +0000 2024-04-06T03:33:50.715 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-06 03:25:06.361679678 +0000 2024-04-06T03:33:50.715 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-06 03:25:06.361679678 +0000 2024-04-06T03:33:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-06 03:25:06.361679678 +0000 2024-04-06T03:33:50.715 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-06T03:33:50.777 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:50 smithi136 ceph-mon[36436]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:50.779 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-06T03:33:50.779 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-06T03:33:50.780 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000202098 s, 2.5 MB/s 2024-04-06T03:33:50.781 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-06T03:33:50.837 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-04-06T03:33:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-06T03:33:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-06T03:33:50.893 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-06T03:33:50.893 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:33:50.893 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-06T03:33:50.893 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-06 03:33:36.926040300 +0000 2024-04-06T03:33:50.893 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-06 03:25:06.710671821 +0000 2024-04-06T03:33:50.893 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-06 03:25:06.710671821 +0000 2024-04-06T03:33:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-06 03:25:06.710671821 +0000 2024-04-06T03:33:50.893 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-06T03:33:50.959 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-06T03:33:50.959 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-06T03:33:50.959 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000208629 s, 2.5 MB/s 2024-04-06T03:33:50.960 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-06T03:33:51.016 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-04-06T03:33:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-06T03:33:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-06T03:33:51.071 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-06T03:33:51.071 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:33:51.071 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-06T03:33:51.071 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-06 03:33:36.927040277 +0000 2024-04-06T03:33:51.071 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-06 03:25:07.051664145 +0000 2024-04-06T03:33:51.071 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-06 03:25:07.051664145 +0000 2024-04-06T03:33:51.071 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-06 03:25:07.051664145 +0000 2024-04-06T03:33:51.072 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-06T03:33:51.136 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-06T03:33:51.137 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-06T03:33:51.137 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000302328 s, 1.7 MB/s 2024-04-06T03:33:51.138 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-06T03:33:51.193 INFO:tasks.cephadm:Deploying osd.0 on smithi094 with /dev/vg_nvme/lv_4... 2024-04-06T03:33:51.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-06T03:33:51.445 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:33:51.799 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:51 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:51.799 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:51 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:51.800 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:51 smithi094 ceph-mon[29713]: Reconfiguring crash.smithi094 (monmap changed)... 2024-04-06T03:33:51.800 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:51 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:33:51.800 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:51 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:51.800 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:51 smithi094 ceph-mon[29713]: Reconfiguring daemon crash.smithi094 on smithi094 2024-04-06T03:33:51.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:51 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:51.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:51 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:51.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:51 smithi136 ceph-mon[36436]: Reconfiguring crash.smithi094 (monmap changed)... 2024-04-06T03:33:51.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:51 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:33:51.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:51 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:51.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:51 smithi136 ceph-mon[36436]: Reconfiguring daemon crash.smithi094 on smithi094 2024-04-06T03:33:52.831 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:52 smithi094 ceph-mon[29713]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:52.831 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:52 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:52.831 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:52 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:52 smithi136 ceph-mon[36436]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:52 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:52 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:33:53.844 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:53 smithi094 ceph-mon[29713]: Reconfiguring alertmanager.smithi094 (dependencies changed)... 2024-04-06T03:33:53.844 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:53 smithi094 ceph-mon[29713]: Reconfiguring daemon alertmanager.smithi094 on smithi094 2024-04-06T03:33:53.847 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:53 smithi136 ceph-mon[36436]: Reconfiguring alertmanager.smithi094 (dependencies changed)... 2024-04-06T03:33:53.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:53 smithi136 ceph-mon[36436]: Reconfiguring daemon alertmanager.smithi094 on smithi094 2024-04-06T03:33:54.271 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-06T03:33:54.271 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-06T03:33:54.272 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-06T03:33:54.272 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-06T03:33:54.272 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010093 s, 1.0 GB/s 2024-04-06T03:33:54.272 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-06T03:33:54.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:54 smithi094 ceph-mon[29713]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:54.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:54 smithi136 ceph-mon[36436]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:55.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-04-06T03:33:56.398 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:33:56.509 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:56 smithi094 ceph-mon[29713]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:56.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:56 smithi136 ceph-mon[36436]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.278+0000 7f887c450700 1 -- 172.21.15.94:0/75391791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fe060 msgr2=0x7f88740fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.278+0000 7f887c450700 1 --2- 172.21.15.94:0/75391791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fe060 0x7f88740fe4d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8864004940 tx=0x7f8864009e30 comp rx=0 tx=0).stop 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.278+0000 7f887c450700 1 -- 172.21.15.94:0/75391791 shutdown_connections 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.278+0000 7f887c450700 1 --2- 172.21.15.94:0/75391791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fe060 0x7f88740fe4d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.278+0000 7f887c450700 1 --2- 172.21.15.94:0/75391791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88740fcef0 0x7f88740fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.278+0000 7f887c450700 1 -- 172.21.15.94:0/75391791 >> 172.21.15.94:0/75391791 conn(0x7f88740f85b0 msgr2=0x7f88740fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.278+0000 7f887c450700 1 -- 172.21.15.94:0/75391791 shutdown_connections 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.278+0000 7f887c450700 1 -- 172.21.15.94:0/75391791 wait complete. 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.279+0000 7f887c450700 1 Processor -- start 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.279+0000 7f887c450700 1 -- start start 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.279+0000 7f887c450700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fcef0 0x7f887410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.279+0000 7f887c450700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88740fe060 0x7f887410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.279+0000 7f887c450700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887410e580 con 0x7f88740fcef0 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.279+0000 7f887c450700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887410edd0 con 0x7f88740fe060 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.279+0000 7f887a1ec700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fcef0 0x7f887410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.279+0000 7f887a1ec700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fcef0 0x7f887410da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51308/0 (socket says 172.21.15.94:51308) 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.279+0000 7f887a1ec700 1 -- 172.21.15.94:0/2608533618 learned_addr learned my addr 172.21.15.94:0/2608533618 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f88799eb700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88740fe060 0x7f887410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f887a1ec700 1 -- 172.21.15.94:0/2608533618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88740fe060 msgr2=0x7f887410df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f887a1ec700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88740fe060 0x7f887410df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f887a1ec700 1 -- 172.21.15.94:0/2608533618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8864005040 con 0x7f88740fcef0 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f887a1ec700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fcef0 0x7f887410da40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8870002a50 tx=0x7f88700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f886affd700 1 -- 172.21.15.94:0/2608533618 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8870007cc0 con 0x7f88740fcef0 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f886affd700 1 -- 172.21.15.94:0/2608533618 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f887000f040 con 0x7f88740fcef0 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887410f0b0 con 0x7f88740fcef0 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f886affd700 1 -- 172.21.15.94:0/2608533618 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f887000da30 con 0x7f88740fcef0 2024-04-06T03:33:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.280+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887410f5f0 con 0x7f88740fcef0 2024-04-06T03:33:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.281+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887405f5f0 con 0x7f88740fcef0 2024-04-06T03:33:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.282+0000 7f886affd700 1 -- 172.21.15.94:0/2608533618 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f887000db90 con 0x7f88740fcef0 2024-04-06T03:33:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.282+0000 7f886affd700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8860069210 0x7f886006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:33:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.282+0000 7f886affd700 1 -- 172.21.15.94:0/2608533618 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f8870085e70 con 0x7f88740fcef0 2024-04-06T03:33:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.282+0000 7f88799eb700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8860069210 0x7f886006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:33:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.283+0000 7f88799eb700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8860069210 0x7f886006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f886400b970 tx=0x7f8864009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:33:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.287+0000 7f886affd700 1 -- 172.21.15.94:0/2608533618 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8870056c90 con 0x7f88740fcef0 2024-04-06T03:33:58.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:33:58.494+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f887410f970 con 0x7f8860069210 2024-04-06T03:33:58.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:58 smithi094 ceph-mon[29713]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:58.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:58 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:33:58.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:58 smithi136 ceph-mon[36436]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:33:58.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:58 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:33:59.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:59 smithi136 ceph-mon[36436]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:33:59.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:33:59.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:33:59.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:33:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:33:59.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:59 smithi094 ceph-mon[29713]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:33:59.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:33:59.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:33:59.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:33:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:00.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:00 smithi136 ceph-mon[36436]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:00.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:00 smithi094 ceph-mon[29713]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:01.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:01 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:01.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:01 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:01.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:01 smithi136 ceph-mon[36436]: Reconfiguring grafana.smithi094 (dependencies changed)... 2024-04-06T03:34:01.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:01 smithi136 ceph-mon[36436]: Reconfiguring daemon grafana.smithi094 on smithi094 2024-04-06T03:34:01.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:01 smithi136 ceph-mon[36436]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:01.882 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:01 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:01.882 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:01 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:01.882 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:01 smithi094 ceph-mon[29713]: Reconfiguring grafana.smithi094 (dependencies changed)... 2024-04-06T03:34:01.882 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:01 smithi094 ceph-mon[29713]: Reconfiguring daemon grafana.smithi094 on smithi094 2024-04-06T03:34:01.882 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:01 smithi094 ceph-mon[29713]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:02 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3115624952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f51dba0-a74e-444c-b4ab-d5f1fa68ef94"}]: dispatch 2024-04-06T03:34:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:02 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3115624952' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f51dba0-a74e-444c-b4ab-d5f1fa68ef94"}]': finished 2024-04-06T03:34:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:02 smithi136 ceph-mon[36436]: osdmap e6: 1 total, 0 up, 1 in 2024-04-06T03:34:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:02 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3201208618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:34:02.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:02 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3115624952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f51dba0-a74e-444c-b4ab-d5f1fa68ef94"}]: dispatch 2024-04-06T03:34:02.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:02 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3115624952' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f51dba0-a74e-444c-b4ab-d5f1fa68ef94"}]': finished 2024-04-06T03:34:02.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:02 smithi094 ceph-mon[29713]: osdmap e6: 1 total, 0 up, 1 in 2024-04-06T03:34:02.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:02.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:02 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3201208618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:34:03.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:03 smithi136 ceph-mon[36436]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:03.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:03 smithi094 ceph-mon[29713]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:06.087 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:06 smithi094 ceph-mon[29713]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:06.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:06 smithi136 ceph-mon[36436]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:08.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:08 smithi094 ceph-mon[29713]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:08.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:08 smithi136 ceph-mon[36436]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:10.341 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:10 smithi094 ceph-mon[29713]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:10.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:10 smithi136 ceph-mon[36436]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:11.809 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:11 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:11.809 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:11 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:11.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:11 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:11.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:11 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:12.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:12 smithi136 ceph-mon[36436]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-04-06T03:34:12.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:12 smithi136 ceph-mon[36436]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-04-06T03:34:12.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:12 smithi136 ceph-mon[36436]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:12.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-06T03:34:12.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:12.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:12 smithi094 ceph-mon[29713]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-04-06T03:34:12.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:12 smithi094 ceph-mon[29713]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-04-06T03:34:12.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:12 smithi094 ceph-mon[29713]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:12.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-06T03:34:12.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:13.637 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:13 smithi094 ceph-mon[29713]: Deploying daemon osd.0 on smithi094 2024-04-06T03:34:13.637 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:34:13.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:13 smithi136 ceph-mon[36436]: Deploying daemon osd.0 on smithi094 2024-04-06T03:34:13.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:34:14.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:14 smithi136 ceph-mon[36436]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:14.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:14 smithi094 ceph-mon[29713]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:15.847 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:15 smithi094 ceph-mon[29713]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:15.847 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:15 smithi136 ceph-mon[36436]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:17.261 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:17.261 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:17.261 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:16 smithi136 ceph-mon[36436]: Reconfiguring crash.smithi136 (monmap changed)... 2024-04-06T03:34:17.261 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi136", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:34:17.261 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:17.262 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:16 smithi136 ceph-mon[36436]: Reconfiguring daemon crash.smithi136 on smithi136 2024-04-06T03:34:17.305 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:17.305 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:17.305 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:16 smithi094 ceph-mon[29713]: Reconfiguring crash.smithi136 (monmap changed)... 2024-04-06T03:34:17.305 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi136", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:34:17.305 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:17.305 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:16 smithi094 ceph-mon[29713]: Reconfiguring daemon crash.smithi136 on smithi136 2024-04-06T03:34:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:18 smithi094 ceph-mon[29713]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:18 smithi094 ceph-mon[29713]: Reconfiguring mgr.smithi136.jagvwt (monmap changed)... 2024-04-06T03:34:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:34:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:34:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:18 smithi094 ceph-mon[29713]: Reconfiguring daemon mgr.smithi136.jagvwt on smithi136 2024-04-06T03:34:18.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:18 smithi136 ceph-mon[36436]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:18.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:18 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:18.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:18 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:18.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:18 smithi136 ceph-mon[36436]: Reconfiguring mgr.smithi136.jagvwt (monmap changed)... 2024-04-06T03:34:18.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:18 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:34:18.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:18 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:34:18.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:18 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:18.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:18 smithi136 ceph-mon[36436]: Reconfiguring daemon mgr.smithi136.jagvwt on smithi136 2024-04-06T03:34:19.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:19 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:19.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:19 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:19.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:19 smithi094 ceph-mon[29713]: Reconfiguring mon.smithi136 (monmap changed)... 2024-04-06T03:34:19.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:19 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:34:19.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:19 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:34:19.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:19 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:19.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:19 smithi094 ceph-mon[29713]: Reconfiguring daemon mon.smithi136 on smithi136 2024-04-06T03:34:19.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:19 smithi094 ceph-mon[29713]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:19.958 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:19 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:19.958 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:19 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:19.958 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:19 smithi136 ceph-mon[36436]: Reconfiguring mon.smithi136 (monmap changed)... 2024-04-06T03:34:19.958 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:19 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:34:19.958 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:19 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:34:19.958 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:19 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:19.958 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:19 smithi136 ceph-mon[36436]: Reconfiguring daemon mon.smithi136 on smithi136 2024-04-06T03:34:19.958 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:19 smithi136 ceph-mon[36436]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:34:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:34:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi094.front.sepia.ceph.com:9093"}]: dispatch 2024-04-06T03:34:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi094.front.sepia.ceph.com:9093"}]: dispatch 2024-04-06T03:34:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi094.front.sepia.ceph.com:3000"}]: dispatch 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi094.front.sepia.ceph.com:3000"}]: dispatch 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi094.front.sepia.ceph.com:9095"}]: dispatch 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi094.front.sepia.ceph.com:9095"}]: dispatch 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:34:21.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:34:21.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi094.front.sepia.ceph.com:9093"}]: dispatch 2024-04-06T03:34:21.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi094.front.sepia.ceph.com:9093"}]: dispatch 2024-04-06T03:34:21.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:34:21.392 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi094.front.sepia.ceph.com:3000"}]: dispatch 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi094.front.sepia.ceph.com:3000"}]: dispatch 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi094.front.sepia.ceph.com:9095"}]: dispatch 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi094.front.sepia.ceph.com:9095"}]: dispatch 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:21.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:22.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:22 smithi136 ceph-mon[36436]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:22.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:22 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:22.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:22 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:22.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:22 smithi094 ceph-mon[29713]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:22.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:22 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:22.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:22 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:24.097 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:24 smithi094 ceph-mon[29713]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:24.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:24 smithi136 ceph-mon[36436]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.971+0000 7f886affd700 1 -- 172.21.15.94:0/2608533618 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f887410f970 con 0x7f8860069210 2024-04-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 0 on host 'smithi094' 2024-04-06T03:34:24.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8860069210 msgr2=0x7f886006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8860069210 0x7f886006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f886400b970 tx=0x7f8864009e30 comp rx=0 tx=0).stop 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fcef0 msgr2=0x7f887410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fcef0 0x7f887410da40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8870002a50 tx=0x7f88700046b0 comp rx=0 tx=0).stop 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 shutdown_connections 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8860069210 0x7f886006b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88740fcef0 0x7f887410da40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 --2- 172.21.15.94:0/2608533618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88740fe060 0x7f887410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 >> 172.21.15.94:0/2608533618 conn(0x7f88740f85b0 msgr2=0x7f88740f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 shutdown_connections 2024-04-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:24.973+0000 7f887c450700 1 -- 172.21.15.94:0/2608533618 wait complete. 2024-04-06T03:34:25.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:25 smithi136 ceph-mon[36436]: from='osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-06T03:34:25.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:25.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:25.406 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:25 smithi094 ceph-mon[29713]: from='osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-06T03:34:25.406 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:25.406 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:26.083 DEBUG:teuthology.orchestra.run.smithi094:osd.0> sudo journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.0.service 2024-04-06T03:34:26.085 INFO:tasks.cephadm:Deploying osd.1 on smithi094 with /dev/vg_nvme/lv_3... 2024-04-06T03:34:26.085 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-06T03:34:26.108 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:26 smithi094 ceph-mon[29713]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:26.109 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:26 smithi094 ceph-mon[29713]: from='osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-06T03:34:26.109 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:26 smithi094 ceph-mon[29713]: osdmap e7: 1 total, 0 up, 1 in 2024-04-06T03:34:26.109 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:26 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:26.109 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:26 smithi094 ceph-mon[29713]: from='osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:34:26.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:26 smithi136 ceph-mon[36436]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:26.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:26 smithi136 ceph-mon[36436]: from='osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-06T03:34:26.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:26 smithi136 ceph-mon[36436]: osdmap e7: 1 total, 0 up, 1 in 2024-04-06T03:34:26.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:26 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:26.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:26 smithi136 ceph-mon[36436]: from='osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:34:26.723 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:34:27.148 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:27 smithi094 ceph-mon[29713]: from='osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-06T03:34:27.149 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:27 smithi094 ceph-mon[29713]: osdmap e8: 1 total, 0 up, 1 in 2024-04-06T03:34:27.149 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:27.149 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:27.149 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:27.149 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:27 smithi136 ceph-mon[36436]: from='osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-06T03:34:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:27 smithi136 ceph-mon[36436]: osdmap e8: 1 total, 0 up, 1 in 2024-04-06T03:34:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:28.077 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:28 smithi094 ceph-mon[29713]: purged_snaps scrub starts 2024-04-06T03:34:28.077 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:28 smithi094 ceph-mon[29713]: purged_snaps scrub ok 2024-04-06T03:34:28.078 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:28 smithi094 ceph-mon[29713]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:28.078 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:28 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:28.078 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:28 smithi094 ceph-mon[29713]: osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] boot 2024-04-06T03:34:28.078 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:28 smithi094 ceph-mon[29713]: osdmap e9: 1 total, 1 up, 1 in 2024-04-06T03:34:28.078 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:28 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:28.078 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:28 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:34:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:28 smithi136 ceph-mon[36436]: purged_snaps scrub starts 2024-04-06T03:34:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:28 smithi136 ceph-mon[36436]: purged_snaps scrub ok 2024-04-06T03:34:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:28 smithi136 ceph-mon[36436]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-06T03:34:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:28 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:28 smithi136 ceph-mon[36436]: osd.0 [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] boot 2024-04-06T03:34:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:28 smithi136 ceph-mon[36436]: osdmap e9: 1 total, 1 up, 1 in 2024-04-06T03:34:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:28 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:34:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:28 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:34:29.116 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-06T03:34:29.117 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-06T03:34:29.117 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-06T03:34:29.117 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-06T03:34:29.117 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00932147 s, 1.1 GB/s 2024-04-06T03:34:29.117 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-06T03:34:29.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:29 smithi094 ceph-mon[29713]: osdmap e10: 1 total, 1 up, 1 in 2024-04-06T03:34:29.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:29 smithi136 ceph-mon[36436]: osdmap e10: 1 total, 1 up, 1 in 2024-04-06T03:34:30.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:30 smithi094 ceph-mon[29713]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:30.302 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-04-06T03:34:30.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:30 smithi136 ceph-mon[36436]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:34:32.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:32.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: Detected new or changed devices on smithi094 2024-04-06T03:34:32.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:32.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:32.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:34:32.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: Adjusting osd_memory_target on smithi094 to 12864M 2024-04-06T03:34:32.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:32.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:32.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:34:32.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:32.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: Detected new or changed devices on smithi094 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: Adjusting osd_memory_target on smithi094 to 12864M 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:34:32.594 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:32.595 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:34:33.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.734+0000 7f8dadad4700 1 -- 172.21.15.94:0/4099605505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 msgr2=0x7f8da80fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:34:33.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.734+0000 7f8dadad4700 1 --2- 172.21.15.94:0/4099605505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 0x7f8da80fe4c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004940 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-04-06T03:34:33.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.734+0000 7f8dadad4700 1 -- 172.21.15.94:0/4099605505 shutdown_connections 2024-04-06T03:34:33.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.734+0000 7f8dadad4700 1 --2- 172.21.15.94:0/4099605505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 0x7f8da80fe4c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.734+0000 7f8dadad4700 1 --2- 172.21.15.94:0/4099605505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da80fce50 0x7f8da80fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.734+0000 7f8dadad4700 1 -- 172.21.15.94:0/4099605505 >> 172.21.15.94:0/4099605505 conn(0x7f8da80f8580 msgr2=0x7f8da80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.734+0000 7f8dadad4700 1 -- 172.21.15.94:0/4099605505 shutdown_connections 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.734+0000 7f8dadad4700 1 -- 172.21.15.94:0/4099605505 wait complete. 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.734+0000 7f8dadad4700 1 Processor -- start 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8dadad4700 1 -- start start 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8dadad4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da80fce50 0x7f8da8071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8dadad4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 0x7f8da8070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8dadad4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8072260 con 0x7f8da80fe050 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8dadad4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8070850 con 0x7f8da80fce50 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8da6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 0x7f8da8070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da80fce50 0x7f8da8071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:34:33.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8da6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 0x7f8da8070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57218/0 (socket says 172.21.15.94:57218) 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8da6ffd700 1 -- 172.21.15.94:0/2504821413 learned_addr learned my addr 172.21.15.94:0/2504821413 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8da6ffd700 1 -- 172.21.15.94:0/2504821413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da80fce50 msgr2=0x7f8da8071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8da6ffd700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da80fce50 0x7f8da8071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8da6ffd700 1 -- 172.21.15.94:0/2504821413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c005040 con 0x7f8da80fe050 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8da77fe700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da80fce50 0x7f8da8071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8da6ffd700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 0x7f8da8070310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c013870 tx=0x7f8d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.735+0000 7f8d8ffff700 1 -- 172.21.15.94:0/2504821413 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c021070 con 0x7f8da80fe050 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.736+0000 7f8d8ffff700 1 -- 172.21.15.94:0/2504821413 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d9c009430 con 0x7f8da80fe050 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.736+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da8070aa0 con 0x7f8da80fe050 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.736+0000 7f8d8ffff700 1 -- 172.21.15.94:0/2504821413 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c00e070 con 0x7f8da80fe050 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.736+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da8070fe0 con 0x7f8da80fe050 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.737+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da804c710 con 0x7f8da80fe050 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.737+0000 7f8d8ffff700 1 -- 172.21.15.94:0/2504821413 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f8d9c01c750 con 0x7f8da80fe050 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.737+0000 7f8d8ffff700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8d900691a0 0x7f8d9006b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:34:33.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.738+0000 7f8d8ffff700 1 -- 172.21.15.94:0/2504821413 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f8d9c0875a0 con 0x7f8da80fe050 2024-04-06T03:34:33.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.738+0000 7f8da77fe700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8d900691a0 0x7f8d9006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:34:33.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.738+0000 7f8da77fe700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8d900691a0 0x7f8d9006b650 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8da80fdbc0 tx=0x7f8d9800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:34:33.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.742+0000 7f8d8ffff700 1 -- 172.21.15.94:0/2504821413 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8d9c0581e0 con 0x7f8da80fe050 2024-04-06T03:34:33.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:32.945+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8da81022f0 con 0x7f8d900691a0 2024-04-06T03:34:33.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:33 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:34:33.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:33 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:34:33.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:33 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:33.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:33 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:33.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:33 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:33.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:34:33.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:34:33.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:33.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:33.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:33 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:34.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:34 smithi094 ceph-mon[29713]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:34:34.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:34 smithi094 ceph-mon[29713]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:34 smithi136 ceph-mon[36436]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:34:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:34 smithi136 ceph-mon[36436]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:36.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:36 smithi136 ceph-mon[36436]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:36.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:36 smithi094 ceph-mon[29713]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:37 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2698930482' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2dd01097-c6d6-45e9-acee-ca5ea5f4e166"}]: dispatch 2024-04-06T03:34:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:37 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2698930482' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2dd01097-c6d6-45e9-acee-ca5ea5f4e166"}]': finished 2024-04-06T03:34:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:37 smithi136 ceph-mon[36436]: osdmap e11: 2 total, 1 up, 2 in 2024-04-06T03:34:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:37 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:37 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/4050873949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:34:37.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:37 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2698930482' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2dd01097-c6d6-45e9-acee-ca5ea5f4e166"}]: dispatch 2024-04-06T03:34:37.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:37 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2698930482' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2dd01097-c6d6-45e9-acee-ca5ea5f4e166"}]': finished 2024-04-06T03:34:37.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:37 smithi094 ceph-mon[29713]: osdmap e11: 2 total, 1 up, 2 in 2024-04-06T03:34:37.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:37 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:37.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:37 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/4050873949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:34:38.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:38 smithi136 ceph-mon[36436]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:38.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:38.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:38.740 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:38.740 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:34:38.740 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:38.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:38 smithi094 ceph-mon[29713]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:38.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:38.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:38.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:38.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:34:38.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:40.739 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:40 smithi094 ceph-mon[29713]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:40.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:40 smithi136 ceph-mon[36436]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:42.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:42 smithi094 ceph-mon[29713]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:42.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:42 smithi136 ceph-mon[36436]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:43.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:34:43.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-06T03:34:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:34:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-06T03:34:44.728 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:44 smithi094 ceph-mon[29713]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:44.729 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:44 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:44.729 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:44 smithi094 ceph-mon[29713]: Deploying daemon osd.1 on smithi094 2024-04-06T03:34:44.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:44 smithi136 ceph-mon[36436]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:44.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:44 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:34:44.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:44 smithi136 ceph-mon[36436]: Deploying daemon osd.1 on smithi094 2024-04-06T03:34:46.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:46 smithi136 ceph-mon[36436]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:46.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:46 smithi094 ceph-mon[29713]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:48.737 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:48 smithi136 ceph-mon[36436]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:48.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:48 smithi094 ceph-mon[29713]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:50.060 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:50.060 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:50.060 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:34:50.061 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:49 smithi136 ceph-mon[36436]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:50.164 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:50.164 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:50.164 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:34:50.164 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:49 smithi094 ceph-mon[29713]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:51.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:51 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:51.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:51 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:51.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:51 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:51.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:51 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:51.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.657+0000 7f8d8ffff700 1 -- 172.21.15.94:0/2504821413 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8da81022f0 con 0x7f8d900691a0 2024-04-06T03:34:51.659 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 1 on host 'smithi094' 2024-04-06T03:34:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8d900691a0 msgr2=0x7f8d9006b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:34:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8d900691a0 0x7f8d9006b650 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8da80fdbc0 tx=0x7f8d9800b040 comp rx=0 tx=0).stop 2024-04-06T03:34:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 msgr2=0x7f8da8070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:34:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 0x7f8da8070310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c013870 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-04-06T03:34:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 shutdown_connections 2024-04-06T03:34:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8d900691a0 0x7f8d9006b650 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da80fce50 0x7f8da8071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 --2- 172.21.15.94:0/2504821413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8da80fe050 0x7f8da8070310 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:51.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 >> 172.21.15.94:0/2504821413 conn(0x7f8da80f8580 msgr2=0x7f8da80f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:34:51.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 shutdown_connections 2024-04-06T03:34:51.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:51.660+0000 7f8dadad4700 1 -- 172.21.15.94:0/2504821413 wait complete. 2024-04-06T03:34:52.519 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:52 smithi094 ceph-mon[29713]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:52.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:52 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:52.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:52 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:52 smithi136 ceph-mon[36436]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:52 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:52 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:53.115 DEBUG:teuthology.orchestra.run.smithi094:osd.1> sudo journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.1.service 2024-04-06T03:34:53.117 INFO:tasks.cephadm:Deploying osd.2 on smithi094 with /dev/vg_nvme/lv_2... 2024-04-06T03:34:53.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-06T03:34:53.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:53 smithi136 ceph-mon[36436]: from='osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-06T03:34:53.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:53 smithi094 ceph-mon[29713]: from='osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-06T03:34:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:34:54.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:54 smithi094 ceph-mon[29713]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:54.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:54 smithi094 ceph-mon[29713]: from='osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-06T03:34:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:54 smithi094 ceph-mon[29713]: osdmap e12: 2 total, 1 up, 2 in 2024-04-06T03:34:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:54 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:54 smithi094 ceph-mon[29713]: from='osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:34:54.668 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:34:54 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[55408]: 2024-04-06T03:34:54.445+0000 7f8730093700 -1 osd.1 0 waiting for initial osdmap 2024-04-06T03:34:54.668 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:34:54 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[55408]: 2024-04-06T03:34:54.449+0000 7f872ce92700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:34:54.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:54 smithi136 ceph-mon[36436]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:54.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:54 smithi136 ceph-mon[36436]: from='osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-06T03:34:54.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:54 smithi136 ceph-mon[36436]: osdmap e12: 2 total, 1 up, 2 in 2024-04-06T03:34:54.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:54 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:54.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:54 smithi136 ceph-mon[36436]: from='osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:34:55.537 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:55 smithi094 ceph-mon[29713]: from='osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-06T03:34:55.537 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:55 smithi094 ceph-mon[29713]: osdmap e13: 2 total, 1 up, 2 in 2024-04-06T03:34:55.537 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:55 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:55.537 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:55 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:55.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:55 smithi136 ceph-mon[36436]: from='osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-06T03:34:55.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:55 smithi136 ceph-mon[36436]: osdmap e13: 2 total, 1 up, 2 in 2024-04-06T03:34:55.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:55 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:55.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:55 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:56.524 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-06T03:34:56.524 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-06T03:34:56.525 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-06T03:34:56.525 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-06T03:34:56.525 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0104159 s, 1.0 GB/s 2024-04-06T03:34:56.525 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-06T03:34:56.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:56 smithi136 ceph-mon[36436]: purged_snaps scrub starts 2024-04-06T03:34:56.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:56 smithi136 ceph-mon[36436]: purged_snaps scrub ok 2024-04-06T03:34:56.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:56 smithi136 ceph-mon[36436]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:56.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:56 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:56.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:56 smithi136 ceph-mon[36436]: osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] boot 2024-04-06T03:34:56.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:56 smithi136 ceph-mon[36436]: osdmap e14: 2 total, 2 up, 2 in 2024-04-06T03:34:56.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:56 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:56.890 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:56 smithi094 ceph-mon[29713]: purged_snaps scrub starts 2024-04-06T03:34:56.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:56 smithi094 ceph-mon[29713]: purged_snaps scrub ok 2024-04-06T03:34:56.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:56 smithi094 ceph-mon[29713]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-06T03:34:56.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:56 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:56.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:56 smithi094 ceph-mon[29713]: osd.1 [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] boot 2024-04-06T03:34:56.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:56 smithi094 ceph-mon[29713]: osdmap e14: 2 total, 2 up, 2 in 2024-04-06T03:34:56.891 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:56 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:34:57.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-04-06T03:34:57.470 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:34:57.759 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:57 smithi094 ceph-mon[29713]: osdmap e15: 2 total, 2 up, 2 in 2024-04-06T03:34:57.759 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:57 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:57.760 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:57 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:57.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:57 smithi136 ceph-mon[36436]: osdmap e15: 2 total, 2 up, 2 in 2024-04-06T03:34:57.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:57 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:57.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:57 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:34:58.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:58 smithi136 ceph-mon[36436]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:34:58.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:34:58 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:34:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:58 smithi094 ceph-mon[29713]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:34:58.748 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:34:58 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:34:59.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.621+0000 7f1e30b54700 1 -- 172.21.15.94:0/584468851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 msgr2=0x7f1e2c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:34:59.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.621+0000 7f1e30b54700 1 --2- 172.21.15.94:0/584468851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 0x7f1e2c0fc0e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009ce0 tx=0x7f1e1c004d10 comp rx=0 tx=0).stop 2024-04-06T03:34:59.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.621+0000 7f1e30b54700 1 -- 172.21.15.94:0/584468851 shutdown_connections 2024-04-06T03:34:59.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.621+0000 7f1e30b54700 1 --2- 172.21.15.94:0/584468851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e2c0fcf80 0x7f1e2c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:59.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.621+0000 7f1e30b54700 1 --2- 172.21.15.94:0/584468851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 0x7f1e2c0fc0e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:59.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.621+0000 7f1e30b54700 1 -- 172.21.15.94:0/584468851 >> 172.21.15.94:0/584468851 conn(0x7f1e2c0f7430 msgr2=0x7f1e2c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:34:59.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.621+0000 7f1e30b54700 1 -- 172.21.15.94:0/584468851 shutdown_connections 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.621+0000 7f1e30b54700 1 -- 172.21.15.94:0/584468851 wait complete. 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.622+0000 7f1e30b54700 1 Processor -- start 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.622+0000 7f1e30b54700 1 -- start start 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.622+0000 7f1e30b54700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 0x7f1e2c10db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.622+0000 7f1e30b54700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e2c0fcf80 0x7f1e2c10e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.622+0000 7f1e30b54700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c10e710 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.622+0000 7f1e30b54700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c113060 con 0x7f1e2c0fcf80 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.622+0000 7f1e2a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 0x7f1e2c10db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.622+0000 7f1e2a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 0x7f1e2c10db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55258/0 (socket says 172.21.15.94:55258) 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.622+0000 7f1e2a59c700 1 -- 172.21.15.94:0/4164365804 learned_addr learned my addr 172.21.15.94:0/4164365804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e29d9b700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e2c0fcf80 0x7f1e2c10e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e2a59c700 1 -- 172.21.15.94:0/4164365804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e2c0fcf80 msgr2=0x7f1e2c10e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e2a59c700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e2c0fcf80 0x7f1e2c10e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:34:59.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e2a59c700 1 -- 172.21.15.94:0/4164365804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c009940 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e29d9b700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e2c0fcf80 0x7f1e2c10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:34:59.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e2a59c700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 0x7f1e2c10db40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c004740 tx=0x7f1e1c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:34:59.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e22ffd700 1 -- 172.21.15.94:0/4164365804 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1c00e040 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e22ffd700 1 -- 172.21.15.94:0/4164365804 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e1c0128e0 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2c113260 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e22ffd700 1 -- 172.21.15.94:0/4164365804 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1c01ca80 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.623+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2c1137d0 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.624+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2c101b10 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.625+0000 7f1e22ffd700 1 -- 172.21.15.94:0/4164365804 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f1e1c012a50 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.625+0000 7f1e22ffd700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e18069210 0x7f1e1806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:34:59.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.625+0000 7f1e22ffd700 1 -- 172.21.15.94:0/4164365804 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f1e1c0880a0 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.625+0000 7f1e29d9b700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e18069210 0x7f1e1806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:34:59.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.626+0000 7f1e29d9b700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e18069210 0x7f1e1806b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e14006fd0 tx=0x7f1e14008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:34:59.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.630+0000 7f1e22ffd700 1 -- 172.21.15.94:0/4164365804 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1e1c058b30 con 0x7f1e2c0fbcd0 2024-04-06T03:34:59.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:34:59.826+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e2c02b3b0 con 0x7f1e18069210 2024-04-06T03:35:00.730 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:00 smithi094 ceph-mon[29713]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:00.730 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:00 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:35:00.730 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:00 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:35:00.730 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:00 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:00.828 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:00 smithi136 ceph-mon[36436]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:00.828 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:00 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:35:00.828 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:00 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:35:00.828 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:00 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:01.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:01 smithi094 ceph-mon[29713]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:35:01.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:01 smithi136 ceph-mon[36436]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:35:02.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:02 smithi094 ceph-mon[29713]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:02.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:02.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:02.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:02.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:02.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:02.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:02.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:02.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:02 smithi136 ceph-mon[36436]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:02.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:02.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:02.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:02.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:02.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:03.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:03 smithi094 ceph-mon[29713]: Detected new or changed devices on smithi094 2024-04-06T03:35:03.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:03 smithi094 ceph-mon[29713]: Adjusting osd_memory_target on smithi094 to 6432M 2024-04-06T03:35:03.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:03 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:03.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:03 smithi136 ceph-mon[36436]: Detected new or changed devices on smithi094 2024-04-06T03:35:03.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:03 smithi136 ceph-mon[36436]: Adjusting osd_memory_target on smithi094 to 6432M 2024-04-06T03:35:03.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:03 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:04.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:04 smithi136 ceph-mon[36436]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:04.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:04 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3002977490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9645da7c-ffbc-4415-8fa0-0fcb07c53f62"}]: dispatch 2024-04-06T03:35:04.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:04 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3002977490' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9645da7c-ffbc-4415-8fa0-0fcb07c53f62"}]': finished 2024-04-06T03:35:04.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:04 smithi136 ceph-mon[36436]: osdmap e16: 3 total, 2 up, 3 in 2024-04-06T03:35:04.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:04 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:04.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:04 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3488524065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:35:04.896 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:04 smithi094 ceph-mon[29713]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:04.896 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:04 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3002977490' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9645da7c-ffbc-4415-8fa0-0fcb07c53f62"}]: dispatch 2024-04-06T03:35:04.896 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:04 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3002977490' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9645da7c-ffbc-4415-8fa0-0fcb07c53f62"}]': finished 2024-04-06T03:35:04.896 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:04 smithi094 ceph-mon[29713]: osdmap e16: 3 total, 2 up, 3 in 2024-04-06T03:35:04.896 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:04 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:04.896 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:04 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3488524065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:35:06.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:06 smithi136 ceph-mon[36436]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:06.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:06 smithi094 ceph-mon[29713]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:07.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:07 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:07.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:07 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:07.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:07 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:07.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:07 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:07.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:07 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:07.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:07 smithi094 ceph-mon[29713]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:08.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:07 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:08.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:07 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:08.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:07 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:08.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:07 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:08.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:07 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:08.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:07 smithi136 ceph-mon[36436]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:10.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:10 smithi094 ceph-mon[29713]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:10.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:10 smithi136 ceph-mon[36436]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:11.293 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:11 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-06T03:35:11.293 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:11 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:11.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:11 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-06T03:35:11.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:11 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:12.314 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:12 smithi094 ceph-mon[29713]: Deploying daemon osd.2 on smithi094 2024-04-06T03:35:12.314 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:12 smithi094 ceph-mon[29713]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:12.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:12 smithi136 ceph-mon[36436]: Deploying daemon osd.2 on smithi094 2024-04-06T03:35:12.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:12 smithi136 ceph-mon[36436]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:13.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:35:13.370 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:35:14.309 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:14 smithi094 ceph-mon[29713]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:14.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:14 smithi136 ceph-mon[36436]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:16.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:16 smithi136 ceph-mon[36436]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:16.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:16.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:16.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:16.366 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:16 smithi094 ceph-mon[29713]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:16.366 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:16.366 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:16.366 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:18.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:18 smithi136 ceph-mon[36436]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:18.383 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:18 smithi094 ceph-mon[29713]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:19.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.537+0000 7f1e22ffd700 1 -- 172.21.15.94:0/4164365804 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1e2c02b3b0 con 0x7f1e18069210 2024-04-06T03:35:19.539 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 2 on host 'smithi094' 2024-04-06T03:35:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e18069210 msgr2=0x7f1e1806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:35:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e18069210 0x7f1e1806b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e14006fd0 tx=0x7f1e14008040 comp rx=0 tx=0).stop 2024-04-06T03:35:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 msgr2=0x7f1e2c10db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:35:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 0x7f1e2c10db40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c004740 tx=0x7f1e1c004d10 comp rx=0 tx=0).stop 2024-04-06T03:35:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 shutdown_connections 2024-04-06T03:35:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e18069210 0x7f1e1806b6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2c0fbcd0 0x7f1e2c10db40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 --2- 172.21.15.94:0/4164365804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e2c0fcf80 0x7f1e2c10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 >> 172.21.15.94:0/4164365804 conn(0x7f1e2c0f7430 msgr2=0x7f1e2c0667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:35:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.539+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 shutdown_connections 2024-04-06T03:35:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:35:19.540+0000 7f1e30b54700 1 -- 172.21.15.94:0/4164365804 wait complete. 2024-04-06T03:35:20.220 DEBUG:teuthology.orchestra.run.smithi094:osd.2> sudo journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.2.service 2024-04-06T03:35:20.223 INFO:tasks.cephadm:Deploying osd.3 on smithi136 with /dev/vg_nvme/lv_4... 2024-04-06T03:35:20.223 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-06T03:35:20.482 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi136/config 2024-04-06T03:35:20.790 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:20 smithi136 ceph-mon[36436]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:20.790 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:20 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:20.790 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:20 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:20.791 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:20 smithi136 ceph-mon[36436]: from='osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-06T03:35:20.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:20 smithi094 ceph-mon[29713]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:20.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:20.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:20.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:20 smithi094 ceph-mon[29713]: from='osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-06T03:35:21.741 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:21 smithi136 ceph-mon[36436]: from='osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-06T03:35:21.741 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:21 smithi136 ceph-mon[36436]: osdmap e17: 3 total, 2 up, 3 in 2024-04-06T03:35:21.741 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:21 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:21.741 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:21 smithi136 ceph-mon[36436]: from='osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:35:21.777 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:21 smithi094 ceph-mon[29713]: from='osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-06T03:35:21.777 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:21 smithi094 ceph-mon[29713]: osdmap e17: 3 total, 2 up, 3 in 2024-04-06T03:35:21.777 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:21 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:21.777 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:21 smithi094 ceph-mon[29713]: from='osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:35:21.777 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:21 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[62756]: 2024-04-06T03:35:21.517+0000 7f2679898700 -1 osd.2 0 waiting for initial osdmap 2024-04-06T03:35:21.777 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:21 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[62756]: 2024-04-06T03:35:21.522+0000 7f2674e94700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:35:22.295 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-06T03:35:22.295 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-06T03:35:22.296 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-06T03:35:22.296 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-06T03:35:22.296 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00732744 s, 1.4 GB/s 2024-04-06T03:35:22.296 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-06T03:35:22.625 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:22 smithi136 ceph-mon[36436]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:22.625 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:22 smithi136 ceph-mon[36436]: from='osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-06T03:35:22.625 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:22 smithi136 ceph-mon[36436]: osdmap e18: 3 total, 2 up, 3 in 2024-04-06T03:35:22.625 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:22 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:22.625 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:22 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:22.625 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:22 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:22.625 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:22 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:22.832 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:22 smithi094 ceph-mon[29713]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:22.832 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:22 smithi094 ceph-mon[29713]: from='osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-06T03:35:22.833 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:22 smithi094 ceph-mon[29713]: osdmap e18: 3 total, 2 up, 3 in 2024-04-06T03:35:22.833 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:22 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:22.833 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:22 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:22.833 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:22 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:22.833 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:22 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:22.892 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-04-06T03:35:23.147 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi136/config 2024-04-06T03:35:23.840 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:23 smithi136 ceph-mon[36436]: purged_snaps scrub starts 2024-04-06T03:35:23.840 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:23 smithi136 ceph-mon[36436]: purged_snaps scrub ok 2024-04-06T03:35:23.840 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:23 smithi136 ceph-mon[36436]: osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] boot 2024-04-06T03:35:23.840 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:23 smithi136 ceph-mon[36436]: osdmap e19: 3 total, 3 up, 3 in 2024-04-06T03:35:23.840 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:23 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:23.840 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:23 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-06T03:35:23.861 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:23 smithi094 ceph-mon[29713]: purged_snaps scrub starts 2024-04-06T03:35:23.861 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:23 smithi094 ceph-mon[29713]: purged_snaps scrub ok 2024-04-06T03:35:23.861 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:23 smithi094 ceph-mon[29713]: osd.2 [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] boot 2024-04-06T03:35:23.861 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:23 smithi094 ceph-mon[29713]: osdmap e19: 3 total, 3 up, 3 in 2024-04-06T03:35:23.861 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:23 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:35:23.861 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:23 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-06T03:35:24.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.570+0000 7f01a9c9c700 1 -- 172.21.15.136:0/1655029150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fbcd0 msgr2=0x7f01a40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:35:24.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.570+0000 7f01a9c9c700 1 --2- 172.21.15.136:0/1655029150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fbcd0 0x7f01a40fc0e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0194004940 tx=0x7f0194009e30 comp rx=0 tx=0).stop 2024-04-06T03:35:24.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.570+0000 7f01a9c9c700 1 -- 172.21.15.136:0/1655029150 shutdown_connections 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.570+0000 7f01a9c9c700 1 --2- 172.21.15.136:0/1655029150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01a40fcf80 0x7f01a40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.570+0000 7f01a9c9c700 1 --2- 172.21.15.136:0/1655029150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fbcd0 0x7f01a40fc0e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.570+0000 7f01a9c9c700 1 -- 172.21.15.136:0/1655029150 >> 172.21.15.136:0/1655029150 conn(0x7f01a40f7430 msgr2=0x7f01a40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.570+0000 7f01a9c9c700 1 -- 172.21.15.136:0/1655029150 shutdown_connections 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.570+0000 7f01a9c9c700 1 -- 172.21.15.136:0/1655029150 wait complete. 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.570+0000 7f01a9c9c700 1 Processor -- start 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a9c9c700 1 -- start start 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a9c9c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01a40fbcd0 0x7f01a40ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a9c9c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fcf80 0x7f01a40fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a9c9c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40fe850 con 0x7f01a40fbcd0 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a9c9c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a40fe990 con 0x7f01a40fcf80 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f019affd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fcf80 0x7f01a40fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f019affd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fcf80 0x7f01a40fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:34980/0 (socket says 172.21.15.136:34980) 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f019affd700 1 -- 172.21.15.136:0/3852246675 learned_addr learned my addr 172.21.15.136:0/3852246675 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:35:24.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01a40fbcd0 0x7f01a40ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a37fe700 1 -- 172.21.15.136:0/3852246675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fcf80 msgr2=0x7f01a40fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a37fe700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fcf80 0x7f01a40fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a37fe700 1 -- 172.21.15.136:0/3852246675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194005040 con 0x7f01a40fbcd0 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f019affd700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fcf80 0x7f01a40fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.571+0000 7f01a37fe700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01a40fbcd0 0x7f01a40ffd80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f019400b970 tx=0x7f0194009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.572+0000 7f01a0ff9700 1 -- 172.21.15.136:0/3852246675 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019400e040 con 0x7f01a40fbcd0 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.572+0000 7f01a0ff9700 1 -- 172.21.15.136:0/3852246675 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0194004290 con 0x7f01a40fbcd0 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.572+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a40febe0 con 0x7f01a40fbcd0 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.572+0000 7f01a0ff9700 1 -- 172.21.15.136:0/3852246675 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019401c3f0 con 0x7f01a40fbcd0 2024-04-06T03:35:24.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.572+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a41159c0 con 0x7f01a40fbcd0 2024-04-06T03:35:24.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.573+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a4101b10 con 0x7f01a40fbcd0 2024-04-06T03:35:24.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.573+0000 7f01a0ff9700 1 -- 172.21.15.136:0/3852246675 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f019400bbf0 con 0x7f01a40fbcd0 2024-04-06T03:35:24.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.574+0000 7f01a0ff9700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0190069210 0x7f019006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:35:24.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.574+0000 7f01a0ff9700 1 -- 172.21.15.136:0/3852246675 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3228+0+0 (secure 0 0 0) 0x7f0194088560 con 0x7f01a40fbcd0 2024-04-06T03:35:24.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.574+0000 7f019affd700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0190069210 0x7f019006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:35:24.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.574+0000 7f019affd700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0190069210 0x7f019006b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f018c00e5e0 tx=0x7f018c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:35:24.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.578+0000 7f01a0ff9700 1 -- 172.21.15.136:0/3852246675 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0194055210 con 0x7f01a40fbcd0 2024-04-06T03:35:24.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:24 smithi094 ceph-mon[29713]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:24.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:24 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-06T03:35:24.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:24 smithi094 ceph-mon[29713]: osdmap e20: 3 total, 3 up, 3 in 2024-04-06T03:35:24.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:24 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-06T03:35:24.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:24.771+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f01a405e400 con 0x7f0190069210 2024-04-06T03:35:24.773 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:24 smithi136 ceph-mon[36436]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-06T03:35:24.773 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:24 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-06T03:35:24.773 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:24 smithi136 ceph-mon[36436]: osdmap e20: 3 total, 3 up, 3 in 2024-04-06T03:35:24.773 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:24 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-06T03:35:25.103 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:35:24 smithi094 sudo[66877]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-06T03:35:25.103 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:35:24 smithi094 sudo[66877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:25.103 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66877]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:25.104 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66877]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:25.417 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66882]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-06T03:35:25.417 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66882]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:25.417 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66882]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:25.417 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66882]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:25.800 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: osdmap e21: 3 total, 3 up, 3 in 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='client.14296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: Detected new or changed devices on smithi094 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:25.801 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: Adjusting osd_memory_target on smithi094 to 4288M 2024-04-06T03:35:25.802 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:25.802 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:25.802 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:25.802 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:25.810 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66918]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-06T03:35:25.810 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66918]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:25.810 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66918]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:25.811 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66918]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: osdmap e21: 3 total, 3 up, 3 in 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='client.14296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: Detected new or changed devices on smithi094 2024-04-06T03:35:25.811 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:25.812 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:25.812 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:25.812 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:25.812 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:25.812 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: Adjusting osd_memory_target on smithi094 to 4288M 2024-04-06T03:35:25.812 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:25.812 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:25.812 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:25.812 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:26.068 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66999]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-06T03:35:26.068 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66999]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:26.068 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66999]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:26.068 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:35:25 smithi094 sudo[66999]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:26.331 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67079]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-06T03:35:26.331 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67079]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:26.331 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67079]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:26.332 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67079]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:26.594 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67098]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-06T03:35:26.594 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67098]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:26.594 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67098]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:26.594 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67098]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:26.855 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67170]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-06T03:35:26.855 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67170]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:26.855 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67170]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:26.855 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67170]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:27.120 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:26 smithi136 ceph-mon[36436]: osdmap e22: 3 total, 3 up, 3 in 2024-04-06T03:35:27.120 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:26 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:27.120 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:26 smithi136 ceph-mon[36436]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-06T03:35:27.120 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 sudo[39785]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-06T03:35:27.120 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 sudo[39785]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:27.120 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 sudo[39785]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:27.120 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 sudo[39785]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:27.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67208]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-06T03:35:27.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67208]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:27.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67208]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:27.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 sudo[67208]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:27.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 ceph-mon[29713]: osdmap e22: 3 total, 3 up, 3 in 2024-04-06T03:35:27.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:27.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:26 smithi094 ceph-mon[29713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-06T03:35:27.371 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 sudo[39788]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-06T03:35:27.371 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 sudo[39788]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-06T03:35:27.371 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 sudo[39788]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-06T03:35:27.371 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 sudo[39788]: pam_unix(sudo:session): session closed for user root 2024-04-06T03:35:27.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:27.368+0000 7f01a0ff9700 1 -- 172.21.15.136:0/3852246675 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f01940555e0 con 0x7f01a40fbcd0 2024-04-06T03:35:28.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-06T03:35:28.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:35:28.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:35:28.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-06T03:35:28.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:35:28.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:35:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-06T03:35:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: mgrmap e19: smithi094.purlzp(active, since 2m), standbys: smithi136.jagvwt 2024-04-06T03:35:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/3398503265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43321ebd-f195-4738-a2a5-f4d55a375bbc"}]: dispatch 2024-04-06T03:35:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43321ebd-f195-4738-a2a5-f4d55a375bbc"}]: dispatch 2024-04-06T03:35:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43321ebd-f195-4738-a2a5-f4d55a375bbc"}]': finished 2024-04-06T03:35:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: osdmap e23: 4 total, 3 up, 4 in 2024-04-06T03:35:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:27 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: mgrmap e19: smithi094.purlzp(active, since 2m), standbys: smithi136.jagvwt 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='client.? 172.21.15.136:0/3398503265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43321ebd-f195-4738-a2a5-f4d55a375bbc"}]: dispatch 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43321ebd-f195-4738-a2a5-f4d55a375bbc"}]: dispatch 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43321ebd-f195-4738-a2a5-f4d55a375bbc"}]': finished 2024-04-06T03:35:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: osdmap e23: 4 total, 3 up, 4 in 2024-04-06T03:35:28.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:27 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:29.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:28 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:35:29.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:28 smithi136 ceph-mon[36436]: from='client.? 172.21.15.136:0/2546343856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:35:29.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:28 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:35:29.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:28 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/2546343856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:35:30.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:29 smithi094 ceph-mon[29713]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:30.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:30.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:29 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:30.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:29 smithi136 ceph-mon[36436]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:30.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:29 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:30.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:29 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:31.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:30 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:31.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:30 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:31.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:30 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:31.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:30 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:31.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:30 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:31.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:30 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:32.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:31 smithi136 ceph-mon[36436]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:32.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:31 smithi094 ceph-mon[29713]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:34.298 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:34 smithi136 ceph-mon[36436]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-06T03:35:34.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:34 smithi094 ceph-mon[29713]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-06T03:35:35.310 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:35 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-06T03:35:35.310 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:35 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:35.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:35 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-06T03:35:35.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:35 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:36.292 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:36 smithi136 ceph-mon[36436]: Deploying daemon osd.3 on smithi136 2024-04-06T03:35:36.292 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:36 smithi136 ceph-mon[36436]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-06T03:35:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:36 smithi094 ceph-mon[29713]: Deploying daemon osd.3 on smithi136 2024-04-06T03:35:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:36 smithi094 ceph-mon[29713]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-06T03:35:38.332 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:38 smithi136 ceph-mon[36436]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-06T03:35:38.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:38 smithi094 ceph-mon[29713]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-06T03:35:40.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:40 smithi094 ceph-mon[29713]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-04-06T03:35:40.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:40 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:40.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:40 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:40.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:40 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:40.794 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:40 smithi136 ceph-mon[36436]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-04-06T03:35:40.794 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:40 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:40.794 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:40 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:40.794 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:40 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:42.484 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:42 smithi136 ceph-mon[36436]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-06T03:35:42.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.533+0000 7f01a0ff9700 1 -- 172.21.15.136:0/3852246675 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f01a405e400 con 0x7f0190069210 2024-04-06T03:35:42.536 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 3 on host 'smithi136' 2024-04-06T03:35:42.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.535+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0190069210 msgr2=0x7f019006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.535+0000 7f01a9c9c700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0190069210 0x7f019006b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f018c00e5e0 tx=0x7f018c00b040 comp rx=0 tx=0).stop 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.535+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01a40fbcd0 msgr2=0x7f01a40ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.535+0000 7f01a9c9c700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01a40fbcd0 0x7f01a40ffd80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f019400b970 tx=0x7f0194009e30 comp rx=0 tx=0).stop 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.536+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 shutdown_connections 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.536+0000 7f01a9c9c700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0190069210 0x7f019006b6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.536+0000 7f01a9c9c700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01a40fbcd0 0x7f01a40ffd80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.536+0000 7f01a9c9c700 1 --2- 172.21.15.136:0/3852246675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a40fcf80 0x7f01a40fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.536+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 >> 172.21.15.136:0/3852246675 conn(0x7f01a40f7430 msgr2=0x7f01a4065b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.536+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 shutdown_connections 2024-04-06T03:35:42.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:42.536+0000 7f01a9c9c700 1 -- 172.21.15.136:0/3852246675 wait complete. 2024-04-06T03:35:42.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:42 smithi094 ceph-mon[29713]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-06T03:35:43.405 DEBUG:teuthology.orchestra.run.smithi136:osd.3> sudo journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.3.service 2024-04-06T03:35:43.408 INFO:tasks.cephadm:Deploying osd.4 on smithi136 with /dev/vg_nvme/lv_3... 2024-04-06T03:35:43.408 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-06T03:35:43.708 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi136/config 2024-04-06T03:35:43.830 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:43.831 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:43.831 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:35:43.831 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:43 smithi136 ceph-mon[36436]: from='osd.3 [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-06T03:35:43.831 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:43 smithi136 ceph-mon[36436]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-06T03:35:43.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:43.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:43.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:35:43.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:43 smithi094 ceph-mon[29713]: from='osd.3 [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-06T03:35:43.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:43 smithi094 ceph-mon[29713]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-06T03:35:44.807 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-mon[36436]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-06T03:35:44.807 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-mon[36436]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-06T03:35:44.807 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-mon[36436]: osdmap e24: 4 total, 3 up, 4 in 2024-04-06T03:35:44.807 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:44.807 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-mon[36436]: from='osd.3 [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:35:44.807 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-mon[36436]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:35:44.807 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:44.807 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:44.808 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[41886]: 2024-04-06T03:35:44.540+0000 7f624d1ee700 -1 osd.3 0 waiting for initial osdmap 2024-04-06T03:35:44.808 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:35:44 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[41886]: 2024-04-06T03:35:44.545+0000 7f62487ea700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:35:44.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:44 smithi094 ceph-mon[29713]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-06T03:35:44.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:44 smithi094 ceph-mon[29713]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-06T03:35:44.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:44 smithi094 ceph-mon[29713]: osdmap e24: 4 total, 3 up, 4 in 2024-04-06T03:35:44.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:44 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:44.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:44 smithi094 ceph-mon[29713]: from='osd.3 [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:35:44.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:44 smithi094 ceph-mon[29713]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:35:44.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:44 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:44.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:44 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:45.746 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:45 smithi136 ceph-mon[36436]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-06T03:35:45.746 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:45 smithi136 ceph-mon[36436]: osdmap e25: 4 total, 3 up, 4 in 2024-04-06T03:35:45.746 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:45 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:45.746 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:45 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:45.746 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:45 smithi136 ceph-mon[36436]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:45.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:45 smithi094 ceph-mon[29713]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-06T03:35:45.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:45 smithi094 ceph-mon[29713]: osdmap e25: 4 total, 3 up, 4 in 2024-04-06T03:35:45.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:45 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:45.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:45 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:45.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:45 smithi094 ceph-mon[29713]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-06T03:35:46.109 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-06T03:35:46.109 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-06T03:35:46.109 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-06T03:35:46.109 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-06T03:35:46.109 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00928201 s, 1.1 GB/s 2024-04-06T03:35:46.109 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 2024-04-06T03:35:46.109 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-06T03:35:46.769 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:46 smithi136 ceph-mon[36436]: purged_snaps scrub starts 2024-04-06T03:35:46.770 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:46 smithi136 ceph-mon[36436]: purged_snaps scrub ok 2024-04-06T03:35:46.770 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:46 smithi136 ceph-mon[36436]: osd.3 [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] boot 2024-04-06T03:35:46.770 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:46 smithi136 ceph-mon[36436]: osdmap e26: 4 total, 4 up, 4 in 2024-04-06T03:35:46.770 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:46 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:46.793 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-04-06T03:35:46.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:46 smithi094 ceph-mon[29713]: purged_snaps scrub starts 2024-04-06T03:35:46.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:46 smithi094 ceph-mon[29713]: purged_snaps scrub ok 2024-04-06T03:35:46.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:46 smithi094 ceph-mon[29713]: osd.3 [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] boot 2024-04-06T03:35:46.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:46 smithi094 ceph-mon[29713]: osdmap e26: 4 total, 4 up, 4 in 2024-04-06T03:35:46.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:46 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:35:47.315 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi136/config 2024-04-06T03:35:47.710 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:47 smithi136 ceph-mon[36436]: osdmap e27: 4 total, 4 up, 4 in 2024-04-06T03:35:47.710 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:47 smithi136 ceph-mon[36436]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:35:47.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:47 smithi094 ceph-mon[29713]: osdmap e27: 4 total, 4 up, 4 in 2024-04-06T03:35:47.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:47 smithi094 ceph-mon[29713]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:35:48.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:48 smithi136 ceph-mon[36436]: osdmap e28: 4 total, 4 up, 4 in 2024-04-06T03:35:48.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:48 smithi136 ceph-mon[36436]: osdmap e29: 4 total, 4 up, 4 in 2024-04-06T03:35:48.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:48 smithi094 ceph-mon[29713]: osdmap e28: 4 total, 4 up, 4 in 2024-04-06T03:35:48.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:48 smithi094 ceph-mon[29713]: osdmap e29: 4 total, 4 up, 4 in 2024-04-06T03:35:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.353+0000 7f072edd1700 1 -- 172.21.15.136:0/3968758777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 msgr2=0x7f07280ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:35:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.353+0000 7f072edd1700 1 --2- 172.21.15.136:0/3968758777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 0x7f07280ffa00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0720009ce0 tx=0x7f0720004d10 comp rx=0 tx=0).stop 2024-04-06T03:35:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.353+0000 7f072edd1700 1 -- 172.21.15.136:0/3968758777 shutdown_connections 2024-04-06T03:35:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.353+0000 7f072edd1700 1 --2- 172.21.15.136:0/3968758777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07280fff40 0x7f0728102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.353+0000 7f072edd1700 1 --2- 172.21.15.136:0/3968758777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 0x7f07280ffa00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.353+0000 7f072edd1700 1 -- 172.21.15.136:0/3968758777 >> 172.21.15.136:0/3968758777 conn(0x7f07280f7430 msgr2=0x7f07280f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:35:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.353+0000 7f072edd1700 1 -- 172.21.15.136:0/3968758777 shutdown_connections 2024-04-06T03:35:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.353+0000 7f072edd1700 1 -- 172.21.15.136:0/3968758777 wait complete. 2024-04-06T03:35:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072edd1700 1 Processor -- start 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072edd1700 1 -- start start 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072edd1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 0x7f072810dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072edd1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07280fff40 0x7f072810e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072edd1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072810e6f0 con 0x7f07280fff40 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072edd1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072810ef40 con 0x7f07280662a0 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072cb6d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 0x7f072810dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072cb6d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 0x7f072810dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:39732/0 (socket says 172.21.15.136:39732) 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072cb6d700 1 -- 172.21.15.136:0/1784508493 learned_addr learned my addr 172.21.15.136:0/1784508493 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f0727fff700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07280fff40 0x7f072810e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072cb6d700 1 -- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07280fff40 msgr2=0x7f072810e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072cb6d700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07280fff40 0x7f072810e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f072cb6d700 1 -- 172.21.15.136:0/1784508493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720009940 con 0x7f07280662a0 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.354+0000 7f0727fff700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07280fff40 0x7f072810e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.355+0000 7f072cb6d700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 0x7f072810dbb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0720009380 tx=0x7f0720004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:35:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.355+0000 7f07257fa700 1 -- 172.21.15.136:0/1784508493 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f072000e040 con 0x7f07280662a0 2024-04-06T03:35:49.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.355+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072810f1c0 con 0x7f07280662a0 2024-04-06T03:35:49.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.355+0000 7f07257fa700 1 -- 172.21.15.136:0/1784508493 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07200128e0 con 0x7f07280662a0 2024-04-06T03:35:49.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.355+0000 7f07257fa700 1 -- 172.21.15.136:0/1784508493 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f072001c910 con 0x7f07280662a0 2024-04-06T03:35:49.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.355+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072810f700 con 0x7f07280662a0 2024-04-06T03:35:49.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.356+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072805f5f0 con 0x7f07280662a0 2024-04-06T03:35:49.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.356+0000 7f07257fa700 1 -- 172.21.15.136:0/1784508493 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0720012a50 con 0x7f07280662a0 2024-04-06T03:35:49.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.357+0000 7f07257fa700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f07100692a0 0x7f071006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:35:49.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.357+0000 7f07257fa700 1 -- 172.21.15.136:0/1784508493 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f0720088770 con 0x7f07280662a0 2024-04-06T03:35:49.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.357+0000 7f0727fff700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f07100692a0 0x7f071006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:35:49.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.357+0000 7f0727fff700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f07100692a0 0x7f071006b750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f071c009fd0 tx=0x7f071c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:35:49.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.361+0000 7f07257fa700 1 -- 172.21.15.136:0/1784508493 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0720058b50 con 0x7f07280662a0 2024-04-06T03:35:49.555 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:35:49.553+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f072810fa90 con 0x7f07100692a0 2024-04-06T03:35:50.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: Detected new or changed devices on smithi136 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: Adjusting osd_memory_target on smithi136 to 15936M 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: osdmap e30: 4 total, 4 up, 4 in 2024-04-06T03:35:50.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:35:50.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:35:50.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:50.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: Detected new or changed devices on smithi136 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: Adjusting osd_memory_target on smithi136 to 15936M 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:35:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:35:50.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: osdmap e30: 4 total, 4 up, 4 in 2024-04-06T03:35:50.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:35:50.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:35:50.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:35:51.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:50 smithi136 ceph-mon[36436]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:35:51.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:50 smithi094 ceph-mon[29713]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:35:52.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:52 smithi136 ceph-mon[36436]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:35:52.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:52 smithi136 ceph-mon[36436]: from='client.? 172.21.15.136:0/3203094849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "572f9cc0-e83c-4354-babb-519be04069c3"}]: dispatch 2024-04-06T03:35:52.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:52 smithi136 ceph-mon[36436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "572f9cc0-e83c-4354-babb-519be04069c3"}]: dispatch 2024-04-06T03:35:52.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:52 smithi136 ceph-mon[36436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "572f9cc0-e83c-4354-babb-519be04069c3"}]': finished 2024-04-06T03:35:52.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:52 smithi136 ceph-mon[36436]: osdmap e31: 5 total, 4 up, 5 in 2024-04-06T03:35:52.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:52 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:35:52.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:52 smithi094 ceph-mon[29713]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:35:52.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:52 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/3203094849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "572f9cc0-e83c-4354-babb-519be04069c3"}]: dispatch 2024-04-06T03:35:52.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:52 smithi094 ceph-mon[29713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "572f9cc0-e83c-4354-babb-519be04069c3"}]: dispatch 2024-04-06T03:35:52.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:52 smithi094 ceph-mon[29713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "572f9cc0-e83c-4354-babb-519be04069c3"}]': finished 2024-04-06T03:35:52.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:52 smithi094 ceph-mon[29713]: osdmap e31: 5 total, 4 up, 5 in 2024-04-06T03:35:52.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:52 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:35:53.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:53 smithi136 ceph-mon[36436]: from='client.? 172.21.15.136:0/3053427598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:35:53.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:53 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/3053427598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:35:54.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:54 smithi136 ceph-mon[36436]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-06T03:35:54.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:54 smithi094 ceph-mon[29713]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-06T03:35:56.203 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:56 smithi136 ceph-mon[36436]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-06T03:35:56.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:56 smithi094 ceph-mon[29713]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-06T03:35:58.272 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:58 smithi136 ceph-mon[36436]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-06T03:35:58.273 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:35:58 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:35:58.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:58 smithi094 ceph-mon[29713]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-06T03:35:58.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:35:58 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:36:00.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:00 smithi094 ceph-mon[29713]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-06T03:36:00.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:00 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-06T03:36:00.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:00 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:00.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:00 smithi094 ceph-mon[29713]: Deploying daemon osd.4 on smithi136 2024-04-06T03:36:00.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:00 smithi136 ceph-mon[36436]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-06T03:36:00.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:00 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-06T03:36:00.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:00 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:00.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:00 smithi136 ceph-mon[36436]: Deploying daemon osd.4 on smithi136 2024-04-06T03:36:02.234 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:02 smithi136 ceph-mon[36436]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-06T03:36:02.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:02 smithi094 ceph-mon[29713]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-06T03:36:04.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:04 smithi136 ceph-mon[36436]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:04 smithi094 ceph-mon[29713]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:05.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:05 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:05.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:05 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:05.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:05 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:36:05.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:05 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:05.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:05 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:05.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:05 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:36:06.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:06 smithi136 ceph-mon[36436]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:06.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:06 smithi094 ceph-mon[29713]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:08.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.016+0000 7f07257fa700 1 -- 172.21.15.136:0/1784508493 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f072810fa90 con 0x7f07100692a0 2024-04-06T03:36:08.018 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 4 on host 'smithi136' 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f07100692a0 msgr2=0x7f071006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f07100692a0 0x7f071006b750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f071c009fd0 tx=0x7f071c00b040 comp rx=0 tx=0).stop 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 msgr2=0x7f072810dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 0x7f072810dbb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0720009380 tx=0x7f0720004d10 comp rx=0 tx=0).stop 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 shutdown_connections 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f07100692a0 0x7f071006b750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07280662a0 0x7f072810dbb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 --2- 172.21.15.136:0/1784508493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07280fff40 0x7f072810e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 >> 172.21.15.136:0/1784508493 conn(0x7f07280f7430 msgr2=0x7f07280f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 shutdown_connections 2024-04-06T03:36:08.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:08.018+0000 7f072edd1700 1 -- 172.21.15.136:0/1784508493 wait complete. 2024-04-06T03:36:08.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:08 smithi136 ceph-mon[36436]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:08.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:08 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:08.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:08 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:08.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:08 smithi136 ceph-mon[36436]: from='osd.4 [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-06T03:36:08.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:08 smithi136 ceph-mon[36436]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-06T03:36:08.654 DEBUG:teuthology.orchestra.run.smithi136:osd.4> sudo journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.4.service 2024-04-06T03:36:08.656 INFO:tasks.cephadm:Deploying osd.5 on smithi136 with /dev/vg_nvme/lv_2... 2024-04-06T03:36:08.656 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-06T03:36:08.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:08 smithi094 ceph-mon[29713]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:08.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:08 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:08.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:08 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:08.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:08 smithi094 ceph-mon[29713]: from='osd.4 [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-06T03:36:08.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:08 smithi094 ceph-mon[29713]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-06T03:36:08.961 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi136/config 2024-04-06T03:36:09.997 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:09 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:09.997 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:09 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:09.997 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:09 smithi136 ceph-mon[36436]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:09.998 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:09 smithi136 ceph-mon[36436]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-06T03:36:09.998 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:09 smithi136 ceph-mon[36436]: osdmap e32: 5 total, 4 up, 5 in 2024-04-06T03:36:09.998 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:09 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:36:09.998 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:09 smithi136 ceph-mon[36436]: from='osd.4 [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:36:09.998 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:09 smithi136 ceph-mon[36436]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:36:10.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:09 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:10.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:09 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:10.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:09 smithi094 ceph-mon[29713]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:10.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:09 smithi094 ceph-mon[29713]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-06T03:36:10.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:09 smithi094 ceph-mon[29713]: osdmap e32: 5 total, 4 up, 5 in 2024-04-06T03:36:10.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:09 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:36:10.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:09 smithi094 ceph-mon[29713]: from='osd.4 [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:36:10.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:09 smithi094 ceph-mon[29713]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:36:10.348 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:36:09 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[48296]: 2024-04-06T03:36:09.994+0000 7f1f9b0eb700 -1 osd.4 0 waiting for initial osdmap 2024-04-06T03:36:10.349 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:36:10 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[48296]: 2024-04-06T03:36:10.003+0000 7f1f966e7700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:36:11.227 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:10 smithi136 ceph-mon[36436]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-06T03:36:11.227 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:10 smithi136 ceph-mon[36436]: osdmap e33: 5 total, 4 up, 5 in 2024-04-06T03:36:11.227 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:10 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:36:11.227 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:10 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:36:11.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:10 smithi094 ceph-mon[29713]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-06T03:36:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:10 smithi094 ceph-mon[29713]: osdmap e33: 5 total, 4 up, 5 in 2024-04-06T03:36:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:10 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:36:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:10 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:36:12.258 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-06T03:36:12.259 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-06T03:36:12.259 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-06T03:36:12.259 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-06T03:36:12.259 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0103236 s, 1.0 GB/s 2024-04-06T03:36:12.259 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-06T03:36:12.259 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:11 smithi136 ceph-mon[36436]: purged_snaps scrub starts 2024-04-06T03:36:12.259 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:11 smithi136 ceph-mon[36436]: purged_snaps scrub ok 2024-04-06T03:36:12.259 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:11 smithi136 ceph-mon[36436]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:12.259 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:11 smithi136 ceph-mon[36436]: osd.4 [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] boot 2024-04-06T03:36:12.260 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:11 smithi136 ceph-mon[36436]: osdmap e34: 5 total, 5 up, 5 in 2024-04-06T03:36:12.260 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:11 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:36:12.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:11 smithi094 ceph-mon[29713]: purged_snaps scrub starts 2024-04-06T03:36:12.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:11 smithi094 ceph-mon[29713]: purged_snaps scrub ok 2024-04-06T03:36:12.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:11 smithi094 ceph-mon[29713]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-06T03:36:12.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:11 smithi094 ceph-mon[29713]: osd.4 [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] boot 2024-04-06T03:36:12.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:11 smithi094 ceph-mon[29713]: osdmap e34: 5 total, 5 up, 5 in 2024-04-06T03:36:12.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:11 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:36:13.207 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: osdmap e35: 5 total, 5 up, 5 in 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:36:13.208 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.232 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-04-06T03:36:13.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: osdmap e35: 5 total, 5 up, 5 in 2024-04-06T03:36:13.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:36:13.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:13.483 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi136/config 2024-04-06T03:36:14.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:14 smithi136 ceph-mon[36436]: Detected new or changed devices on smithi136 2024-04-06T03:36:14.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:14 smithi136 ceph-mon[36436]: Adjusting osd_memory_target on smithi136 to 7968M 2024-04-06T03:36:14.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:14 smithi136 ceph-mon[36436]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:14.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:14 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:36:14.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:14 smithi094 ceph-mon[29713]: Detected new or changed devices on smithi136 2024-04-06T03:36:14.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:14 smithi094 ceph-mon[29713]: Adjusting osd_memory_target on smithi136 to 7968M 2024-04-06T03:36:14.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:14 smithi094 ceph-mon[29713]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:14.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:14 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:36:15.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.039+0000 7f9e38bcf700 1 -- 172.21.15.136:0/1686705000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340fe450 msgr2=0x7f9e340fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.039+0000 7f9e38bcf700 1 --2- 172.21.15.136:0/1686705000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340fe450 0x7f9e340fe860 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9e24004940 tx=0x7f9e24009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 -- 172.21.15.136:0/1686705000 shutdown_connections 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 --2- 172.21.15.136:0/1686705000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e340feda0 0x7f9e340faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 --2- 172.21.15.136:0/1686705000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340fe450 0x7f9e340fe860 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 -- 172.21.15.136:0/1686705000 >> 172.21.15.136:0/1686705000 conn(0x7f9e340f6c10 msgr2=0x7f9e340f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 -- 172.21.15.136:0/1686705000 shutdown_connections 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 -- 172.21.15.136:0/1686705000 wait complete. 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 Processor -- start 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 -- start start 2024-04-06T03:36:15.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e340fe450 0x7f9e340fda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340feda0 0x7f9e340fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340fc650 con 0x7f9e340fe450 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e38bcf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340fc7c0 con 0x7f9e340feda0 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e31d9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340feda0 0x7f9e340fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e31d9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340feda0 0x7f9e340fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:35432/0 (socket says 172.21.15.136:35432) 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.040+0000 7f9e31d9b700 1 -- 172.21.15.136:0/3615796096 learned_addr learned my addr 172.21.15.136:0/3615796096 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e3259c700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e340fe450 0x7f9e340fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e31d9b700 1 -- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e340fe450 msgr2=0x7f9e340fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e31d9b700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e340fe450 0x7f9e340fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e31d9b700 1 -- 172.21.15.136:0/3615796096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e24005040 con 0x7f9e340feda0 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e31d9b700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340feda0 0x7f9e340fc110 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800b790 tx=0x7f9e28009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e22ffd700 1 -- 172.21.15.136:0/3615796096 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e2800f770 con 0x7f9e340feda0 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e340fcaa0 con 0x7f9e340feda0 2024-04-06T03:36:15.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e22ffd700 1 -- 172.21.15.136:0/3615796096 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e2800fda0 con 0x7f9e340feda0 2024-04-06T03:36:15.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e22ffd700 1 -- 172.21.15.136:0/3615796096 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e2800d310 con 0x7f9e340feda0 2024-04-06T03:36:15.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.041+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e34115a50 con 0x7f9e340feda0 2024-04-06T03:36:15.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.042+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3404c710 con 0x7f9e340feda0 2024-04-06T03:36:15.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.042+0000 7f9e22ffd700 1 -- 172.21.15.136:0/3615796096 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9e28010420 con 0x7f9e340feda0 2024-04-06T03:36:15.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.043+0000 7f9e22ffd700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9e1c0692a0 0x7f9e1c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:15.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.043+0000 7f9e22ffd700 1 -- 172.21.15.136:0/3615796096 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f9e28086cb0 con 0x7f9e340feda0 2024-04-06T03:36:15.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.043+0000 7f9e3259c700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9e1c0692a0 0x7f9e1c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:15.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.043+0000 7f9e3259c700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9e1c0692a0 0x7f9e1c06b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b3b0 tx=0x7f9e24009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:15.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.047+0000 7f9e22ffd700 1 -- 172.21.15.136:0/3615796096 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9e28056f60 con 0x7f9e340feda0 2024-04-06T03:36:15.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:15.241+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e341024a0 con 0x7f9e1c0692a0 2024-04-06T03:36:16.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:16 smithi136 ceph-mon[36436]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:16.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:16 smithi136 ceph-mon[36436]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:36:16.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:36:16.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:36:16.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:16 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:16.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:16 smithi094 ceph-mon[29713]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:16.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:16 smithi094 ceph-mon[29713]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:36:16.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-06T03:36:16.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-06T03:36:16.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:16 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:18.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:18 smithi136 ceph-mon[36436]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:18.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:18 smithi136 ceph-mon[36436]: from='client.? 172.21.15.136:0/3069128063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a38e98d8-dfb7-4083-aae8-e5853a896d50"}]: dispatch 2024-04-06T03:36:18.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:18 smithi136 ceph-mon[36436]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a38e98d8-dfb7-4083-aae8-e5853a896d50"}]: dispatch 2024-04-06T03:36:18.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:18 smithi136 ceph-mon[36436]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a38e98d8-dfb7-4083-aae8-e5853a896d50"}]': finished 2024-04-06T03:36:18.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:18 smithi136 ceph-mon[36436]: osdmap e36: 6 total, 5 up, 6 in 2024-04-06T03:36:18.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:18 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:18 smithi094 ceph-mon[29713]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:18 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/3069128063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a38e98d8-dfb7-4083-aae8-e5853a896d50"}]: dispatch 2024-04-06T03:36:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:18 smithi094 ceph-mon[29713]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a38e98d8-dfb7-4083-aae8-e5853a896d50"}]: dispatch 2024-04-06T03:36:18.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:18 smithi094 ceph-mon[29713]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a38e98d8-dfb7-4083-aae8-e5853a896d50"}]': finished 2024-04-06T03:36:18.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:18 smithi094 ceph-mon[29713]: osdmap e36: 6 total, 5 up, 6 in 2024-04-06T03:36:18.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:18 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:19.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:19 smithi136 ceph-mon[36436]: from='client.? 172.21.15.136:0/2439010476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:36:19.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:19 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/2439010476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-06T03:36:20.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:20 smithi094 ceph-mon[29713]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:20.294 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:20 smithi136 ceph-mon[36436]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:22.292 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:22 smithi136 ceph-mon[36436]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:22.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:22 smithi094 ceph-mon[29713]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:24.284 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:24 smithi136 ceph-mon[36436]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:24.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:24 smithi094 ceph-mon[29713]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:25.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-06T03:36:25.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:25 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:25.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-06T03:36:25.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:25 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:26.222 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:26 smithi136 ceph-mon[36436]: Deploying daemon osd.5 on smithi136 2024-04-06T03:36:26.222 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:26 smithi136 ceph-mon[36436]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:26.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:26 smithi094 ceph-mon[29713]: Deploying daemon osd.5 on smithi136 2024-04-06T03:36:26.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:26 smithi094 ceph-mon[29713]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:28.342 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:28 smithi136 ceph-mon[36436]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:28.342 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:28 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:36:28.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:28 smithi094 ceph-mon[29713]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:28.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:28 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:36:30.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:30 smithi094 ceph-mon[29713]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:30.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:30 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:30.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:30 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:30.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:30 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:36:30.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:30 smithi136 ceph-mon[36436]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:30.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:30 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:30.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:30 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:30.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:30 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:36:32.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:32 smithi136 ceph-mon[36436]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:32.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:32 smithi094 ceph-mon[29713]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:33.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.204+0000 7f9e22ffd700 1 -- 172.21.15.136:0/3615796096 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9e341024a0 con 0x7f9e1c0692a0 2024-04-06T03:36:33.206 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 5 on host 'smithi136' 2024-04-06T03:36:33.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9e1c0692a0 msgr2=0x7f9e1c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:33.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9e1c0692a0 0x7f9e1c06b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b3b0 tx=0x7f9e24009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:33.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340feda0 msgr2=0x7f9e340fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:33.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340feda0 0x7f9e340fc110 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800b790 tx=0x7f9e28009250 comp rx=0 tx=0).stop 2024-04-06T03:36:33.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 shutdown_connections 2024-04-06T03:36:33.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9e1c0692a0 0x7f9e1c06b750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:33.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e340fe450 0x7f9e340fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:33.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 --2- 172.21.15.136:0/3615796096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e340feda0 0x7f9e340fc110 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:33.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 >> 172.21.15.136:0/3615796096 conn(0x7f9e340f6c10 msgr2=0x7f9e340f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:33.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 shutdown_connections 2024-04-06T03:36:33.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:33.206+0000 7f9e38bcf700 1 -- 172.21.15.136:0/3615796096 wait complete. 2024-04-06T03:36:34.124 DEBUG:teuthology.orchestra.run.smithi136:osd.5> sudo journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.5.service 2024-04-06T03:36:34.126 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-06T03:36:34.127 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd stat -f json 2024-04-06T03:36:34.396 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:34.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:34 smithi094 ceph-mon[29713]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:34.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:34 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:34.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:34 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:34.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:34 smithi094 ceph-mon[29713]: from='osd.5 [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-06T03:36:34.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:34 smithi094 ceph-mon[29713]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-06T03:36:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:34 smithi136 ceph-mon[36436]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:34 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:34 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:34 smithi136 ceph-mon[36436]: from='osd.5 [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-06T03:36:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:34 smithi136 ceph-mon[36436]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-06T03:36:35.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:35 smithi136 ceph-mon[36436]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-06T03:36:35.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:35 smithi136 ceph-mon[36436]: osdmap e37: 6 total, 5 up, 6 in 2024-04-06T03:36:35.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:35 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:35.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:35 smithi136 ceph-mon[36436]: from='osd.5 [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:36:35.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:35 smithi136 ceph-mon[36436]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:36:35.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:35 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:35.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:35 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:35.349 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:36:35 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[54793]: 2024-04-06T03:36:35.171+0000 7fcf2801d700 -1 osd.5 0 waiting for initial osdmap 2024-04-06T03:36:35.349 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:36:35 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[54793]: 2024-04-06T03:36:35.180+0000 7fcf22e18700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:36:35.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:35 smithi094 ceph-mon[29713]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-06T03:36:35.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:35 smithi094 ceph-mon[29713]: osdmap e37: 6 total, 5 up, 6 in 2024-04-06T03:36:35.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:35 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:35.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:35 smithi094 ceph-mon[29713]: from='osd.5 [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:36:35.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:35 smithi094 ceph-mon[29713]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:36:35.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:35 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:35.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:35 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.015+0000 7f03522da700 1 -- 172.21.15.94:0/3973693179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fdfe0 msgr2=0x7f034c0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.015+0000 7f03522da700 1 --2- 172.21.15.94:0/3973693179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fdfe0 0x7f034c0fe450 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f033c004940 tx=0x7f033c009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.015+0000 7f03522da700 1 -- 172.21.15.94:0/3973693179 shutdown_connections 2024-04-06T03:36:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.015+0000 7f03522da700 1 --2- 172.21.15.94:0/3973693179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fdfe0 0x7f034c0fe450 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.015+0000 7f03522da700 1 --2- 172.21.15.94:0/3973693179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f034c0fce70 0x7f034c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.015+0000 7f03522da700 1 -- 172.21.15.94:0/3973693179 >> 172.21.15.94:0/3973693179 conn(0x7f034c0f8550 msgr2=0x7f034c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.015+0000 7f03522da700 1 -- 172.21.15.94:0/3973693179 shutdown_connections 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.015+0000 7f03522da700 1 -- 172.21.15.94:0/3973693179 wait complete. 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.015+0000 7f03522da700 1 Processor -- start 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f03522da700 1 -- start start 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f03522da700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fce70 0x7f034c10fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f03522da700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f034c0fdfe0 0x7f034c110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f03522da700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c110670 con 0x7f034c0fce70 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f03522da700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c1107e0 con 0x7f034c0fdfe0 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f034bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fce70 0x7f034c10fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f034bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fce70 0x7f034c10fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46294/0 (socket says 172.21.15.94:46294) 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f034bfff700 1 -- 172.21.15.94:0/1048876141 learned_addr learned my addr 172.21.15.94:0/1048876141 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:36:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f034b7fe700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f034c0fdfe0 0x7f034c110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f034bfff700 1 -- 172.21.15.94:0/1048876141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f034c0fdfe0 msgr2=0x7f034c110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f034bfff700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f034c0fdfe0 0x7f034c110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f034bfff700 1 -- 172.21.15.94:0/1048876141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c005040 con 0x7f034c0fce70 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f034b7fe700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f034c0fdfe0 0x7f034c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f034bfff700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fce70 0x7f034c10fb30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0340002a50 tx=0x7f03400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f0348ff9700 1 -- 172.21.15.94:0/1048876141 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0340007cc0 con 0x7f034c0fce70 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c115240 con 0x7f034c0fce70 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.016+0000 7f0348ff9700 1 -- 172.21.15.94:0/1048876141 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f034000f040 con 0x7f034c0fce70 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.017+0000 7f0348ff9700 1 -- 172.21.15.94:0/1048876141 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0340009640 con 0x7f034c0fce70 2024-04-06T03:36:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.017+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c115800 con 0x7f034c0fce70 2024-04-06T03:36:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.017+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c101970 con 0x7f034c0fce70 2024-04-06T03:36:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.018+0000 7f0348ff9700 1 -- 172.21.15.94:0/1048876141 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f03400097a0 con 0x7f034c0fce70 2024-04-06T03:36:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.018+0000 7f0348ff9700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f03340692a0 0x7f033406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.018+0000 7f034b7fe700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f03340692a0 0x7f033406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.018+0000 7f0348ff9700 1 -- 172.21.15.94:0/1048876141 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f0340086b70 con 0x7f034c0fce70 2024-04-06T03:36:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.019+0000 7f034b7fe700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f03340692a0 0x7f033406b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f033c00b970 tx=0x7f033c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:36.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.023+0000 7f0348ff9700 1 -- 172.21.15.94:0/1048876141 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0340056d00 con 0x7f034c0fce70 2024-04-06T03:36:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.216+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f034c04c710 con 0x7f034c0fce70 2024-04-06T03:36:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.216+0000 7f0348ff9700 1 -- 172.21.15.94:0/1048876141 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f0340056890 con 0x7f034c0fce70 2024-04-06T03:36:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.218+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f03340692a0 msgr2=0x7f033406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.218+0000 7f03522da700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f03340692a0 0x7f033406b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f033c00b970 tx=0x7f033c009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.218+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fce70 msgr2=0x7f034c10fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.218+0000 7f03522da700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fce70 0x7f034c10fb30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0340002a50 tx=0x7f03400046b0 comp rx=0 tx=0).stop 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.219+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 shutdown_connections 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.219+0000 7f03522da700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f03340692a0 0x7f033406b750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.219+0000 7f03522da700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f034c0fce70 0x7f034c10fb30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.219+0000 7f03522da700 1 --2- 172.21.15.94:0/1048876141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f034c0fdfe0 0x7f034c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.219+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 >> 172.21.15.94:0/1048876141 conn(0x7f034c0f8550 msgr2=0x7f034c0f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.219+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 shutdown_connections 2024-04-06T03:36:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:36.219+0000 7f03522da700 1 -- 172.21.15.94:0/1048876141 wait complete. 2024-04-06T03:36:36.310 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:36 smithi136 ceph-mon[36436]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:36.310 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:36 smithi136 ceph-mon[36436]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-06T03:36:36.310 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:36 smithi136 ceph-mon[36436]: osdmap e38: 6 total, 5 up, 6 in 2024-04-06T03:36:36.310 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:36 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:36.310 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:36 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:36.444 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:36 smithi094 ceph-mon[29713]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:36.444 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:36 smithi094 ceph-mon[29713]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-06T03:36:36.444 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:36 smithi094 ceph-mon[29713]: osdmap e38: 6 total, 5 up, 6 in 2024-04-06T03:36:36.444 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:36 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:36.444 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:36 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:36.801 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1712374596,"num_in_osds":6,"osd_in_since":1712374577,"num_remapped_pgs":0} 2024-04-06T03:36:36.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-06T03:36:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:37.454 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:37 smithi136 ceph-mon[36436]: purged_snaps scrub starts 2024-04-06T03:36:37.454 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:37 smithi136 ceph-mon[36436]: purged_snaps scrub ok 2024-04-06T03:36:37.454 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:37 smithi136 ceph-mon[36436]: osd.5 [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] boot 2024-04-06T03:36:37.454 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:37 smithi136 ceph-mon[36436]: osdmap e39: 6 total, 6 up, 6 in 2024-04-06T03:36:37.454 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:37 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:37.455 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:37 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1048876141' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-06T03:36:37.470 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:37 smithi094 ceph-mon[29713]: purged_snaps scrub starts 2024-04-06T03:36:37.471 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:37 smithi094 ceph-mon[29713]: purged_snaps scrub ok 2024-04-06T03:36:37.471 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:37 smithi094 ceph-mon[29713]: osd.5 [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] boot 2024-04-06T03:36:37.471 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:37 smithi094 ceph-mon[29713]: osdmap e39: 6 total, 6 up, 6 in 2024-04-06T03:36:37.471 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:37 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:36:37.471 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:37 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1048876141' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: osdmap e40: 6 total, 6 up, 6 in 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:38.376 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:38 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:36:38.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.586+0000 7f2bec2e9700 1 -- 172.21.15.94:0/2312051466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 msgr2=0x7f2be40fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:38.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.586+0000 7f2bec2e9700 1 --2- 172.21.15.94:0/2312051466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 0x7f2be40fbd30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4004940 tx=0x7f2bd4009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:38.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.586+0000 7f2bec2e9700 1 -- 172.21.15.94:0/2312051466 shutdown_connections 2024-04-06T03:36:38.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.586+0000 7f2bec2e9700 1 --2- 172.21.15.94:0/2312051466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 0x7f2be40fbd30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:38.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.586+0000 7f2bec2e9700 1 --2- 172.21.15.94:0/2312051466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2be4101d20 0x7f2be41020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:38.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.586+0000 7f2bec2e9700 1 -- 172.21.15.94:0/2312051466 >> 172.21.15.94:0/2312051466 conn(0x7f2be40f78d0 msgr2=0x7f2be40f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:38.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.586+0000 7f2bec2e9700 1 -- 172.21.15.94:0/2312051466 shutdown_connections 2024-04-06T03:36:38.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.586+0000 7f2bec2e9700 1 -- 172.21.15.94:0/2312051466 wait complete. 2024-04-06T03:36:38.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.587+0000 7f2bec2e9700 1 Processor -- start 2024-04-06T03:36:38.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.587+0000 7f2bec2e9700 1 -- start start 2024-04-06T03:36:38.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.587+0000 7f2bec2e9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2be4101d20 0x7f2be4112ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:38.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.587+0000 7f2bec2e9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 0x7f2be4113010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:38.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.587+0000 7f2bec2e9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41136f0 con 0x7f2be41026c0 2024-04-06T03:36:38.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.587+0000 7f2bec2e9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be410ca50 con 0x7f2be4101d20 2024-04-06T03:36:38.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.587+0000 7f2be9884700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 0x7f2be4113010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:38.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.587+0000 7f2be9884700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 0x7f2be4113010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43080/0 (socket says 172.21.15.94:43080) 2024-04-06T03:36:38.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.587+0000 7f2be9884700 1 -- 172.21.15.94:0/3455712780 learned_addr learned my addr 172.21.15.94:0/3455712780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:36:38.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2bea085700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2be4101d20 0x7f2be4112ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:38.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2be9884700 1 -- 172.21.15.94:0/3455712780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2be4101d20 msgr2=0x7f2be4112ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:38.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2be9884700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2be4101d20 0x7f2be4112ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2be9884700 1 -- 172.21.15.94:0/3455712780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd4005040 con 0x7f2be41026c0 2024-04-06T03:36:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2be9884700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 0x7f2be4113010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009100 tx=0x7f2bd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2bdaffd700 1 -- 172.21.15.94:0/3455712780 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd400e040 con 0x7f2be41026c0 2024-04-06T03:36:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be410ccd0 con 0x7f2be41026c0 2024-04-06T03:36:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2bdaffd700 1 -- 172.21.15.94:0/3455712780 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bd4013420 con 0x7f2be41026c0 2024-04-06T03:36:38.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2bdaffd700 1 -- 172.21.15.94:0/3455712780 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd401c610 con 0x7f2be41026c0 2024-04-06T03:36:38.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.588+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be410d290 con 0x7f2be41026c0 2024-04-06T03:36:38.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.589+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be40fd470 con 0x7f2be41026c0 2024-04-06T03:36:38.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.590+0000 7f2bdaffd700 1 -- 172.21.15.94:0/3455712780 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2bd4013590 con 0x7f2be41026c0 2024-04-06T03:36:38.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.590+0000 7f2bdaffd700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f2bd00692a0 0x7f2bd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:38.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.590+0000 7f2bdaffd700 1 -- 172.21.15.94:0/3455712780 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2bd40880b0 con 0x7f2be41026c0 2024-04-06T03:36:38.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.590+0000 7f2bea085700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f2bd00692a0 0x7f2bd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:38.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.590+0000 7f2bea085700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f2bd00692a0 0x7f2bd006b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be0008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-06T03:36:38.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: osdmap e40: 6 total, 6 up, 6 in 2024-04-06T03:36:38.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:38.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:38.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:38.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:38.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:36:38.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:38.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:38.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:36:38.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:38.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:38 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:36:38.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.595+0000 7f2bdaffd700 1 -- 172.21.15.94:0/3455712780 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2bd40581b0 con 0x7f2be41026c0 2024-04-06T03:36:38.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.787+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2be4108e70 con 0x7f2be41026c0 2024-04-06T03:36:38.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.788+0000 7f2bdaffd700 1 -- 172.21.15.94:0/3455712780 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11094 (secure 0 0 0) 0x7f2bd4057d40 con 0x7f2be41026c0 2024-04-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:36:38.791 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","created":"2024-04-06T03:31:12.411006+0000","modified":"2024-04-06T03:36:37.175616+0000","last_up_change":"2024-04-06T03:36:36.170053+0000","last_in_change":"2024-04-06T03:36:17.627724+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-06T03:35:23.271684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7f51dba0-a74e-444c-b4ab-d5f1fa68ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6803","nonce":2289372883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6805","nonce":2289372883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6809","nonce":2289372883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6807","nonce":2289372883}]},"public_addr":"172.21.15.94:6803/2289372883","cluster_addr":"172.21.15.94:6805/2289372883","heartbeat_back_addr":"172.21.15.94:6809/2289372883","heartbeat_front_addr":"172.21.15.94:6807/2289372883","state":["exists","up"]},{"osd":1,"uuid":"2dd01097-c6d6-45e9-acee-ca5ea5f4e166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6811","nonce":1337787373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6813","nonce":1337787373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6817","nonce":1337787373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6815","nonce":1337787373}]},"public_addr":"172.21.15.94:6811/1337787373","cluster_addr":"172.21.15.94:6813/1337787373","heartbeat_back_addr":"172.21.15.94:6817/1337787373","heartbeat_front_addr":"172.21.15.94:6815/1337787373","state":["exists","up"]},{"osd":2,"uuid":"9645da7c-ffbc-4415-8fa0-0fcb07c53f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6819","nonce":1538420111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6821","nonce":1538420111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6825","nonce":1538420111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6823","nonce":1538420111}]},"public_addr":"172.21.15.94:6819/1538420111","cluster_addr":"172.21.15.94:6821/1538420111","heartbeat_back_addr":"172.21.15.94:6825/1538420111","heartbeat_front_addr":"172.21.15.94:6823/1538420111","state":["exists","up"]},{"osd":3,"uuid":"43321ebd-f195-4738-a2a5-f4d55a375bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271372986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271372986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271372986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271372986}]},"public_addr":"172.21.15.136:6801/1271372986","cluster_addr":"172.21.15.136:6803/1271372986","heartbeat_back_addr":"172.21.15.136:6807/1271372986","heartbeat_front_addr":"172.21.15.136:6805/1271372986","state":["exists","up"]},{"osd":4,"uuid":"572f9cc0-e83c-4354-babb-519be04069c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6809","nonce":2873260774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6811","nonce":2873260774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6815","nonce":2873260774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6813","nonce":2873260774}]},"public_addr":"172.21.15.136:6809/2873260774","cluster_addr":"172.21.15.136:6811/2873260774","heartbeat_back_addr":"172.21.15.136:6815/2873260774","heartbeat_front_addr":"172.21.15.136:6813/2873260774","state":["exists","up"]},{"osd":5,"uuid":"a38e98d8-dfb7-4083-aae8-e5853a896d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6817","nonce":3055268575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6819","nonce":3055268575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6823","nonce":3055268575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6821","nonce":3055268575}]},"public_addr":"172.21.15.136:6817/3055268575","cluster_addr":"172.21.15.136:6819/3055268575","heartbeat_back_addr":"172.21.15.136:6823/3055268575","heartbeat_front_addr":"172.21.15.136:6821/3055268575","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:25.784070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:53.656615+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:20.675464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:44.290139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:09.122272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:34.741771+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1085015108":"2024-04-07T03:33:12.953920+0000","172.21.15.94:6801/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1082665650":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6801/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/3195038611":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/852522634":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/2412091380":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/2779591805":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6801/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/1247691830":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/3724179185":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/1084447585":"2024-04-07T03:32:10.456958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f2bd00692a0 msgr2=0x7f2bd006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f2bd00692a0 0x7f2bd006b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be0008040 comp rx=0 tx=0).stop 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 msgr2=0x7f2be4113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 0x7f2be4113010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009100 tx=0x7f2bd4009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 shutdown_connections 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f2bd00692a0 0x7f2bd006b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2be4101d20 0x7f2be4112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 --2- 172.21.15.94:0/3455712780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2be41026c0 0x7f2be4113010 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 >> 172.21.15.94:0/3455712780 conn(0x7f2be40f78d0 msgr2=0x7f2be40f8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 shutdown_connections 2024-04-06T03:36:38.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:38.790+0000 7f2bec2e9700 1 -- 172.21.15.94:0/3455712780 wait complete. 2024-04-06T03:36:39.374 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:39 smithi094 ceph-mon[29713]: Detected new or changed devices on smithi136 2024-04-06T03:36:39.374 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:39 smithi094 ceph-mon[29713]: Adjusting osd_memory_target on smithi136 to 5312M 2024-04-06T03:36:39.374 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:39 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:39.374 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:39 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:36:39.374 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:39 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:39.375 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:39 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3455712780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:36:39.399 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-06T03:35:23.271684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-04-06T03:36:39.400 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-04-06T03:36:39.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:39 smithi136 ceph-mon[36436]: Detected new or changed devices on smithi136 2024-04-06T03:36:39.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:39 smithi136 ceph-mon[36436]: Adjusting osd_memory_target on smithi136 to 5312M 2024-04-06T03:36:39.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:39 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:36:39.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:39 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:36:39.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:39 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:36:39.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:39 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3455712780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:36:39.653 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:40.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:40 smithi094 ceph-mon[29713]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:40.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:40 smithi136 ceph-mon[36436]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:41.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.281+0000 7fdfaade6700 1 -- 172.21.15.94:0/1296277347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4101d20 msgr2=0x7fdfa41020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:41.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.281+0000 7fdfaade6700 1 --2- 172.21.15.94:0/1296277347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4101d20 0x7fdfa41020f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf98004940 tx=0x7fdf98009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:41.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.281+0000 7fdfaade6700 1 -- 172.21.15.94:0/1296277347 shutdown_connections 2024-04-06T03:36:41.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.281+0000 7fdfaade6700 1 --2- 172.21.15.94:0/1296277347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfa41026c0 0x7fdfa40fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:41.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.281+0000 7fdfaade6700 1 --2- 172.21.15.94:0/1296277347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa4101d20 0x7fdfa41020f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:41.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.281+0000 7fdfaade6700 1 -- 172.21.15.94:0/1296277347 >> 172.21.15.94:0/1296277347 conn(0x7fdfa40f7870 msgr2=0x7fdfa40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:41.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.282+0000 7fdfaade6700 1 -- 172.21.15.94:0/1296277347 shutdown_connections 2024-04-06T03:36:41.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.282+0000 7fdfaade6700 1 -- 172.21.15.94:0/1296277347 wait complete. 2024-04-06T03:36:41.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.282+0000 7fdfaade6700 1 Processor -- start 2024-04-06T03:36:41.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.282+0000 7fdfaade6700 1 -- start start 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.282+0000 7fdfaade6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfa4101d20 0x7fdfa410eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.282+0000 7fdfaade6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa41026c0 0x7fdfa410f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.282+0000 7fdfaade6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa410f5b0 con 0x7fdfa41026c0 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.282+0000 7fdfaade6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa410f6f0 con 0x7fdfa4101d20 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.282+0000 7fdfa3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa41026c0 0x7fdfa410f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa41026c0 0x7fdfa410f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43090/0 (socket says 172.21.15.94:43090) 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa3fff700 1 -- 172.21.15.94:0/1885500674 learned_addr learned my addr 172.21.15.94:0/1885500674 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa8b82700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfa4101d20 0x7fdfa410eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa3fff700 1 -- 172.21.15.94:0/1885500674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfa4101d20 msgr2=0x7fdfa410eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa3fff700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfa4101d20 0x7fdfa410eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:41.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa3fff700 1 -- 172.21.15.94:0/1885500674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98005040 con 0x7fdfa41026c0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa3fff700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa41026c0 0x7fdfa410f070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000b790 tx=0x7fdf90009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa17fa700 1 -- 172.21.15.94:0/1885500674 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf9000f6f0 con 0x7fdfa41026c0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa4117ba0 con 0x7fdfa41026c0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa17fa700 1 -- 172.21.15.94:0/1885500674 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf9000fd20 con 0x7fdfa41026c0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfa17fa700 1 -- 172.21.15.94:0/1885500674 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf900103a0 con 0x7fdfa41026c0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.283+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa4118020 con 0x7fdfa41026c0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.284+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa40fd470 con 0x7fdfa41026c0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.284+0000 7fdfa17fa700 1 -- 172.21.15.94:0/1885500674 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdf90010500 con 0x7fdfa41026c0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.285+0000 7fdfa17fa700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdf940692a0 0x7fdf9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.285+0000 7fdfa17fa700 1 -- 172.21.15.94:0/1885500674 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdf90086c30 con 0x7fdfa41026c0 2024-04-06T03:36:41.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.285+0000 7fdfa8b82700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdf940692a0 0x7fdf9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.285+0000 7fdfa8b82700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdf940692a0 0x7fdf9406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf98004f90 tx=0x7fdf98009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:41.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.289+0000 7fdfa17fa700 1 -- 172.21.15.94:0/1885500674 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdf900532a0 con 0x7fdfa41026c0 2024-04-06T03:36:41.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.471+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdfa4118270 con 0x7fdfa41026c0 2024-04-06T03:36:41.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.471+0000 7fdfa17fa700 1 -- 172.21.15.94:0/1885500674 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf900568c0 con 0x7fdfa41026c0 2024-04-06T03:36:41.474 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-06T03:36:41.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdf940692a0 msgr2=0x7fdf9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdf940692a0 0x7fdf9406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf98004f90 tx=0x7fdf98009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa41026c0 msgr2=0x7fdfa410f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa41026c0 0x7fdfa410f070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000b790 tx=0x7fdf90009220 comp rx=0 tx=0).stop 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 shutdown_connections 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdf940692a0 0x7fdf9406b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfa4101d20 0x7fdfa410eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 --2- 172.21.15.94:0/1885500674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfa41026c0 0x7fdfa410f070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 >> 172.21.15.94:0/1885500674 conn(0x7fdfa40f7870 msgr2=0x7fdfa40f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 shutdown_connections 2024-04-06T03:36:41.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:41.473+0000 7fdfaade6700 1 -- 172.21.15.94:0/1885500674 wait complete. 2024-04-06T03:36:42.155 INFO:tasks.cephadm:Setting up client nodes... 2024-04-06T03:36:42.156 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-06T03:36:42.410 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:42.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:42 smithi094 ceph-mon[29713]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:42.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:42 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1885500674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:36:42.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:42 smithi136 ceph-mon[36436]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:42.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:42 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1885500674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:36:43.463 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:36:43.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:36:43.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.946+0000 7fc121e3e700 1 -- 172.21.15.94:0/3431223408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fce80 msgr2=0x7fc11c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:43.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.946+0000 7fc121e3e700 1 --2- 172.21.15.94:0/3431223408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fce80 0x7fc11c0fd290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc10c004940 tx=0x7fc10c009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:43.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.946+0000 7fc121e3e700 1 -- 172.21.15.94:0/3431223408 shutdown_connections 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.946+0000 7fc121e3e700 1 --2- 172.21.15.94:0/3431223408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0fdff0 0x7fc11c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.946+0000 7fc121e3e700 1 --2- 172.21.15.94:0/3431223408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fce80 0x7fc11c0fd290 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.946+0000 7fc121e3e700 1 -- 172.21.15.94:0/3431223408 >> 172.21.15.94:0/3431223408 conn(0x7fc11c0f8580 msgr2=0x7fc11c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.946+0000 7fc121e3e700 1 -- 172.21.15.94:0/3431223408 shutdown_connections 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.946+0000 7fc121e3e700 1 -- 172.21.15.94:0/3431223408 wait complete. 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc121e3e700 1 Processor -- start 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc121e3e700 1 -- start start 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc121e3e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0fce80 0x7fc11c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc121e3e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fdff0 0x7fc11c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc121e3e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c10e440 con 0x7fc11c0fdff0 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc121e3e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c10e5b0 con 0x7fc11c0fce80 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc11affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fdff0 0x7fc11c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc11affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fdff0 0x7fc11c10de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43104/0 (socket says 172.21.15.94:43104) 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc11b7fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0fce80 0x7fc11c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.947+0000 7fc11affd700 1 -- 172.21.15.94:0/887600971 learned_addr learned my addr 172.21.15.94:0/887600971 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:36:43.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.948+0000 7fc11affd700 1 -- 172.21.15.94:0/887600971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0fce80 msgr2=0x7fc11c10d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:43.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.948+0000 7fc11affd700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0fce80 0x7fc11c10d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:43.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.948+0000 7fc11affd700 1 -- 172.21.15.94:0/887600971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c005040 con 0x7fc11c0fdff0 2024-04-06T03:36:43.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.948+0000 7fc11affd700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fdff0 0x7fc11c10de40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc110002a80 tx=0x7fc1100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:43.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.948+0000 7fc103fff700 1 -- 172.21.15.94:0/887600971 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc110007cc0 con 0x7fc11c0fdff0 2024-04-06T03:36:43.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.948+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c112fc0 con 0x7fc11c0fdff0 2024-04-06T03:36:43.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.948+0000 7fc103fff700 1 -- 172.21.15.94:0/887600971 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc11000f040 con 0x7fc11c0fdff0 2024-04-06T03:36:43.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.948+0000 7fc103fff700 1 -- 172.21.15.94:0/887600971 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11000dac0 con 0x7fc11c0fdff0 2024-04-06T03:36:43.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.948+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c113550 con 0x7fc11c0fdff0 2024-04-06T03:36:43.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.949+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c04c710 con 0x7fc11c0fdff0 2024-04-06T03:36:43.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.950+0000 7fc103fff700 1 -- 172.21.15.94:0/887600971 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc11000dc20 con 0x7fc11c0fdff0 2024-04-06T03:36:43.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.950+0000 7fc103fff700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc1040692a0 0x7fc10406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:43.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.950+0000 7fc103fff700 1 -- 172.21.15.94:0/887600971 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc110086ab0 con 0x7fc11c0fdff0 2024-04-06T03:36:43.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.950+0000 7fc11b7fe700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc1040692a0 0x7fc10406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:43.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.950+0000 7fc11b7fe700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc1040692a0 0x7fc10406b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00b3b0 tx=0x7fc10c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:43.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:43.954+0000 7fc103fff700 1 -- 172.21.15.94:0/887600971 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc110056b30 con 0x7fc11c0fdff0 2024-04-06T03:36:44.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.216+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc11c113770 con 0x7fc11c0fdff0 2024-04-06T03:36:44.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.220+0000 7fc103fff700 1 -- 172.21.15.94:0/887600971 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc1100566c0 con 0x7fc11c0fdff0 2024-04-06T03:36:44.222 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-04-06T03:36:44.222 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBMwxBmCq4CDRAAoGqCradqDfY3zRzqohqZpw== 2024-04-06T03:36:44.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc1040692a0 msgr2=0x7fc10406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:44.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc1040692a0 0x7fc10406b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00b3b0 tx=0x7fc10c009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:44.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fdff0 msgr2=0x7fc11c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:44.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fdff0 0x7fc11c10de40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc110002a80 tx=0x7fc1100046b0 comp rx=0 tx=0).stop 2024-04-06T03:36:44.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 shutdown_connections 2024-04-06T03:36:44.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc1040692a0 0x7fc10406b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:44.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0fce80 0x7fc11c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:44.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 --2- 172.21.15.94:0/887600971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc11c0fdff0 0x7fc11c10de40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:44.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 >> 172.21.15.94:0/887600971 conn(0x7fc11c0f8580 msgr2=0x7fc11c0f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:44.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 shutdown_connections 2024-04-06T03:36:44.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:44.222+0000 7fc121e3e700 1 -- 172.21.15.94:0/887600971 wait complete. 2024-04-06T03:36:44.520 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:44 smithi094 ceph-mon[29713]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:44.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:44 smithi136 ceph-mon[36436]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:44.903 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:36:44.903 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-06T03:36:44.903 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-06T03:36:44.945 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-06T03:36:45.207 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi136/config 2024-04-06T03:36:45.330 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:45 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/887600971' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-06T03:36:45.330 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:45 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/887600971' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-06T03:36:45.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:45 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/887600971' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-06T03:36:45.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:45 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/887600971' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-06T03:36:46.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:46 smithi136 ceph-mon[36436]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:46.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:46 smithi094 ceph-mon[29713]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:46.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.793+0000 7fb5bb948700 1 -- 172.21.15.136:0/2351464017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fbd20 msgr2=0x7fb5b40fc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.793+0000 7fb5bb948700 1 --2- 172.21.15.136:0/2351464017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fbd20 0x7fb5b40fc130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0004940 tx=0x7fb5b0009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.793+0000 7fb5bb948700 1 -- 172.21.15.136:0/2351464017 shutdown_connections 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.793+0000 7fb5bb948700 1 --2- 172.21.15.136:0/2351464017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b40fcfd0 0x7fb5b40fd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.793+0000 7fb5bb948700 1 --2- 172.21.15.136:0/2351464017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fbd20 0x7fb5b40fc130 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.793+0000 7fb5bb948700 1 -- 172.21.15.136:0/2351464017 >> 172.21.15.136:0/2351464017 conn(0x7fb5b40f7480 msgr2=0x7fb5b40f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.793+0000 7fb5bb948700 1 -- 172.21.15.136:0/2351464017 shutdown_connections 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.793+0000 7fb5bb948700 1 -- 172.21.15.136:0/2351464017 wait complete. 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5bb948700 1 Processor -- start 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5bb948700 1 -- start start 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5bb948700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b40fbd20 0x7fb5b410c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5bb948700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fcfd0 0x7fb5b410cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:46.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5bb948700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b410d3f0 con 0x7fb5b40fbd20 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5bb948700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b4111d90 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5b8ee3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fcfd0 0x7fb5b410cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5b96e4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b40fbd20 0x7fb5b410c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5b8ee3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fcfd0 0x7fb5b410cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:54890/0 (socket says 172.21.15.136:54890) 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5b8ee3700 1 -- 172.21.15.136:0/1849482191 learned_addr learned my addr 172.21.15.136:0/1849482191 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5b8ee3700 1 -- 172.21.15.136:0/1849482191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b40fbd20 msgr2=0x7fb5b410c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5b8ee3700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b40fbd20 0x7fb5b410c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5b8ee3700 1 -- 172.21.15.136:0/1849482191 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b0005040 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5b8ee3700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fcfd0 0x7fb5b410cd60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b790 tx=0x7fb5a4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.794+0000 7fb5a9ffb700 1 -- 172.21.15.136:0/1849482191 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a400f6f0 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.795+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b4111ff0 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.795+0000 7fb5a9ffb700 1 -- 172.21.15.136:0/1849482191 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5a400fd20 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.795+0000 7fb5a9ffb700 1 -- 172.21.15.136:0/1849482191 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a40103a0 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.795+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b4112530 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.796+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b404c710 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.796+0000 7fb5a9ffb700 1 -- 172.21.15.136:0/1849482191 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb5a4011400 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.796+0000 7fb5a9ffb700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb5a00692a0 0x7fb5a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:46.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.796+0000 7fb5a9ffb700 1 -- 172.21.15.136:0/1849482191 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb5a4086c50 con 0x7fb5b40fcfd0 2024-04-06T03:36:46.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.797+0000 7fb5b96e4700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb5a00692a0 0x7fb5a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:46.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.797+0000 7fb5b96e4700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb5a00692a0 0x7fb5a006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000b970 tx=0x7fb5b0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:46.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:46.801+0000 7fb5a9ffb700 1 -- 172.21.15.136:0/1849482191 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb5a4056d50 con 0x7fb5b40fcfd0 2024-04-06T03:36:47.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.058+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb5b4104890 con 0x7fb5b40fcfd0 2024-04-06T03:36:47.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.064+0000 7fb5a9ffb700 1 -- 172.21.15.136:0/1849482191 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb5a40568e0 con 0x7fb5b40fcfd0 2024-04-06T03:36:47.066 INFO:teuthology.orchestra.run.smithi136.stdout:[client.1] 2024-04-06T03:36:47.066 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBPwxBmc02eAxAAOPbwiF4w6spA+mxAh0UQzQ== 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb5a00692a0 msgr2=0x7fb5a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb5a00692a0 0x7fb5a006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000b970 tx=0x7fb5b0009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fcfd0 msgr2=0x7fb5b410cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fcfd0 0x7fb5b410cd60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b790 tx=0x7fb5a4009250 comp rx=0 tx=0).stop 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 shutdown_connections 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb5a00692a0 0x7fb5a006b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5b40fbd20 0x7fb5b410c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 --2- 172.21.15.136:0/1849482191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5b40fcfd0 0x7fb5b410cd60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 >> 172.21.15.136:0/1849482191 conn(0x7fb5b40f7480 msgr2=0x7fb5b40f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:47.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 shutdown_connections 2024-04-06T03:36:47.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:36:47.066+0000 7fb5bb948700 1 -- 172.21.15.136:0/1849482191 wait complete. 2024-04-06T03:36:47.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:47 smithi136 ceph-mon[36436]: from='client.? 172.21.15.136:0/1849482191' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-06T03:36:47.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:47 smithi136 ceph-mon[36436]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-06T03:36:47.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:47 smithi136 ceph-mon[36436]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-06T03:36:47.487 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:47 smithi094 ceph-mon[29713]: from='client.? 172.21.15.136:0/1849482191' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-06T03:36:47.487 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:47 smithi094 ceph-mon[29713]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-06T03:36:47.487 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:47 smithi094 ceph-mon[29713]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-06T03:36:47.675 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:36:47.675 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-06T03:36:47.675 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-06T03:36:47.716 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-06T03:36:47.716 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-06T03:36:47.716 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph mgr dump --format=json 2024-04-06T03:36:47.977 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:48.391 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:48 smithi094 ceph-mon[29713]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:48.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:48 smithi136 ceph-mon[36436]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.544+0000 7ff9dd859700 1 -- 172.21.15.94:0/512130248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 msgr2=0x7ff9d81020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.544+0000 7ff9dd859700 1 --2- 172.21.15.94:0/512130248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 0x7ff9d81020c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8004940 tx=0x7ff9c8009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.544+0000 7ff9dd859700 1 -- 172.21.15.94:0/512130248 shutdown_connections 2024-04-06T03:36:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.544+0000 7ff9dd859700 1 --2- 172.21.15.94:0/512130248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9d8102690 0x7ff9d80fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.544+0000 7ff9dd859700 1 --2- 172.21.15.94:0/512130248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 0x7ff9d81020c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.544+0000 7ff9dd859700 1 -- 172.21.15.94:0/512130248 >> 172.21.15.94:0/512130248 conn(0x7ff9d80f7860 msgr2=0x7ff9d80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.544+0000 7ff9dd859700 1 -- 172.21.15.94:0/512130248 shutdown_connections 2024-04-06T03:36:49.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.544+0000 7ff9dd859700 1 -- 172.21.15.94:0/512130248 wait complete. 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9dd859700 1 Processor -- start 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9dd859700 1 -- start start 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9dd859700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 0x7ff9d8112a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9dd859700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9d8102690 0x7ff9d8112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9dd859700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d8113610 con 0x7ff9d8101cf0 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9dd859700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d810ca00 con 0x7ff9d8102690 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9d6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 0x7ff9d8112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9d6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 0x7ff9d8112a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48232/0 (socket says 172.21.15.94:48232) 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9d6ffd700 1 -- 172.21.15.94:0/1624839854 learned_addr learned my addr 172.21.15.94:0/1624839854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:36:49.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.545+0000 7ff9d67fc700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9d8102690 0x7ff9d8112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.546+0000 7ff9d6ffd700 1 -- 172.21.15.94:0/1624839854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9d8102690 msgr2=0x7ff9d8112fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.546+0000 7ff9d6ffd700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9d8102690 0x7ff9d8112fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.546+0000 7ff9d6ffd700 1 -- 172.21.15.94:0/1624839854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8005040 con 0x7ff9d8101cf0 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.546+0000 7ff9d6ffd700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 0x7ff9d8112a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b3b0 tx=0x7ff9c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.546+0000 7ff9bf7fe700 1 -- 172.21.15.94:0/1624839854 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c800e040 con 0x7ff9d8101cf0 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.546+0000 7ff9bf7fe700 1 -- 172.21.15.94:0/1624839854 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9c8013420 con 0x7ff9d8101cf0 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.546+0000 7ff9bf7fe700 1 -- 172.21.15.94:0/1624839854 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c801c630 con 0x7ff9d8101cf0 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.546+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d810cc80 con 0x7ff9d8101cf0 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.546+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d810d240 con 0x7ff9d8101cf0 2024-04-06T03:36:49.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.547+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d80fd440 con 0x7ff9d8101cf0 2024-04-06T03:36:49.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.547+0000 7ff9bf7fe700 1 -- 172.21.15.94:0/1624839854 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff9c8013590 con 0x7ff9d8101cf0 2024-04-06T03:36:49.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.548+0000 7ff9bf7fe700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9c00692a0 0x7ff9c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:49.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.548+0000 7ff9d67fc700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9c00692a0 0x7ff9c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:49.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.548+0000 7ff9bf7fe700 1 -- 172.21.15.94:0/1624839854 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff9c8088030 con 0x7ff9d8101cf0 2024-04-06T03:36:49.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.548+0000 7ff9d67fc700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9c00692a0 0x7ff9c006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9d810dbc0 tx=0x7ff9cc00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:49.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.552+0000 7ff9bf7fe700 1 -- 172.21.15.94:0/1624839854 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff9c8058130 con 0x7ff9d8101cf0 2024-04-06T03:36:49.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.786+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff9d804bed0 con 0x7ff9d8101cf0 2024-04-06T03:36:49.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.789+0000 7ff9bf7fe700 1 -- 172.21.15.94:0/1624839854 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167168 (secure 0 0 0) 0x7ff9c8057cc0 con 0x7ff9d8101cf0 2024-04-06T03:36:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9c00692a0 msgr2=0x7ff9c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9c00692a0 0x7ff9c006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9d810dbc0 tx=0x7ff9cc00b040 comp rx=0 tx=0).stop 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 msgr2=0x7ff9d8112a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 0x7ff9d8112a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b3b0 tx=0x7ff9c8009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 shutdown_connections 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9c00692a0 0x7ff9c006b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101cf0 0x7ff9d8112a80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 --2- 172.21.15.94:0/1624839854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9d8102690 0x7ff9d8112fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 >> 172.21.15.94:0/1624839854 conn(0x7ff9d80f7860 msgr2=0x7ff9d80f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 shutdown_connections 2024-04-06T03:36:49.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:49.792+0000 7ff9dd859700 1 -- 172.21.15.94:0/1624839854 wait complete. 2024-04-06T03:36:50.345 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"active_gid":14208,"active_name":"smithi094.purlzp","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3627366865},{"type":"v1","addr":"172.21.15.94:6801","nonce":3627366865}]},"active_addr":"172.21.15.94:6801/3627366865","active_change":"2024-04-06T03:33:12.954031+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14228,"name":"smithi136.jagvwt","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.94:8443/","prometheus":"http://172.21.15.94:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3705686069}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":4131567976}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":457112173}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1319147349}]}]} 2024-04-06T03:36:50.348 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-06T03:36:50.349 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-06T03:36:50.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-06T03:36:50.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:50 smithi094 ceph-mon[29713]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:50.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:50 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1624839854' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-06T03:36:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:50 smithi136 ceph-mon[36436]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:50 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1624839854' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-06T03:36:50.600 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 -- 172.21.15.94:0/2913980293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4348102f50 msgr2=0x7f4348103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 --2- 172.21.15.94:0/2913980293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4348102f50 0x7f4348103320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f433c004940 tx=0x7f433c009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 -- 172.21.15.94:0/2913980293 shutdown_connections 2024-04-06T03:36:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 --2- 172.21.15.94:0/2913980293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43480fb4d0 0x7f43480fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:52.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 --2- 172.21.15.94:0/2913980293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4348102f50 0x7f4348103320 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 -- 172.21.15.94:0/2913980293 >> 172.21.15.94:0/2913980293 conn(0x7f43480714c0 msgr2=0x7f43480718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 -- 172.21.15.94:0/2913980293 shutdown_connections 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 -- 172.21.15.94:0/2913980293 wait complete. 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 Processor -- start 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.084+0000 7f434ebd8700 1 -- start start 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434ebd8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43480fb4d0 0x7f4348113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434ebd8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4348102f50 0x7f4348113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434ebd8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348114670 con 0x7f43480fb4d0 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434ebd8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434810dc10 con 0x7f4348102f50 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434c974700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43480fb4d0 0x7f4348113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:52.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434c974700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43480fb4d0 0x7f4348113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48238/0 (socket says 172.21.15.94:48238) 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434c974700 1 -- 172.21.15.94:0/2351448339 learned_addr learned my addr 172.21.15.94:0/2351448339 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f4347fff700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4348102f50 0x7f4348113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434c974700 1 -- 172.21.15.94:0/2351448339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4348102f50 msgr2=0x7f4348113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434c974700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4348102f50 0x7f4348113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434c974700 1 -- 172.21.15.94:0/2351448339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c005040 con 0x7f43480fb4d0 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434c974700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43480fb4d0 0x7f4348113a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f433c009100 tx=0x7f433c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f43457fa700 1 -- 172.21.15.94:0/2351448339 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f433c00e040 con 0x7f43480fb4d0 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f43457fa700 1 -- 172.21.15.94:0/2351448339 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f433c013420 con 0x7f43480fb4d0 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434810de30 con 0x7f43480fb4d0 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.085+0000 7f43457fa700 1 -- 172.21.15.94:0/2351448339 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f433c01c610 con 0x7f43480fb4d0 2024-04-06T03:36:52.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.086+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434810e370 con 0x7f43480fb4d0 2024-04-06T03:36:52.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.086+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4348100200 con 0x7f43480fb4d0 2024-04-06T03:36:52.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.087+0000 7f43457fa700 1 -- 172.21.15.94:0/2351448339 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f433c013590 con 0x7f43480fb4d0 2024-04-06T03:36:52.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.088+0000 7f43457fa700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f43380692a0 0x7f433806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:52.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.088+0000 7f43457fa700 1 -- 172.21.15.94:0/2351448339 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f433c0880d0 con 0x7f43480fb4d0 2024-04-06T03:36:52.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.088+0000 7f4347fff700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f43380692a0 0x7f433806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:52.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.088+0000 7f4347fff700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f43380692a0 0x7f433806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:52.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.092+0000 7f43457fa700 1 -- 172.21.15.94:0/2351448339 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f433c0581d0 con 0x7f43480fb4d0 2024-04-06T03:36:52.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.283+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f434810a120 con 0x7f43480fb4d0 2024-04-06T03:36:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.284+0000 7f43457fa700 1 -- 172.21.15.94:0/2351448339 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11094 (secure 0 0 0) 0x7f433c057d60 con 0x7f43480fb4d0 2024-04-06T03:36:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:36:52.286 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","created":"2024-04-06T03:31:12.411006+0000","modified":"2024-04-06T03:36:37.175616+0000","last_up_change":"2024-04-06T03:36:36.170053+0000","last_in_change":"2024-04-06T03:36:17.627724+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-06T03:35:23.271684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7f51dba0-a74e-444c-b4ab-d5f1fa68ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6803","nonce":2289372883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6805","nonce":2289372883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6809","nonce":2289372883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6807","nonce":2289372883}]},"public_addr":"172.21.15.94:6803/2289372883","cluster_addr":"172.21.15.94:6805/2289372883","heartbeat_back_addr":"172.21.15.94:6809/2289372883","heartbeat_front_addr":"172.21.15.94:6807/2289372883","state":["exists","up"]},{"osd":1,"uuid":"2dd01097-c6d6-45e9-acee-ca5ea5f4e166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6811","nonce":1337787373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6813","nonce":1337787373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6817","nonce":1337787373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6815","nonce":1337787373}]},"public_addr":"172.21.15.94:6811/1337787373","cluster_addr":"172.21.15.94:6813/1337787373","heartbeat_back_addr":"172.21.15.94:6817/1337787373","heartbeat_front_addr":"172.21.15.94:6815/1337787373","state":["exists","up"]},{"osd":2,"uuid":"9645da7c-ffbc-4415-8fa0-0fcb07c53f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6819","nonce":1538420111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6821","nonce":1538420111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6825","nonce":1538420111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6823","nonce":1538420111}]},"public_addr":"172.21.15.94:6819/1538420111","cluster_addr":"172.21.15.94:6821/1538420111","heartbeat_back_addr":"172.21.15.94:6825/1538420111","heartbeat_front_addr":"172.21.15.94:6823/1538420111","state":["exists","up"]},{"osd":3,"uuid":"43321ebd-f195-4738-a2a5-f4d55a375bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271372986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271372986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271372986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271372986}]},"public_addr":"172.21.15.136:6801/1271372986","cluster_addr":"172.21.15.136:6803/1271372986","heartbeat_back_addr":"172.21.15.136:6807/1271372986","heartbeat_front_addr":"172.21.15.136:6805/1271372986","state":["exists","up"]},{"osd":4,"uuid":"572f9cc0-e83c-4354-babb-519be04069c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6809","nonce":2873260774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6811","nonce":2873260774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6815","nonce":2873260774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6813","nonce":2873260774}]},"public_addr":"172.21.15.136:6809/2873260774","cluster_addr":"172.21.15.136:6811/2873260774","heartbeat_back_addr":"172.21.15.136:6815/2873260774","heartbeat_front_addr":"172.21.15.136:6813/2873260774","state":["exists","up"]},{"osd":5,"uuid":"a38e98d8-dfb7-4083-aae8-e5853a896d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6817","nonce":3055268575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6819","nonce":3055268575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6823","nonce":3055268575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6821","nonce":3055268575}]},"public_addr":"172.21.15.136:6817/3055268575","cluster_addr":"172.21.15.136:6819/3055268575","heartbeat_back_addr":"172.21.15.136:6823/3055268575","heartbeat_front_addr":"172.21.15.136:6821/3055268575","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:25.784070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:53.656615+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:20.675464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:44.290139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:09.122272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:34.741771+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1085015108":"2024-04-07T03:33:12.953920+0000","172.21.15.94:6801/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1082665650":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6801/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/3195038611":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/852522634":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/2412091380":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/2779591805":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6801/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/1247691830":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/3724179185":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/1084447585":"2024-04-07T03:32:10.456958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-06T03:36:52.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f43380692a0 msgr2=0x7f433806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:52.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f43380692a0 0x7f433806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334008040 comp rx=0 tx=0).stop 2024-04-06T03:36:52.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43480fb4d0 msgr2=0x7f4348113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43480fb4d0 0x7f4348113a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f433c009100 tx=0x7f433c009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 shutdown_connections 2024-04-06T03:36:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f43380692a0 0x7f433806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43480fb4d0 0x7f4348113a50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 --2- 172.21.15.94:0/2351448339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4348102f50 0x7f4348113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 >> 172.21.15.94:0/2351448339 conn(0x7f43480714c0 msgr2=0x7f43480fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 shutdown_connections 2024-04-06T03:36:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:52.286+0000 7f434ebd8700 1 -- 172.21.15.94:0/2351448339 wait complete. 2024-04-06T03:36:52.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:52 smithi094 ceph-mon[29713]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:52.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:52 smithi136 ceph-mon[36436]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:53.032 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-06T03:36:53.032 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-06T03:36:53.290 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:53.411 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:53 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2351448339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:36:53.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:53 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2351448339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:36:54.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:54 smithi094 ceph-mon[29713]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:54.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:54 smithi136 ceph-mon[36436]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.801+0000 7fa516dcf700 1 -- 172.21.15.94:0/650222012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510102800 msgr2=0x7fa5100fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.801+0000 7fa516dcf700 1 --2- 172.21.15.94:0/650222012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510102800 0x7fa5100fc650 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc004940 tx=0x7fa4fc009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.801+0000 7fa516dcf700 1 -- 172.21.15.94:0/650222012 shutdown_connections 2024-04-06T03:36:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.801+0000 7fa516dcf700 1 --2- 172.21.15.94:0/650222012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510102800 0x7fa5100fc650 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.801+0000 7fa516dcf700 1 --2- 172.21.15.94:0/650222012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa510101e60 0x7fa510102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:54.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.801+0000 7fa516dcf700 1 -- 172.21.15.94:0/650222012 >> 172.21.15.94:0/650222012 conn(0x7fa510071530 msgr2=0x7fa510071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.801+0000 7fa516dcf700 1 -- 172.21.15.94:0/650222012 shutdown_connections 2024-04-06T03:36:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.801+0000 7fa516dcf700 1 -- 172.21.15.94:0/650222012 wait complete. 2024-04-06T03:36:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa516dcf700 1 Processor -- start 2024-04-06T03:36:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa516dcf700 1 -- start start 2024-04-06T03:36:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa516dcf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510101e60 0x7fa5101128a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa516dcf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa510102800 0x7fa510112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa516dcf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101134c0 con 0x7fa510101e60 2024-04-06T03:36:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa516dcf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51010ca80 con 0x7fa510102800 2024-04-06T03:36:54.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa514b6b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510101e60 0x7fa5101128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa514b6b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510101e60 0x7fa5101128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48262/0 (socket says 172.21.15.94:48262) 2024-04-06T03:36:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa514b6b700 1 -- 172.21.15.94:0/1156000822 learned_addr learned my addr 172.21.15.94:0/1156000822 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:36:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa514b6b700 1 -- 172.21.15.94:0/1156000822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa510102800 msgr2=0x7fa510112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa50ffff700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa510102800 0x7fa510112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.802+0000 7fa514b6b700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa510102800 0x7fa510112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.803+0000 7fa514b6b700 1 -- 172.21.15.94:0/1156000822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc005040 con 0x7fa510101e60 2024-04-06T03:36:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.803+0000 7fa514b6b700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510101e60 0x7fa5101128a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa50400b790 tx=0x7fa504009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.803+0000 7fa50d7fa700 1 -- 172.21.15.94:0/1156000822 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa50400f770 con 0x7fa510101e60 2024-04-06T03:36:54.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.803+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51010cd60 con 0x7fa510101e60 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.803+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51010d2a0 con 0x7fa510101e60 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.803+0000 7fa50d7fa700 1 -- 172.21.15.94:0/1156000822 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa50400fda0 con 0x7fa510101e60 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.803+0000 7fa50d7fa700 1 -- 172.21.15.94:0/1156000822 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa50400d310 con 0x7fa510101e60 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.804+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5100ff1c0 con 0x7fa510101e60 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.805+0000 7fa50d7fa700 1 -- 172.21.15.94:0/1156000822 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa504010420 con 0x7fa510101e60 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.805+0000 7fa50d7fa700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa5000692a0 0x7fa50006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.805+0000 7fa50d7fa700 1 -- 172.21.15.94:0/1156000822 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa504086e70 con 0x7fa510101e60 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.805+0000 7fa50ffff700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa5000692a0 0x7fa50006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.805+0000 7fa50ffff700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa5000692a0 0x7fa50006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009fd0 tx=0x7fa4fc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:54.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.809+0000 7fa50d7fa700 1 -- 172.21.15.94:0/1156000822 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa504056f70 con 0x7fa510101e60 2024-04-06T03:36:54.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.995+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa51010df50 con 0x7fa510101e60 2024-04-06T03:36:54.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.996+0000 7fa50d7fa700 1 -- 172.21.15.94:0/1156000822 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11094 (secure 0 0 0) 0x7fa504056b00 con 0x7fa510101e60 2024-04-06T03:36:54.999 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:36:54.999 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":40,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","created":"2024-04-06T03:31:12.411006+0000","modified":"2024-04-06T03:36:37.175616+0000","last_up_change":"2024-04-06T03:36:36.170053+0000","last_in_change":"2024-04-06T03:36:17.627724+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-06T03:35:23.271684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"7f51dba0-a74e-444c-b4ab-d5f1fa68ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6803","nonce":2289372883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6805","nonce":2289372883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6809","nonce":2289372883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6807","nonce":2289372883}]},"public_addr":"172.21.15.94:6803/2289372883","cluster_addr":"172.21.15.94:6805/2289372883","heartbeat_back_addr":"172.21.15.94:6809/2289372883","heartbeat_front_addr":"172.21.15.94:6807/2289372883","state":["exists","up"]},{"osd":1,"uuid":"2dd01097-c6d6-45e9-acee-ca5ea5f4e166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6811","nonce":1337787373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6813","nonce":1337787373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6817","nonce":1337787373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6815","nonce":1337787373}]},"public_addr":"172.21.15.94:6811/1337787373","cluster_addr":"172.21.15.94:6813/1337787373","heartbeat_back_addr":"172.21.15.94:6817/1337787373","heartbeat_front_addr":"172.21.15.94:6815/1337787373","state":["exists","up"]},{"osd":2,"uuid":"9645da7c-ffbc-4415-8fa0-0fcb07c53f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6819","nonce":1538420111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6821","nonce":1538420111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6825","nonce":1538420111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6823","nonce":1538420111}]},"public_addr":"172.21.15.94:6819/1538420111","cluster_addr":"172.21.15.94:6821/1538420111","heartbeat_back_addr":"172.21.15.94:6825/1538420111","heartbeat_front_addr":"172.21.15.94:6823/1538420111","state":["exists","up"]},{"osd":3,"uuid":"43321ebd-f195-4738-a2a5-f4d55a375bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271372986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271372986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271372986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271372986}]},"public_addr":"172.21.15.136:6801/1271372986","cluster_addr":"172.21.15.136:6803/1271372986","heartbeat_back_addr":"172.21.15.136:6807/1271372986","heartbeat_front_addr":"172.21.15.136:6805/1271372986","state":["exists","up"]},{"osd":4,"uuid":"572f9cc0-e83c-4354-babb-519be04069c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6809","nonce":2873260774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6811","nonce":2873260774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6815","nonce":2873260774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6813","nonce":2873260774}]},"public_addr":"172.21.15.136:6809/2873260774","cluster_addr":"172.21.15.136:6811/2873260774","heartbeat_back_addr":"172.21.15.136:6815/2873260774","heartbeat_front_addr":"172.21.15.136:6813/2873260774","state":["exists","up"]},{"osd":5,"uuid":"a38e98d8-dfb7-4083-aae8-e5853a896d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6817","nonce":3055268575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6819","nonce":3055268575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6823","nonce":3055268575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6821","nonce":3055268575}]},"public_addr":"172.21.15.136:6817/3055268575","cluster_addr":"172.21.15.136:6819/3055268575","heartbeat_back_addr":"172.21.15.136:6823/3055268575","heartbeat_front_addr":"172.21.15.136:6821/3055268575","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:25.784070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:53.656615+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:20.675464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:44.290139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:09.122272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:34.741771+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1085015108":"2024-04-07T03:33:12.953920+0000","172.21.15.94:6801/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1082665650":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6801/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/3195038611":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/852522634":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/2412091380":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/2779591805":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6801/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/1247691830":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/3724179185":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/1084447585":"2024-04-07T03:32:10.456958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-06T03:36:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa5000692a0 msgr2=0x7fa50006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa5000692a0 0x7fa50006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009fd0 tx=0x7fa4fc009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510101e60 msgr2=0x7fa5101128a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510101e60 0x7fa5101128a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa50400b790 tx=0x7fa504009250 comp rx=0 tx=0).stop 2024-04-06T03:36:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 shutdown_connections 2024-04-06T03:36:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa5000692a0 0x7fa50006b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa510101e60 0x7fa5101128a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 --2- 172.21.15.94:0/1156000822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa510102800 0x7fa510112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 >> 172.21.15.94:0/1156000822 conn(0x7fa510071530 msgr2=0x7fa5100fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 shutdown_connections 2024-04-06T03:36:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:54.998+0000 7fa516dcf700 1 -- 172.21.15.94:0/1156000822 wait complete. 2024-04-06T03:36:55.448 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:55 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1156000822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:36:55.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:55 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1156000822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:36:55.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-04-06T03:36:55.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-04-06T03:36:55.619 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-04-06T03:36:55.619 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-04-06T03:36:55.620 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-04-06T03:36:55.620 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-04-06T03:36:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:56.347 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:56 smithi094 ceph-mon[29713]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:56.595 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:56.596 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:56.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:56 smithi136 ceph-mon[36436]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:57.858 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:57.865 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:36:58.358 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:58 smithi094 ceph-mon[29713]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:58.359 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:36:58 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:36:58.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:58 smithi136 ceph-mon[36436]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:36:58.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:36:58 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 -- 172.21.15.94:0/1196878033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9600708a0 msgr2=0x7fb960070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 --2- 172.21.15.94:0/1196878033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9600708a0 0x7fb960070d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb954004970 tx=0x7fb954009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 -- 172.21.15.94:0/1196878033 shutdown_connections 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 --2- 172.21.15.94:0/1196878033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9600708a0 0x7fb960070d10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 --2- 172.21.15.94:0/1196878033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb960071f90 0x7fb960070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 -- 172.21.15.94:0/1196878033 >> 172.21.15.94:0/1196878033 conn(0x7fb9600f89b0 msgr2=0x7fb9600fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 -- 172.21.15.94:0/1196878033 shutdown_connections 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 -- 172.21.15.94:0/1196878033 wait complete. 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 Processor -- start 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.885+0000 7fb96649a700 1 -- start start 2024-04-06T03:36:58.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb96649a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9600708a0 0x7fb960119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb96649a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960071f90 0x7fb96011a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb96649a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96011ac00 con 0x7fb960071f90 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb96649a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb960114150 con 0x7fb9600708a0 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960071f90 0x7fb96011a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960071f90 0x7fb96011a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52514/0 (socket says 172.21.15.94:52514) 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95f7fe700 1 -- 172.21.15.94:0/1044335578 learned_addr learned my addr 172.21.15.94:0/1044335578 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95f7fe700 1 -- 172.21.15.94:0/1044335578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9600708a0 msgr2=0x7fb960119fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95f7fe700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9600708a0 0x7fb960119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95f7fe700 1 -- 172.21.15.94:0/1044335578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb954005040 con 0x7fb960071f90 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95f7fe700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960071f90 0x7fb96011a520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb954004f50 tx=0x7fb954009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95cff9700 1 -- 172.21.15.94:0/1044335578 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95400e040 con 0x7fb960071f90 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95cff9700 1 -- 172.21.15.94:0/1044335578 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb954004290 con 0x7fb960071f90 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb960114370 con 0x7fb960071f90 2024-04-06T03:36:58.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb95cff9700 1 -- 172.21.15.94:0/1044335578 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95401c630 con 0x7fb960071f90 2024-04-06T03:36:58.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.886+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9601148b0 con 0x7fb960071f90 2024-04-06T03:36:58.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.887+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb960114de0 con 0x7fb960071f90 2024-04-06T03:36:58.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.888+0000 7fb95cff9700 1 -- 172.21.15.94:0/1044335578 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb95400bc10 con 0x7fb960071f90 2024-04-06T03:36:58.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.888+0000 7fb95cff9700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb9480692a0 0x7fb94806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:58.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.888+0000 7fb95cff9700 1 -- 172.21.15.94:0/1044335578 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb954087ff0 con 0x7fb960071f90 2024-04-06T03:36:58.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.888+0000 7fb95ffff700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb9480692a0 0x7fb94806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:58.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.888+0000 7fb95cff9700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] conn(0x7fb94806ef70 0x7fb948071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:36:58.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.888+0000 7fb95cff9700 1 -- 172.21.15.94:0/1044335578 --> [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb948071a30 con 0x7fb94806ef70 2024-04-06T03:36:58.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.888+0000 7fb95cff9700 1 -- 172.21.15.94:0/1044335578 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb95408a600 con 0x7fb960071f90 2024-04-06T03:36:58.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.888+0000 7fb964a37700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] conn(0x7fb94806ef70 0x7fb948071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:36:58.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.889+0000 7fb95ffff700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb9480692a0 0x7fb94806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb950006fd0 tx=0x7fb950006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:58.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.889+0000 7fb964a37700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] conn(0x7fb94806ef70 0x7fb948071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:36:58.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.890+0000 7fb95cff9700 1 -- 172.21.15.94:0/1044335578 <== osd.4 v2:172.21.15.136:6808/2873260774 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb948071a30 con 0x7fb94806ef70 2024-04-06T03:36:58.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.904+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 --> [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9600fa3f0 con 0x7fb94806ef70 2024-04-06T03:36:58.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb95cff9700 1 -- 172.21.15.94:0/1044335578 <== osd.4 v2:172.21.15.136:6808/2873260774 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb9600fa3f0 con 0x7fb94806ef70 2024-04-06T03:36:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 >> [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] conn(0x7fb94806ef70 msgr2=0x7fb948071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] conn(0x7fb94806ef70 0x7fb948071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb9480692a0 msgr2=0x7fb94806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb9480692a0 0x7fb94806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb950006fd0 tx=0x7fb950006db0 comp rx=0 tx=0).stop 2024-04-06T03:36:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960071f90 msgr2=0x7fb96011a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:36:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960071f90 0x7fb96011a520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb954004f50 tx=0x7fb954009e30 comp rx=0 tx=0).stop 2024-04-06T03:36:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 shutdown_connections 2024-04-06T03:36:58.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb9480692a0 0x7fb94806b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.136:6808/2873260774,v1:172.21.15.136:6809/2873260774] conn(0x7fb94806ef70 0x7fb948071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9600708a0 0x7fb960119fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 --2- 172.21.15.94:0/1044335578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb960071f90 0x7fb96011a520 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:36:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 >> 172.21.15.94:0/1044335578 conn(0x7fb9600f89b0 msgr2=0x7fb9600f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:36:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 shutdown_connections 2024-04-06T03:36:58.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:36:58.905+0000 7fb96649a700 1 -- 172.21.15.94:0/1044335578 wait complete. 2024-04-06T03:37:00.387 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:00 smithi094 ceph-mon[29713]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:00.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:00 smithi136 ceph-mon[36436]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.552+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/2012498460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 msgr2=0x7f6f58102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.552+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/2012498460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 0x7f6f58102140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f48004940 tx=0x7f6f48009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.552+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/2012498460 shutdown_connections 2024-04-06T03:37:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.552+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/2012498460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f58102710 0x7f6f580fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.552+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/2012498460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 0x7f6f58102140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.552+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/2012498460 >> 172.21.15.94:0/2012498460 conn(0x7f6f58072bc0 msgr2=0x7f6f58072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.552+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/2012498460 shutdown_connections 2024-04-06T03:37:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.552+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/2012498460 wait complete. 2024-04-06T03:37:01.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.552+0000 7f6f5e2ca700 1 Processor -- start 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f5e2ca700 1 -- start start 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f5e2ca700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 0x7f6f5810eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f5e2ca700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f58102710 0x7f6f5810eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f5e2ca700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5810f6d0 con 0x7f6f58101d70 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f5e2ca700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5810f810 con 0x7f6f58102710 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 0x7f6f5810eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 0x7f6f5810eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52526/0 (socket says 172.21.15.94:52526) 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f57fff700 1 -- 172.21.15.94:0/1079695880 learned_addr learned my addr 172.21.15.94:0/1079695880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f57fff700 1 -- 172.21.15.94:0/1079695880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f58102710 msgr2=0x7f6f5810eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f57fff700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f58102710 0x7f6f5810eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f57fff700 1 -- 172.21.15.94:0/1079695880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f48005040 con 0x7f6f58101d70 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f57fff700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 0x7f6f5810eab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800b3b0 tx=0x7f6f48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f54ff9700 1 -- 172.21.15.94:0/1079695880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f4800e040 con 0x7f6f58101d70 2024-04-06T03:37:01.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f54ff9700 1 -- 172.21.15.94:0/1079695880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f4800bbc0 con 0x7f6f58101d70 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.553+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f58117b20 con 0x7f6f58101d70 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.554+0000 7f6f54ff9700 1 -- 172.21.15.94:0/1079695880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f4801c3f0 con 0x7f6f58101d70 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.554+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f581180b0 con 0x7f6f58101d70 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.554+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f5805e700 con 0x7f6f58101d70 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.555+0000 7f6f54ff9700 1 -- 172.21.15.94:0/1079695880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6f4800bd30 con 0x7f6f58101d70 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.555+0000 7f6f54ff9700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f6f400692a0 0x7f6f4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.555+0000 7f6f54ff9700 1 -- 172.21.15.94:0/1079695880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6f48088b60 con 0x7f6f58101d70 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.555+0000 7f6f577fe700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f6f400692a0 0x7f6f4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.555+0000 7f6f54ff9700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] conn(0x7f6f4006ef70 0x7f6f40071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.555+0000 7f6f54ff9700 1 -- 172.21.15.94:0/1079695880 --> [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f40071a30 con 0x7f6f4006ef70 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.555+0000 7f6f54ff9700 1 -- 172.21.15.94:0/1079695880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f48088f40 con 0x7f6f58101d70 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.556+0000 7f6f5c867700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] conn(0x7f6f4006ef70 0x7f6f40071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.556+0000 7f6f577fe700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f6f400692a0 0x7f6f4006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c006fd0 tx=0x7f6f4c006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:01.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.556+0000 7f6f5c867700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] conn(0x7f6f4006ef70 0x7f6f40071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.556+0000 7f6f54ff9700 1 -- 172.21.15.94:0/1079695880 <== osd.0 v2:172.21.15.94:6802/2289372883 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6f40071a30 con 0x7f6f4006ef70 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.571+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 --> [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f5805eb80 con 0x7f6f4006ef70 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.571+0000 7f6f54ff9700 1 -- 172.21.15.94:0/1079695880 <== osd.0 v2:172.21.15.94:6802/2289372883 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6f5805eb80 con 0x7f6f4006ef70 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.572+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] conn(0x7f6f4006ef70 msgr2=0x7f6f40071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.572+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] conn(0x7f6f4006ef70 0x7f6f40071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.572+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f6f400692a0 msgr2=0x7f6f4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.572+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f6f400692a0 0x7f6f4006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c006fd0 tx=0x7f6f4c006db0 comp rx=0 tx=0).stop 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.572+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 msgr2=0x7f6f5810eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.572+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 0x7f6f5810eab0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800b3b0 tx=0x7f6f48009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.573+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 shutdown_connections 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.573+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6802/2289372883,v1:172.21.15.94:6803/2289372883] conn(0x7f6f4006ef70 0x7f6f40071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.573+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f6f400692a0 0x7f6f4006b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.573+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f58101d70 0x7f6f5810eab0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.573+0000 7f6f5e2ca700 1 --2- 172.21.15.94:0/1079695880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f58102710 0x7f6f5810eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:01.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.573+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 >> 172.21.15.94:0/1079695880 conn(0x7f6f58072bc0 msgr2=0x7f6f580fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.573+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 shutdown_connections 2024-04-06T03:37:01.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:01.573+0000 7f6f5e2ca700 1 -- 172.21.15.94:0/1079695880 wait complete. 2024-04-06T03:37:02.504 INFO:teuthology.orchestra.run.smithi094.stdout:38654705698 2024-04-06T03:37:02.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-04-06T03:37:02.531 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:02 smithi094 ceph-mon[29713]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:02.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:02 smithi136 ceph-mon[36436]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.684+0000 7f35d9f05700 1 -- 172.21.15.94:0/2740027290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d41026b0 msgr2=0x7f35d40fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.684+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2740027290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d41026b0 0x7f35d40fbd20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f35c8004940 tx=0x7f35c8009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:02.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.684+0000 7f35d9f05700 1 -- 172.21.15.94:0/2740027290 shutdown_connections 2024-04-06T03:37:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.684+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2740027290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d41026b0 0x7f35d40fbd20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.684+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2740027290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d4101d10 0x7f35d41020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.684+0000 7f35d9f05700 1 -- 172.21.15.94:0/2740027290 >> 172.21.15.94:0/2740027290 conn(0x7f35d40f7880 msgr2=0x7f35d40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.685+0000 7f35d9f05700 1 -- 172.21.15.94:0/2740027290 shutdown_connections 2024-04-06T03:37:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.685+0000 7f35d9f05700 1 -- 172.21.15.94:0/2740027290 wait complete. 2024-04-06T03:37:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.685+0000 7f35d9f05700 1 Processor -- start 2024-04-06T03:37:02.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.685+0000 7f35d9f05700 1 -- start start 2024-04-06T03:37:02.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.685+0000 7f35d9f05700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d4101d10 0x7f35d4112db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:02.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d9f05700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d41026b0 0x7f35d41132f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:02.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d9f05700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d41139d0 con 0x7f35d4101d10 2024-04-06T03:37:02.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d9f05700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d410cd10 con 0x7f35d41026b0 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d4101d10 0x7f35d4112db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d4101d10 0x7f35d4112db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52534/0 (socket says 172.21.15.94:52534) 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d37fe700 1 -- 172.21.15.94:0/2183599554 learned_addr learned my addr 172.21.15.94:0/2183599554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d37fe700 1 -- 172.21.15.94:0/2183599554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d41026b0 msgr2=0x7f35d41132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d2ffd700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d41026b0 0x7f35d41132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d37fe700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d41026b0 0x7f35d41132f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d37fe700 1 -- 172.21.15.94:0/2183599554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8005040 con 0x7f35d4101d10 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d2ffd700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d41026b0 0x7f35d41132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d37fe700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d4101d10 0x7f35d4112db0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35c400b790 tx=0x7f35c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35bbfff700 1 -- 172.21.15.94:0/2183599554 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35c400f770 con 0x7f35d4101d10 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35bbfff700 1 -- 172.21.15.94:0/2183599554 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35c400fda0 con 0x7f35d4101d10 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d410cf90 con 0x7f35d4101d10 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35bbfff700 1 -- 172.21.15.94:0/2183599554 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35c400d310 con 0x7f35d4101d10 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.686+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d410d500 con 0x7f35d4101d10 2024-04-06T03:37:02.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.687+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35d40fd460 con 0x7f35d4101d10 2024-04-06T03:37:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.688+0000 7f35bbfff700 1 -- 172.21.15.94:0/2183599554 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f35c4010420 con 0x7f35d4101d10 2024-04-06T03:37:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.688+0000 7f35bbfff700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35bc0692a0 0x7f35bc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.688+0000 7f35bbfff700 1 -- 172.21.15.94:0/2183599554 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f35c4086e70 con 0x7f35d4101d10 2024-04-06T03:37:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.688+0000 7f35d2ffd700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35bc0692a0 0x7f35bc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.688+0000 7f35bbfff700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] conn(0x7f35bc06ef70 0x7f35bc071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.688+0000 7f35bbfff700 1 -- 172.21.15.94:0/2183599554 --> [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35bc071a30 con 0x7f35bc06ef70 2024-04-06T03:37:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.688+0000 7f35bbfff700 1 -- 172.21.15.94:0/2183599554 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35c408c4d0 con 0x7f35d4101d10 2024-04-06T03:37:02.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.688+0000 7f35d3fff700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] conn(0x7f35bc06ef70 0x7f35bc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:02.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.689+0000 7f35d2ffd700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35bc0692a0 0x7f35bc06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009fd0 tx=0x7f35c8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:02.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.689+0000 7f35d3fff700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] conn(0x7f35bc06ef70 0x7f35bc071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:02.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.690+0000 7f35bbfff700 1 -- 172.21.15.94:0/2183599554 <== osd.2 v2:172.21.15.94:6818/1538420111 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f35bc071a30 con 0x7f35bc06ef70 2024-04-06T03:37:02.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.705+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 --> [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35d405f5f0 con 0x7f35bc06ef70 2024-04-06T03:37:02.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35bbfff700 1 -- 172.21.15.94:0/2183599554 <== osd.2 v2:172.21.15.94:6818/1538420111 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f35d405f5f0 con 0x7f35bc06ef70 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] conn(0x7f35bc06ef70 msgr2=0x7f35bc071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] conn(0x7f35bc06ef70 0x7f35bc071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35bc0692a0 msgr2=0x7f35bc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35bc0692a0 0x7f35bc06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009fd0 tx=0x7f35c8009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d4101d10 msgr2=0x7f35d4112db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d4101d10 0x7f35d4112db0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35c400b790 tx=0x7f35c4009250 comp rx=0 tx=0).stop 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 shutdown_connections 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6818/1538420111,v1:172.21.15.94:6819/1538420111] conn(0x7f35bc06ef70 0x7f35bc071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35bc0692a0 0x7f35bc06b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35d4101d10 0x7f35d4112db0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 --2- 172.21.15.94:0/2183599554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35d41026b0 0x7f35d41132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 >> 172.21.15.94:0/2183599554 conn(0x7f35d40f7880 msgr2=0x7f35d4104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 shutdown_connections 2024-04-06T03:37:02.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:02.706+0000 7f35d9f05700 1 -- 172.21.15.94:0/2183599554 wait complete. 2024-04-06T03:37:03.063 INFO:teuthology.orchestra.run.smithi094.stdout:146028888075 2024-04-06T03:37:03.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-04-06T03:37:04.337 INFO:teuthology.orchestra.run.smithi094.stdout:81604378645 2024-04-06T03:37:04.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-04-06T03:37:04.521 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:04 smithi094 ceph-mon[29713]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:04.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:04 smithi136 ceph-mon[36436]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:06.317 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:06 smithi094 ceph-mon[29713]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:06.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:06 smithi136 ceph-mon[36436]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:07.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.585+0000 7f0c80adc700 1 -- 172.21.15.94:0/328769914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c1037a0 msgr2=0x7f0c7c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:07.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.585+0000 7f0c80adc700 1 --2- 172.21.15.94:0/328769914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c1037a0 0x7f0c7c0fce10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0c70004940 tx=0x7f0c70009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:07.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.585+0000 7f0c80adc700 1 -- 172.21.15.94:0/328769914 shutdown_connections 2024-04-06T03:37:07.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.585+0000 7f0c80adc700 1 --2- 172.21.15.94:0/328769914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c1037a0 0x7f0c7c0fce10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:07.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.585+0000 7f0c80adc700 1 --2- 172.21.15.94:0/328769914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c7c102e00 0x7f0c7c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:07.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.585+0000 7f0c80adc700 1 -- 172.21.15.94:0/328769914 >> 172.21.15.94:0/328769914 conn(0x7f0c7c0f89b0 msgr2=0x7f0c7c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:07.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.585+0000 7f0c80adc700 1 -- 172.21.15.94:0/328769914 shutdown_connections 2024-04-06T03:37:07.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.585+0000 7f0c80adc700 1 -- 172.21.15.94:0/328769914 wait complete. 2024-04-06T03:37:07.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c80adc700 1 Processor -- start 2024-04-06T03:37:07.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c80adc700 1 -- start start 2024-04-06T03:37:07.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c80adc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c102e00 0x7f0c7c113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:07.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c80adc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c7c1037a0 0x7f0c7c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:07.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c80adc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c114510 con 0x7f0c7c102e00 2024-04-06T03:37:07.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c80adc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c10db10 con 0x7f0c7c1037a0 2024-04-06T03:37:07.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c7a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c102e00 0x7f0c7c113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:07.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c7a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c102e00 0x7f0c7c113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37202/0 (socket says 172.21.15.94:37202) 2024-04-06T03:37:07.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c7a59c700 1 -- 172.21.15.94:0/2234237799 learned_addr learned my addr 172.21.15.94:0/2234237799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:07.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c79d9b700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c7c1037a0 0x7f0c7c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:07.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c7a59c700 1 -- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c7c1037a0 msgr2=0x7f0c7c113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:07.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c7a59c700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c7c1037a0 0x7f0c7c113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:07.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c7a59c700 1 -- 172.21.15.94:0/2234237799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70005040 con 0x7f0c7c102e00 2024-04-06T03:37:07.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.586+0000 7f0c7a59c700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c102e00 0x7f0c7c113980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c002a50 tx=0x7f0c6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:07.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.587+0000 7f0c6affd700 1 -- 172.21.15.94:0/2234237799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c6c007cc0 con 0x7f0c7c102e00 2024-04-06T03:37:07.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.587+0000 7f0c6affd700 1 -- 172.21.15.94:0/2234237799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c6c00f040 con 0x7f0c7c102e00 2024-04-06T03:37:07.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.587+0000 7f0c6affd700 1 -- 172.21.15.94:0/2234237799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c6c009640 con 0x7f0c7c102e00 2024-04-06T03:37:07.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.587+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c10ddf0 con 0x7f0c7c102e00 2024-04-06T03:37:07.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.587+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c10e3b0 con 0x7f0c7c102e00 2024-04-06T03:37:07.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.587+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c7c10e8e0 con 0x7f0c7c102e00 2024-04-06T03:37:07.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.588+0000 7f0c6affd700 1 -- 172.21.15.94:0/2234237799 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0c6c0097a0 con 0x7f0c7c102e00 2024-04-06T03:37:07.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.588+0000 7f0c6affd700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0c640692a0 0x7f0c6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:07.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.588+0000 7f0c6affd700 1 -- 172.21.15.94:0/2234237799 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0c6c086b70 con 0x7f0c7c102e00 2024-04-06T03:37:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.588+0000 7f0c79d9b700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0c640692a0 0x7f0c6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.588+0000 7f0c6affd700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] conn(0x7f0c6406ef70 0x7f0c64071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.588+0000 7f0c6affd700 1 -- 172.21.15.94:0/2234237799 --> [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c64071a30 con 0x7f0c6406ef70 2024-04-06T03:37:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.589+0000 7f0c6affd700 1 -- 172.21.15.94:0/2234237799 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c6c08b220 con 0x7f0c7c102e00 2024-04-06T03:37:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.589+0000 7f0c79d9b700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0c640692a0 0x7f0c6406b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c10ecd0 tx=0x7f0c70009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.589+0000 7f0c7ad9d700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] conn(0x7f0c6406ef70 0x7f0c64071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.589+0000 7f0c7ad9d700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] conn(0x7f0c6406ef70 0x7f0c64071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.590+0000 7f0c6affd700 1 -- 172.21.15.94:0/2234237799 <== osd.5 v2:172.21.15.136:6816/3055268575 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0c64071a30 con 0x7f0c6406ef70 2024-04-06T03:37:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.605+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 --> [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c7c05e670 con 0x7f0c6406ef70 2024-04-06T03:37:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.605+0000 7f0c6affd700 1 -- 172.21.15.94:0/2234237799 <== osd.5 v2:172.21.15.136:6816/3055268575 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0c7c05e670 con 0x7f0c6406ef70 2024-04-06T03:37:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] conn(0x7f0c6406ef70 msgr2=0x7f0c64071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] conn(0x7f0c6406ef70 0x7f0c64071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:07.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0c640692a0 msgr2=0x7f0c6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:07.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0c640692a0 0x7f0c6406b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c10ecd0 tx=0x7f0c70009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:07.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c102e00 msgr2=0x7f0c7c113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:07.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c102e00 0x7f0c7c113980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c002a50 tx=0x7f0c6c0046b0 comp rx=0 tx=0).stop 2024-04-06T03:37:07.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 shutdown_connections 2024-04-06T03:37:07.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0c640692a0 0x7f0c6406b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:6816/3055268575,v1:172.21.15.136:6817/3055268575] conn(0x7f0c6406ef70 0x7f0c64071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c7c102e00 0x7f0c7c113980 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 --2- 172.21.15.94:0/2234237799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c7c1037a0 0x7f0c7c113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 >> 172.21.15.94:0/2234237799 conn(0x7f0c7c0f89b0 msgr2=0x7f0c7c1021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 shutdown_connections 2024-04-06T03:37:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:07.606+0000 7f0c80adc700 1 -- 172.21.15.94:0/2234237799 wait complete. 2024-04-06T03:37:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.013+0000 7fa191315700 1 -- 172.21.15.94:0/366490889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c101d00 msgr2=0x7fa18c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.013+0000 7fa191315700 1 --2- 172.21.15.94:0/366490889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c101d00 0x7fa18c1020d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa17c004940 tx=0x7fa17c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.013+0000 7fa191315700 1 -- 172.21.15.94:0/366490889 shutdown_connections 2024-04-06T03:37:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.013+0000 7fa191315700 1 --2- 172.21.15.94:0/366490889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c1026a0 0x7fa18c0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.013+0000 7fa191315700 1 --2- 172.21.15.94:0/366490889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c101d00 0x7fa18c1020d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.013+0000 7fa191315700 1 -- 172.21.15.94:0/366490889 >> 172.21.15.94:0/366490889 conn(0x7fa18c0f78d0 msgr2=0x7fa18c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.013+0000 7fa191315700 1 -- 172.21.15.94:0/366490889 shutdown_connections 2024-04-06T03:37:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.013+0000 7fa191315700 1 -- 172.21.15.94:0/366490889 wait complete. 2024-04-06T03:37:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa191315700 1 Processor -- start 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa191315700 1 -- start start 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa191315700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c101d00 0x7fa18c112ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa191315700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c1026a0 0x7fa18c113000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa191315700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c1136e0 con 0x7fa18c1026a0 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa191315700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c10ca40 con 0x7fa18c101d00 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa18a7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c1026a0 0x7fa18c113000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa18a7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c1026a0 0x7fa18c113000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37216/0 (socket says 172.21.15.94:37216) 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa18a7fc700 1 -- 172.21.15.94:0/1166697538 learned_addr learned my addr 172.21.15.94:0/1166697538 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa18affd700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c101d00 0x7fa18c112ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa18a7fc700 1 -- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c101d00 msgr2=0x7fa18c112ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa18a7fc700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c101d00 0x7fa18c112ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa18a7fc700 1 -- 172.21.15.94:0/1166697538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c005040 con 0x7fa18c1026a0 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa18affd700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c101d00 0x7fa18c112ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:37:08.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa18a7fc700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c1026a0 0x7fa18c113000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa18000b790 tx=0x7fa180009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.014+0000 7fa1737fe700 1 -- 172.21.15.94:0/1166697538 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa18000f6f0 con 0x7fa18c1026a0 2024-04-06T03:37:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.015+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c10cd20 con 0x7fa18c1026a0 2024-04-06T03:37:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.015+0000 7fa1737fe700 1 -- 172.21.15.94:0/1166697538 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa18000fd20 con 0x7fa18c1026a0 2024-04-06T03:37:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.015+0000 7fa1737fe700 1 -- 172.21.15.94:0/1166697538 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1800103a0 con 0x7fa18c1026a0 2024-04-06T03:37:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.015+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c10d2e0 con 0x7fa18c1026a0 2024-04-06T03:37:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.015+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa18c10d860 con 0x7fa18c1026a0 2024-04-06T03:37:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.016+0000 7fa1737fe700 1 -- 172.21.15.94:0/1166697538 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa180011400 con 0x7fa18c1026a0 2024-04-06T03:37:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.016+0000 7fa1737fe700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa1740692a0 0x7fa17406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.016+0000 7fa1737fe700 1 -- 172.21.15.94:0/1166697538 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa180086cd0 con 0x7fa18c1026a0 2024-04-06T03:37:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.016+0000 7fa18affd700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa1740692a0 0x7fa17406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.016+0000 7fa1737fe700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] conn(0x7fa17406ef70 0x7fa174071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.016+0000 7fa1737fe700 1 -- 172.21.15.94:0/1166697538 --> [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa174071a30 con 0x7fa17406ef70 2024-04-06T03:37:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.016+0000 7fa1737fe700 1 -- 172.21.15.94:0/1166697538 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa18008b030 con 0x7fa18c1026a0 2024-04-06T03:37:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.017+0000 7fa18affd700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa1740692a0 0x7fa17406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa17c004560 tx=0x7fa17c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.017+0000 7fa18b7fe700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] conn(0x7fa17406ef70 0x7fa174071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.017+0000 7fa18b7fe700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] conn(0x7fa17406ef70 0x7fa174071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.018+0000 7fa1737fe700 1 -- 172.21.15.94:0/1166697538 <== osd.3 v2:172.21.15.136:6800/1271372986 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa174071a30 con 0x7fa17406ef70 2024-04-06T03:37:08.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.033+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 --> [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa18c0f91b0 con 0x7fa17406ef70 2024-04-06T03:37:08.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa1737fe700 1 -- 172.21.15.94:0/1166697538 <== osd.3 v2:172.21.15.136:6800/1271372986 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa18c0f91b0 con 0x7fa17406ef70 2024-04-06T03:37:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] conn(0x7fa17406ef70 msgr2=0x7fa174071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] conn(0x7fa17406ef70 0x7fa174071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa1740692a0 msgr2=0x7fa17406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa1740692a0 0x7fa17406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa17c004560 tx=0x7fa17c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c1026a0 msgr2=0x7fa18c113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c1026a0 0x7fa18c113000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa18000b790 tx=0x7fa180009220 comp rx=0 tx=0).stop 2024-04-06T03:37:08.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 shutdown_connections 2024-04-06T03:37:08.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa1740692a0 0x7fa17406b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:08.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:6800/1271372986,v1:172.21.15.136:6801/1271372986] conn(0x7fa17406ef70 0x7fa174071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:08.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c101d00 0x7fa18c112ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:08.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 --2- 172.21.15.94:0/1166697538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa18c1026a0 0x7fa18c113000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:08.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.034+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 >> 172.21.15.94:0/1166697538 conn(0x7fa18c0f78d0 msgr2=0x7fa18c0f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:08.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.035+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 shutdown_connections 2024-04-06T03:37:08.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:08.035+0000 7fa191315700 1 -- 172.21.15.94:0/1166697538 wait complete. 2024-04-06T03:37:08.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:08 smithi136 ceph-mon[36436]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:08.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:08 smithi094 ceph-mon[29713]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:08.814 INFO:teuthology.orchestra.run.smithi094.stdout:167503724554 2024-04-06T03:37:08.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-04-06T03:37:09.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.007+0000 7fe990f51700 1 -- 172.21.15.94:0/3052291190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c101d70 msgr2=0x7fe98c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:09.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.007+0000 7fe990f51700 1 --2- 172.21.15.94:0/3052291190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c101d70 0x7fe98c102140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe97c004940 tx=0x7fe97c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:09.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 -- 172.21.15.94:0/3052291190 shutdown_connections 2024-04-06T03:37:09.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 --2- 172.21.15.94:0/3052291190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe98c102710 0x7fe98c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 --2- 172.21.15.94:0/3052291190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c101d70 0x7fe98c102140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 -- 172.21.15.94:0/3052291190 >> 172.21.15.94:0/3052291190 conn(0x7fe98c072bc0 msgr2=0x7fe98c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 -- 172.21.15.94:0/3052291190 shutdown_connections 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 -- 172.21.15.94:0/3052291190 wait complete. 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 Processor -- start 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 -- start start 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe98c101d70 0x7fe98c113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c102710 0x7fe98c114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c114690 con 0x7fe98c102710 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.008+0000 7fe990f51700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c10dac0 con 0x7fe98c101d70 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe989d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c102710 0x7fe98c114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe989d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c102710 0x7fe98c114040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37240/0 (socket says 172.21.15.94:37240) 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe989d9b700 1 -- 172.21.15.94:0/3787676100 learned_addr learned my addr 172.21.15.94:0/3787676100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe98a59c700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe98c101d70 0x7fe98c113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe989d9b700 1 -- 172.21.15.94:0/3787676100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe98c101d70 msgr2=0x7fe98c113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe989d9b700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe98c101d70 0x7fe98c113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe989d9b700 1 -- 172.21.15.94:0/3787676100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c005040 con 0x7fe98c102710 2024-04-06T03:37:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe989d9b700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c102710 0x7fe98c114040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c890 tx=0x7fe9800044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe97affd700 1 -- 172.21.15.94:0/3787676100 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe98000a700 con 0x7fe98c102710 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe97affd700 1 -- 172.21.15.94:0/3787676100 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe98000ad30 con 0x7fe98c102710 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c10dda0 con 0x7fe98c102710 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe97affd700 1 -- 172.21.15.94:0/3787676100 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9800113a0 con 0x7fe98c102710 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.009+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c10e360 con 0x7fe98c102710 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.010+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe98c10e8a0 con 0x7fe98c102710 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.011+0000 7fe97affd700 1 -- 172.21.15.94:0/3787676100 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe98000a860 con 0x7fe98c102710 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.011+0000 7fe97affd700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe9740692a0 0x7fe97406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.011+0000 7fe97affd700 1 -- 172.21.15.94:0/3787676100 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe980086f40 con 0x7fe98c102710 2024-04-06T03:37:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.011+0000 7fe98a59c700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe9740692a0 0x7fe97406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.011+0000 7fe97affd700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] conn(0x7fe97406ef70 0x7fe974071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.011+0000 7fe97affd700 1 -- 172.21.15.94:0/3787676100 --> [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe974071a30 con 0x7fe97406ef70 2024-04-06T03:37:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.011+0000 7fe97affd700 1 -- 172.21.15.94:0/3787676100 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe980089530 con 0x7fe98c102710 2024-04-06T03:37:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.012+0000 7fe98ad9d700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] conn(0x7fe97406ef70 0x7fe974071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.012+0000 7fe98a59c700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe9740692a0 0x7fe97406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe97c004f90 tx=0x7fe97c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.012+0000 7fe98ad9d700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] conn(0x7fe97406ef70 0x7fe974071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.013+0000 7fe97affd700 1 -- 172.21.15.94:0/3787676100 <== osd.1 v2:172.21.15.94:6810/1337787373 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fe974071a30 con 0x7fe97406ef70 2024-04-06T03:37:09.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.032+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 --> [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe98c106020 con 0x7fe97406ef70 2024-04-06T03:37:09.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe97affd700 1 -- 172.21.15.94:0/3787676100 <== osd.1 v2:172.21.15.94:6810/1337787373 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe98c106020 con 0x7fe97406ef70 2024-04-06T03:37:09.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] conn(0x7fe97406ef70 msgr2=0x7fe974071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:09.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] conn(0x7fe97406ef70 0x7fe974071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe9740692a0 msgr2=0x7fe97406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:09.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe9740692a0 0x7fe97406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe97c004f90 tx=0x7fe97c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:09.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c102710 msgr2=0x7fe98c114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:09.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c102710 0x7fe98c114040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c890 tx=0x7fe9800044c0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 shutdown_connections 2024-04-06T03:37:09.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6810/1337787373,v1:172.21.15.94:6811/1337787373] conn(0x7fe97406ef70 0x7fe974071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe9740692a0 0x7fe97406b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe98c101d70 0x7fe98c113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 --2- 172.21.15.94:0/3787676100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe98c102710 0x7fe98c114040 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 >> 172.21.15.94:0/3787676100 conn(0x7fe98c072bc0 msgr2=0x7fe98c065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:09.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 shutdown_connections 2024-04-06T03:37:09.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.033+0000 7fe990f51700 1 -- 172.21.15.94:0/3787676100 wait complete. 2024-04-06T03:37:09.240 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:09.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.258+0000 7fe48d4d2700 1 -- 172.21.15.94:0/3275791832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4881026c0 msgr2=0x7fe4880fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:09.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.258+0000 7fe48d4d2700 1 --2- 172.21.15.94:0/3275791832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4881026c0 0x7fe4880fbd30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe47c004940 tx=0x7fe47c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:09.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.258+0000 7fe48d4d2700 1 -- 172.21.15.94:0/3275791832 shutdown_connections 2024-04-06T03:37:09.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.258+0000 7fe48d4d2700 1 --2- 172.21.15.94:0/3275791832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4881026c0 0x7fe4880fbd30 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe47c004940 tx=0x7fe47c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:09.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.258+0000 7fe48d4d2700 1 --2- 172.21.15.94:0/3275791832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe488101d20 0x7fe4881020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.258+0000 7fe48d4d2700 1 -- 172.21.15.94:0/3275791832 >> 172.21.15.94:0/3275791832 conn(0x7fe4880f78d0 msgr2=0x7fe4880f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:09.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.258+0000 7fe48d4d2700 1 -- 172.21.15.94:0/3275791832 shutdown_connections 2024-04-06T03:37:09.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.258+0000 7fe48d4d2700 1 -- 172.21.15.94:0/3275791832 wait complete. 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe48d4d2700 1 Processor -- start 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe48d4d2700 1 -- start start 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe48d4d2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe488101d20 0x7fe488115f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe48d4d2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881164c0 0x7fe488110560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe48d4d2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488116a40 con 0x7fe488101d20 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe48d4d2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488116b80 con 0x7fe4881164c0 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe486ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe488101d20 0x7fe488115f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe486ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe488101d20 0x7fe488115f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37246/0 (socket says 172.21.15.94:37246) 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe486ffd700 1 -- 172.21.15.94:0/800970392 learned_addr learned my addr 172.21.15.94:0/800970392 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.259+0000 7fe4867fc700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881164c0 0x7fe488110560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:09.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.260+0000 7fe486ffd700 1 -- 172.21.15.94:0/800970392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881164c0 msgr2=0x7fe488110560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.260+0000 7fe486ffd700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881164c0 0x7fe488110560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.260+0000 7fe486ffd700 1 -- 172.21.15.94:0/800970392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c005040 con 0x7fe488101d20 2024-04-06T03:37:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.260+0000 7fe486ffd700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe488101d20 0x7fe488115f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe47800c810 tx=0x7fe478004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.260+0000 7fe46f7fe700 1 -- 172.21.15.94:0/800970392 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe478012810 con 0x7fe488101d20 2024-04-06T03:37:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.260+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe488110b00 con 0x7fe488101d20 2024-04-06T03:37:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.260+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe488111090 con 0x7fe488101d20 2024-04-06T03:37:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.261+0000 7fe46f7fe700 1 -- 172.21.15.94:0/800970392 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe478012e40 con 0x7fe488101d20 2024-04-06T03:37:09.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.261+0000 7fe46f7fe700 1 -- 172.21.15.94:0/800970392 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe47801ae50 con 0x7fe488101d20 2024-04-06T03:37:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.262+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4740052b0 con 0x7fe488101d20 2024-04-06T03:37:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.262+0000 7fe46f7fe700 1 -- 172.21.15.94:0/800970392 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe478011380 con 0x7fe488101d20 2024-04-06T03:37:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.262+0000 7fe46f7fe700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe4700692a0 0x7fe47006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.263+0000 7fe46f7fe700 1 -- 172.21.15.94:0/800970392 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe478086f90 con 0x7fe488101d20 2024-04-06T03:37:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.263+0000 7fe4867fc700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe4700692a0 0x7fe47006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:09.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.263+0000 7fe4867fc700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe4700692a0 0x7fe47006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009040 tx=0x7fe47c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:09.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.268+0000 7fe46f7fe700 1 -- 172.21.15.94:0/800970392 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe478054460 con 0x7fe488101d20 2024-04-06T03:37:09.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.460+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe474005560 con 0x7fe488101d20 2024-04-06T03:37:09.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.461+0000 7fe46f7fe700 1 -- 172.21.15.94:0/800970392 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe478016020 con 0x7fe488101d20 2024-04-06T03:37:09.463 INFO:teuthology.orchestra.run.smithi094.stdout:38654705700 2024-04-06T03:37:09.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.462+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe4700692a0 msgr2=0x7fe47006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:09.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.462+0000 7fe48d4d2700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe4700692a0 0x7fe47006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009040 tx=0x7fe47c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:09.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.462+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe488101d20 msgr2=0x7fe488115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:09.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.462+0000 7fe48d4d2700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe488101d20 0x7fe488115f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe47800c810 tx=0x7fe478004470 comp rx=0 tx=0).stop 2024-04-06T03:37:09.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.462+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 shutdown_connections 2024-04-06T03:37:09.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.462+0000 7fe48d4d2700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe4700692a0 0x7fe47006b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.462+0000 7fe48d4d2700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe488101d20 0x7fe488115f80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.463+0000 7fe48d4d2700 1 --2- 172.21.15.94:0/800970392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4881164c0 0x7fe488110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.463+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 >> 172.21.15.94:0/800970392 conn(0x7fe4880f78d0 msgr2=0x7fe488109f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.463+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 shutdown_connections 2024-04-06T03:37:09.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:09.463+0000 7fe48d4d2700 1 -- 172.21.15.94:0/800970392 wait complete. 2024-04-06T03:37:09.790 INFO:teuthology.orchestra.run.smithi094.stdout:111669149715 2024-04-06T03:37:09.790 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-04-06T03:37:10.310 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:10 smithi094 ceph-mon[29713]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:10.310 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:10 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/800970392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-06T03:37:10.340 INFO:teuthology.orchestra.run.smithi094.stdout:60129542174 2024-04-06T03:37:10.340 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-04-06T03:37:10.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:10 smithi136 ceph-mon[36436]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:10.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:10 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/800970392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-06T03:37:11.396 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:11.914 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705698 got 38654705700 for osd.0 2024-04-06T03:37:11.914 DEBUG:teuthology.parallel:result is None 2024-04-06T03:37:11.976 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:11.977 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:11.977 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:12.530 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:12 smithi094 ceph-mon[29713]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:12.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:12 smithi136 ceph-mon[36436]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:13.354 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:37:13.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:37:14.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:14 smithi136 ceph-mon[36436]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:14.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:14 smithi094 ceph-mon[29713]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.310+0000 7f334c04c700 1 -- 172.21.15.94:0/4163953519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 msgr2=0x7f33440fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.310+0000 7f334c04c700 1 --2- 172.21.15.94:0/4163953519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 0x7f33440fbd30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3334004940 tx=0x7f3334009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.310+0000 7f334c04c700 1 -- 172.21.15.94:0/4163953519 shutdown_connections 2024-04-06T03:37:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.310+0000 7f334c04c700 1 --2- 172.21.15.94:0/4163953519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 0x7f33440fbd30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.310+0000 7f334c04c700 1 --2- 172.21.15.94:0/4163953519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3344101d20 0x7f33441020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:15.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.310+0000 7f334c04c700 1 -- 172.21.15.94:0/4163953519 >> 172.21.15.94:0/4163953519 conn(0x7f33440f7870 msgr2=0x7f33440f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.310+0000 7f334c04c700 1 -- 172.21.15.94:0/4163953519 shutdown_connections 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.310+0000 7f334c04c700 1 -- 172.21.15.94:0/4163953519 wait complete. 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f334c04c700 1 Processor -- start 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f334c04c700 1 -- start start 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f334c04c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3344101d20 0x7f334410eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f334c04c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 0x7f334410f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f334c04c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334410f5e0 con 0x7f33441026c0 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f334c04c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334410f720 con 0x7f3344101d20 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f33495e7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 0x7f334410f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f33495e7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 0x7f334410f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37262/0 (socket says 172.21.15.94:37262) 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f33495e7700 1 -- 172.21.15.94:0/2281317935 learned_addr learned my addr 172.21.15.94:0/2281317935 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.311+0000 7f3349de8700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3344101d20 0x7f334410eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.312+0000 7f33495e7700 1 -- 172.21.15.94:0/2281317935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3344101d20 msgr2=0x7f334410eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.312+0000 7f33495e7700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3344101d20 0x7f334410eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:15.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.312+0000 7f33495e7700 1 -- 172.21.15.94:0/2281317935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3334005040 con 0x7f33441026c0 2024-04-06T03:37:15.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.312+0000 7f33495e7700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 0x7f334410f0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3334009100 tx=0x7f3334009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:15.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.312+0000 7f333a7fc700 1 -- 172.21.15.94:0/2281317935 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333400e040 con 0x7f33441026c0 2024-04-06T03:37:15.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.312+0000 7f333a7fc700 1 -- 172.21.15.94:0/2281317935 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3334013420 con 0x7f33441026c0 2024-04-06T03:37:15.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.312+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334410f9a0 con 0x7f33441026c0 2024-04-06T03:37:15.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.312+0000 7f333a7fc700 1 -- 172.21.15.94:0/2281317935 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f333401c610 con 0x7f33441026c0 2024-04-06T03:37:15.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.312+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3344117ed0 con 0x7f33441026c0 2024-04-06T03:37:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.313+0000 7f333a7fc700 1 -- 172.21.15.94:0/2281317935 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3334013590 con 0x7f33441026c0 2024-04-06T03:37:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.313+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33440fd470 con 0x7f33441026c0 2024-04-06T03:37:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.314+0000 7f333a7fc700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f33300692a0 0x7f333006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.314+0000 7f333a7fc700 1 -- 172.21.15.94:0/2281317935 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3334088070 con 0x7f33441026c0 2024-04-06T03:37:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.314+0000 7f3349de8700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f33300692a0 0x7f333006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.314+0000 7f3349de8700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f33300692a0 0x7f333006b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f33400041d0 tx=0x7f3340009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:15.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.318+0000 7f333a7fc700 1 -- 172.21.15.94:0/2281317935 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f33340546c0 con 0x7f33441026c0 2024-04-06T03:37:15.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.522+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3344108e00 con 0x7f33441026c0 2024-04-06T03:37:15.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.522+0000 7f333a7fc700 1 -- 172.21.15.94:0/2281317935 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3334057ce0 con 0x7f33441026c0 2024-04-06T03:37:15.523 INFO:teuthology.orchestra.run.smithi094.stdout:60129542175 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f33300692a0 msgr2=0x7f333006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f33300692a0 0x7f333006b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f33400041d0 tx=0x7f3340009040 comp rx=0 tx=0).stop 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 msgr2=0x7f334410f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 0x7f334410f0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3334009100 tx=0x7f3334009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 shutdown_connections 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f33300692a0 0x7f333006b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3344101d20 0x7f334410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 --2- 172.21.15.94:0/2281317935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33441026c0 0x7f334410f0a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 >> 172.21.15.94:0/2281317935 conn(0x7f33440f7870 msgr2=0x7f33440f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 shutdown_connections 2024-04-06T03:37:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:15.524+0000 7f334c04c700 1 -- 172.21.15.94:0/2281317935 wait complete. 2024-04-06T03:37:16.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.184+0000 7f8596ef9700 1 -- 172.21.15.94:0/1973008775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8590101d20 msgr2=0x7f85901020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.184+0000 7f8596ef9700 1 --2- 172.21.15.94:0/1973008775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8590101d20 0x7f85901020f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8584004940 tx=0x7f8584009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.184+0000 7f8596ef9700 1 -- 172.21.15.94:0/1973008775 shutdown_connections 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.184+0000 7f8596ef9700 1 --2- 172.21.15.94:0/1973008775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85901026c0 0x7f85900fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.184+0000 7f8596ef9700 1 --2- 172.21.15.94:0/1973008775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8590101d20 0x7f85901020f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.184+0000 7f8596ef9700 1 -- 172.21.15.94:0/1973008775 >> 172.21.15.94:0/1973008775 conn(0x7f85900f78b0 msgr2=0x7f85900f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.185+0000 7f8596ef9700 1 -- 172.21.15.94:0/1973008775 shutdown_connections 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.185+0000 7f8596ef9700 1 -- 172.21.15.94:0/1973008775 wait complete. 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.185+0000 7f8596ef9700 1 Processor -- start 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.185+0000 7f8596ef9700 1 -- start start 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.185+0000 7f8596ef9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8590101d20 0x7f8590112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.185+0000 7f8596ef9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85901026c0 0x7f8590112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:16.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.185+0000 7f8596ef9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8590113420 con 0x7f85901026c0 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.185+0000 7f8596ef9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859010ca20 con 0x7f8590101d20 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85901026c0 0x7f8590112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85901026c0 0x7f8590112dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37286/0 (socket says 172.21.15.94:37286) 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858ffff700 1 -- 172.21.15.94:0/1227178670 learned_addr learned my addr 172.21.15.94:0/1227178670 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f8594c95700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8590101d20 0x7f8590112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858ffff700 1 -- 172.21.15.94:0/1227178670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8590101d20 msgr2=0x7f8590112890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858ffff700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8590101d20 0x7f8590112890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858ffff700 1 -- 172.21.15.94:0/1227178670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8584005040 con 0x7f85901026c0 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858ffff700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85901026c0 0x7f8590112dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c800 tx=0x7f857c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858d7fa700 1 -- 172.21.15.94:0/1227178670 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f857c0106f0 con 0x7f85901026c0 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859010cd00 con 0x7f85901026c0 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858d7fa700 1 -- 172.21.15.94:0/1227178670 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f857c010d20 con 0x7f85901026c0 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f858d7fa700 1 -- 172.21.15.94:0/1227178670 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f857c0113a0 con 0x7f85901026c0 2024-04-06T03:37:16.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.186+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859010d2c0 con 0x7f85901026c0 2024-04-06T03:37:16.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.187+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85900fd470 con 0x7f85901026c0 2024-04-06T03:37:16.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.188+0000 7f858d7fa700 1 -- 172.21.15.94:0/1227178670 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f857c010850 con 0x7f85901026c0 2024-04-06T03:37:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.188+0000 7f858d7fa700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f85800692a0 0x7f858006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.188+0000 7f858d7fa700 1 -- 172.21.15.94:0/1227178670 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f857c086e10 con 0x7f85901026c0 2024-04-06T03:37:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.188+0000 7f8594c95700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f85800692a0 0x7f858006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.188+0000 7f8594c95700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f85800692a0 0x7f858006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8584004f90 tx=0x7f8584009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:16.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.192+0000 7f858d7fa700 1 -- 172.21.15.94:0/1227178670 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f857c056f10 con 0x7f85901026c0 2024-04-06T03:37:16.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.383+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8590108e20 con 0x7f85901026c0 2024-04-06T03:37:16.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.383+0000 7f858d7fa700 1 -- 172.21.15.94:0/1227178670 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f857c056aa0 con 0x7f85901026c0 2024-04-06T03:37:16.386 INFO:teuthology.orchestra.run.smithi094.stdout:146028888078 2024-04-06T03:37:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.385+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f85800692a0 msgr2=0x7f858006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.385+0000 7f8596ef9700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f85800692a0 0x7f858006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8584004f90 tx=0x7f8584009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.385+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85901026c0 msgr2=0x7f8590112dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.385+0000 7f8596ef9700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85901026c0 0x7f8590112dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f857c00c800 tx=0x7f857c004470 comp rx=0 tx=0).stop 2024-04-06T03:37:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.386+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 shutdown_connections 2024-04-06T03:37:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.386+0000 7f8596ef9700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f85800692a0 0x7f858006b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.386+0000 7f8596ef9700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8590101d20 0x7f8590112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.386+0000 7f8596ef9700 1 --2- 172.21.15.94:0/1227178670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85901026c0 0x7f8590112dd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:16.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.386+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 >> 172.21.15.94:0/1227178670 conn(0x7f85900f78b0 msgr2=0x7f8590104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:16.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.386+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 shutdown_connections 2024-04-06T03:37:16.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:16.386+0000 7f8596ef9700 1 -- 172.21.15.94:0/1227178670 wait complete. 2024-04-06T03:37:16.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:16 smithi136 ceph-mon[36436]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:16.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:16 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2281317935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-06T03:37:16.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:16 smithi094 ceph-mon[29713]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:16.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:16 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2281317935' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-06T03:37:16.847 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542175 for osd.1 2024-04-06T03:37:16.847 DEBUG:teuthology.parallel:result is None 2024-04-06T03:37:17.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:17 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1227178670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-06T03:37:17.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:17 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1227178670' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-06T03:37:18.398 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:18 smithi094 ceph-mon[29713]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:18.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:18 smithi136 ceph-mon[36436]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:18.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.613+0000 7faac56e5700 1 -- 172.21.15.94:0/4256485090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fded0 msgr2=0x7faac00fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:18.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.613+0000 7faac56e5700 1 --2- 172.21.15.94:0/4256485090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fded0 0x7faac00fe2a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faab4004940 tx=0x7faab4009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:18.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.613+0000 7faac56e5700 1 -- 172.21.15.94:0/4256485090 shutdown_connections 2024-04-06T03:37:18.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.613+0000 7faac56e5700 1 --2- 172.21.15.94:0/4256485090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faac00fe870 0x7faac0106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:18.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.613+0000 7faac56e5700 1 --2- 172.21.15.94:0/4256485090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fded0 0x7faac00fe2a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:18.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.613+0000 7faac56e5700 1 -- 172.21.15.94:0/4256485090 >> 172.21.15.94:0/4256485090 conn(0x7faac0072bc0 msgr2=0x7faac0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:18.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.613+0000 7faac56e5700 1 -- 172.21.15.94:0/4256485090 shutdown_connections 2024-04-06T03:37:18.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.613+0000 7faac56e5700 1 -- 172.21.15.94:0/4256485090 wait complete. 2024-04-06T03:37:18.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faac56e5700 1 Processor -- start 2024-04-06T03:37:18.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faac56e5700 1 -- start start 2024-04-06T03:37:18.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faac56e5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faac00fded0 0x7faac0113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:18.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faac56e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fe870 0x7faac0114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:18.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faac56e5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01146b0 con 0x7faac00fe870 2024-04-06T03:37:18.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faac56e5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac010daa0 con 0x7faac00fded0 2024-04-06T03:37:18.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faabe7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fe870 0x7faac0114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:18.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faabe7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fe870 0x7faac0114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43578/0 (socket says 172.21.15.94:43578) 2024-04-06T03:37:18.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faabe7fc700 1 -- 172.21.15.94:0/3919981693 learned_addr learned my addr 172.21.15.94:0/3919981693 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.614+0000 7faabeffd700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faac00fded0 0x7faac0113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.615+0000 7faabe7fc700 1 -- 172.21.15.94:0/3919981693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faac00fded0 msgr2=0x7faac0113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.615+0000 7faabe7fc700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faac00fded0 0x7faac0113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.615+0000 7faabe7fc700 1 -- 172.21.15.94:0/3919981693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab4005040 con 0x7faac00fe870 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.615+0000 7faabe7fc700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fe870 0x7faac0114060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faab0002a50 tx=0x7faab00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.615+0000 7faaa77fe700 1 -- 172.21.15.94:0/3919981693 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab0007cc0 con 0x7faac00fe870 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.615+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac010dd80 con 0x7faac00fe870 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.615+0000 7faaa77fe700 1 -- 172.21.15.94:0/3919981693 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faab000f040 con 0x7faac00fe870 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.615+0000 7faaa77fe700 1 -- 172.21.15.94:0/3919981693 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab000da30 con 0x7faac00fe870 2024-04-06T03:37:18.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.615+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac010e2c0 con 0x7faac00fe870 2024-04-06T03:37:18.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.616+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac01019a0 con 0x7faac00fe870 2024-04-06T03:37:18.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.616+0000 7faaa77fe700 1 -- 172.21.15.94:0/3919981693 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7faab000db90 con 0x7faac00fe870 2024-04-06T03:37:18.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.617+0000 7faaa77fe700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faaa80692b0 0x7faaa806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:18.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.617+0000 7faaa77fe700 1 -- 172.21.15.94:0/3919981693 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7faab0086be0 con 0x7faac00fe870 2024-04-06T03:37:18.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.617+0000 7faabeffd700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faaa80692b0 0x7faaa806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:18.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.617+0000 7faabeffd700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faaa80692b0 0x7faaa806b760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faab4009fd0 tx=0x7faab4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:18.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.621+0000 7faaa77fe700 1 -- 172.21.15.94:0/3919981693 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7faab0056c60 con 0x7faac00fe870 2024-04-06T03:37:18.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.814+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7faac0065df0 con 0x7faac00fe870 2024-04-06T03:37:18.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.814+0000 7faaa77fe700 1 -- 172.21.15.94:0/3919981693 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faab00567f0 con 0x7faac00fe870 2024-04-06T03:37:18.817 INFO:teuthology.orchestra.run.smithi094.stdout:167503724555 2024-04-06T03:37:18.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faaa80692b0 msgr2=0x7faaa806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faaa80692b0 0x7faaa806b760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faab4009fd0 tx=0x7faab4009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fe870 msgr2=0x7faac0114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fe870 0x7faac0114060 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faab0002a50 tx=0x7faab00046b0 comp rx=0 tx=0).stop 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 shutdown_connections 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faaa80692b0 0x7faaa806b760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faac00fded0 0x7faac0113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 --2- 172.21.15.94:0/3919981693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faac00fe870 0x7faac0114060 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 >> 172.21.15.94:0/3919981693 conn(0x7faac0072bc0 msgr2=0x7faac00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 shutdown_connections 2024-04-06T03:37:18.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:18.817+0000 7faac56e5700 1 -- 172.21.15.94:0/3919981693 wait complete. 2024-04-06T03:37:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.091+0000 7f79d5e11700 1 -- 172.21.15.94:0/1194504751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0102de0 msgr2=0x7f79d01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.091+0000 7f79d5e11700 1 --2- 172.21.15.94:0/1194504751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0102de0 0x7f79d01031b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79c0004940 tx=0x7f79c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.091+0000 7f79d5e11700 1 -- 172.21.15.94:0/1194504751 shutdown_connections 2024-04-06T03:37:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.091+0000 7f79d5e11700 1 --2- 172.21.15.94:0/1194504751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79d0103780 0x7f79d00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.091+0000 7f79d5e11700 1 --2- 172.21.15.94:0/1194504751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0102de0 0x7f79d01031b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.092+0000 7f79d5e11700 1 -- 172.21.15.94:0/1194504751 >> 172.21.15.94:0/1194504751 conn(0x7f79d00f8950 msgr2=0x7f79d00fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.092+0000 7f79d5e11700 1 -- 172.21.15.94:0/1194504751 shutdown_connections 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.092+0000 7f79d5e11700 1 -- 172.21.15.94:0/1194504751 wait complete. 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.092+0000 7f79d5e11700 1 Processor -- start 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.092+0000 7f79d5e11700 1 -- start start 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.092+0000 7f79d5e11700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79d0102de0 0x7f79d0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.092+0000 7f79d5e11700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0103780 0x7f79d0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.092+0000 7f79d5e11700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d0114590 con 0x7f79d0103780 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.092+0000 7f79d5e11700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d010db00 con 0x7f79d0102de0 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79ceffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0103780 0x7f79d0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79ceffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0103780 0x7f79d0113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43596/0 (socket says 172.21.15.94:43596) 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79ceffd700 1 -- 172.21.15.94:0/3193839942 learned_addr learned my addr 172.21.15.94:0/3193839942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:19.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79cf7fe700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79d0102de0 0x7f79d0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79ceffd700 1 -- 172.21.15.94:0/3193839942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79d0102de0 msgr2=0x7f79d0113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79ceffd700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79d0102de0 0x7f79d0113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79ceffd700 1 -- 172.21.15.94:0/3193839942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c0005040 con 0x7f79d0103780 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79ceffd700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0103780 0x7f79d0113eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79c400b790 tx=0x7f79c4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79b7fff700 1 -- 172.21.15.94:0/3193839942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79c400f6f0 con 0x7f79d0103780 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79b7fff700 1 -- 172.21.15.94:0/3193839942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79c400fd20 con 0x7f79d0103780 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d010dde0 con 0x7f79d0103780 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79b7fff700 1 -- 172.21.15.94:0/3193839942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79c40103a0 con 0x7f79d0103780 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.093+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d010e3a0 con 0x7f79d0103780 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.094+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d01000f0 con 0x7f79d0103780 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.095+0000 7f79b7fff700 1 -- 172.21.15.94:0/3193839942 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f79c4011400 con 0x7f79d0103780 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.095+0000 7f79b7fff700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f79b80692a0 0x7f79b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.095+0000 7f79b7fff700 1 -- 172.21.15.94:0/3193839942 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f79c4086c40 con 0x7f79d0103780 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.095+0000 7f79cf7fe700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f79b80692a0 0x7f79b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:19.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.095+0000 7f79cf7fe700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f79b80692a0 0x7f79b806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b3b0 tx=0x7f79c0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:19.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.099+0000 7f79b7fff700 1 -- 172.21.15.94:0/3193839942 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f79c4056d40 con 0x7f79d0103780 2024-04-06T03:37:19.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.294+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f79d00fe530 con 0x7f79d0103780 2024-04-06T03:37:19.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.294+0000 7f79b7fff700 1 -- 172.21.15.94:0/3193839942 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f79c40568d0 con 0x7f79d0103780 2024-04-06T03:37:19.297 INFO:teuthology.orchestra.run.smithi094.stdout:81604378649 2024-04-06T03:37:19.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f79b80692a0 msgr2=0x7f79b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:19.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f79b80692a0 0x7f79b806b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b3b0 tx=0x7f79c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:19.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0103780 msgr2=0x7f79d0113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:19.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0103780 0x7f79d0113eb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79c400b790 tx=0x7f79c4009220 comp rx=0 tx=0).stop 2024-04-06T03:37:19.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 shutdown_connections 2024-04-06T03:37:19.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f79b80692a0 0x7f79b806b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:19.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79d0102de0 0x7f79d0113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:19.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 --2- 172.21.15.94:0/3193839942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79d0103780 0x7f79d0113eb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:19.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 >> 172.21.15.94:0/3193839942 conn(0x7f79d00f8950 msgr2=0x7f79d0107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:19.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 shutdown_connections 2024-04-06T03:37:19.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.297+0000 7f79d5e11700 1 -- 172.21.15.94:0/3193839942 wait complete. 2024-04-06T03:37:19.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:19 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3919981693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-06T03:37:19.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:19 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3919981693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-06T03:37:20.093 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724555 for osd.5 2024-04-06T03:37:20.093 DEBUG:teuthology.parallel:result is None 2024-04-06T03:37:20.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.468+0000 7fa7a7044700 1 -- 172.21.15.94:0/1925685709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a01037a0 msgr2=0x7fa7a00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.468+0000 7fa7a7044700 1 --2- 172.21.15.94:0/1925685709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a01037a0 0x7fa7a00fce10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa78c004940 tx=0x7fa78c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.468+0000 7fa7a7044700 1 -- 172.21.15.94:0/1925685709 shutdown_connections 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.468+0000 7fa7a7044700 1 --2- 172.21.15.94:0/1925685709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a01037a0 0x7fa7a00fce10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.468+0000 7fa7a7044700 1 --2- 172.21.15.94:0/1925685709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7a0102e00 0x7fa7a01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.468+0000 7fa7a7044700 1 -- 172.21.15.94:0/1925685709 >> 172.21.15.94:0/1925685709 conn(0x7fa7a00f89b0 msgr2=0x7fa7a00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.468+0000 7fa7a7044700 1 -- 172.21.15.94:0/1925685709 shutdown_connections 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.468+0000 7fa7a7044700 1 -- 172.21.15.94:0/1925685709 wait complete. 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.469+0000 7fa7a7044700 1 Processor -- start 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.469+0000 7fa7a7044700 1 -- start start 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.469+0000 7fa7a7044700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0102e00 0x7fa7a0115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:20.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.469+0000 7fa7a7044700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7a01037a0 0x7fa7a01160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.469+0000 7fa7a7044700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a01167d0 con 0x7fa7a0102e00 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.469+0000 7fa7a7044700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a010fd20 con 0x7fa7a01037a0 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.469+0000 7fa7a4de0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0102e00 0x7fa7a0115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.469+0000 7fa7a4de0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0102e00 0x7fa7a0115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43610/0 (socket says 172.21.15.94:43610) 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.469+0000 7fa7a4de0700 1 -- 172.21.15.94:0/657299097 learned_addr learned my addr 172.21.15.94:0/657299097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa79ffff700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7a01037a0 0x7fa7a01160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa7a4de0700 1 -- 172.21.15.94:0/657299097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7a01037a0 msgr2=0x7fa7a01160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa7a4de0700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7a01037a0 0x7fa7a01160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa7a4de0700 1 -- 172.21.15.94:0/657299097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c005040 con 0x7fa7a0102e00 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa79ffff700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7a01037a0 0x7fa7a01160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa7a4de0700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0102e00 0x7fa7a0115bb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa79400b790 tx=0x7fa794009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa79d7fa700 1 -- 172.21.15.94:0/657299097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79400f770 con 0x7fa7a0102e00 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa79d7fa700 1 -- 172.21.15.94:0/657299097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa79400fda0 con 0x7fa7a0102e00 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a010ffa0 con 0x7fa7a0102e00 2024-04-06T03:37:20.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa79d7fa700 1 -- 172.21.15.94:0/657299097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79400d310 con 0x7fa7a0102e00 2024-04-06T03:37:20.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.470+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a0110560 con 0x7fa7a0102e00 2024-04-06T03:37:20.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.471+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a00fe550 con 0x7fa7a0102e00 2024-04-06T03:37:20.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.472+0000 7fa79d7fa700 1 -- 172.21.15.94:0/657299097 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa794010420 con 0x7fa7a0102e00 2024-04-06T03:37:20.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.472+0000 7fa79d7fa700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7900692a0 0x7fa79006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:20.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.472+0000 7fa79d7fa700 1 -- 172.21.15.94:0/657299097 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa794086e70 con 0x7fa7a0102e00 2024-04-06T03:37:20.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.472+0000 7fa79ffff700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7900692a0 0x7fa79006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:20.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.472+0000 7fa79ffff700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7900692a0 0x7fa79006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00b3b0 tx=0x7fa78c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:20.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.476+0000 7fa79d7fa700 1 -- 172.21.15.94:0/657299097 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa794056f70 con 0x7fa7a0102e00 2024-04-06T03:37:20.197 INFO:teuthology.orchestra.run.smithi094.stdout:111669149717 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.667+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa7a0111230 con 0x7fa7a0102e00 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.668+0000 7fa79d7fa700 1 -- 172.21.15.94:0/657299097 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa794056b00 con 0x7fa7a0102e00 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.669+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7900692a0 msgr2=0x7fa79006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.669+0000 7fa7a7044700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7900692a0 0x7fa79006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00b3b0 tx=0x7fa78c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.669+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0102e00 msgr2=0x7fa7a0115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.669+0000 7fa7a7044700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0102e00 0x7fa7a0115bb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa79400b790 tx=0x7fa794009250 comp rx=0 tx=0).stop 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.670+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 shutdown_connections 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.670+0000 7fa7a7044700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7900692a0 0x7fa79006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.670+0000 7fa7a7044700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0102e00 0x7fa7a0115bb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.670+0000 7fa7a7044700 1 --2- 172.21.15.94:0/657299097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7a01037a0 0x7fa7a01160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.670+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 >> 172.21.15.94:0/657299097 conn(0x7fa7a00f89b0 msgr2=0x7fa7a00fa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:20.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.670+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 shutdown_connections 2024-04-06T03:37:20.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:19.670+0000 7fa7a7044700 1 -- 172.21.15.94:0/657299097 wait complete. 2024-04-06T03:37:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:20 smithi094 ceph-mon[29713]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:20 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3193839942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-06T03:37:20.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:20 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/657299097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-06T03:37:20.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:20 smithi136 ceph-mon[36436]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:20.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:20 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3193839942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-06T03:37:20.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:20 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/657299097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-06T03:37:20.867 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888075 got 146028888078 for osd.4 2024-04-06T03:37:20.867 DEBUG:teuthology.parallel:result is None 2024-04-06T03:37:21.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378645 got 81604378649 for osd.2 2024-04-06T03:37:21.320 DEBUG:teuthology.parallel:result is None 2024-04-06T03:37:21.800 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149717 for osd.3 2024-04-06T03:37:21.801 DEBUG:teuthology.parallel:result is None 2024-04-06T03:37:21.801 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-06T03:37:21.801 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-06T03:37:22.061 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:22.465 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:22 smithi094 ceph-mon[29713]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:22.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:22 smithi136 ceph-mon[36436]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:23.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.600+0000 7f688e56e700 1 -- 172.21.15.94:0/1673110678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 msgr2=0x7f68880fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:23.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.600+0000 7f688e56e700 1 --2- 172.21.15.94:0/1673110678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 0x7f68880fce10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f687c004940 tx=0x7f687c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:23.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 -- 172.21.15.94:0/1673110678 shutdown_connections 2024-04-06T03:37:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 --2- 172.21.15.94:0/1673110678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 0x7f68880fce10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 --2- 172.21.15.94:0/1673110678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6888102e00 0x7f68881031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 -- 172.21.15.94:0/1673110678 >> 172.21.15.94:0/1673110678 conn(0x7f68880f89b0 msgr2=0x7f68880fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 -- 172.21.15.94:0/1673110678 shutdown_connections 2024-04-06T03:37:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 -- 172.21.15.94:0/1673110678 wait complete. 2024-04-06T03:37:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 Processor -- start 2024-04-06T03:37:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 -- start start 2024-04-06T03:37:23.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6888102e00 0x7f68880702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:23.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 0x7f6888070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:23.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68880722a0 con 0x7f68881037a0 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.601+0000 7f688e56e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6888070d40 con 0x7f6888102e00 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f68877fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 0x7f6888070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f68877fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 0x7f6888070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43626/0 (socket says 172.21.15.94:43626) 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f68877fe700 1 -- 172.21.15.94:0/2876910787 learned_addr learned my addr 172.21.15.94:0/2876910787 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f6887fff700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6888102e00 0x7f68880702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f68877fe700 1 -- 172.21.15.94:0/2876910787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6888102e00 msgr2=0x7f68880702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f68877fe700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6888102e00 0x7f68880702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f68877fe700 1 -- 172.21.15.94:0/2876910787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c005040 con 0x7f68881037a0 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f68877fe700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 0x7f6888070800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f687c009100 tx=0x7f687c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f6884ff9700 1 -- 172.21.15.94:0/2876910787 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f687c00e040 con 0x7f68881037a0 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6888070fc0 con 0x7f68881037a0 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f6884ff9700 1 -- 172.21.15.94:0/2876910787 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f687c009af0 con 0x7f68881037a0 2024-04-06T03:37:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f6884ff9700 1 -- 172.21.15.94:0/2876910787 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f687c01c3f0 con 0x7f68881037a0 2024-04-06T03:37:23.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.602+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688811a1d0 con 0x7f68881037a0 2024-04-06T03:37:23.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.603+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6888100110 con 0x7f68881037a0 2024-04-06T03:37:23.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.604+0000 7f6884ff9700 1 -- 172.21.15.94:0/2876910787 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f687c00baf0 con 0x7f68881037a0 2024-04-06T03:37:23.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.604+0000 7f6884ff9700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f68700692a0 0x7f687006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:23.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.604+0000 7f6884ff9700 1 -- 172.21.15.94:0/2876910787 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f687c0880d0 con 0x7f68881037a0 2024-04-06T03:37:23.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.604+0000 7f6887fff700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f68700692a0 0x7f687006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:23.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.604+0000 7f6887fff700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f68700692a0 0x7f687006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6878009b60 tx=0x7f68780091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.608+0000 7f6884ff9700 1 -- 172.21.15.94:0/2876910787 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f687c054740 con 0x7f68881037a0 2024-04-06T03:37:23.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.792+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6888109f50 con 0x7f68700692a0 2024-04-06T03:37:23.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.794+0000 7f6884ff9700 1 -- 172.21.15.94:0/2876910787 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28191 (secure 0 0 0) 0x7f6888109f50 con 0x7f68700692a0 2024-04-06T03:37:23.797 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:37:23.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f68700692a0 msgr2=0x7f687006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:23.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f68700692a0 0x7f687006b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6878009b60 tx=0x7f68780091a0 comp rx=0 tx=0).stop 2024-04-06T03:37:23.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 msgr2=0x7f6888070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:23.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 0x7f6888070800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f687c009100 tx=0x7f687c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:23.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 shutdown_connections 2024-04-06T03:37:23.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f68700692a0 0x7f687006b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:23.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6888102e00 0x7f68880702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:23.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 --2- 172.21.15.94:0/2876910787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68881037a0 0x7f6888070800 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:23.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 >> 172.21.15.94:0/2876910787 conn(0x7f68880f89b0 msgr2=0x7f68880fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:23.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 shutdown_connections 2024-04-06T03:37:23.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:23.796+0000 7f688e56e700 1 -- 172.21.15.94:0/2876910787 wait complete. 2024-04-06T03:37:23.801 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-06T03:37:24.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:24 smithi094 ceph-mon[29713]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:24.560 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-06T03:37:23.000147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788020,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682892,"statfs":{"total":575970213888,"available":574139281408,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001915"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-06T03:36:36.173003+0000","last_change":"2024-04-06T03:36:10.996741+0000","last_active":"2024-04-06T03:36:36.173003+0000","last_peered":"2024-04-06T03:36:36.173003+0000","last_clean":"2024-04-06T03:36:36.173003+0000","last_became_active":"2024-04-06T03:36:10.996290+0000","last_became_peered":"2024-04-06T03:36:10.996290+0000","last_unstale":"2024-04-06T03:36:36.173003+0000","last_undegraded":"2024-04-06T03:36:36.173003+0000","last_fullsized":"2024-04-06T03:36:36.173003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-06T03:35:23.522849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-06T03:35:23.522849+0000","last_clean_scrub_stamp":"2024-04-06T03:35:23.522849+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:26:16.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 6 03:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.55900000000000005}]},{"osd":1,"last update":"Sat Apr 6 03:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Sat Apr 6 03:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.505}]},{"osd":3,"last update":"Sat Apr 6 03:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 6 03:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Sat Apr 6 03:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.437}]},{"osd":2,"last update":"Sat Apr 6 03:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.505}]},{"osd":4,"last update":"Sat Apr 6 03:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378649,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 6 03:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.28999999999999998}]},{"osd":1,"last update":"Sat Apr 6 03:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.38700000000000001}]},{"osd":3,"last update":"Sat Apr 6 03:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Apr 6 03:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 6 03:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.34699999999999998,"15min":0.35299999999999998},"min":{"1min":0.255,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39800000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.316},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.33900000000000002,"15min":0.34200000000000003},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.41999999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.27400000000000002}]},{"osd":2,"last update":"Sat Apr 6 03:36:25 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.35799999999999998}]},{"osd":3,"last update":"Sat Apr 6 03:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.69699999999999995}]},{"osd":4,"last update":"Sat Apr 6 03:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 6 03:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.246,"5min":0.186,"15min":0.186},"max":{"1min":0.49299999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.33200000000000002,"15min":0.32700000000000001},"min":{"1min":0.26800000000000002,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.46700000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Sat Apr 6 03:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.313},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.27800000000000002}]},{"osd":3,"last update":"Sat Apr 6 03:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52600000000000002}]},{"osd":4,"last update":"Sat Apr 6 03:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-06T03:37:24.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-06T03:37:24.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:24 smithi136 ceph-mon[36436]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:24.816 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:25.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:25 smithi136 ceph-mon[36436]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-06T03:37:25.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:25 smithi094 ceph-mon[29713]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-06T03:37:26.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.424+0000 7f931b0c8700 1 -- 172.21.15.94:0/3498389208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9314101d10 msgr2=0x7f93141020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:26.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.424+0000 7f931b0c8700 1 --2- 172.21.15.94:0/3498389208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9314101d10 0x7f93141020e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9308004940 tx=0x7f9308009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:26.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.424+0000 7f931b0c8700 1 -- 172.21.15.94:0/3498389208 shutdown_connections 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.424+0000 7f931b0c8700 1 --2- 172.21.15.94:0/3498389208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93141026b0 0x7f93140fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.424+0000 7f931b0c8700 1 --2- 172.21.15.94:0/3498389208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9314101d10 0x7f93141020e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.424+0000 7f931b0c8700 1 -- 172.21.15.94:0/3498389208 >> 172.21.15.94:0/3498389208 conn(0x7f93140f78e0 msgr2=0x7f93140f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.424+0000 7f931b0c8700 1 -- 172.21.15.94:0/3498389208 shutdown_connections 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.424+0000 7f931b0c8700 1 -- 172.21.15.94:0/3498389208 wait complete. 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f931b0c8700 1 Processor -- start 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f931b0c8700 1 -- start start 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f931b0c8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9314101d10 0x7f9314113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f931b0c8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93141026b0 0x7f9314113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f931b0c8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9314114510 con 0x7f93141026b0 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f931b0c8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931410db10 con 0x7f9314101d10 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f9313fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93141026b0 0x7f9314113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f9313fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93141026b0 0x7f9314113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43644/0 (socket says 172.21.15.94:43644) 2024-04-06T03:37:26.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f9313fff700 1 -- 172.21.15.94:0/827274825 learned_addr learned my addr 172.21.15.94:0/827274825 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f9318e64700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9314101d10 0x7f9314113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f9313fff700 1 -- 172.21.15.94:0/827274825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9314101d10 msgr2=0x7f9314113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f9313fff700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9314101d10 0x7f9314113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.425+0000 7f9313fff700 1 -- 172.21.15.94:0/827274825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9308005040 con 0x7f93141026b0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.426+0000 7f9313fff700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93141026b0 0x7f9314113ec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f930000c800 tx=0x7f9300004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.426+0000 7f93117fa700 1 -- 172.21.15.94:0/827274825 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f930000a760 con 0x7f93141026b0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.426+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931410ddf0 con 0x7f93141026b0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.426+0000 7f93117fa700 1 -- 172.21.15.94:0/827274825 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f930000ad90 con 0x7f93141026b0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.426+0000 7f93117fa700 1 -- 172.21.15.94:0/827274825 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93000113a0 con 0x7f93141026b0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.426+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931410e3b0 con 0x7f93141026b0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.427+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93140fd460 con 0x7f93141026b0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.427+0000 7f93117fa700 1 -- 172.21.15.94:0/827274825 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9300012400 con 0x7f93141026b0 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.427+0000 7f93117fa700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93040692a0 0x7f930406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:26.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.428+0000 7f93117fa700 1 -- 172.21.15.94:0/827274825 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9300086e90 con 0x7f93141026b0 2024-04-06T03:37:26.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.428+0000 7f9318e64700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93040692a0 0x7f930406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:26.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.428+0000 7f9318e64700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93040692a0 0x7f930406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9308004f90 tx=0x7f9308009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:26.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.432+0000 7f93117fa700 1 -- 172.21.15.94:0/827274825 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9300056f90 con 0x7f93141026b0 2024-04-06T03:37:26.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:26 smithi136 ceph-mon[36436]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:26.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.617+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9314108e10 con 0x7f93040692a0 2024-04-06T03:37:26.619 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:26 smithi094 ceph-mon[29713]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:26.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.618+0000 7f93117fa700 1 -- 172.21.15.94:0/827274825 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28191 (secure 0 0 0) 0x7f9314108e10 con 0x7f93040692a0 2024-04-06T03:37:26.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:37:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.620+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93040692a0 msgr2=0x7f930406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.620+0000 7f931b0c8700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93040692a0 0x7f930406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9308004f90 tx=0x7f9308009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.620+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93141026b0 msgr2=0x7f9314113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.620+0000 7f931b0c8700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93141026b0 0x7f9314113ec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f930000c800 tx=0x7f9300004470 comp rx=0 tx=0).stop 2024-04-06T03:37:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.621+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 shutdown_connections 2024-04-06T03:37:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.621+0000 7f931b0c8700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93040692a0 0x7f930406b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.621+0000 7f931b0c8700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9314101d10 0x7f9314113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:26.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.621+0000 7f931b0c8700 1 --2- 172.21.15.94:0/827274825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93141026b0 0x7f9314113ec0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:26.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.621+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 >> 172.21.15.94:0/827274825 conn(0x7f93140f78e0 msgr2=0x7f9314066250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:26.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.621+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 shutdown_connections 2024-04-06T03:37:26.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:26.621+0000 7f931b0c8700 1 -- 172.21.15.94:0/827274825 wait complete. 2024-04-06T03:37:26.624 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-06T03:37:27.308 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-04-06T03:37:25.000466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788020,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682892,"statfs":{"total":575970213888,"available":574139281408,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001926"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-06T03:36:36.173003+0000","last_change":"2024-04-06T03:36:10.996741+0000","last_active":"2024-04-06T03:36:36.173003+0000","last_peered":"2024-04-06T03:36:36.173003+0000","last_clean":"2024-04-06T03:36:36.173003+0000","last_became_active":"2024-04-06T03:36:10.996290+0000","last_became_peered":"2024-04-06T03:36:10.996290+0000","last_unstale":"2024-04-06T03:36:36.173003+0000","last_undegraded":"2024-04-06T03:36:36.173003+0000","last_fullsized":"2024-04-06T03:36:36.173003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-06T03:35:23.522849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-06T03:35:23.522849+0000","last_clean_scrub_stamp":"2024-04-06T03:35:23.522849+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-07T10:26:16.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 6 03:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.55900000000000005}]},{"osd":1,"last update":"Sat Apr 6 03:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Sat Apr 6 03:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.505}]},{"osd":3,"last update":"Sat Apr 6 03:37:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 6 03:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Sat Apr 6 03:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.437}]},{"osd":2,"last update":"Sat Apr 6 03:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.505}]},{"osd":4,"last update":"Sat Apr 6 03:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378650,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 6 03:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.28199999999999997}]},{"osd":1,"last update":"Sat Apr 6 03:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.29899999999999999}]},{"osd":3,"last update":"Sat Apr 6 03:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51500000000000001}]},{"osd":4,"last update":"Sat Apr 6 03:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.45100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 6 03:37:01 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.34699999999999998,"15min":0.35299999999999998},"min":{"1min":0.255,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39800000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.316},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.33900000000000002,"15min":0.34200000000000003},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.41999999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.27400000000000002}]},{"osd":2,"last update":"Sat Apr 6 03:36:25 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.35799999999999998}]},{"osd":3,"last update":"Sat Apr 6 03:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.69699999999999995}]},{"osd":4,"last update":"Sat Apr 6 03:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 6 03:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.246,"5min":0.186,"15min":0.186},"max":{"1min":0.49299999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.33200000000000002,"15min":0.32700000000000001},"min":{"1min":0.26800000000000002,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.46700000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Sat Apr 6 03:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.313},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.27800000000000002}]},{"osd":3,"last update":"Sat Apr 6 03:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52600000000000002}]},{"osd":4,"last update":"Sat Apr 6 03:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-06T03:37:27.309 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-06T03:37:27.309 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-06T03:37:27.309 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-06T03:37:27.310 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph health --format=json 2024-04-06T03:37:27.565 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:28.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:28 smithi136 ceph-mon[36436]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-06T03:37:28.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:28 smithi136 ceph-mon[36436]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:28.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:28 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:37:28.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:28 smithi094 ceph-mon[29713]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-06T03:37:28.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:28 smithi094 ceph-mon[29713]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:28.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:28 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:37:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 -- 172.21.15.94:0/3097265667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fde90 msgr2=0x7fea4c0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 --2- 172.21.15.94:0/3097265667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fde90 0x7fea4c0fe260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fea3c004940 tx=0x7fea3c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 -- 172.21.15.94:0/3097265667 shutdown_connections 2024-04-06T03:37:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 --2- 172.21.15.94:0/3097265667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea4c0fe830 0x7fea4c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:29.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 --2- 172.21.15.94:0/3097265667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fde90 0x7fea4c0fe260 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 -- 172.21.15.94:0/3097265667 >> 172.21.15.94:0/3097265667 conn(0x7fea4c072bc0 msgr2=0x7fea4c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 -- 172.21.15.94:0/3097265667 shutdown_connections 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 -- 172.21.15.94:0/3097265667 wait complete. 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 Processor -- start 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.135+0000 7fea532c8700 1 -- start start 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea532c8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea4c0fde90 0x7fea4c0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea532c8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fe830 0x7fea4c0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea532c8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0fc8b0 con 0x7fea4c0fe830 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea532c8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0fc9f0 con 0x7fea4c0fde90 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea50863700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fe830 0x7fea4c0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea50863700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fe830 0x7fea4c0fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49216/0 (socket says 172.21.15.94:49216) 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea50863700 1 -- 172.21.15.94:0/2464160902 learned_addr learned my addr 172.21.15.94:0/2464160902 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea51064700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea4c0fde90 0x7fea4c0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea50863700 1 -- 172.21.15.94:0/2464160902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea4c0fde90 msgr2=0x7fea4c0fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:29.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea50863700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea4c0fde90 0x7fea4c0fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea50863700 1 -- 172.21.15.94:0/2464160902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c005040 con 0x7fea4c0fe830 2024-04-06T03:37:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea51064700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea4c0fde90 0x7fea4c0fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:37:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea50863700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fe830 0x7fea4c0fc290 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fea4800b790 tx=0x7fea48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea41ffb700 1 -- 172.21.15.94:0/2464160902 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea4800f770 con 0x7fea4c0fe830 2024-04-06T03:37:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea41ffb700 1 -- 172.21.15.94:0/2464160902 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea4800fda0 con 0x7fea4c0fe830 2024-04-06T03:37:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c119ed0 con 0x7fea4c0fe830 2024-04-06T03:37:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.136+0000 7fea41ffb700 1 -- 172.21.15.94:0/2464160902 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea4800d310 con 0x7fea4c0fe830 2024-04-06T03:37:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.137+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c11a380 con 0x7fea4c0fe830 2024-04-06T03:37:29.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.137+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c065e30 con 0x7fea4c0fe830 2024-04-06T03:37:29.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.138+0000 7fea41ffb700 1 -- 172.21.15.94:0/2464160902 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fea48010420 con 0x7fea4c0fe830 2024-04-06T03:37:29.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.138+0000 7fea41ffb700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea380692a0 0x7fea3806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:29.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.138+0000 7fea41ffb700 1 -- 172.21.15.94:0/2464160902 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fea48086da0 con 0x7fea4c0fe830 2024-04-06T03:37:29.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.138+0000 7fea51064700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea380692a0 0x7fea3806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:29.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.139+0000 7fea51064700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea380692a0 0x7fea3806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea3c004f90 tx=0x7fea3c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:29.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.143+0000 7fea41ffb700 1 -- 172.21.15.94:0/2464160902 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fea480534b0 con 0x7fea4c0fe830 2024-04-06T03:37:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.367+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fea4c05f5f0 con 0x7fea4c0fe830 2024-04-06T03:37:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.367+0000 7fea41ffb700 1 -- 172.21.15.94:0/2464160902 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fea48056ad0 con 0x7fea4c0fe830 2024-04-06T03:37:29.370 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:37:29.370 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-06T03:37:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.370+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea380692a0 msgr2=0x7fea3806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.370+0000 7fea532c8700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea380692a0 0x7fea3806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea3c004f90 tx=0x7fea3c009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.370+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fe830 msgr2=0x7fea4c0fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.370+0000 7fea532c8700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fe830 0x7fea4c0fc290 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fea4800b790 tx=0x7fea48009250 comp rx=0 tx=0).stop 2024-04-06T03:37:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.371+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 shutdown_connections 2024-04-06T03:37:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.371+0000 7fea532c8700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea380692a0 0x7fea3806b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.371+0000 7fea532c8700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea4c0fde90 0x7fea4c0fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.371+0000 7fea532c8700 1 --2- 172.21.15.94:0/2464160902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea4c0fe830 0x7fea4c0fc290 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.371+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 >> 172.21.15.94:0/2464160902 conn(0x7fea4c072bc0 msgr2=0x7fea4c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.371+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 shutdown_connections 2024-04-06T03:37:29.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:29.371+0000 7fea532c8700 1 -- 172.21.15.94:0/2464160902 wait complete. 2024-04-06T03:37:29.928 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-06T03:37:29.928 INFO:tasks.cephadm:Setup complete, yielding 2024-04-06T03:37:29.929 INFO:teuthology.run_tasks:Running task print... 2024-04-06T03:37:29.939 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-04-06T03:37:29.939 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-06T03:37:29.949 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:37:29.950 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-06T03:37:30.202 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:30.431 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:30 smithi094 ceph-mon[29713]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:30.432 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:30 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2464160902' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-06T03:37:30.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:30 smithi136 ceph-mon[36436]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:30.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:30 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2464160902' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-06T03:37:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.598+0000 7f37053df700 1 -- 172.21.15.94:0/562813555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fbcd0 msgr2=0x7f37000fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.598+0000 7f37053df700 1 --2- 172.21.15.94:0/562813555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fbcd0 0x7f37000fc0e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f36f4004940 tx=0x7f36f4009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.598+0000 7f37053df700 1 -- 172.21.15.94:0/562813555 shutdown_connections 2024-04-06T03:37:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.598+0000 7f37053df700 1 --2- 172.21.15.94:0/562813555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37000fcf80 0x7f37000fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.598+0000 7f37053df700 1 --2- 172.21.15.94:0/562813555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fbcd0 0x7f37000fc0e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:31.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.598+0000 7f37053df700 1 -- 172.21.15.94:0/562813555 >> 172.21.15.94:0/562813555 conn(0x7f37000f7430 msgr2=0x7f37000f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.598+0000 7f37053df700 1 -- 172.21.15.94:0/562813555 shutdown_connections 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.598+0000 7f37053df700 1 -- 172.21.15.94:0/562813555 wait complete. 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f37053df700 1 Processor -- start 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f37053df700 1 -- start start 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f37053df700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37000fbcd0 0x7f3700110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f37053df700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fcf80 0x7f3700112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f37053df700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37000663e0 con 0x7f37000fcf80 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f37053df700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3700066520 con 0x7f37000fbcd0 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f36fe7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fcf80 0x7f3700112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f36fe7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fcf80 0x7f3700112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49224/0 (socket says 172.21.15.94:49224) 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f36fe7fc700 1 -- 172.21.15.94:0/488396244 learned_addr learned my addr 172.21.15.94:0/488396244 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f36fe7fc700 1 -- 172.21.15.94:0/488396244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37000fbcd0 msgr2=0x7f3700110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f36fe7fc700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37000fbcd0 0x7f3700110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f36fe7fc700 1 -- 172.21.15.94:0/488396244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f4005040 con 0x7f37000fcf80 2024-04-06T03:37:31.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f36fe7fc700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fcf80 0x7f3700112c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f36f000b790 tx=0x7f36f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:31.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f36e77fe700 1 -- 172.21.15.94:0/488396244 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36f000f770 con 0x7f37000fcf80 2024-04-06T03:37:31.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.599+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37001132e0 con 0x7f37000fcf80 2024-04-06T03:37:31.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.600+0000 7f36e77fe700 1 -- 172.21.15.94:0/488396244 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36f000fda0 con 0x7f37000fcf80 2024-04-06T03:37:31.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.600+0000 7f36e77fe700 1 -- 172.21.15.94:0/488396244 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36f000d310 con 0x7f37000fcf80 2024-04-06T03:37:31.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.600+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3700113870 con 0x7f37000fcf80 2024-04-06T03:37:31.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.600+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370005f5f0 con 0x7f37000fcf80 2024-04-06T03:37:31.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.601+0000 7f36e77fe700 1 -- 172.21.15.94:0/488396244 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f36f000f8d0 con 0x7f37000fcf80 2024-04-06T03:37:31.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.601+0000 7f36e77fe700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f36e80692a0 0x7f36e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:31.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.601+0000 7f36e77fe700 1 -- 172.21.15.94:0/488396244 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f36f0086ec0 con 0x7f37000fcf80 2024-04-06T03:37:31.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.601+0000 7f36feffd700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f36e80692a0 0x7f36e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:31.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.602+0000 7f36feffd700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f36e80692a0 0x7f36e806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36f4004f90 tx=0x7f36f4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:31.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.606+0000 7f36e77fe700 1 -- 172.21.15.94:0/488396244 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f36f0053530 con 0x7f37000fcf80 2024-04-06T03:37:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.797+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f3700104940 con 0x7f37000fcf80 2024-04-06T03:37:31.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.801+0000 7f36e77fe700 1 -- 172.21.15.94:0/488396244 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f36f0020250 con 0x7f37000fcf80 2024-04-06T03:37:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.804+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f36e80692a0 msgr2=0x7f36e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.804+0000 7f37053df700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f36e80692a0 0x7f36e806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36f4004f90 tx=0x7f36f4009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.804+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fcf80 msgr2=0x7f3700112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.804+0000 7f37053df700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fcf80 0x7f3700112c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f36f000b790 tx=0x7f36f0009250 comp rx=0 tx=0).stop 2024-04-06T03:37:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.804+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 shutdown_connections 2024-04-06T03:37:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.804+0000 7f37053df700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f36e80692a0 0x7f36e806b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:31.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.804+0000 7f37053df700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37000fbcd0 0x7f3700110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.804+0000 7f37053df700 1 --2- 172.21.15.94:0/488396244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37000fcf80 0x7f3700112c80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.804+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 >> 172.21.15.94:0/488396244 conn(0x7f37000f7430 msgr2=0x7f37000f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.805+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 shutdown_connections 2024-04-06T03:37:31.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:31.805+0000 7f37053df700 1 -- 172.21.15.94:0/488396244 wait complete. 2024-04-06T03:37:32.550 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:32 smithi094 ceph-mon[29713]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:32.550 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:32 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/488396244' entity='client.admin' 2024-04-06T03:37:32.550 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:37:32.550 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:32.550 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:37:32.550 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:32 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:32.574 INFO:teuthology.run_tasks:Running task print... 2024-04-06T03:37:32.584 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-06T03:37:32.584 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-06T03:37:32.596 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:37:32.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-04-06T03:37:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:32 smithi136 ceph-mon[36436]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:32 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/488396244' entity='client.admin' 2024-04-06T03:37:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:37:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:37:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:32 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:32.850 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:34.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.314+0000 7f050eb50700 1 -- 172.21.15.94:0/472132652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 msgr2=0x7f05080fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:34.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.314+0000 7f050eb50700 1 --2- 172.21.15.94:0/472132652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 0x7f05080fd280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f04fc004940 tx=0x7f04fc009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:34.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.314+0000 7f050eb50700 1 -- 172.21.15.94:0/472132652 shutdown_connections 2024-04-06T03:37:34.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.314+0000 7f050eb50700 1 --2- 172.21.15.94:0/472132652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05080fdfe0 0x7f05080fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:34.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.314+0000 7f050eb50700 1 --2- 172.21.15.94:0/472132652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 0x7f05080fd280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:34.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.314+0000 7f050eb50700 1 -- 172.21.15.94:0/472132652 >> 172.21.15.94:0/472132652 conn(0x7f05080f8550 msgr2=0x7f05080fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:34.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.314+0000 7f050eb50700 1 -- 172.21.15.94:0/472132652 shutdown_connections 2024-04-06T03:37:34.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.314+0000 7f050eb50700 1 -- 172.21.15.94:0/472132652 wait complete. 2024-04-06T03:37:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.315+0000 7f050eb50700 1 Processor -- start 2024-04-06T03:37:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.315+0000 7f050eb50700 1 -- start start 2024-04-06T03:37:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.315+0000 7f050eb50700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 0x7f050810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.315+0000 7f050eb50700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05080fdfe0 0x7f050810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:34.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.315+0000 7f050eb50700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050810e490 con 0x7f05080fce70 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.315+0000 7f050eb50700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050810e600 con 0x7f05080fdfe0 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.315+0000 7f050c8ec700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 0x7f050810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.315+0000 7f050c8ec700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 0x7f050810d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49242/0 (socket says 172.21.15.94:49242) 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.315+0000 7f050c8ec700 1 -- 172.21.15.94:0/795532524 learned_addr learned my addr 172.21.15.94:0/795532524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f0507fff700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05080fdfe0 0x7f050810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f050c8ec700 1 -- 172.21.15.94:0/795532524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05080fdfe0 msgr2=0x7f050810de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f050c8ec700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05080fdfe0 0x7f050810de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f050c8ec700 1 -- 172.21.15.94:0/795532524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc005040 con 0x7f05080fce70 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f050c8ec700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 0x7f050810d950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009150 tx=0x7f04fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f05057fa700 1 -- 172.21.15.94:0/795532524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04fc00e040 con 0x7f05080fce70 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f05057fa700 1 -- 172.21.15.94:0/795532524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04fc00bbf0 con 0x7f05080fce70 2024-04-06T03:37:34.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0508113000 con 0x7f05080fce70 2024-04-06T03:37:34.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f05057fa700 1 -- 172.21.15.94:0/795532524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04fc01c630 con 0x7f05080fce70 2024-04-06T03:37:34.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.316+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05081135c0 con 0x7f05080fce70 2024-04-06T03:37:34.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.317+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050804c710 con 0x7f05080fce70 2024-04-06T03:37:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.318+0000 7f05057fa700 1 -- 172.21.15.94:0/795532524 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f04fc00bd60 con 0x7f05080fce70 2024-04-06T03:37:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.318+0000 7f05057fa700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f04f80692a0 0x7f04f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.318+0000 7f05057fa700 1 -- 172.21.15.94:0/795532524 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f04fc0881b0 con 0x7f05080fce70 2024-04-06T03:37:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.318+0000 7f0507fff700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f04f80692a0 0x7f04f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:34.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.318+0000 7f0507fff700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f04f80692a0 0x7f04f806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f04f4004f40 tx=0x7f04f400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:34.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.322+0000 7f05057fa700 1 -- 172.21.15.94:0/795532524 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f04fc058360 con 0x7f05080fce70 2024-04-06T03:37:34.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:34 smithi094 ceph-mon[29713]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:34.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.526+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0508102340 con 0x7f04f80692a0 2024-04-06T03:37:34.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.527+0000 7f05057fa700 1 -- 172.21.15.94:0/795532524 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0508102340 con 0x7f04f80692a0 2024-04-06T03:37:34.529 INFO:teuthology.orchestra.run.smithi094.stdout:Backend: cephadm 2024-04-06T03:37:34.529 INFO:teuthology.orchestra.run.smithi094.stdout:Available: Yes 2024-04-06T03:37:34.529 INFO:teuthology.orchestra.run.smithi094.stdout:Paused: No 2024-04-06T03:37:34.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.528+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f04f80692a0 msgr2=0x7f04f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:34.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.528+0000 7f050eb50700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f04f80692a0 0x7f04f806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f04f4004f40 tx=0x7f04f400b040 comp rx=0 tx=0).stop 2024-04-06T03:37:34.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.528+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 msgr2=0x7f050810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:34.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.528+0000 7f050eb50700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 0x7f050810d950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009150 tx=0x7f04fc009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:34.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.529+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 shutdown_connections 2024-04-06T03:37:34.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.529+0000 7f050eb50700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f04f80692a0 0x7f04f806b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:34.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.529+0000 7f050eb50700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05080fce70 0x7f050810d950 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:34.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.529+0000 7f050eb50700 1 --2- 172.21.15.94:0/795532524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05080fdfe0 0x7f050810de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:34.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.529+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 >> 172.21.15.94:0/795532524 conn(0x7f05080f8550 msgr2=0x7f05080f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:34.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.529+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 shutdown_connections 2024-04-06T03:37:34.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:34.529+0000 7f050eb50700 1 -- 172.21.15.94:0/795532524 wait complete. 2024-04-06T03:37:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:34 smithi136 ceph-mon[36436]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:35.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-04-06T03:37:35.473 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:35.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:35 smithi094 ceph-mon[29713]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:35.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:35 smithi136 ceph-mon[36436]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:36.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:36 smithi136 ceph-mon[36436]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:36.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:36 smithi094 ceph-mon[29713]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:36.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.951+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/4206643868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fbd00 msgr2=0x7f1e480fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.951+0000 7f1e4e7f2700 1 --2- 172.21.15.94:0/4206643868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fbd00 0x7f1e480fc110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1e38004940 tx=0x7f1e38009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.951+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/4206643868 shutdown_connections 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.951+0000 7f1e4e7f2700 1 --2- 172.21.15.94:0/4206643868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e480fcfb0 0x7f1e480fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.951+0000 7f1e4e7f2700 1 --2- 172.21.15.94:0/4206643868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fbd00 0x7f1e480fc110 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.951+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/4206643868 >> 172.21.15.94:0/4206643868 conn(0x7f1e480f7480 msgr2=0x7f1e480f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.951+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/4206643868 shutdown_connections 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.951+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/4206643868 wait complete. 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.952+0000 7f1e4e7f2700 1 Processor -- start 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.952+0000 7f1e4e7f2700 1 -- start start 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.952+0000 7f1e4e7f2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e480fbd00 0x7f1e4810c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.952+0000 7f1e4e7f2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fcfb0 0x7f1e4810cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.952+0000 7f1e4e7f2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4810d400 con 0x7f1e480fcfb0 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.952+0000 7f1e4e7f2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e48111da0 con 0x7f1e480fbd00 2024-04-06T03:37:36.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.952+0000 7f1e477fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fcfb0 0x7f1e4810cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.952+0000 7f1e477fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fcfb0 0x7f1e4810cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49254/0 (socket says 172.21.15.94:49254) 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e477fe700 1 -- 172.21.15.94:0/1732860482 learned_addr learned my addr 172.21.15.94:0/1732860482 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e47fff700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e480fbd00 0x7f1e4810c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e477fe700 1 -- 172.21.15.94:0/1732860482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e480fbd00 msgr2=0x7f1e4810c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e477fe700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e480fbd00 0x7f1e4810c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e477fe700 1 -- 172.21.15.94:0/1732860482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38005040 con 0x7f1e480fcfb0 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e477fe700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fcfb0 0x7f1e4810cd70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00b790 tx=0x7f1e3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e44ff9700 1 -- 172.21.15.94:0/1732860482 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e3c00f6f0 con 0x7f1e480fcfb0 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e44ff9700 1 -- 172.21.15.94:0/1732860482 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e3c00fd20 con 0x7f1e480fcfb0 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e48112000 con 0x7f1e480fcfb0 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e44ff9700 1 -- 172.21.15.94:0/1732860482 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e3c0103a0 con 0x7f1e480fcfb0 2024-04-06T03:37:36.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.953+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e48112540 con 0x7f1e480fcfb0 2024-04-06T03:37:36.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.954+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4804c710 con 0x7f1e480fcfb0 2024-04-06T03:37:36.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.954+0000 7f1e44ff9700 1 -- 172.21.15.94:0/1732860482 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1e3c00f850 con 0x7f1e480fcfb0 2024-04-06T03:37:36.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.955+0000 7f1e44ff9700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e300692a0 0x7f1e3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:36.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.955+0000 7f1e44ff9700 1 -- 172.21.15.94:0/1732860482 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1e3c086c20 con 0x7f1e480fcfb0 2024-04-06T03:37:36.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.955+0000 7f1e47fff700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e300692a0 0x7f1e3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:36.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.956+0000 7f1e47fff700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e300692a0 0x7f1e3006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800b970 tx=0x7f1e38009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:36.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:36.959+0000 7f1e44ff9700 1 -- 172.21.15.94:0/1732860482 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1e3c056d20 con 0x7f1e480fcfb0 2024-04-06T03:37:37.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.152+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e48101250 con 0x7f1e300692a0 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.160+0000 7f1e44ff9700 1 -- 172.21.15.94:0/1732860482 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2718 (secure 0 0 0) 0x7f1e48101250 con 0x7f1e300692a0 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (3m) 2m ago 4m 26.9M - 0.25.0 c8568f914cd2 b63c91af7786 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (5m) 2m ago 5m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (4m) 62s ago 4m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (3m) 2m ago 4m 80.2M - 9.4.7 954c08fa6188 78ad0d4cf52b 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:9283 running (6m) 2m ago 6m 472M - 17.2.7-819-g6699495c ec300eaa768b d83dfb730a39 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283 running (4m) 62s ago 4m 433M - 17.2.7-819-g6699495c ec300eaa768b 9e1727db1762 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (6m) 2m ago 6m 42.0M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:37:37.163 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (3m) 62s ago 3m 38.4M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:37:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (5m) 2m ago 5m 16.3M - 1.5.0 0da6a335fe13 f8f823e89f23 2024-04-06T03:37:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (4m) 62s ago 4m 17.2M - 1.5.0 0da6a335fe13 e5324db1ea82 2024-04-06T03:37:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (3m) 2m ago 3m 37.7M 4288M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:37:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (2m) 2m ago 2m 38.6M 4288M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:37:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (2m) 2m ago 2m 38.2M 4288M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:37:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (118s) 62s ago 117s 38.0M 5312M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:37:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (93s) 62s ago 92s 36.5M 5312M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:37:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (67s) 62s ago 67s 11.0M 5312M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:37:37.164 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (3m) 2m ago 4m 37.8M - 2.43.0 a07b618ecd1d 82a078c9e43c 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.162+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e300692a0 msgr2=0x7f1e3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.162+0000 7f1e4e7f2700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e300692a0 0x7f1e3006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800b970 tx=0x7f1e38009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.162+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fcfb0 msgr2=0x7f1e4810cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.162+0000 7f1e4e7f2700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fcfb0 0x7f1e4810cd70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00b790 tx=0x7f1e3c009250 comp rx=0 tx=0).stop 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.162+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 shutdown_connections 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.162+0000 7f1e4e7f2700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1e300692a0 0x7f1e3006b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.162+0000 7f1e4e7f2700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e480fbd00 0x7f1e4810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.162+0000 7f1e4e7f2700 1 --2- 172.21.15.94:0/1732860482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e480fcfb0 0x7f1e4810cd70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.162+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 >> 172.21.15.94:0/1732860482 conn(0x7f1e480f7480 msgr2=0x7f1e480f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.163+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 shutdown_connections 2024-04-06T03:37:37.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:37.163+0000 7f1e4e7f2700 1 -- 172.21.15.94:0/1732860482 wait complete. 2024-04-06T03:37:37.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-04-06T03:37:38.251 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:38.597 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:38 smithi094 ceph-mon[29713]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:38.598 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:38 smithi094 ceph-mon[29713]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:38.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:38 smithi136 ceph-mon[36436]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:38.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:38 smithi136 ceph-mon[36436]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 -- 172.21.15.94:0/3573285700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fbcd0 msgr2=0x7fb6640fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 --2- 172.21.15.94:0/3573285700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fbcd0 0x7fb6640fc0e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb654004940 tx=0x7fb654009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 -- 172.21.15.94:0/3573285700 shutdown_connections 2024-04-06T03:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 --2- 172.21.15.94:0/3573285700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6640fcf80 0x7fb6640fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 --2- 172.21.15.94:0/3573285700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fbcd0 0x7fb6640fc0e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 -- 172.21.15.94:0/3573285700 >> 172.21.15.94:0/3573285700 conn(0x7fb6640f7430 msgr2=0x7fb6640f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 -- 172.21.15.94:0/3573285700 shutdown_connections 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 -- 172.21.15.94:0/3573285700 wait complete. 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 Processor -- start 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.918+0000 7fb6690c7700 1 -- start start 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb6690c7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6640fbcd0 0x7fb6640ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb6690c7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fcf80 0x7fb6640fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb6690c7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640fe870 con 0x7fb6640fcf80 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb6690c7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640fe9e0 con 0x7fb6640fbcd0 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fcf80 0x7fb6640fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fcf80 0x7fb6640fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53476/0 (socket says 172.21.15.94:53476) 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65a59c700 1 -- 172.21.15.94:0/2908052750 learned_addr learned my addr 172.21.15.94:0/2908052750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb662d9d700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6640fbcd0 0x7fb6640ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:39.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65a59c700 1 -- 172.21.15.94:0/2908052750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6640fbcd0 msgr2=0x7fb6640ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:39.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65a59c700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6640fbcd0 0x7fb6640ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:39.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65a59c700 1 -- 172.21.15.94:0/2908052750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb654005040 con 0x7fb6640fcf80 2024-04-06T03:37:39.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65a59c700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fcf80 0x7fb6640fe2c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb64c002a80 tx=0x7fb64c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:39.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65bfff700 1 -- 172.21.15.94:0/2908052750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64c007cc0 con 0x7fb6640fcf80 2024-04-06T03:37:39.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6640fecc0 con 0x7fb6640fcf80 2024-04-06T03:37:39.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65bfff700 1 -- 172.21.15.94:0/2908052750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb64c00f040 con 0x7fb6640fcf80 2024-04-06T03:37:39.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb65bfff700 1 -- 172.21.15.94:0/2908052750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64c0096f0 con 0x7fb6640fcf80 2024-04-06T03:37:39.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.919+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6641159b0 con 0x7fb6640fcf80 2024-04-06T03:37:39.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.920+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb664101b10 con 0x7fb6640fcf80 2024-04-06T03:37:39.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.921+0000 7fb65bfff700 1 -- 172.21.15.94:0/2908052750 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb64c00d690 con 0x7fb6640fcf80 2024-04-06T03:37:39.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.921+0000 7fb65bfff700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb6500692a0 0x7fb65006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:39.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.921+0000 7fb65bfff700 1 -- 172.21.15.94:0/2908052750 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb64c086610 con 0x7fb6640fcf80 2024-04-06T03:37:39.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.921+0000 7fb662d9d700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb6500692a0 0x7fb65006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:39.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.922+0000 7fb662d9d700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb6500692a0 0x7fb65006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb65400b970 tx=0x7fb654009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:39.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:39.926+0000 7fb65bfff700 1 -- 172.21.15.94:0/2908052750 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb64c04d190 con 0x7fb6640fcf80 2024-04-06T03:37:40.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.118+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb664104970 con 0x7fb6500692a0 2024-04-06T03:37:40.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.122+0000 7fb65bfff700 1 -- 172.21.15.94:0/2908052750 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7fb654007cc0 con 0x7fb6500692a0 2024-04-06T03:37:40.124 INFO:teuthology.orchestra.run.smithi094.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-06T03:37:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-06T03:37:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:crash 2/2 2m ago 5m * 2024-04-06T03:37:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-04-06T03:37:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:mgr 2/2 2m ago 5m count:2 2024-04-06T03:37:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:mon 2/2 2m ago 4m smithi094:172.21.15.94=smithi094;smithi136:172.21.15.136=smithi136;count:2 2024-04-06T03:37:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-06T03:37:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd 6 2m ago - 2024-04-06T03:37:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-04-06T03:37:40.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb6500692a0 msgr2=0x7fb65006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb6500692a0 0x7fb65006b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb65400b970 tx=0x7fb654009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fcf80 msgr2=0x7fb6640fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fcf80 0x7fb6640fe2c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb64c002a80 tx=0x7fb64c0046b0 comp rx=0 tx=0).stop 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 shutdown_connections 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb6500692a0 0x7fb65006b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6640fbcd0 0x7fb6640ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 --2- 172.21.15.94:0/2908052750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6640fcf80 0x7fb6640fe2c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 >> 172.21.15.94:0/2908052750 conn(0x7fb6640f7430 msgr2=0x7fb6640f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 shutdown_connections 2024-04-06T03:37:40.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:40.124+0000 7fb6690c7700 1 -- 172.21.15.94:0/2908052750 wait complete. 2024-04-06T03:37:40.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:40 smithi136 ceph-mon[36436]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:40.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:40 smithi094 ceph-mon[29713]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:40.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-04-06T03:37:41.062 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:41.444 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:41 smithi094 ceph-mon[29713]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:41.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:41 smithi136 ceph-mon[36436]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:42.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 -- 172.21.15.94:0/63206656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fbcd0 msgr2=0x7ff2780fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:42.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 --2- 172.21.15.94:0/63206656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fbcd0 0x7ff2780fc0e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff268004940 tx=0x7ff268009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:42.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 -- 172.21.15.94:0/63206656 shutdown_connections 2024-04-06T03:37:42.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 --2- 172.21.15.94:0/63206656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2780fcf80 0x7ff2780fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:42.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 --2- 172.21.15.94:0/63206656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fbcd0 0x7ff2780fc0e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:42.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 -- 172.21.15.94:0/63206656 >> 172.21.15.94:0/63206656 conn(0x7ff2780f7430 msgr2=0x7ff2780f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:42.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 -- 172.21.15.94:0/63206656 shutdown_connections 2024-04-06T03:37:42.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 -- 172.21.15.94:0/63206656 wait complete. 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 Processor -- start 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.506+0000 7ff27e947700 1 -- start start 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff27e947700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2780fbcd0 0x7ff278110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff27e947700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fcf80 0x7ff278112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff27e947700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780663e0 con 0x7ff2780fcf80 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff27e947700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278066520 con 0x7ff2780fbcd0 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff2777fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fcf80 0x7ff278112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff2777fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fcf80 0x7ff278112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53490/0 (socket says 172.21.15.94:53490) 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff2777fe700 1 -- 172.21.15.94:0/4018592024 learned_addr learned my addr 172.21.15.94:0/4018592024 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff277fff700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2780fbcd0 0x7ff278110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff2777fe700 1 -- 172.21.15.94:0/4018592024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2780fbcd0 msgr2=0x7ff278110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff2777fe700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2780fbcd0 0x7ff278110730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff2777fe700 1 -- 172.21.15.94:0/4018592024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff268005040 con 0x7ff2780fcf80 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff2777fe700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fcf80 0x7ff278112c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00b790 tx=0x7ff26c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:42.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff274ff9700 1 -- 172.21.15.94:0/4018592024 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26c00f770 con 0x7ff2780fcf80 2024-04-06T03:37:42.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff274ff9700 1 -- 172.21.15.94:0/4018592024 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff26c00fda0 con 0x7ff2780fcf80 2024-04-06T03:37:42.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff274ff9700 1 -- 172.21.15.94:0/4018592024 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26c00d310 con 0x7ff2780fcf80 2024-04-06T03:37:42.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.507+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2781132e0 con 0x7ff2780fcf80 2024-04-06T03:37:42.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.508+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff278113820 con 0x7ff2780fcf80 2024-04-06T03:37:42.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.508+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27805f5f0 con 0x7ff2780fcf80 2024-04-06T03:37:42.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.509+0000 7ff274ff9700 1 -- 172.21.15.94:0/4018592024 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff26c010420 con 0x7ff2780fcf80 2024-04-06T03:37:42.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.509+0000 7ff274ff9700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff2600692a0 0x7ff26006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:42.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.509+0000 7ff274ff9700 1 -- 172.21.15.94:0/4018592024 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff26c086e70 con 0x7ff2780fcf80 2024-04-06T03:37:42.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.509+0000 7ff277fff700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff2600692a0 0x7ff26006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:42.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.510+0000 7ff277fff700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff2600692a0 0x7ff26006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b970 tx=0x7ff268009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:42.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.514+0000 7ff274ff9700 1 -- 172.21.15.94:0/4018592024 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff26c0534e0 con 0x7ff2780fcf80 2024-04-06T03:37:42.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:42 smithi136 ceph-mon[36436]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:42.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:42 smithi094 ceph-mon[29713]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:42.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.706+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff278101220 con 0x7ff2600692a0 2024-04-06T03:37:42.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.707+0000 7ff274ff9700 1 -- 172.21.15.94:0/4018592024 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7ff278101220 con 0x7ff2600692a0 2024-04-06T03:37:42.709 INFO:teuthology.orchestra.run.smithi094.stdout:HOST ADDR LABELS STATUS 2024-04-06T03:37:42.710 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 172.21.15.94 2024-04-06T03:37:42.710 INFO:teuthology.orchestra.run.smithi094.stdout:smithi136 172.21.15.136 2024-04-06T03:37:42.710 INFO:teuthology.orchestra.run.smithi094.stdout:2 hosts in cluster 2024-04-06T03:37:42.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.709+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff2600692a0 msgr2=0x7ff26006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:42.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.709+0000 7ff27e947700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff2600692a0 0x7ff26006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b970 tx=0x7ff268009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:42.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.709+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fcf80 msgr2=0x7ff278112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:42.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.709+0000 7ff27e947700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fcf80 0x7ff278112c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00b790 tx=0x7ff26c009250 comp rx=0 tx=0).stop 2024-04-06T03:37:42.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.709+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 shutdown_connections 2024-04-06T03:37:42.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.709+0000 7ff27e947700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff2600692a0 0x7ff26006b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:42.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.709+0000 7ff27e947700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2780fbcd0 0x7ff278110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:42.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.709+0000 7ff27e947700 1 --2- 172.21.15.94:0/4018592024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2780fcf80 0x7ff278112c80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:42.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.709+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 >> 172.21.15.94:0/4018592024 conn(0x7ff2780f7430 msgr2=0x7ff2780f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:42.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.710+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 shutdown_connections 2024-04-06T03:37:42.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:42.710+0000 7ff27e947700 1 -- 172.21.15.94:0/4018592024 wait complete. 2024-04-06T03:37:43.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:37:43.574 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-04-06T03:37:43.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:37:43.830 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:44.578 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:44 smithi094 ceph-mon[29713]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:44.578 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:44 smithi094 ceph-mon[29713]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:44.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:44 smithi136 ceph-mon[36436]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:44.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:44 smithi136 ceph-mon[36436]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:45.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 -- 172.21.15.94:0/1106219784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fe040 msgr2=0x7f35880fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:45.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 --2- 172.21.15.94:0/1106219784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fe040 0x7f35880fe4b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3578004940 tx=0x7f3578009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:45.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 -- 172.21.15.94:0/1106219784 shutdown_connections 2024-04-06T03:37:45.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 --2- 172.21.15.94:0/1106219784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fe040 0x7f35880fe4b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:45.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 --2- 172.21.15.94:0/1106219784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35880fce40 0x7f35880fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:45.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 -- 172.21.15.94:0/1106219784 >> 172.21.15.94:0/1106219784 conn(0x7f35880f8570 msgr2=0x7f35880fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:45.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 -- 172.21.15.94:0/1106219784 shutdown_connections 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 -- 172.21.15.94:0/1106219784 wait complete. 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 Processor -- start 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.318+0000 7f358faf8700 1 -- start start 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358faf8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fce40 0x7f358810da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358faf8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35880fe040 0x7f358810dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358faf8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358810e5d0 con 0x7f35880fce40 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358faf8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358810edd0 con 0x7f35880fe040 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358d894700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fce40 0x7f358810da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358d894700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fce40 0x7f358810da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53500/0 (socket says 172.21.15.94:53500) 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358d894700 1 -- 172.21.15.94:0/2448361407 learned_addr learned my addr 172.21.15.94:0/2448361407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358d093700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35880fe040 0x7f358810dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358d894700 1 -- 172.21.15.94:0/2448361407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35880fe040 msgr2=0x7f358810dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358d894700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35880fe040 0x7f358810dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358d894700 1 -- 172.21.15.94:0/2448361407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578005040 con 0x7f35880fce40 2024-04-06T03:37:45.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358d093700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35880fe040 0x7f358810dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:37:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358d894700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fce40 0x7f358810da90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f358000cae0 tx=0x7f3580007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f357e7fc700 1 -- 172.21.15.94:0/2448361407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3580017070 con 0x7f35880fce40 2024-04-06T03:37:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358810f0b0 con 0x7f35880fce40 2024-04-06T03:37:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f357e7fc700 1 -- 172.21.15.94:0/2448361407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3580004210 con 0x7f35880fce40 2024-04-06T03:37:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f357e7fc700 1 -- 172.21.15.94:0/2448361407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3580003c70 con 0x7f35880fce40 2024-04-06T03:37:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.319+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358810f5f0 con 0x7f35880fce40 2024-04-06T03:37:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.320+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358805f5f0 con 0x7f35880fce40 2024-04-06T03:37:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.321+0000 7f357e7fc700 1 -- 172.21.15.94:0/2448361407 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3580003430 con 0x7f35880fce40 2024-04-06T03:37:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.321+0000 7f357e7fc700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35740692a0 0x7f357406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.321+0000 7f357e7fc700 1 -- 172.21.15.94:0/2448361407 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3580087030 con 0x7f35880fce40 2024-04-06T03:37:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.321+0000 7f358d093700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35740692a0 0x7f357406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:45.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.321+0000 7f358d093700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35740692a0 0x7f357406b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f357800b3b0 tx=0x7f3578009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:45.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.325+0000 7f357e7fc700 1 -- 172.21.15.94:0/2448361407 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f35800570b0 con 0x7f35880fce40 2024-04-06T03:37:45.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.520+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f358810f9e0 con 0x7f35740692a0 2024-04-06T03:37:45.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.522+0000 7f357e7fc700 1 -- 172.21.15.94:0/2448361407 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f358810f9e0 con 0x7f35740692a0 2024-04-06T03:37:45.524 INFO:teuthology.orchestra.run.smithi094.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-06T03:37:45.524 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-06T03:37:45.524 INFO:teuthology.orchestra.run.smithi094.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 67s ago Has a FileSystem, LVM detected 2024-04-06T03:37:45.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.523+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35740692a0 msgr2=0x7f357406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:45.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35740692a0 0x7f357406b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f357800b3b0 tx=0x7f3578009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:45.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fce40 msgr2=0x7f358810da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:45.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fce40 0x7f358810da90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f358000cae0 tx=0x7f3580007f60 comp rx=0 tx=0).stop 2024-04-06T03:37:45.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 shutdown_connections 2024-04-06T03:37:45.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f35740692a0 0x7f357406b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:45.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35880fce40 0x7f358810da90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:45.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 --2- 172.21.15.94:0/2448361407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35880fe040 0x7f358810dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:45.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 >> 172.21.15.94:0/2448361407 conn(0x7f35880f8570 msgr2=0x7f35880f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:45.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 shutdown_connections 2024-04-06T03:37:45.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:45.524+0000 7f358faf8700 1 -- 172.21.15.94:0/2448361407 wait complete. 2024-04-06T03:37:46.127 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-06T03:37:46.137 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:37:46.137 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-06T03:37:46.391 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:46.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:46 smithi136 ceph-mon[36436]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:46.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:46 smithi136 ceph-mon[36436]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:46.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:46 smithi094 ceph-mon[29713]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:46.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:46 smithi094 ceph-mon[29713]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.876+0000 7f86457e3700 1 -- 172.21.15.94:0/4177567876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 msgr2=0x7f86400ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.876+0000 7f86457e3700 1 --2- 172.21.15.94:0/4177567876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 0x7f86400ffa40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8630004940 tx=0x7f8630009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.876+0000 7f86457e3700 1 -- 172.21.15.94:0/4177567876 shutdown_connections 2024-04-06T03:37:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.876+0000 7f86457e3700 1 --2- 172.21.15.94:0/4177567876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86400fff80 0x7f8640102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.876+0000 7f86457e3700 1 --2- 172.21.15.94:0/4177567876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 0x7f86400ffa40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.876+0000 7f86457e3700 1 -- 172.21.15.94:0/4177567876 >> 172.21.15.94:0/4177567876 conn(0x7f86400f7410 msgr2=0x7f86400f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.876+0000 7f86457e3700 1 -- 172.21.15.94:0/4177567876 shutdown_connections 2024-04-06T03:37:47.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.876+0000 7f86457e3700 1 -- 172.21.15.94:0/4177567876 wait complete. 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f86457e3700 1 Processor -- start 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f86457e3700 1 -- start start 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f86457e3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 0x7f86400fd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f86457e3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86400fff80 0x7f86400fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f86457e3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400fbef0 con 0x7f8640066af0 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f86457e3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400fc060 con 0x7f86400fff80 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f863effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 0x7f86400fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f863effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 0x7f86400fd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58108/0 (socket says 172.21.15.94:58108) 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f863effd700 1 -- 172.21.15.94:0/1438351594 learned_addr learned my addr 172.21.15.94:0/1438351594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f863effd700 1 -- 172.21.15.94:0/1438351594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86400fff80 msgr2=0x7f86400fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f863effd700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86400fff80 0x7f86400fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f863effd700 1 -- 172.21.15.94:0/1438351594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8630005040 con 0x7f8640066af0 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.877+0000 7f863effd700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 0x7f86400fd470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8630000c00 tx=0x7f8630009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.878+0000 7f86277fe700 1 -- 172.21.15.94:0/1438351594 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f863000e040 con 0x7f8640066af0 2024-04-06T03:37:47.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.878+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86400fc2e0 con 0x7f8640066af0 2024-04-06T03:37:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.878+0000 7f86277fe700 1 -- 172.21.15.94:0/1438351594 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f863000bcc0 con 0x7f8640066af0 2024-04-06T03:37:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.878+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8640115a80 con 0x7f8640066af0 2024-04-06T03:37:47.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.878+0000 7f86277fe700 1 -- 172.21.15.94:0/1438351594 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f863001c630 con 0x7f8640066af0 2024-04-06T03:37:47.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.879+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864005f5f0 con 0x7f8640066af0 2024-04-06T03:37:47.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.879+0000 7f86277fe700 1 -- 172.21.15.94:0/1438351594 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8630013980 con 0x7f8640066af0 2024-04-06T03:37:47.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.880+0000 7f86277fe700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f86280692a0 0x7f862806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:47.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.880+0000 7f863e7fc700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f86280692a0 0x7f862806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:47.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.880+0000 7f86277fe700 1 -- 172.21.15.94:0/1438351594 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8630088230 con 0x7f8640066af0 2024-04-06T03:37:47.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.880+0000 7f863e7fc700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f86280692a0 0x7f862806b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8634004c10 tx=0x7f8634007d60 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:47.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:47.884+0000 7f86277fe700 1 -- 172.21.15.94:0/1438351594 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8630054950 con 0x7f8640066af0 2024-04-06T03:37:48.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:48.098+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f86400f8f70 con 0x7f86280692a0 2024-04-06T03:37:48.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:48 smithi136 ceph-mon[36436]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:48.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:48 smithi094 ceph-mon[29713]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:49.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.418+0000 7f86277fe700 1 -- 172.21.15.94:0/1438351594 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f86400f8f70 con 0x7f86280692a0 2024-04-06T03:37:49.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f86280692a0 msgr2=0x7f862806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f86280692a0 0x7f862806b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8634004c10 tx=0x7f8634007d60 comp rx=0 tx=0).stop 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 msgr2=0x7f86400fd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 0x7f86400fd470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8630000c00 tx=0x7f8630009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 shutdown_connections 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f86280692a0 0x7f862806b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8640066af0 0x7f86400fd470 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 --2- 172.21.15.94:0/1438351594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86400fff80 0x7f86400fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 >> 172.21.15.94:0/1438351594 conn(0x7f86400f7410 msgr2=0x7f86400f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 shutdown_connections 2024-04-06T03:37:49.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:49.422+0000 7f86457e3700 1 -- 172.21.15.94:0/1438351594 wait complete. 2024-04-06T03:37:49.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:49 smithi136 ceph-mon[36436]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:49.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-06T03:37:49.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-06T03:37:49.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:49 smithi136 ceph-mon[36436]: osdmap e41: 6 total, 6 up, 6 in 2024-04-06T03:37:49.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-06T03:37:49.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:49 smithi094 ceph-mon[29713]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:37:49.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-06T03:37:49.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-06T03:37:49.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:49 smithi094 ceph-mon[29713]: osdmap e41: 6 total, 6 up, 6 in 2024-04-06T03:37:49.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-06T03:37:49.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:49 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[29709]: 2024-04-06T03:37:49.395+0000 7f4528a40700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:37:50.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: pgmap v167: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: osdmap e42: 6 total, 6 up, 6 in 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: osdmap e43: 6 total, 6 up, 6 in 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: fsmap cephfs:0 2024-04-06T03:37:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: Saving service mds.cephfs spec with placement count:4 2024-04-06T03:37:50.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:50.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:37:50.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:50.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:37:50.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:50.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.qogyzw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:37:50.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.qogyzw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-06T03:37:50.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:50 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: pgmap v167: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: osdmap e42: 6 total, 6 up, 6 in 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: osdmap e43: 6 total, 6 up, 6 in 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: fsmap cephfs:0 2024-04-06T03:37:50.609 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: Saving service mds.cephfs spec with placement count:4 2024-04-06T03:37:50.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:50.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:37:50.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:50.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:37:50.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:50.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.qogyzw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:37:50.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.qogyzw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-06T03:37:50.610 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:50 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:51.309 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:51.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:51 smithi094 ceph-mon[29713]: Deploying daemon mds.cephfs.smithi094.qogyzw on smithi094 2024-04-06T03:37:51.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:51 smithi094 ceph-mon[29713]: osdmap e44: 6 total, 6 up, 6 in 2024-04-06T03:37:51.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:51 smithi136 ceph-mon[36436]: Deploying daemon mds.cephfs.smithi094.qogyzw on smithi094 2024-04-06T03:37:51.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:51 smithi136 ceph-mon[36436]: osdmap e44: 6 total, 6 up, 6 in 2024-04-06T03:37:52.620 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:52 smithi094 ceph-mon[29713]: pgmap v171: 65 pgs: 5 creating+peering, 1 active+clean, 59 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:52.620 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:52 smithi094 ceph-mon[29713]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-06T03:37:52.620 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:52 smithi094 ceph-mon[29713]: osdmap e45: 6 total, 6 up, 6 in 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 -- 172.21.15.94:0/3439207845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 msgr2=0x7f675c0fc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 --2- 172.21.15.94:0/3439207845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 0x7f675c0fc580 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6758004940 tx=0x7f6758009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 -- 172.21.15.94:0/3439207845 shutdown_connections 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 --2- 172.21.15.94:0/3439207845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f675c0fcac0 0x7f675c0faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 --2- 172.21.15.94:0/3439207845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 0x7f675c0fc580 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 -- 172.21.15.94:0/3439207845 >> 172.21.15.94:0/3439207845 conn(0x7f675c0f6be0 msgr2=0x7f675c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 -- 172.21.15.94:0/3439207845 shutdown_connections 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 -- 172.21.15.94:0/3439207845 wait complete. 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 Processor -- start 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 -- start start 2024-04-06T03:37:52.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 0x7f675c113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f675c0fcac0 0x7f675c114500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c114b00 con 0x7f675c0fc170 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.814+0000 7f6763ad3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c114c70 con 0x7f675c0fcac0 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f676186f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 0x7f675c113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f676186f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 0x7f675c113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58124/0 (socket says 172.21.15.94:58124) 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f676186f700 1 -- 172.21.15.94:0/1609854750 learned_addr learned my addr 172.21.15.94:0/1609854750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f676106e700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f675c0fcac0 0x7f675c114500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f676186f700 1 -- 172.21.15.94:0/1609854750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f675c0fcac0 msgr2=0x7f675c114500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f676186f700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f675c0fcac0 0x7f675c114500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f676186f700 1 -- 172.21.15.94:0/1609854750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758005040 con 0x7f675c0fc170 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f676186f700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 0x7f675c113fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f675800b3b0 tx=0x7f6758009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f67527fc700 1 -- 172.21.15.94:0/1609854750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f675800e040 con 0x7f675c0fc170 2024-04-06T03:37:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c119620 con 0x7f675c0fc170 2024-04-06T03:37:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f67527fc700 1 -- 172.21.15.94:0/1609854750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f675800bcc0 con 0x7f675c0fc170 2024-04-06T03:37:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f67527fc700 1 -- 172.21.15.94:0/1609854750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f675801c630 con 0x7f675c0fc170 2024-04-06T03:37:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.815+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c119be0 con 0x7f675c0fc170 2024-04-06T03:37:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.816+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675c04c710 con 0x7f675c0fc170 2024-04-06T03:37:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.817+0000 7f67527fc700 1 -- 172.21.15.94:0/1609854750 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6758013980 con 0x7f675c0fc170 2024-04-06T03:37:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.817+0000 7f67527fc700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f67480692a0 0x7f674806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.817+0000 7f67527fc700 1 -- 172.21.15.94:0/1609854750 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6758087f00 con 0x7f675c0fc170 2024-04-06T03:37:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.817+0000 7f676106e700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f67480692a0 0x7f674806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.818+0000 7f676106e700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f67480692a0 0x7f674806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f674c0041d0 tx=0x7f674c009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:52.822+0000 7f67527fc700 1 -- 172.21.15.94:0/1609854750 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6758057cc0 con 0x7f675c0fc170 2024-04-06T03:37:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:52 smithi136 ceph-mon[36436]: pgmap v171: 65 pgs: 5 creating+peering, 1 active+clean, 59 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:52 smithi136 ceph-mon[36436]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-06T03:37:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:52 smithi136 ceph-mon[36436]: osdmap e45: 6 total, 6 up, 6 in 2024-04-06T03:37:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.059+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f675c11a030 con 0x7f675c0fc170 2024-04-06T03:37:53.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.060+0000 7f67527fc700 1 -- 172.21.15.94:0/1609854750 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f6758057850 con 0x7f675c0fc170 2024-04-06T03:37:53.062 INFO:teuthology.orchestra.run.smithi094.stdout:e2 2024-04-06T03:37:53.062 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:37:53.062 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 2 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:37:49.396319+0000 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:37:53.063 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:in 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:up {} 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 0 2024-04-06T03:37:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:37:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:37:53.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f67480692a0 msgr2=0x7f674806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:53.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f67480692a0 0x7f674806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f674c0041d0 tx=0x7f674c009040 comp rx=0 tx=0).stop 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 msgr2=0x7f675c113fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 0x7f675c113fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f675800b3b0 tx=0x7f6758009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 shutdown_connections 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f67480692a0 0x7f674806b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f675c0fc170 0x7f675c113fc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 --2- 172.21.15.94:0/1609854750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f675c0fcac0 0x7f675c114500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 >> 172.21.15.94:0/1609854750 conn(0x7f675c0f6be0 msgr2=0x7f675c0f8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 shutdown_connections 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:53.062+0000 7f6763ad3700 1 -- 172.21.15.94:0/1609854750 wait complete. 2024-04-06T03:37:53.066 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-04-06T03:37:53.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:53 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1609854750' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:37:53.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:53 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1609854750' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:37:54.004 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-06T03:37:54.019 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:37:54.020 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-06T03:37:54.275 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:37:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: pgmap v173: 65 pgs: 5 creating+peering, 47 active+clean, 13 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: daemon mds.cephfs.smithi094.qogyzw assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-06T03:37:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:37:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-06T03:37:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: Cluster is now healthy 2024-04-06T03:37:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: mds.? [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] up:boot 2024-04-06T03:37:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:creating} 2024-04-06T03:37:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:37:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: daemon mds.cephfs.smithi094.qogyzw is now active in filesystem cephfs as rank 0 2024-04-06T03:37:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.hbjgtw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:37:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.hbjgtw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-06T03:37:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:54 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:54.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: pgmap v173: 65 pgs: 5 creating+peering, 47 active+clean, 13 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:54.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: daemon mds.cephfs.smithi094.qogyzw assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-06T03:37:54.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:37:54.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-06T03:37:54.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: Cluster is now healthy 2024-04-06T03:37:54.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: mds.? [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] up:boot 2024-04-06T03:37:54.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:creating} 2024-04-06T03:37:54.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:37:54.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:54.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: daemon mds.cephfs.smithi094.qogyzw is now active in filesystem cephfs as rank 0 2024-04-06T03:37:54.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:54.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:54.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.hbjgtw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:37:54.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.hbjgtw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-06T03:37:54.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:54 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:55.724 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:55 smithi094 ceph-mon[29713]: Deploying daemon mds.cephfs.smithi136.hbjgtw on smithi136 2024-04-06T03:37:55.724 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:55 smithi094 ceph-mon[29713]: mds.? [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] up:active 2024-04-06T03:37:55.725 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:55 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 2024-04-06T03:37:55.725 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:55 smithi136 ceph-mon[36436]: Deploying daemon mds.cephfs.smithi136.hbjgtw on smithi136 2024-04-06T03:37:55.725 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:55 smithi136 ceph-mon[36436]: mds.? [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] up:active 2024-04-06T03:37:55.725 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:55 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 2024-04-06T03:37:55.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.824+0000 7fe012a77700 1 -- 172.21.15.94:0/55837538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fbcd0 msgr2=0x7fe00c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:55.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.824+0000 7fe012a77700 1 --2- 172.21.15.94:0/55837538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fbcd0 0x7fe00c0fc0e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe000004940 tx=0x7fe000009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:55.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.824+0000 7fe012a77700 1 -- 172.21.15.94:0/55837538 shutdown_connections 2024-04-06T03:37:55.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.824+0000 7fe012a77700 1 --2- 172.21.15.94:0/55837538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe00c0fcf80 0x7fe00c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:55.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.824+0000 7fe012a77700 1 --2- 172.21.15.94:0/55837538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fbcd0 0x7fe00c0fc0e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:55.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.824+0000 7fe012a77700 1 -- 172.21.15.94:0/55837538 >> 172.21.15.94:0/55837538 conn(0x7fe00c0f7430 msgr2=0x7fe00c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:55.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.824+0000 7fe012a77700 1 -- 172.21.15.94:0/55837538 shutdown_connections 2024-04-06T03:37:55.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.824+0000 7fe012a77700 1 -- 172.21.15.94:0/55837538 wait complete. 2024-04-06T03:37:55.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe012a77700 1 Processor -- start 2024-04-06T03:37:55.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe012a77700 1 -- start start 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe012a77700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe00c0fbcd0 0x7fe00c0ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe012a77700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fcf80 0x7fe00c0fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe012a77700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c100210 con 0x7fe00c0fcf80 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe012a77700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0fe800 con 0x7fe00c0fbcd0 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe00bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fcf80 0x7fe00c0fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe00bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fcf80 0x7fe00c0fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58166/0 (socket says 172.21.15.94:58166) 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe00bfff700 1 -- 172.21.15.94:0/671288144 learned_addr learned my addr 172.21.15.94:0/671288144 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.825+0000 7fe010813700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe00c0fbcd0 0x7fe00c0ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.826+0000 7fe00bfff700 1 -- 172.21.15.94:0/671288144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe00c0fbcd0 msgr2=0x7fe00c0ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:55.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.826+0000 7fe00bfff700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe00c0fbcd0 0x7fe00c0ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:55.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.826+0000 7fe00bfff700 1 -- 172.21.15.94:0/671288144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000005040 con 0x7fe00c0fcf80 2024-04-06T03:37:55.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.826+0000 7fe00bfff700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fcf80 0x7fe00c0fe2c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdff8002a50 tx=0x7fdff80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:55.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.826+0000 7fe0097fa700 1 -- 172.21.15.94:0/671288144 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdff8007cc0 con 0x7fe00c0fcf80 2024-04-06T03:37:55.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.826+0000 7fe0097fa700 1 -- 172.21.15.94:0/671288144 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdff800f040 con 0x7fe00c0fcf80 2024-04-06T03:37:55.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.826+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c0feab0 con 0x7fe00c0fcf80 2024-04-06T03:37:55.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.826+0000 7fe0097fa700 1 -- 172.21.15.94:0/671288144 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdff800da30 con 0x7fe00c0fcf80 2024-04-06T03:37:55.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.826+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c0ff070 con 0x7fe00c0fcf80 2024-04-06T03:37:55.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.827+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00c101b10 con 0x7fe00c0fcf80 2024-04-06T03:37:55.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.827+0000 7fe0097fa700 1 -- 172.21.15.94:0/671288144 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdff800db90 con 0x7fe00c0fcf80 2024-04-06T03:37:55.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.828+0000 7fe0097fa700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdffc0692a0 0x7fdffc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:37:55.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.828+0000 7fe010813700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdffc0692a0 0x7fdffc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:37:55.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.828+0000 7fe0097fa700 1 -- 172.21.15.94:0/671288144 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fdff8086f40 con 0x7fe00c0fcf80 2024-04-06T03:37:55.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.828+0000 7fe010813700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdffc0692a0 0x7fdffc06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe00000b970 tx=0x7fe000009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:37:55.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:55.832+0000 7fe0097fa700 1 -- 172.21.15.94:0/671288144 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdff8056c80 con 0x7fe00c0fcf80 2024-04-06T03:37:56.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.063+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7fe00c02b390 con 0x7fe00c0fcf80 2024-04-06T03:37:56.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.662+0000 7fe0097fa700 1 -- 172.21.15.94:0/671288144 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v5) v1 ==== 105+0+0 (secure 0 0 0) 0x7fdff8056810 con 0x7fe00c0fcf80 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdffc0692a0 msgr2=0x7fdffc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdffc0692a0 0x7fdffc06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe00000b970 tx=0x7fe000009e30 comp rx=0 tx=0).stop 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fcf80 msgr2=0x7fe00c0fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fcf80 0x7fe00c0fe2c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdff8002a50 tx=0x7fdff80046b0 comp rx=0 tx=0).stop 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 shutdown_connections 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdffc0692a0 0x7fdffc06b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe00c0fbcd0 0x7fe00c0ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 --2- 172.21.15.94:0/671288144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe00c0fcf80 0x7fe00c0fe2c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 >> 172.21.15.94:0/671288144 conn(0x7fe00c0f7430 msgr2=0x7fe00c066850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 shutdown_connections 2024-04-06T03:37:56.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:37:56.664+0000 7fe012a77700 1 -- 172.21.15.94:0/671288144 wait complete. 2024-04-06T03:37:56.678 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:56 smithi094 ceph-mon[29713]: pgmap v174: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:56.678 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:56 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/671288144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-06T03:37:56.678 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:56 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:56.678 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:56 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:56.678 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:56 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:56.678 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:56 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.kvxcqo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:37:56.678 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:56 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.kvxcqo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-06T03:37:56.678 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:56 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:56 smithi136 ceph-mon[36436]: pgmap v174: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:37:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:56 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/671288144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-06T03:37:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:56 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:56 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:56 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:37:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:56 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.kvxcqo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:37:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:56 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.kvxcqo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-06T03:37:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:56 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:37:57.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:57 smithi136 ceph-mon[36436]: Deploying daemon mds.cephfs.smithi094.kvxcqo on smithi094 2024-04-06T03:37:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:57 smithi136 ceph-mon[36436]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:boot 2024-04-06T03:37:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:57 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/671288144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-06T03:37:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:57 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 1 up:standby 2024-04-06T03:37:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:57 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:37:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:57 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 1 up:standby 2024-04-06T03:37:57.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:57 smithi094 ceph-mon[29713]: Deploying daemon mds.cephfs.smithi094.kvxcqo on smithi094 2024-04-06T03:37:57.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:57 smithi094 ceph-mon[29713]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:boot 2024-04-06T03:37:57.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:57 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/671288144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-06T03:37:57.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:57 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 1 up:standby 2024-04-06T03:37:57.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:57 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:37:57.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:57 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 1 up:standby 2024-04-06T03:37:58.069 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-06T03:37:58.080 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:37:58.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-06T03:37:58.567 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:58 smithi094 ceph-mon[29713]: pgmap v175: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 807 B/s wr, 2 op/s 2024-04-06T03:37:58.567 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:58 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:37:58.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:58 smithi136 ceph-mon[36436]: pgmap v175: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 807 B/s wr, 2 op/s 2024-04-06T03:37:58.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:58 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:37:58.738 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:00.039 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.trdonf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.trdonf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: Deploying daemon mds.cephfs.smithi136.trdonf on smithi136 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: pgmap v176: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:boot 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 2 up:standby 2024-04-06T03:38:00.040 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:37:59 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:38:00.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:00.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:00.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:00.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.trdonf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:38:00.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.trdonf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-06T03:38:00.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:38:00.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: Deploying daemon mds.cephfs.smithi136.trdonf on smithi136 2024-04-06T03:38:00.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: pgmap v176: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-06T03:38:00.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:boot 2024-04-06T03:38:00.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 2 up:standby 2024-04-06T03:38:00.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:37:59 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:38:00.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.415+0000 7f0a11909700 1 -- 172.21.15.94:0/1599468137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 msgr2=0x7f0a0c0fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.415+0000 7f0a11909700 1 --2- 172.21.15.94:0/1599468137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 0x7f0a0c0fc110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09fc004940 tx=0x7f09fc009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 -- 172.21.15.94:0/1599468137 shutdown_connections 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 --2- 172.21.15.94:0/1599468137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a0c0fcfb0 0x7f0a0c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 --2- 172.21.15.94:0/1599468137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 0x7f0a0c0fc110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 -- 172.21.15.94:0/1599468137 >> 172.21.15.94:0/1599468137 conn(0x7f0a0c0f7480 msgr2=0x7f0a0c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 -- 172.21.15.94:0/1599468137 shutdown_connections 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 -- 172.21.15.94:0/1599468137 wait complete. 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 Processor -- start 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 -- start start 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 0x7f0a0c066d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a0c0fcfb0 0x7f0a0c112690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c112bd0 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a11909700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c112d10 con 0x7f0a0c0fcfb0 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a0affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 0x7f0a0c066d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a0affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 0x7f0a0c066d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40400/0 (socket says 172.21.15.94:40400) 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a0affd700 1 -- 172.21.15.94:0/902157966 learned_addr learned my addr 172.21.15.94:0/902157966 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.416+0000 7f0a0a7fc700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a0c0fcfb0 0x7f0a0c112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.417+0000 7f0a0affd700 1 -- 172.21.15.94:0/902157966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a0c0fcfb0 msgr2=0x7f0a0c112690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.417+0000 7f0a0affd700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a0c0fcfb0 0x7f0a0c112690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.417+0000 7f0a0affd700 1 -- 172.21.15.94:0/902157966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc005040 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.417+0000 7f0a0affd700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 0x7f0a0c066d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009150 tx=0x7f09fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.417+0000 7f09f37fe700 1 -- 172.21.15.94:0/902157966 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09fc00e040 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.417+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c112f90 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.417+0000 7f09f37fe700 1 -- 172.21.15.94:0/902157966 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09fc004290 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.417+0000 7f09f37fe700 1 -- 172.21.15.94:0/902157966 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09fc01c630 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.417+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1134c0 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.418+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c04c710 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.418+0000 7f09f37fe700 1 -- 172.21.15.94:0/902157966 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f09fc00bbf0 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.419+0000 7f09f37fe700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f09f40692a0 0x7f09f406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:00.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.419+0000 7f09f37fe700 1 -- 172.21.15.94:0/902157966 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f09fc087ea0 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.419+0000 7f0a0a7fc700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f09f40692a0 0x7f09f406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:00.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.419+0000 7f0a0a7fc700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f09f40692a0 0x7f09f406b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009fd0 tx=0x7f0a0000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:00.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.423+0000 7f09f37fe700 1 -- 172.21.15.94:0/902157966 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f09fc057c00 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.656+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0a0c1137f0 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.730+0000 7f09f37fe700 1 -- 172.21.15.94:0/902157966 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f09fc057790 con 0x7f0a0c0fbd00 2024-04-06T03:38:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f09f40692a0 msgr2=0x7f09f406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f09f40692a0 0x7f09f406b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009fd0 tx=0x7f0a0000b040 comp rx=0 tx=0).stop 2024-04-06T03:38:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 msgr2=0x7f0a0c066d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 0x7f0a0c066d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009150 tx=0x7f09fc009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 shutdown_connections 2024-04-06T03:38:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f09f40692a0 0x7f09f406b750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a0c0fbd00 0x7f0a0c066d70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:00.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 --2- 172.21.15.94:0/902157966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a0c0fcfb0 0x7f0a0c112690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:00.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 >> 172.21.15.94:0/902157966 conn(0x7f0a0c0f7480 msgr2=0x7f0a0c0f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:00.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 shutdown_connections 2024-04-06T03:38:00.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:00.732+0000 7f0a11909700 1 -- 172.21.15.94:0/902157966 wait complete. 2024-04-06T03:38:00.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:00 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/902157966' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-06T03:38:01.050 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:00 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/902157966' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-06T03:38:01.376 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-06T03:38:01.386 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:38:01.386 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph fs set cephfs inline_data false' 2024-04-06T03:38:01.674 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:02.013 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:01 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/902157966' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-06T03:38:02.013 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:01 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 2 up:standby 2024-04-06T03:38:02.013 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:01 smithi094 ceph-mon[29713]: pgmap v177: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-06T03:38:02.013 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:01 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.013 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:01 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.013 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:01 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.014 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:01 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.014 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:01 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.014 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:01 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:38:02.018 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:01 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/902157966' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-06T03:38:02.018 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:01 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 2 up:standby 2024-04-06T03:38:02.019 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:01 smithi136 ceph-mon[36436]: pgmap v177: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-06T03:38:02.019 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:01 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.019 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:01 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.019 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:01 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.019 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:01 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.019 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:01 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:02.019 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:01 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:38:02.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:02 smithi136 ceph-mon[36436]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:boot 2024-04-06T03:38:02.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:02 smithi136 ceph-mon[36436]: mds.? [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] up:active 2024-04-06T03:38:02.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:02 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:38:02.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:38:03.003 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:02 smithi094 ceph-mon[29713]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:boot 2024-04-06T03:38:03.004 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:02 smithi094 ceph-mon[29713]: mds.? [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] up:active 2024-04-06T03:38:03.004 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:02 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:38:03.004 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:38:03.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.307+0000 7f8027d30700 1 -- 172.21.15.94:0/3322248846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fe040 msgr2=0x7f80200fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:03.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.307+0000 7f8027d30700 1 --2- 172.21.15.94:0/3322248846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fe040 0x7f80200fe4b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8010004940 tx=0x7f8010009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:03.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.307+0000 7f8027d30700 1 -- 172.21.15.94:0/3322248846 shutdown_connections 2024-04-06T03:38:03.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.307+0000 7f8027d30700 1 --2- 172.21.15.94:0/3322248846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fe040 0x7f80200fe4b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:03.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.307+0000 7f8027d30700 1 --2- 172.21.15.94:0/3322248846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80200fce40 0x7f80200fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:03.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.307+0000 7f8027d30700 1 -- 172.21.15.94:0/3322248846 >> 172.21.15.94:0/3322248846 conn(0x7f80200f8570 msgr2=0x7f80200fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:03.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.307+0000 7f8027d30700 1 -- 172.21.15.94:0/3322248846 shutdown_connections 2024-04-06T03:38:03.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.307+0000 7f8027d30700 1 -- 172.21.15.94:0/3322248846 wait complete. 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8027d30700 1 Processor -- start 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8027d30700 1 -- start start 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8027d30700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fce40 0x7f8020071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8027d30700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80200fe040 0x7f8020070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8027d30700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020070950 con 0x7f80200fce40 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8027d30700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020070ac0 con 0x7f80200fe040 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8025acc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fce40 0x7f8020071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8025acc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fce40 0x7f8020071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40420/0 (socket says 172.21.15.94:40420) 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8025acc700 1 -- 172.21.15.94:0/772723663 learned_addr learned my addr 172.21.15.94:0/772723663 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f80252cb700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80200fe040 0x7f8020070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8025acc700 1 -- 172.21.15.94:0/772723663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80200fe040 msgr2=0x7f8020070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8025acc700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80200fe040 0x7f8020070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.308+0000 7f8025acc700 1 -- 172.21.15.94:0/772723663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8010005040 con 0x7f80200fce40 2024-04-06T03:38:03.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.309+0000 7f8025acc700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fce40 0x7f8020071e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f801c00b790 tx=0x7f801c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:03.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.310+0000 7f80167fc700 1 -- 172.21.15.94:0/772723663 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f801c00f6f0 con 0x7f80200fce40 2024-04-06T03:38:03.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.310+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8020070da0 con 0x7f80200fce40 2024-04-06T03:38:03.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.310+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8020115af0 con 0x7f80200fce40 2024-04-06T03:38:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.310+0000 7f80167fc700 1 -- 172.21.15.94:0/772723663 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f801c00fd20 con 0x7f80200fce40 2024-04-06T03:38:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.310+0000 7f80167fc700 1 -- 172.21.15.94:0/772723663 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f801c0103a0 con 0x7f80200fce40 2024-04-06T03:38:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.311+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80040052e0 con 0x7f80200fce40 2024-04-06T03:38:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.311+0000 7f80167fc700 1 -- 172.21.15.94:0/772723663 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f801c00f850 con 0x7f80200fce40 2024-04-06T03:38:03.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.312+0000 7f80167fc700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f800c0692b0 0x7f800c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:03.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.312+0000 7f80167fc700 1 -- 172.21.15.94:0/772723663 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f801c086ec0 con 0x7f80200fce40 2024-04-06T03:38:03.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.312+0000 7f80252cb700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f800c0692b0 0x7f800c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:03.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.312+0000 7f80252cb700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f800c0692b0 0x7f800c06b760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8010009fd0 tx=0x7f8010009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.316+0000 7f80167fc700 1 -- 172.21.15.94:0/772723663 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f801c056c80 con 0x7f80200fce40 2024-04-06T03:38:03.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:03.557+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f8004005b10 con 0x7f80200fce40 2024-04-06T03:38:03.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:03 smithi094 ceph-mon[29713]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:standby 2024-04-06T03:38:03.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:03 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:38:03.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:03 smithi094 ceph-mon[29713]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-06T03:38:03.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:03 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:03.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:03 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:03.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:03 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:03.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:03 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/772723663' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-06T03:38:04.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:03 smithi136 ceph-mon[36436]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:standby 2024-04-06T03:38:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:03 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:38:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:03 smithi136 ceph-mon[36436]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-06T03:38:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:03 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:03 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:03 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:03 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/772723663' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-06T03:38:04.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.311+0000 7f80167fc700 1 -- 172.21.15.94:0/772723663 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v11) v1 ==== 133+0+0 (secure 0 0 0) 0x7f801c056810 con 0x7f80200fce40 2024-04-06T03:38:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f800c0692b0 msgr2=0x7f800c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f800c0692b0 0x7f800c06b760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8010009fd0 tx=0x7f8010009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fce40 msgr2=0x7f8020071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fce40 0x7f8020071e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f801c00b790 tx=0x7f801c009220 comp rx=0 tx=0).stop 2024-04-06T03:38:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 shutdown_connections 2024-04-06T03:38:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f800c0692b0 0x7f800c06b760 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80200fce40 0x7f8020071e40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 --2- 172.21.15.94:0/772723663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80200fe040 0x7f8020070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 >> 172.21.15.94:0/772723663 conn(0x7f80200f8570 msgr2=0x7f80200f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 shutdown_connections 2024-04-06T03:38:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:04.313+0000 7f8027d30700 1 -- 172.21.15.94:0/772723663 wait complete. 2024-04-06T03:38:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:inline data disabled 2024-04-06T03:38:05.052 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-06T03:38:05.069 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:38:05.069 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-06T03:38:05.514 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:05.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:05 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/772723663' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-06T03:38:05.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:05 smithi136 ceph-mon[36436]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:standby 2024-04-06T03:38:05.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:05 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:38:05.638 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:05 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/772723663' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-06T03:38:05.638 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:05 smithi094 ceph-mon[29713]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:standby 2024-04-06T03:38:05.638 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:05 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:38:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:06 smithi136 ceph-mon[36436]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-06T03:38:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:06 smithi136 ceph-mon[36436]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:standby 2024-04-06T03:38:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:06 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:38:06.602 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:06 smithi094 ceph-mon[29713]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-06T03:38:06.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:06 smithi094 ceph-mon[29713]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:standby 2024-04-06T03:38:06.603 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:06 smithi094 ceph-mon[29713]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.154+0000 7f8c89ea8700 1 -- 172.21.15.94:0/625321752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fe500 msgr2=0x7f8c840fe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.154+0000 7f8c89ea8700 1 --2- 172.21.15.94:0/625321752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fe500 0x7f8c840fe910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8c74004940 tx=0x7f8c74009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.154+0000 7f8c89ea8700 1 -- 172.21.15.94:0/625321752 shutdown_connections 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.154+0000 7f8c89ea8700 1 --2- 172.21.15.94:0/625321752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c840fee50 0x7f8c840faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.154+0000 7f8c89ea8700 1 --2- 172.21.15.94:0/625321752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fe500 0x7f8c840fe910 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.154+0000 7f8c89ea8700 1 -- 172.21.15.94:0/625321752 >> 172.21.15.94:0/625321752 conn(0x7f8c840f6bc0 msgr2=0x7f8c840f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.154+0000 7f8c89ea8700 1 -- 172.21.15.94:0/625321752 shutdown_connections 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.154+0000 7f8c89ea8700 1 -- 172.21.15.94:0/625321752 wait complete. 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.155+0000 7f8c89ea8700 1 Processor -- start 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.155+0000 7f8c89ea8700 1 -- start start 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.155+0000 7f8c89ea8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c840fe500 0x7f8c8410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:08.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.155+0000 7f8c89ea8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fee50 0x7f8c8410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.155+0000 7f8c89ea8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8410e520 con 0x7f8c840fee50 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.155+0000 7f8c89ea8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c84112ec0 con 0x7f8c840fe500 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.155+0000 7f8c82ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fee50 0x7f8c8410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c82ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fee50 0x7f8c8410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37280/0 (socket says 172.21.15.94:37280) 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c82ffd700 1 -- 172.21.15.94:0/156022159 learned_addr learned my addr 172.21.15.94:0/156022159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c837fe700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c840fe500 0x7f8c8410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c82ffd700 1 -- 172.21.15.94:0/156022159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c840fe500 msgr2=0x7f8c8410d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c82ffd700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c840fe500 0x7f8c8410d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c82ffd700 1 -- 172.21.15.94:0/156022159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c74005040 con 0x7f8c840fee50 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c82ffd700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fee50 0x7f8c8410de90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800b790 tx=0x7f8c78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c6bfff700 1 -- 172.21.15.94:0/156022159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c7800f6f0 con 0x7f8c840fee50 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c6bfff700 1 -- 172.21.15.94:0/156022159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c7800fd20 con 0x7f8c840fee50 2024-04-06T03:38:08.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c84113120 con 0x7f8c840fee50 2024-04-06T03:38:08.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c6bfff700 1 -- 172.21.15.94:0/156022159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c780103a0 con 0x7f8c840fee50 2024-04-06T03:38:08.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.156+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c84113690 con 0x7f8c840fee50 2024-04-06T03:38:08.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.157+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8404c710 con 0x7f8c840fee50 2024-04-06T03:38:08.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.158+0000 7f8c6bfff700 1 -- 172.21.15.94:0/156022159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8c78010500 con 0x7f8c840fee50 2024-04-06T03:38:08.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.158+0000 7f8c6bfff700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8c6c0692a0 0x7f8c6c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:08.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.158+0000 7f8c6bfff700 1 -- 172.21.15.94:0/156022159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8c78086fa0 con 0x7f8c840fee50 2024-04-06T03:38:08.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.158+0000 7f8c837fe700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8c6c0692a0 0x7f8c6c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:08.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.159+0000 7f8c837fe700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8c6c0692a0 0x7f8c6c06b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400b3b0 tx=0x7f8c74009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:08.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.163+0000 7f8c6bfff700 1 -- 172.21.15.94:0/156022159 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8c78056d60 con 0x7f8c840fee50 2024-04-06T03:38:08.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.402+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8c84105990 con 0x7f8c840fee50 2024-04-06T03:38:08.405 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:08 smithi094 ceph-mon[29713]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-06T03:38:08.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.403+0000 7f8c6bfff700 1 -- 172.21.15.94:0/156022159 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7f8c780568f0 con 0x7f8c840fee50 2024-04-06T03:38:08.406 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:38:08.406 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:38:08.406 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:38:08.407 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:38:08.408 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:38:08.409 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:38:08.409 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:38:08.409 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:08.409 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:08.409 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:38:08.409 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:08.409 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:38:08.409 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:38:08.409 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:38:08.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8c6c0692a0 msgr2=0x7f8c6c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8c6c0692a0 0x7f8c6c06b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400b3b0 tx=0x7f8c74009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fee50 msgr2=0x7f8c8410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fee50 0x7f8c8410de90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800b790 tx=0x7f8c78009250 comp rx=0 tx=0).stop 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 shutdown_connections 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f8c6c0692a0 0x7f8c6c06b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c840fe500 0x7f8c8410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 --2- 172.21.15.94:0/156022159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c840fee50 0x7f8c8410de90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 >> 172.21.15.94:0/156022159 conn(0x7f8c840f6bc0 msgr2=0x7f8c840f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 shutdown_connections 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:08.406+0000 7f8c89ea8700 1 -- 172.21.15.94:0/156022159 wait complete. 2024-04-06T03:38:08.411 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:38:08.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:08 smithi136 ceph-mon[36436]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-06T03:38:08.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-06T03:38:09.345 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:09.587 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:09 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/156022159' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:38:09.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:09 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/156022159' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:38:10.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:10 smithi136 ceph-mon[36436]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 1 op/s 2024-04-06T03:38:10.623 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:10 smithi094 ceph-mon[29713]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 1 op/s 2024-04-06T03:38:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 -- 172.21.15.94:0/2637355043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fcf80 msgr2=0x7f5b380fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 --2- 172.21.15.94:0/2637355043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fcf80 0x7f5b380fd3f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5b28004940 tx=0x7f5b28009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 -- 172.21.15.94:0/2637355043 shutdown_connections 2024-04-06T03:38:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 --2- 172.21.15.94:0/2637355043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fcf80 0x7f5b380fd3f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 --2- 172.21.15.94:0/2637355043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b380fbcd0 0x7f5b380fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 -- 172.21.15.94:0/2637355043 >> 172.21.15.94:0/2637355043 conn(0x7f5b380f7450 msgr2=0x7f5b380f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 -- 172.21.15.94:0/2637355043 shutdown_connections 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 -- 172.21.15.94:0/2637355043 wait complete. 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 Processor -- start 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.948+0000 7f5b3fd14700 1 -- start start 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3fd14700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fbcd0 0x7f5b38066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3fd14700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b380fcf80 0x7f5b38112720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3fd14700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b38112c60 con 0x7f5b380fbcd0 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3fd14700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b38112da0 con 0x7f5b380fcf80 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3dab0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fbcd0 0x7f5b38066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3dab0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fbcd0 0x7f5b38066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37302/0 (socket says 172.21.15.94:37302) 2024-04-06T03:38:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3dab0700 1 -- 172.21.15.94:0/3740690490 learned_addr learned my addr 172.21.15.94:0/3740690490 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3d2af700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b380fcf80 0x7f5b38112720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3dab0700 1 -- 172.21.15.94:0/3740690490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b380fcf80 msgr2=0x7f5b38112720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3dab0700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b380fcf80 0x7f5b38112720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3dab0700 1 -- 172.21.15.94:0/3740690490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28005040 con 0x7f5b380fbcd0 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3dab0700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fbcd0 0x7f5b38066cd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5b34002a50 tx=0x7f5b340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b2e7fc700 1 -- 172.21.15.94:0/3740690490 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b34007cc0 con 0x7f5b380fbcd0 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b38112fa0 con 0x7f5b380fbcd0 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b2e7fc700 1 -- 172.21.15.94:0/3740690490 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b3400f040 con 0x7f5b380fbcd0 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b2e7fc700 1 -- 172.21.15.94:0/3740690490 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b3400da30 con 0x7f5b380fbcd0 2024-04-06T03:38:10.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.949+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b38113530 con 0x7f5b380fbcd0 2024-04-06T03:38:10.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.950+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3804c710 con 0x7f5b380fbcd0 2024-04-06T03:38:10.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.951+0000 7f5b2e7fc700 1 -- 172.21.15.94:0/3740690490 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5b3400db90 con 0x7f5b380fbcd0 2024-04-06T03:38:10.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.951+0000 7f5b2e7fc700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5b240692a0 0x7f5b2406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:10.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.951+0000 7f5b3d2af700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5b240692a0 0x7f5b2406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:10.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.951+0000 7f5b2e7fc700 1 -- 172.21.15.94:0/3740690490 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5b34086f40 con 0x7f5b380fbcd0 2024-04-06T03:38:10.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.952+0000 7f5b3d2af700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5b240692a0 0x7f5b2406b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800b970 tx=0x7f5b28009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:10.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:10.956+0000 7f5b2e7fc700 1 -- 172.21.15.94:0/3740690490 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5b34056c80 con 0x7f5b380fbcd0 2024-04-06T03:38:11.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.202+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b381047c0 con 0x7f5b380fbcd0 2024-04-06T03:38:11.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.202+0000 7f5b2e7fc700 1 -- 172.21.15.94:0/3740690490 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4686 (secure 0 0 0) 0x7f5b34056810 con 0x7f5b380fbcd0 2024-04-06T03:38:11.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5b240692a0 msgr2=0x7f5b2406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:11.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5b240692a0 0x7f5b2406b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800b970 tx=0x7f5b28009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:11.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fbcd0 msgr2=0x7f5b38066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:11.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fbcd0 0x7f5b38066cd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5b34002a50 tx=0x7f5b340046b0 comp rx=0 tx=0).stop 2024-04-06T03:38:11.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 shutdown_connections 2024-04-06T03:38:11.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5b240692a0 0x7f5b2406b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:11.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b380fbcd0 0x7f5b38066cd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:11.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 --2- 172.21.15.94:0/3740690490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b380fcf80 0x7f5b38112720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:11.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 >> 172.21.15.94:0/3740690490 conn(0x7f5b380f7450 msgr2=0x7f5b380f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:11.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.205+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 shutdown_connections 2024-04-06T03:38:11.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:11.206+0000 7f5b3fd14700 1 -- 172.21.15.94:0/3740690490 wait complete. 2024-04-06T03:38:11.209 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:38:11.218 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:38:11.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-06T03:38:12.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:12 smithi094 ceph-mon[29713]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-06T03:38:12.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:12.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:12.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:38:12.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:38:12.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:12.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:38:12.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:12 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3740690490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-06T03:38:12.294 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:12 smithi136 ceph-mon[36436]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-06T03:38:12.294 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:12.294 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:12.294 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:38:12.294 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:38:12.294 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:12.294 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:38:12.295 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:12 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3740690490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-06T03:38:12.665 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:13.114 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:38:13.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:38:14.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:14 smithi094 ceph-mon[29713]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-06T03:38:14.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:14 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:14.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:14 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:14.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:14 smithi136 ceph-mon[36436]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-06T03:38:14.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:14 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:14.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:14 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:14.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 -- 172.21.15.94:0/5897267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fe510 msgr2=0x7ff9980fe920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:14.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 --2- 172.21.15.94:0/5897267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fe510 0x7ff9980fe920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff988004940 tx=0x7ff988009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:14.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 -- 172.21.15.94:0/5897267 shutdown_connections 2024-04-06T03:38:14.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 --2- 172.21.15.94:0/5897267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9980fee60 0x7ff9980fafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:14.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 --2- 172.21.15.94:0/5897267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fe510 0x7ff9980fe920 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:14.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 -- 172.21.15.94:0/5897267 >> 172.21.15.94:0/5897267 conn(0x7ff9980f6c10 msgr2=0x7ff9980f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:14.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 -- 172.21.15.94:0/5897267 shutdown_connections 2024-04-06T03:38:14.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 -- 172.21.15.94:0/5897267 wait complete. 2024-04-06T03:38:14.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 Processor -- start 2024-04-06T03:38:14.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 -- start start 2024-04-06T03:38:14.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.630+0000 7ff99d922700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9980fe510 0x7ff99810d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:14.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff99d922700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fee60 0x7ff99810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:14.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff99d922700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99810e4d0 con 0x7ff9980fee60 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff99d922700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99810e640 con 0x7ff9980fe510 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff9967fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fee60 0x7ff99810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff9967fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fee60 0x7ff99810ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37322/0 (socket says 172.21.15.94:37322) 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff9967fc700 1 -- 172.21.15.94:0/1617335719 learned_addr learned my addr 172.21.15.94:0/1617335719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff996ffd700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9980fe510 0x7ff99810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff9967fc700 1 -- 172.21.15.94:0/1617335719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9980fe510 msgr2=0x7ff99810d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff9967fc700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9980fe510 0x7ff99810d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff9967fc700 1 -- 172.21.15.94:0/1617335719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff988005040 con 0x7ff9980fee60 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff996ffd700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9980fe510 0x7ff99810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:38:14.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff9967fc700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fee60 0x7ff99810ded0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c810 tx=0x7ff98c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:14.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff97f7fe700 1 -- 172.21.15.94:0/1617335719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff98c012810 con 0x7ff9980fee60 2024-04-06T03:38:14.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9981130a0 con 0x7ff9980fee60 2024-04-06T03:38:14.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff97f7fe700 1 -- 172.21.15.94:0/1617335719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff98c012e40 con 0x7ff9980fee60 2024-04-06T03:38:14.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff97f7fe700 1 -- 172.21.15.94:0/1617335719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff98c00a370 con 0x7ff9980fee60 2024-04-06T03:38:14.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.631+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff998113660 con 0x7ff9980fee60 2024-04-06T03:38:14.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.632+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99804c710 con 0x7ff9980fee60 2024-04-06T03:38:14.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.633+0000 7ff97f7fe700 1 -- 172.21.15.94:0/1617335719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff98c011810 con 0x7ff9980fee60 2024-04-06T03:38:14.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.633+0000 7ff97f7fe700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9800692a0 0x7ff98006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:14.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.633+0000 7ff97f7fe700 1 -- 172.21.15.94:0/1617335719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff98c087620 con 0x7ff9980fee60 2024-04-06T03:38:14.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.633+0000 7ff996ffd700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9800692a0 0x7ff98006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:14.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.634+0000 7ff996ffd700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9800692a0 0x7ff98006b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff98800b3b0 tx=0x7ff988009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:14.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.638+0000 7ff97f7fe700 1 -- 172.21.15.94:0/1617335719 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff98c057360 con 0x7ff9980fee60 2024-04-06T03:38:14.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.903+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7ff998113af0 con 0x7ff9980fee60 2024-04-06T03:38:14.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.904+0000 7ff97f7fe700 1 -- 172.21.15.94:0/1617335719 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v12) v1 ==== 78+0+99 (secure 0 0 0) 0x7ff98c016080 con 0x7ff9980fee60 2024-04-06T03:38:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9800692a0 msgr2=0x7ff98006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9800692a0 0x7ff98006b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff98800b3b0 tx=0x7ff988009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fee60 msgr2=0x7ff99810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:14.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fee60 0x7ff99810ded0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c810 tx=0x7ff98c004470 comp rx=0 tx=0).stop 2024-04-06T03:38:14.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 shutdown_connections 2024-04-06T03:38:14.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff9800692a0 0x7ff98006b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:14.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9980fe510 0x7ff99810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:14.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 --2- 172.21.15.94:0/1617335719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9980fee60 0x7ff99810ded0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:14.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 >> 172.21.15.94:0/1617335719 conn(0x7ff9980f6c10 msgr2=0x7ff9980f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:14.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 shutdown_connections 2024-04-06T03:38:14.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:14.906+0000 7ff99d922700 1 -- 172.21.15.94:0/1617335719 wait complete. 2024-04-06T03:38:14.917 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:38:15.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:15 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1617335719' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-06T03:38:15.588 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-06T03:38:15.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-06T03:38:15.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:15 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1617335719' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-06T03:38:15.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.709+0000 7f9ce9aee640 1 -- 172.21.15.94:0/3644082765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 msgr2=0x7f9ce40fc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:15.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.709+0000 7f9ce9aee640 1 --2- 172.21.15.94:0/3644082765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 0x7f9ce40fc7c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9cd80048d0 tx=0x7f9cd8009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.709+0000 7f9ce9aee640 1 -- 172.21.15.94:0/3644082765 shutdown_connections 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.709+0000 7f9ce9aee640 1 --2- 172.21.15.94:0/3644082765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 0x7f9ce40fc7c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.709+0000 7f9ce9aee640 1 --2- 172.21.15.94:0/3644082765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce40fb080 0x7f9ce40fb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.709+0000 7f9ce9aee640 1 -- 172.21.15.94:0/3644082765 >> 172.21.15.94:0/3644082765 conn(0x7f9ce40f6860 msgr2=0x7f9ce40f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.709+0000 7f9ce9aee640 1 -- 172.21.15.94:0/3644082765 shutdown_connections 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.709+0000 7f9ce9aee640 1 -- 172.21.15.94:0/3644082765 wait complete. 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.709+0000 7f9ce9aee640 1 Processor -- start 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce9aee640 1 -- start start 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce9aee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce40fb080 0x7f9ce410bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce9aee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 0x7f9ce410c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce9aee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce410c820 con 0x7f9ce40fc350 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce9aee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4111190 con 0x7f9ce40fb080 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 0x7f9ce410c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:15.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 0x7f9ce410c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37344/0 (socket says 172.21.15.94:37344) 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce2ffd640 1 -- 172.21.15.94:0/2332324567 learned_addr learned my addr 172.21.15.94:0/2332324567 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce2ffd640 1 -- 172.21.15.94:0/2332324567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce40fb080 msgr2=0x7f9ce410bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce2ffd640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce40fb080 0x7f9ce410bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce2ffd640 1 -- 172.21.15.94:0/2332324567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd8005040 con 0x7f9ce40fc350 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce2ffd640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 0x7f9ce410c140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8007d20 tx=0x7f9cd8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ccbfff640 1 -- 172.21.15.94:0/2332324567 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cd8009590 con 0x7f9ce40fc350 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce4111390 con 0x7f9ce40fc350 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ccbfff640 1 -- 172.21.15.94:0/2332324567 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cd80096f0 con 0x7f9ce40fc350 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.710+0000 7f9ccbfff640 1 -- 172.21.15.94:0/2332324567 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cd803b6d0 con 0x7f9ce40fc350 2024-04-06T03:38:15.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.711+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce41118b0 con 0x7f9ce40fc350 2024-04-06T03:38:15.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.711+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce404bcc0 con 0x7f9ce40fc350 2024-04-06T03:38:15.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.712+0000 7f9ccbfff640 1 -- 172.21.15.94:0/2332324567 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9cd802f070 con 0x7f9ce40fc350 2024-04-06T03:38:15.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.712+0000 7f9ccbfff640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ccc069210 0x7f9ccc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:15.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.712+0000 7f9ccbfff640 1 -- 172.21.15.94:0/2332324567 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9cd80a9870 con 0x7f9ce40fc350 2024-04-06T03:38:15.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.712+0000 7f9ce37fe640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ccc069210 0x7f9ccc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:15.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.712+0000 7f9ce37fe640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ccc069210 0x7f9ccc06b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400acf0 tx=0x7f9cd4009370 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:15.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.716+0000 7f9ccbfff640 1 -- 172.21.15.94:0/2332324567 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9cd8075b20 con 0x7f9ce40fc350 2024-04-06T03:38:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.888+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ce4103c50 con 0x7f9ce40fc350 2024-04-06T03:38:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.889+0000 7f9ccbfff640 1 -- 172.21.15.94:0/2332324567 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14279 (secure 0 0 0) 0x7f9cd8079140 con 0x7f9ce40fc350 2024-04-06T03:38:15.892 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:15.892 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":45,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","created":"2024-04-06T03:31:12.411006+0000","modified":"2024-04-06T03:37:51.403565+0000","last_up_change":"2024-04-06T03:36:36.170053+0000","last_in_change":"2024-04-06T03:36:17.627724+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-06T03:35:23.271684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-06T03:37:48.333560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-06T03:37:48.624904+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7f51dba0-a74e-444c-b4ab-d5f1fa68ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6803","nonce":2289372883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6805","nonce":2289372883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6809","nonce":2289372883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2289372883},{"type":"v1","addr":"172.21.15.94:6807","nonce":2289372883}]},"public_addr":"172.21.15.94:6803/2289372883","cluster_addr":"172.21.15.94:6805/2289372883","heartbeat_back_addr":"172.21.15.94:6809/2289372883","heartbeat_front_addr":"172.21.15.94:6807/2289372883","state":["exists","up"]},{"osd":1,"uuid":"2dd01097-c6d6-45e9-acee-ca5ea5f4e166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6811","nonce":1337787373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6813","nonce":1337787373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6817","nonce":1337787373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1337787373},{"type":"v1","addr":"172.21.15.94:6815","nonce":1337787373}]},"public_addr":"172.21.15.94:6811/1337787373","cluster_addr":"172.21.15.94:6813/1337787373","heartbeat_back_addr":"172.21.15.94:6817/1337787373","heartbeat_front_addr":"172.21.15.94:6815/1337787373","state":["exists","up"]},{"osd":2,"uuid":"9645da7c-ffbc-4415-8fa0-0fcb07c53f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6819","nonce":1538420111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6821","nonce":1538420111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6825","nonce":1538420111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1538420111},{"type":"v1","addr":"172.21.15.94:6823","nonce":1538420111}]},"public_addr":"172.21.15.94:6819/1538420111","cluster_addr":"172.21.15.94:6821/1538420111","heartbeat_back_addr":"172.21.15.94:6825/1538420111","heartbeat_front_addr":"172.21.15.94:6823/1538420111","state":["exists","up"]},{"osd":3,"uuid":"43321ebd-f195-4738-a2a5-f4d55a375bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6801","nonce":1271372986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6803","nonce":1271372986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6807","nonce":1271372986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1271372986},{"type":"v1","addr":"172.21.15.136:6805","nonce":1271372986}]},"public_addr":"172.21.15.136:6801/1271372986","cluster_addr":"172.21.15.136:6803/1271372986","heartbeat_back_addr":"172.21.15.136:6807/1271372986","heartbeat_front_addr":"172.21.15.136:6805/1271372986","state":["exists","up"]},{"osd":4,"uuid":"572f9cc0-e83c-4354-babb-519be04069c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6809","nonce":2873260774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6811","nonce":2873260774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6815","nonce":2873260774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":2873260774},{"type":"v1","addr":"172.21.15.136:6813","nonce":2873260774}]},"public_addr":"172.21.15.136:6809/2873260774","cluster_addr":"172.21.15.136:6811/2873260774","heartbeat_back_addr":"172.21.15.136:6815/2873260774","heartbeat_front_addr":"172.21.15.136:6813/2873260774","state":["exists","up"]},{"osd":5,"uuid":"a38e98d8-dfb7-4083-aae8-e5853a896d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6817","nonce":3055268575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6819","nonce":3055268575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6823","nonce":3055268575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3055268575},{"type":"v1","addr":"172.21.15.136:6821","nonce":3055268575}]},"public_addr":"172.21.15.136:6817/3055268575","cluster_addr":"172.21.15.136:6819/3055268575","heartbeat_back_addr":"172.21.15.136:6823/3055268575","heartbeat_front_addr":"172.21.15.136:6821/3055268575","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:25.784070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:53.656615+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:20.675464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:44.290139+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:09.122272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:34.741771+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1085015108":"2024-04-07T03:33:12.953920+0000","172.21.15.94:6801/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1082665650":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6801/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/3195038611":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/852522634":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/2412091380":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/2779591805":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6800/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6801/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/1247691830":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/3724179185":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/1084447585":"2024-04-07T03:32:10.456958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-06T03:38:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ccc069210 msgr2=0x7f9ccc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ccc069210 0x7f9ccc06b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400acf0 tx=0x7f9cd4009370 comp rx=0 tx=0).stop 2024-04-06T03:38:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 msgr2=0x7f9ce410c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 0x7f9ce410c140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8007d20 tx=0x7f9cd8009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 shutdown_connections 2024-04-06T03:38:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9ccc069210 0x7f9ccc06b6c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ce40fc350 0x7f9ce410c140 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 --2- 172.21.15.94:0/2332324567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce40fb080 0x7f9ce410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 >> 172.21.15.94:0/2332324567 conn(0x7f9ce40f6860 msgr2=0x7f9ce40f7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 shutdown_connections 2024-04-06T03:38:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:15.891+0000 7f9ce9aee640 1 -- 172.21.15.94:0/2332324567 wait complete. 2024-04-06T03:38:15.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-06T03:35:23.271684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-06T03:37:48.333560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-06T03:37:48.624904+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-06T03:38:15.905 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-06T03:38:16.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 -- 172.21.15.94:0/2673224992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fe140 msgr2=0x7f93e00fa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 --2- 172.21.15.94:0/2673224992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fe140 0x7f93e00fa2a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f93c80048d0 tx=0x7f93c8009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 -- 172.21.15.94:0/2673224992 shutdown_connections 2024-04-06T03:38:16.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 --2- 172.21.15.94:0/2673224992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fe140 0x7f93e00fa2a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 --2- 172.21.15.94:0/2673224992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93e00fd7f0 0x7f93e00fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 -- 172.21.15.94:0/2673224992 >> 172.21.15.94:0/2673224992 conn(0x7f93e00f5ff0 msgr2=0x7f93e00f8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 -- 172.21.15.94:0/2673224992 shutdown_connections 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 -- 172.21.15.94:0/2673224992 wait complete. 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 Processor -- start 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.005+0000 7f93e4aee640 1 -- start start 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93e4aee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fd7f0 0x7f93e010cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93e4aee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93e00fe140 0x7f93e010d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93e4aee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e010d890 con 0x7f93e00fd7f0 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93e4aee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e0112250 con 0x7f93e00fe140 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93de59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fd7f0 0x7f93e010cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93de59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fd7f0 0x7f93e010cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37356/0 (socket says 172.21.15.94:37356) 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93de59c640 1 -- 172.21.15.94:0/4013141165 learned_addr learned my addr 172.21.15.94:0/4013141165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93de59c640 1 -- 172.21.15.94:0/4013141165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93e00fe140 msgr2=0x7f93e010d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93de59c640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93e00fe140 0x7f93e010d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93de59c640 1 -- 172.21.15.94:0/4013141165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c8005040 con 0x7f93e00fd7f0 2024-04-06T03:38:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93de59c640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fd7f0 0x7f93e010cd00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f93d000c830 tx=0x7f93d0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93d6ffd640 1 -- 172.21.15.94:0/4013141165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d000a850 con 0x7f93e00fd7f0 2024-04-06T03:38:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93d6ffd640 1 -- 172.21.15.94:0/4013141165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93d000ae80 con 0x7f93e00fd7f0 2024-04-06T03:38:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93d6ffd640 1 -- 172.21.15.94:0/4013141165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d0012ea0 con 0x7f93e00fd7f0 2024-04-06T03:38:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e01123f0 con 0x7f93e00fd7f0 2024-04-06T03:38:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.006+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e01128c0 con 0x7f93e00fd7f0 2024-04-06T03:38:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.007+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e004bcc0 con 0x7f93e00fd7f0 2024-04-06T03:38:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.008+0000 7f93d6ffd640 1 -- 172.21.15.94:0/4013141165 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f93d0012400 con 0x7f93e00fd7f0 2024-04-06T03:38:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.008+0000 7f93d6ffd640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93cc069210 0x7f93cc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.008+0000 7f93d7fff640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93cc069210 0x7f93cc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.008+0000 7f93d6ffd640 1 -- 172.21.15.94:0/4013141165 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f93d0087400 con 0x7f93e00fd7f0 2024-04-06T03:38:16.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.008+0000 7f93d7fff640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93cc069210 0x7f93cc06b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009040 tx=0x7f93c8009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:16.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.012+0000 7f93d6ffd640 1 -- 172.21.15.94:0/4013141165 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f93d0053730 con 0x7f93e00fd7f0 2024-04-06T03:38:16.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.181+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93e0112b40 con 0x7f93e00fd7f0 2024-04-06T03:38:16.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.182+0000 7f93d6ffd640 1 -- 172.21.15.94:0/4013141165 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93d00126b0 con 0x7f93e00fd7f0 2024-04-06T03:38:16.184 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-06T03:38:16.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.183+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93cc069210 msgr2=0x7f93cc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.183+0000 7f93e4aee640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93cc069210 0x7f93cc06b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009040 tx=0x7f93c8009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.183+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fd7f0 msgr2=0x7f93e010cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.183+0000 7f93e4aee640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fd7f0 0x7f93e010cd00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f93d000c830 tx=0x7f93d0004430 comp rx=0 tx=0).stop 2024-04-06T03:38:16.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.184+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 shutdown_connections 2024-04-06T03:38:16.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.184+0000 7f93e4aee640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f93cc069210 0x7f93cc06b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.184+0000 7f93e4aee640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93e00fe140 0x7f93e010d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.184+0000 7f93e4aee640 1 --2- 172.21.15.94:0/4013141165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e00fd7f0 0x7f93e010cd00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.184+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 >> 172.21.15.94:0/4013141165 conn(0x7f93e00f5ff0 msgr2=0x7f93e00f76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:16.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.184+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 shutdown_connections 2024-04-06T03:38:16.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.184+0000 7f93e4aee640 1 -- 172.21.15.94:0/4013141165 wait complete. 2024-04-06T03:38:16.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-06T03:38:16.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.295+0000 7f3e3a99d640 1 -- 172.21.15.94:0/3445507112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 msgr2=0x7f3e34101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.295+0000 7f3e3a99d640 1 --2- 172.21.15.94:0/3445507112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 0x7f3e34101610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3e240048d0 tx=0x7f3e24009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 -- 172.21.15.94:0/3445507112 shutdown_connections 2024-04-06T03:38:16.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 --2- 172.21.15.94:0/3445507112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 0x7f3e34101610 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 --2- 172.21.15.94:0/3445507112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e340fa3d0 0x7f3e340fa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 -- 172.21.15.94:0/3445507112 >> 172.21.15.94:0/3445507112 conn(0x7f3e340f5dd0 msgr2=0x7f3e340f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 -- 172.21.15.94:0/3445507112 shutdown_connections 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 -- 172.21.15.94:0/3445507112 wait complete. 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 Processor -- start 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 -- start start 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e340fa3d0 0x7f3e340fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 0x7f3e340fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e340fd350 con 0x7f3e340fadb0 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.296+0000 7f3e3a99d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e340fb910 con 0x7f3e340fa3d0 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3919a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 0x7f3e340fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3999b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e340fa3d0 0x7f3e340fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3919a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 0x7f3e340fb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37364/0 (socket says 172.21.15.94:37364) 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3919a640 1 -- 172.21.15.94:0/324301553 learned_addr learned my addr 172.21.15.94:0/324301553 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3919a640 1 -- 172.21.15.94:0/324301553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e340fa3d0 msgr2=0x7f3e340fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3919a640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e340fa3d0 0x7f3e340fcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3919a640 1 -- 172.21.15.94:0/324301553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e24005040 con 0x7f3e340fadb0 2024-04-06T03:38:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3999b640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e340fa3d0 0x7f3e340fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:38:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3919a640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 0x7f3e340fb3d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e2402bda0 tx=0x7f3e24009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e2a7fc640 1 -- 172.21.15.94:0/324301553 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e240094c0 con 0x7f3e340fadb0 2024-04-06T03:38:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e2a7fc640 1 -- 172.21.15.94:0/324301553 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e24009a80 con 0x7f3e340fadb0 2024-04-06T03:38:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e340fbb90 con 0x7f3e340fadb0 2024-04-06T03:38:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e2a7fc640 1 -- 172.21.15.94:0/324301553 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e2403b640 con 0x7f3e340fadb0 2024-04-06T03:38:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.297+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e340fc0b0 con 0x7f3e340fadb0 2024-04-06T03:38:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.298+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e341053a0 con 0x7f3e340fadb0 2024-04-06T03:38:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.299+0000 7f3e2a7fc640 1 -- 172.21.15.94:0/324301553 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3e2402f070 con 0x7f3e340fadb0 2024-04-06T03:38:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.299+0000 7f3e2a7fc640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3e20069210 0x7f3e2006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.299+0000 7f3e2a7fc640 1 -- 172.21.15.94:0/324301553 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3e240a9610 con 0x7f3e340fadb0 2024-04-06T03:38:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.299+0000 7f3e3999b640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3e20069210 0x7f3e2006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.300+0000 7f3e3999b640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3e20069210 0x7f3e2006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009fd0 tx=0x7f3e3000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:16.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.304+0000 7f3e2a7fc640 1 -- 172.21.15.94:0/324301553 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3e240758c0 con 0x7f3e340fadb0 2024-04-06T03:38:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:16 smithi094 ceph-mon[29713]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:16 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2332324567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:38:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.479+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f3e3404bcc0 con 0x7f3e340fadb0 2024-04-06T03:38:16.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.480+0000 7f3e2a7fc640 1 -- 172.21.15.94:0/324301553 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3e24078ee0 con 0x7f3e340fadb0 2024-04-06T03:38:16.483 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-06T03:38:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3e20069210 msgr2=0x7f3e2006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3e20069210 0x7f3e2006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009fd0 tx=0x7f3e3000b040 comp rx=0 tx=0).stop 2024-04-06T03:38:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 msgr2=0x7f3e340fb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 0x7f3e340fb3d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e2402bda0 tx=0x7f3e24009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 shutdown_connections 2024-04-06T03:38:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3e20069210 0x7f3e2006b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e340fadb0 0x7f3e340fb3d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 --2- 172.21.15.94:0/324301553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e340fa3d0 0x7f3e340fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 >> 172.21.15.94:0/324301553 conn(0x7f3e340f5dd0 msgr2=0x7f3e340ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 shutdown_connections 2024-04-06T03:38:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.482+0000 7f3e3a99d640 1 -- 172.21.15.94:0/324301553 wait complete. 2024-04-06T03:38:16.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-06T03:38:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 -- 172.21.15.94:0/3117653645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 msgr2=0x7f5ab00fa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 --2- 172.21.15.94:0/3117653645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 0x7f5ab00fa0f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00048d0 tx=0x7f5aa0009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 -- 172.21.15.94:0/3117653645 shutdown_connections 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 --2- 172.21.15.94:0/3117653645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 0x7f5ab00fa0f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 --2- 172.21.15.94:0/3117653645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab00fd640 0x7f5ab00fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 -- 172.21.15.94:0/3117653645 >> 172.21.15.94:0/3117653645 conn(0x7f5ab00f5e40 msgr2=0x7f5ab00f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 -- 172.21.15.94:0/3117653645 shutdown_connections 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 -- 172.21.15.94:0/3117653645 wait complete. 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 Processor -- start 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.596+0000 7f5ab639d640 1 -- start start 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab639d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab00fd640 0x7f5ab010cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab639d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 0x7f5ab010d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab639d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab010d720 con 0x7f5ab00fdf90 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab639d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab0112120 con 0x7f5ab00fd640 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab4b9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 0x7f5ab010d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab4b9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 0x7f5ab010d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37384/0 (socket says 172.21.15.94:37384) 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab4b9a640 1 -- 172.21.15.94:0/3507930626 learned_addr learned my addr 172.21.15.94:0/3507930626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab539b640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab00fd640 0x7f5ab010cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab4b9a640 1 -- 172.21.15.94:0/3507930626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab00fd640 msgr2=0x7f5ab010cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab4b9a640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab00fd640 0x7f5ab010cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab4b9a640 1 -- 172.21.15.94:0/3507930626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa0005040 con 0x7f5ab00fdf90 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab4b9a640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 0x7f5ab010d0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00048a0 tx=0x7f5aa0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5aa5ffb640 1 -- 172.21.15.94:0/3507930626 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aa00094c0 con 0x7f5ab00fdf90 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5aa5ffb640 1 -- 172.21.15.94:0/3507930626 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5aa0007cc0 con 0x7f5ab00fdf90 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5aa5ffb640 1 -- 172.21.15.94:0/3507930626 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aa003b400 con 0x7f5ab00fdf90 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.597+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab0112260 con 0x7f5ab00fdf90 2024-04-06T03:38:16.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.598+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab0112750 con 0x7f5ab00fdf90 2024-04-06T03:38:16.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:16 smithi136 ceph-mon[36436]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:16.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:16 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2332324567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:38:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.599+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab004bcc0 con 0x7f5ab00fdf90 2024-04-06T03:38:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.599+0000 7f5aa5ffb640 1 -- 172.21.15.94:0/3507930626 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5aa002f070 con 0x7f5ab00fdf90 2024-04-06T03:38:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.599+0000 7f5aa5ffb640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5a9c069210 0x7f5a9c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.599+0000 7f5aa5ffb640 1 -- 172.21.15.94:0/3507930626 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5aa00a9610 con 0x7f5ab00fdf90 2024-04-06T03:38:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.599+0000 7f5ab539b640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5a9c069210 0x7f5a9c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.600+0000 7f5ab539b640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5a9c069210 0x7f5a9c06b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009af0 tx=0x7f5aac00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:16.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.604+0000 7f5aa5ffb640 1 -- 172.21.15.94:0/3507930626 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5aa00758c0 con 0x7f5ab00fdf90 2024-04-06T03:38:16.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.783+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f5ab0104b80 con 0x7f5ab00fdf90 2024-04-06T03:38:16.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.783+0000 7f5aa5ffb640 1 -- 172.21.15.94:0/3507930626 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5aa0078ee0 con 0x7f5ab00fdf90 2024-04-06T03:38:16.786 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-06T03:38:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.785+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5a9c069210 msgr2=0x7f5a9c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.785+0000 7f5ab639d640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5a9c069210 0x7f5a9c06b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009af0 tx=0x7f5aac00b040 comp rx=0 tx=0).stop 2024-04-06T03:38:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.785+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 msgr2=0x7f5ab010d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.785+0000 7f5ab639d640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 0x7f5ab010d0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00048a0 tx=0x7f5aa0009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.785+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 shutdown_connections 2024-04-06T03:38:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.785+0000 7f5ab639d640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f5a9c069210 0x7f5a9c06b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.785+0000 7f5ab639d640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ab00fdf90 0x7f5ab010d0d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.785+0000 7f5ab639d640 1 --2- 172.21.15.94:0/3507930626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ab00fd640 0x7f5ab010cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.785+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 >> 172.21.15.94:0/3507930626 conn(0x7f5ab00f5e40 msgr2=0x7f5ab00f7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.786+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 shutdown_connections 2024-04-06T03:38:16.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.786+0000 7f5ab639d640 1 -- 172.21.15.94:0/3507930626 wait complete. 2024-04-06T03:38:16.799 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-06T03:38:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.895+0000 7fea842ee640 1 -- 172.21.15.94:0/1394817085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fb0a0 msgr2=0x7fea7c0fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.895+0000 7fea842ee640 1 --2- 172.21.15.94:0/1394817085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fb0a0 0x7fea7c0fb4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fea780048d0 tx=0x7fea78009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.895+0000 7fea842ee640 1 -- 172.21.15.94:0/1394817085 shutdown_connections 2024-04-06T03:38:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.895+0000 7fea842ee640 1 --2- 172.21.15.94:0/1394817085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea7c0fc370 0x7fea7c0fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.895+0000 7fea842ee640 1 --2- 172.21.15.94:0/1394817085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fb0a0 0x7fea7c0fb4b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.895+0000 7fea842ee640 1 -- 172.21.15.94:0/1394817085 >> 172.21.15.94:0/1394817085 conn(0x7fea7c0f6840 msgr2=0x7fea7c0f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.895+0000 7fea842ee640 1 -- 172.21.15.94:0/1394817085 shutdown_connections 2024-04-06T03:38:16.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.895+0000 7fea842ee640 1 -- 172.21.15.94:0/1394817085 wait complete. 2024-04-06T03:38:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.895+0000 7fea842ee640 1 Processor -- start 2024-04-06T03:38:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea842ee640 1 -- start start 2024-04-06T03:38:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea842ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea7c0fb0a0 0x7fea7c064dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea842ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fc370 0x7fea7c111ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea842ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c111ff0 con 0x7fea7c0fc370 2024-04-06T03:38:16.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea842ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c112130 con 0x7fea7c0fb0a0 2024-04-06T03:38:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea81889640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fc370 0x7fea7c111ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea81889640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fc370 0x7fea7c111ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37390/0 (socket says 172.21.15.94:37390) 2024-04-06T03:38:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea81889640 1 -- 172.21.15.94:0/3013674203 learned_addr learned my addr 172.21.15.94:0/3013674203 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea8208a640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea7c0fb0a0 0x7fea7c064dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea81889640 1 -- 172.21.15.94:0/3013674203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea7c0fb0a0 msgr2=0x7fea7c064dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:16.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea81889640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea7c0fb0a0 0x7fea7c064dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea81889640 1 -- 172.21.15.94:0/3013674203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea78005040 con 0x7fea7c0fc370 2024-04-06T03:38:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea81889640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fc370 0x7fea7c111ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fea6c002a50 tx=0x7fea6c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea72ffd640 1 -- 172.21.15.94:0/3013674203 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea6c007c30 con 0x7fea7c0fc370 2024-04-06T03:38:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea72ffd640 1 -- 172.21.15.94:0/3013674203 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea6c00f040 con 0x7fea7c0fc370 2024-04-06T03:38:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c112390 con 0x7fea7c0fc370 2024-04-06T03:38:16.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea72ffd640 1 -- 172.21.15.94:0/3013674203 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea6c00db50 con 0x7fea7c0fc370 2024-04-06T03:38:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.896+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c1128b0 con 0x7fea7c0fc370 2024-04-06T03:38:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.897+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7c1055a0 con 0x7fea7c0fc370 2024-04-06T03:38:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.898+0000 7fea72ffd640 1 -- 172.21.15.94:0/3013674203 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fea6c00dcf0 con 0x7fea7c0fc370 2024-04-06T03:38:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.898+0000 7fea72ffd640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea68069210 0x7fea6806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.898+0000 7fea72ffd640 1 -- 172.21.15.94:0/3013674203 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fea6c0869f0 con 0x7fea7c0fc370 2024-04-06T03:38:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.898+0000 7fea8208a640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea68069210 0x7fea6806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:16.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.898+0000 7fea8208a640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea68069210 0x7fea6806b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fea780317e0 tx=0x7fea78009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:16.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:16.902+0000 7fea72ffd640 1 -- 172.21.15.94:0/3013674203 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fea6c04d190 con 0x7fea7c0fc370 2024-04-06T03:38:17.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.115+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea7c103c10 con 0x7fea7c0fc370 2024-04-06T03:38:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.116+0000 7fea72ffd640 1 -- 172.21.15.94:0/3013674203 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4686 (secure 0 0 0) 0x7fea6c088e60 con 0x7fea7c0fc370 2024-04-06T03:38:17.119 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:17.119 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.94:6829/3794043365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3794043365},{"type":"v1","addr":"172.21.15.94:6829","nonce":3794043365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:38:04.308946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14460":{"gid":14460,"name":"cephfs.smithi094.qogyzw","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.94:6827/1913624529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1913624529},{"type":"v1","addr":"172.21.15.94:6827","nonce":1913624529}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-04-06T03:38:17.121 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:38:17.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea68069210 msgr2=0x7fea6806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:17.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea68069210 0x7fea6806b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fea780317e0 tx=0x7fea78009da0 comp rx=0 tx=0).stop 2024-04-06T03:38:17.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fc370 msgr2=0x7fea7c111ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:17.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fc370 0x7fea7c111ab0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fea6c002a50 tx=0x7fea6c004600 comp rx=0 tx=0).stop 2024-04-06T03:38:17.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 shutdown_connections 2024-04-06T03:38:17.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fea68069210 0x7fea6806b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea7c0fc370 0x7fea7c111ab0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 --2- 172.21.15.94:0/3013674203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea7c0fb0a0 0x7fea7c064dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 >> 172.21.15.94:0/3013674203 conn(0x7fea7c0f6840 msgr2=0x7fea7c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 shutdown_connections 2024-04-06T03:38:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:17.118+0000 7fea842ee640 1 -- 172.21.15.94:0/3013674203 wait complete. 2024-04-06T03:38:17.131 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 11, 'max_mds': 1, 'flags': 18} 2024-04-06T03:38:17.131 INFO:teuthology.run_tasks:Running task kclient... 2024-04-06T03:38:17.171 INFO:tasks.kclient:Mounting kernel clients... 2024-04-06T03:38:17.171 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-04-06T03:38:17.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:17.172 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-06T03:38:17.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:17.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-06T03:38:17.257 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-04-06T03:38:17.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:38:17.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:17.259 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2024-04-06T03:38:17.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:17.278 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link delete ceph-brx 2024-04-06T03:38:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:Cannot find device "ceph-brx" 2024-04-06T03:38:17.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:38:17.356 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-06T03:38:17.356 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-06T03:38:17.356 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-06T03:38:17.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph fs ls 2024-04-06T03:38:17.381 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:17 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/4013141165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:38:17.381 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:17 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/324301553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-06T03:38:17.381 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:17 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3507930626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-06T03:38:17.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:17 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/4013141165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:38:17.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:17 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/324301553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-06T03:38:17.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:17 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3507930626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-06T03:38:17.852 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:18.221 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:18 smithi094 ceph-mon[29713]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:18.222 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:18 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3013674203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-06T03:38:18.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:18 smithi136 ceph-mon[36436]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:18.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:18 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3013674203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-06T03:38:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 -- 172.21.15.94:0/1289353562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 msgr2=0x7f59940fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 --2- 172.21.15.94:0/1289353562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 0x7f59940fce10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5984004940 tx=0x7f5984009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 -- 172.21.15.94:0/1289353562 shutdown_connections 2024-04-06T03:38:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 --2- 172.21.15.94:0/1289353562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 0x7f59940fce10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:19.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 --2- 172.21.15.94:0/1289353562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5994102e00 0x7f59941031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 -- 172.21.15.94:0/1289353562 >> 172.21.15.94:0/1289353562 conn(0x7f59940f89b0 msgr2=0x7f59940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 -- 172.21.15.94:0/1289353562 shutdown_connections 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 -- 172.21.15.94:0/1289353562 wait complete. 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 Processor -- start 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 -- start start 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5994102e00 0x7f59940702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 0x7f5994070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59940721c0 con 0x7f59941037a0 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.395+0000 7f599ba66700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5994070d40 con 0x7f5994102e00 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999001700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 0x7f5994070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999802700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5994102e00 0x7f59940702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999001700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 0x7f5994070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41736/0 (socket says 172.21.15.94:41736) 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999802700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5994102e00 0x7f59940702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:41188/0 (socket says 172.21.15.94:41188) 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999001700 1 -- 172.21.15.94:0/2372307626 learned_addr learned my addr 172.21.15.94:0/2372307626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:19.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999001700 1 -- 172.21.15.94:0/2372307626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5994102e00 msgr2=0x7f59940702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999001700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5994102e00 0x7f59940702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999001700 1 -- 172.21.15.94:0/2372307626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5984005040 con 0x7f59941037a0 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999802700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5994102e00 0x7f59940702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f5999001700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 0x7f5994070800 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5984000c00 tx=0x7f5984009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f598a7fc700 1 -- 172.21.15.94:0/2372307626 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5984021070 con 0x7f59941037a0 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5994070fc0 con 0x7f59941037a0 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f598a7fc700 1 -- 172.21.15.94:0/2372307626 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5984013c00 con 0x7f59941037a0 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f598a7fc700 1 -- 172.21.15.94:0/2372307626 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f598400e070 con 0x7f59941037a0 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.396+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599411a030 con 0x7f59941037a0 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.397+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59941052d0 con 0x7f59941037a0 2024-04-06T03:38:19.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.398+0000 7f598a7fc700 1 -- 172.21.15.94:0/2372307626 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f598400b6e0 con 0x7f59941037a0 2024-04-06T03:38:19.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.398+0000 7f598a7fc700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f59800692a0 0x7f598006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:19.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.398+0000 7f598a7fc700 1 -- 172.21.15.94:0/2372307626 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f59840885b0 con 0x7f59941037a0 2024-04-06T03:38:19.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.398+0000 7f5999802700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f59800692a0 0x7f598006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:19.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.398+0000 7f5999802700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f59800692a0 0x7f598006b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5990009b80 tx=0x7f599000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:19.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.403+0000 7f598a7fc700 1 -- 172.21.15.94:0/2372307626 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5984054950 con 0x7f59941037a0 2024-04-06T03:38:19.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.631+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5994071750 con 0x7f59941037a0 2024-04-06T03:38:19.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.631+0000 7f598a7fc700 1 -- 172.21.15.94:0/2372307626 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+83 (secure 0 0 0) 0x7f5984004490 con 0x7f59941037a0 2024-04-06T03:38:19.634 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-06T03:38:19.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.633+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f59800692a0 msgr2=0x7f598006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:19.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.633+0000 7f599ba66700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f59800692a0 0x7f598006b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5990009b80 tx=0x7f599000b040 comp rx=0 tx=0).stop 2024-04-06T03:38:19.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.633+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 msgr2=0x7f5994070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:19.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.633+0000 7f599ba66700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 0x7f5994070800 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5984000c00 tx=0x7f5984009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:19.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.633+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 shutdown_connections 2024-04-06T03:38:19.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.633+0000 7f599ba66700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f59800692a0 0x7f598006b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:19.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.633+0000 7f599ba66700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5994102e00 0x7f59940702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:19.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.633+0000 7f599ba66700 1 --2- 172.21.15.94:0/2372307626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59941037a0 0x7f5994070800 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:19.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.633+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 >> 172.21.15.94:0/2372307626 conn(0x7f59940f89b0 msgr2=0x7f59941021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:19.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.634+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 shutdown_connections 2024-04-06T03:38:19.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:19.634+0000 7f599ba66700 1 -- 172.21.15.94:0/2372307626 wait complete. 2024-04-06T03:38:20.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:20 smithi094 ceph-mon[29713]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:20.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:20.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:20.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:38:20.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:38:20.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:20 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:20.236 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:20 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2372307626' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-06T03:38:20.237 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-06T03:38:20.237 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-06T03:38:20.237 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-06T03:38:20.237 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-06T03:38:20.237 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-06T03:38:20.237 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-06T03:38:20.237 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-06T03:38:20.238 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-06T03:38:20.238 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-06T03:38:20.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:20.238 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-06T03:38:20.294 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-06T03:38:20.294 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-06T03:38:20.294 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-06T03:38:20.294 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-06T03:38:20.294 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-06T03:38:20.294 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-06T03:38:20.294 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-06T03:38:20.295 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-06T03:38:20.295 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-06T03:38:20.295 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-06T03:38:20.295 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-06T03:38:20.295 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-06T03:38:20.295 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-06T03:38:20.295 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-06T03:38:20.295 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-06T03:38:20.296 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-06T03:38:20.296 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-06T03:38:20.296 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42137sec preferred_lft 42137sec 2024-04-06T03:38:20.296 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-04-06T03:38:20.296 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-06T03:38:20.297 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-06T03:38:20.297 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:20.297 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-06T03:38:20.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-04-06T03:38:20.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-04-06T03:38:20.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-04-06T03:38:20.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-06T03:38:20.298 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-06T03:38:20.299 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:20 smithi136 ceph-mon[36436]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:20.299 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:20 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:20.299 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:20 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:20.299 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:20 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:38:20.300 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:20 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:38:20.300 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:20 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:20.300 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:20 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2372307626' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-06T03:38:20.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:20.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:20.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:20.502 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-06T03:38:20.529 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-06T03:38:20.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:20.531 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-06T03:38:20.616 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-06T03:38:20.616 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-06T03:38:20.616 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-06T03:38:20.616 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-06T03:38:20.617 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-06T03:38:20.617 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:20.617 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-06T03:38:20.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-06T03:38:20.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-06T03:38:20.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-06T03:38:20.617 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-06T03:38:20.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:20.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:20.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:20.898 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-06T03:38:20.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:20.914 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-04-06T03:38:20.968 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:20.969 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-06T03:38:20.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-06T03:38:20.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-06T03:38:20.969 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-06T03:38:21.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:21.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:21.090 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-06T03:38:21.090 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:21.090 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-06T03:38:21.090 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-06T03:38:21.091 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-06T03:38:21.091 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-06T03:38:21.091 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-06T03:38:21.091 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-06T03:38:21.091 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-06T03:38:21.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:21.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:21.229 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:21.229 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-06T03:38:21.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-04-06T03:38:21.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-04-06T03:38:21.229 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-06T03:38:21.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:21.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:21.311 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-06T03:38:21.311 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-06T03:38:21.325 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-06T03:38:21.325 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-06T03:38:21.326 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-06T03:38:21.384 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-06T03:38:21.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:21.384 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-06T03:38:21.475 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-06T03:38:21.475 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-06T03:38:21.476 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-06T03:38:21.595 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-06T03:38:21.595 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-06T03:38:21.595 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-06T03:38:21.595 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.136:3300" 2024-04-06T03:38:21.595 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-06T03:38:21.600 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-06T03:38:21.600 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-06T03:38:21.601 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-04-06T03:38:21.601 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.136:3300" 2024-04-06T03:38:21.601 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-06T03:38:21.601 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:6789,172.21.15.136:6789" 2024-04-06T03:38:21.601 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:6789,172.21.15.136:6789:/ 2024-04-06T03:38:21.601 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c6509676-f3c5-11ee-b649-cb9ed24678a4" will pass to kernel 2024-04-06T03:38:21.602 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-06T03:38:21.602 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-04-06T03:38:21.602 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-06T03:38:21.602 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-04-06T03:38:21.602 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-06T03:38:21.604 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-06T03:38:21.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:21.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-06T03:38:21.631 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-06T03:38:21.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-06T03:38:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:21.730 INFO:teuthology.orchestra.run.smithi094.stdout:client24295 2024-04-06T03:38:21.731 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:38:21.731 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-06T03:38:21.748 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-06T03:38:21.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-06T03:38:21.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:21.847 INFO:teuthology.orchestra.run.smithi094.stdout:client24295 2024-04-06T03:38:21.847 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-06T03:38:21.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-06T03:38:21.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:21.907 INFO:teuthology.orchestra.run.smithi094.stdout:client24295 2024-04-06T03:38:21.907 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4.client24295/status') 2024-04-06T03:38:21.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:21.970 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.24295 (3)172.21.15.94:0/2208962971 2024-04-06T03:38:21.971 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-04-06T03:38:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-04-06T03:38:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-04-06T03:38:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:71 bytes copied, 3.7983e-05 s, 1.9 MB/s 2024-04-06T03:38:21.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:21.978 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-06T03:38:21.978 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-06T03:38:21.978 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-06T03:38:21.978 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- ceph fs ls 2024-04-06T03:38:22.235 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:22.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:22 smithi136 ceph-mon[36436]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:22.357 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:22 smithi094 ceph-mon[29713]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.821+0000 7faec3338700 1 -- 172.21.15.94:0/3333044950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 msgr2=0x7faebc1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.821+0000 7faec3338700 1 --2- 172.21.15.94:0/3333044950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 0x7faebc1020f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faeb8004940 tx=0x7faeb8009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.821+0000 7faec3338700 1 -- 172.21.15.94:0/3333044950 shutdown_connections 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.821+0000 7faec3338700 1 --2- 172.21.15.94:0/3333044950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faebc1026c0 0x7faebc0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.821+0000 7faec3338700 1 --2- 172.21.15.94:0/3333044950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 0x7faebc1020f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.821+0000 7faec3338700 1 -- 172.21.15.94:0/3333044950 >> 172.21.15.94:0/3333044950 conn(0x7faebc0f7870 msgr2=0x7faebc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.821+0000 7faec3338700 1 -- 172.21.15.94:0/3333044950 shutdown_connections 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.821+0000 7faec3338700 1 -- 172.21.15.94:0/3333044950 wait complete. 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec3338700 1 Processor -- start 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec3338700 1 -- start start 2024-04-06T03:38:23.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec3338700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 0x7faebc10eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec3338700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faebc1026c0 0x7faebc10f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec3338700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc10f5a0 con 0x7faebc101d20 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec3338700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc10f6e0 con 0x7faebc1026c0 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec10d4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 0x7faebc10eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec10d4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 0x7faebc10eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41760/0 (socket says 172.21.15.94:41760) 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec10d4700 1 -- 172.21.15.94:0/2294505641 learned_addr learned my addr 172.21.15.94:0/2294505641 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.822+0000 7faec08d3700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faebc1026c0 0x7faebc10f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.823+0000 7faec10d4700 1 -- 172.21.15.94:0/2294505641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faebc1026c0 msgr2=0x7faebc10f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.823+0000 7faec10d4700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faebc1026c0 0x7faebc10f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.823+0000 7faec10d4700 1 -- 172.21.15.94:0/2294505641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8005040 con 0x7faebc101d20 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.823+0000 7faec10d4700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 0x7faebc10eb20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faeb800b3b0 tx=0x7faeb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.823+0000 7faeb1ffb700 1 -- 172.21.15.94:0/2294505641 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeb800e040 con 0x7faebc101d20 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.823+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faebc10f960 con 0x7faebc101d20 2024-04-06T03:38:23.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.823+0000 7faeb1ffb700 1 -- 172.21.15.94:0/2294505641 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faeb8013420 con 0x7faebc101d20 2024-04-06T03:38:23.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.823+0000 7faeb1ffb700 1 -- 172.21.15.94:0/2294505641 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeb801c630 con 0x7faebc101d20 2024-04-06T03:38:23.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.823+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faebc117f30 con 0x7faebc101d20 2024-04-06T03:38:23.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.824+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faebc0fd470 con 0x7faebc101d20 2024-04-06T03:38:23.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.824+0000 7faeb1ffb700 1 -- 172.21.15.94:0/2294505641 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7faeb8013590 con 0x7faebc101d20 2024-04-06T03:38:23.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.825+0000 7faeb1ffb700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faea80692a0 0x7faea806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:23.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.825+0000 7faeb1ffb700 1 -- 172.21.15.94:0/2294505641 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7faeb8088370 con 0x7faebc101d20 2024-04-06T03:38:23.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.825+0000 7faec08d3700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faea80692a0 0x7faea806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:23.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.825+0000 7faec08d3700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faea80692a0 0x7faea806b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faebc066ac0 tx=0x7faeac009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:23.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:23.829+0000 7faeb1ffb700 1 -- 172.21.15.94:0/2294505641 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7faeb80546a0 con 0x7faebc101d20 2024-04-06T03:38:24.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.061+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faebc066870 con 0x7faebc101d20 2024-04-06T03:38:24.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.061+0000 7faeb1ffb700 1 -- 172.21.15.94:0/2294505641 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+83 (secure 0 0 0) 0x7faeb8057cc0 con 0x7faebc101d20 2024-04-06T03:38:24.064 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-06T03:38:24.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faea80692a0 msgr2=0x7faea806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faea80692a0 0x7faea806b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faebc066ac0 tx=0x7faeac009040 comp rx=0 tx=0).stop 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 msgr2=0x7faebc10eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 0x7faebc10eb20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faeb800b3b0 tx=0x7faeb8009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 shutdown_connections 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7faea80692a0 0x7faea806b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faebc101d20 0x7faebc10eb20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 --2- 172.21.15.94:0/2294505641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faebc1026c0 0x7faebc10f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 >> 172.21.15.94:0/2294505641 conn(0x7faebc0f7870 msgr2=0x7faebc0f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 shutdown_connections 2024-04-06T03:38:24.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:24.063+0000 7faec3338700 1 -- 172.21.15.94:0/2294505641 wait complete. 2024-04-06T03:38:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:24 smithi136 ceph-mon[36436]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:24 smithi094 ceph-mon[29713]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:24.756 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-06T03:38:24.757 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-06T03:38:24.757 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2024-04-06T03:38:24.757 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-06T03:38:24.757 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-06T03:38:24.757 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-06T03:38:24.757 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-06T03:38:24.757 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-06T03:38:24.757 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-06T03:38:24.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:24.757 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2024-04-06T03:38:24.778 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-06T03:38:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-06T03:38:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2024-04-06T03:38:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2024-04-06T03:38:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2024-04-06T03:38:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2024-04-06T03:38:24.778 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-06T03:38:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2024-04-06T03:38:24.778 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42160sec preferred_lft 42160sec 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2024-04-06T03:38:24.779 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2024-04-06T03:38:24.779 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-06T03:38:24.780 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:24.780 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2024-04-06T03:38:24.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link add name ceph-brx type bridge 2024-04-06T03:38:24.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip addr flush dev ceph-brx 2024-04-06T03:38:24.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set ceph-brx up 2024-04-06T03:38:24.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-06T03:38:24.780 DEBUG:teuthology.orchestra.run.smithi136:> ') 2024-04-06T03:38:24.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:25.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:25.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:25.012 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-06T03:38:25.040 INFO:teuthology.orchestra.run.smithi136.stdout:1 2024-04-06T03:38:25.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:25.042 DEBUG:teuthology.orchestra.run.smithi136:> route 2024-04-06T03:38:25.114 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2024-04-06T03:38:25.114 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-06T03:38:25.114 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-06T03:38:25.114 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-06T03:38:25.114 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-06T03:38:25.115 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:25.115 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2024-04-06T03:38:25.115 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-06T03:38:25.115 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-06T03:38:25.115 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-06T03:38:25.115 DEBUG:teuthology.orchestra.run.smithi136:> ') 2024-04-06T03:38:25.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:25.212 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:25 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2294505641' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-06T03:38:25.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:25.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:25.416 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2024-04-06T03:38:25.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:25 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2294505641' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-06T03:38:25.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:25.431 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list-id 2024-04-06T03:38:25.486 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:25.486 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2024-04-06T03:38:25.486 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-06T03:38:25.486 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-06T03:38:25.486 DEBUG:teuthology.orchestra.run.smithi136:> ') 2024-04-06T03:38:25.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:25.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:25.613 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-06T03:38:25.613 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:25.613 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2024-04-06T03:38:25.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-06T03:38:25.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-06T03:38:25.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-06T03:38:25.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-06T03:38:25.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-06T03:38:25.614 DEBUG:teuthology.orchestra.run.smithi136:> ') 2024-04-06T03:38:25.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:25.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:25.754 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-06T03:38:25.754 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2024-04-06T03:38:25.754 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2024-04-06T03:38:25.754 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set dev brx.0 master ceph-brx 2024-04-06T03:38:25.754 DEBUG:teuthology.orchestra.run.smithi136:> ') 2024-04-06T03:38:25.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:25.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:25.835 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-06T03:38:25.835 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-06T03:38:25.850 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-06T03:38:25.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-06T03:38:25.850 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-06T03:38:25.904 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-06T03:38:25.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:25.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-06T03:38:25.998 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-06T03:38:25.999 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-06T03:38:25.999 INFO:teuthology.orchestra.run.smithi136.stdout:invalid new device string format 2024-04-06T03:38:26.124 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-06T03:38:26.125 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-06T03:38:26.125 INFO:teuthology.orchestra.run.smithi136.stdout:invalid new device string format 2024-04-06T03:38:26.125 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.136:3300" 2024-04-06T03:38:26.125 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-06T03:38:26.129 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-06T03:38:26.129 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-06T03:38:26.129 INFO:teuthology.orchestra.run.smithi136.stdout:invalid new device string format 2024-04-06T03:38:26.129 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.136:3300" 2024-04-06T03:38:26.130 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-06T03:38:26.130 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.94:6789,172.21.15.136:6789" 2024-04-06T03:38:26.130 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:6789,172.21.15.136:6789:/ 2024-04-06T03:38:26.130 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c6509676-f3c5-11ee-b649-cb9ed24678a4" will pass to kernel 2024-04-06T03:38:26.130 INFO:teuthology.orchestra.run.smithi136.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-06T03:38:26.130 INFO:teuthology.orchestra.run.smithi136.stdout: You just mounted a file system that supports labels which does not 2024-04-06T03:38:26.131 INFO:teuthology.orchestra.run.smithi136.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-06T03:38:26.131 INFO:teuthology.orchestra.run.smithi136.stdout: applications will generate AVC messages and not be allowed access to 2024-04-06T03:38:26.131 INFO:teuthology.orchestra.run.smithi136.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-06T03:38:26.132 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-06T03:38:26.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:38:26.132 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-06T03:38:26.159 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-06T03:38:26.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-06T03:38:26.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:26.254 INFO:teuthology.orchestra.run.smithi136.stdout:client14524 2024-04-06T03:38:26.255 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:38:26.255 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-06T03:38:26.271 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-06T03:38:26.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:26 smithi136 ceph-mon[36436]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:26.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-06T03:38:26.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:26.367 INFO:teuthology.orchestra.run.smithi136.stdout:client14524 2024-04-06T03:38:26.368 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-06T03:38:26.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:26 smithi094 ceph-mon[29713]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:26.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-06T03:38:26.458 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:26.464 INFO:teuthology.orchestra.run.smithi136.stdout:client14524 2024-04-06T03:38:26.464 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4.client14524/status') 2024-04-06T03:38:26.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-06T03:38:26.525 INFO:teuthology.orchestra.run.smithi136.stdout:instance: client.14524 (3)172.21.15.136:0/1309930737 2024-04-06T03:38:26.525 INFO:teuthology.orchestra.run.smithi136.stdout:blocklisted: false 2024-04-06T03:38:26.526 INFO:teuthology.orchestra.run.smithi136.stderr:0+1 records in 2024-04-06T03:38:26.526 INFO:teuthology.orchestra.run.smithi136.stderr:0+1 records out 2024-04-06T03:38:26.526 INFO:teuthology.orchestra.run.smithi136.stderr:72 bytes copied, 3.7555e-05 s, 1.9 MB/s 2024-04-06T03:38:26.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-06T03:38:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-06T03:38:26.533 INFO:teuthology.run_tasks:Running task print... 2024-04-06T03:38:26.544 INFO:teuthology.task.print:**** done client 2024-04-06T03:38:26.544 INFO:teuthology.run_tasks:Running task parallel... 2024-04-06T03:38:26.557 INFO:teuthology.task.parallel:starting parallel... 2024-04-06T03:38:26.557 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-06T03:38:26.558 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-06T03:38:26.558 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:38:26.558 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-04-06T03:38:26.559 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-06T03:38:26.559 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-06T03:38:26.562 INFO:tasks.workunit:Pulling workunits from ref 584a67ccb9eb224c5312dccd96a82e67b0645518 2024-04-06T03:38:26.563 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-06T03:38:26.563 INFO:tasks.workunit:timeout=3h 2024-04-06T03:38:26.563 INFO:tasks.workunit:cleanup=True 2024-04-06T03:38:26.563 DEBUG:teuthology.orchestra.run.smithi094:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-06T03:38:26.583 INFO:teuthology.orchestra.run.smithi094.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-06T03:38:26.583 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-06T03:38:26.583 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 94h/148d Inode: 1 Links: 2 2024-04-06T03:38:26.583 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:38:26.583 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-06T03:38:26.583 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-06T03:38:26.584 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-06 03:37:53.628876077 +0000 2024-04-06T03:38:26.584 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-06 03:38:21.627890536 +0000 2024-04-06T03:38:26.584 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-06 03:37:53.628876077 +0000 2024-04-06T03:38:26.584 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-06T03:38:26.584 DEBUG:teuthology.orchestra.run.smithi094:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-06T03:38:26.656 DEBUG:teuthology.orchestra.run.smithi136:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-06T03:38:26.673 INFO:teuthology.orchestra.run.smithi136.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-06T03:38:26.673 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-04-06T03:38:26.673 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-04-06T03:38:26.673 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-06T03:38:26.673 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-06T03:38:26.673 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-06T03:38:26.673 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-06 03:38:26.649805734 +0000 2024-04-06T03:38:26.674 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-06 03:38:26.649805734 +0000 2024-04-06T03:38:26.674 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-06 03:37:53.628876077 +0000 2024-04-06T03:38:26.674 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-06T03:38:26.674 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-06T03:38:26.744 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 584a67ccb9eb224c5312dccd96a82e67b0645518 2024-04-06T03:38:26.745 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 584a67ccb9eb224c5312dccd96a82e67b0645518 2024-04-06T03:38:26.760 INFO:tasks.workunit.client.0.smithi094.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-06T03:38:26.801 INFO:tasks.workunit.client.1.smithi136.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-06T03:38:26.840 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:28.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:28 smithi094 ceph-mon[29713]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-06T03:38:28.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:28 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:38:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:28 smithi136 ceph-mon[36436]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-06T03:38:28.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:28 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.419+0000 7f3ae798d700 1 -- 172.21.15.94:0/945171175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00fffc0 msgr2=0x7f3ae01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:28.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.419+0000 7f3ae798d700 1 --2- 172.21.15.94:0/945171175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00fffc0 0x7f3ae01023a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4004940 tx=0x7f3ad4009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.419+0000 7f3ae798d700 1 -- 172.21.15.94:0/945171175 shutdown_connections 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.419+0000 7f3ae798d700 1 --2- 172.21.15.94:0/945171175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00fffc0 0x7f3ae01023a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.419+0000 7f3ae798d700 1 --2- 172.21.15.94:0/945171175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0066220 0x7f3ae00ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.419+0000 7f3ae798d700 1 -- 172.21.15.94:0/945171175 >> 172.21.15.94:0/945171175 conn(0x7f3ae00f7430 msgr2=0x7f3ae00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.419+0000 7f3ae798d700 1 -- 172.21.15.94:0/945171175 shutdown_connections 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.419+0000 7f3ae798d700 1 -- 172.21.15.94:0/945171175 wait complete. 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.419+0000 7f3ae798d700 1 Processor -- start 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae798d700 1 -- start start 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae798d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0066220 0x7f3ae00fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae798d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00fffc0 0x7f3ae00fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae798d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00fbf30 con 0x7f3ae00fffc0 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae798d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00fc0a0 con 0x7f3ae0066220 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae4f28700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00fffc0 0x7f3ae00fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:28.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae4f28700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00fffc0 0x7f3ae00fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36424/0 (socket says 172.21.15.94:36424) 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae4f28700 1 -- 172.21.15.94:0/3659575188 learned_addr learned my addr 172.21.15.94:0/3659575188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae5729700 1 --2- 172.21.15.94:0/3659575188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0066220 0x7f3ae00fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae4f28700 1 -- 172.21.15.94:0/3659575188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0066220 msgr2=0x7f3ae00fd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae4f28700 1 --2- 172.21.15.94:0/3659575188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0066220 0x7f3ae00fd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae4f28700 1 -- 172.21.15.94:0/3659575188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad4005040 con 0x7f3ae00fffc0 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3ae4f28700 1 --2- 172.21.15.94:0/3659575188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ae00fffc0 0x7f3ae00fb980 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400b3b0 tx=0x7f3ad4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3addffb700 1 -- 172.21.15.94:0/3659575188 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad400e040 con 0x7f3ae00fffc0 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.420+0000 7f3addffb700 1 -- 172.21.15.94:0/3659575188 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ad400bcc0 con 0x7f3ae00fffc0 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.421+0000 7f3ae798d700 1 -- 172.21.15.94:0/3659575188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae00fc320 con 0x7f3ae00fffc0 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.421+0000 7f3addffb700 1 -- 172.21.15.94:0/3659575188 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad401c630 con 0x7f3ae00fffc0 2024-04-06T03:38:28.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.421+0000 7f3ae798d700 1 -- 172.21.15.94:0/3659575188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae0115a90 con 0x7f3ae00fffc0 2024-04-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.422+0000 7f3ae798d700 1 -- 172.21.15.94:0/3659575188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae005f5f0 con 0x7f3ae00fffc0 2024-04-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.422+0000 7f3addffb700 1 -- 172.21.15.94:0/3659575188 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3ad40139f0 con 0x7f3ae00fffc0 2024-04-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.422+0000 7f3addffb700 1 --2- 172.21.15.94:0/3659575188 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3ac80692a0 0x7f3ac806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.423+0000 7f3addffb700 1 -- 172.21.15.94:0/3659575188 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3ad4087f90 con 0x7f3ae00fffc0 2024-04-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.423+0000 7f3ae5729700 1 --2- 172.21.15.94:0/3659575188 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3ac80692a0 0x7f3ac806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:28.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.423+0000 7f3ae5729700 1 --2- 172.21.15.94:0/3659575188 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3ac80692a0 0x7f3ac806b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00fd240 tx=0x7f3ad8009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:28.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.427+0000 7f3addffb700 1 -- 172.21.15.94:0/3659575188 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3ad40542c0 con 0x7f3ae00fffc0 2024-04-06T03:38:28.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.614+0000 7f3ae798d700 1 -- 172.21.15.94:0/3659575188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f3ae004bed0 con 0x7f3ae00fffc0 2024-04-06T03:38:28.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:28.615+0000 7f3addffb700 1 -- 172.21.15.94:0/3659575188 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7f3ad40578e0 con 0x7f3ae00fffc0 2024-04-06T03:38:28.620 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-04-06T03:38:29.249 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-06T03:38:29.249 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:38:29.249 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-06T03:38:29.548 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:30.289 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:30 smithi094 ceph-mon[29713]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-06T03:38:30.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:30 smithi136 ceph-mon[36436]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-06T03:38:31.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 -- 172.21.15.94:0/3147717657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fdfe0 msgr2=0x7f1c200fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:31.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 --2- 172.21.15.94:0/3147717657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fdfe0 0x7f1c200fe450 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1c10004940 tx=0x7f1c10009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:31.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 -- 172.21.15.94:0/3147717657 shutdown_connections 2024-04-06T03:38:31.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 --2- 172.21.15.94:0/3147717657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fdfe0 0x7f1c200fe450 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:31.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 --2- 172.21.15.94:0/3147717657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c200fce70 0x7f1c200fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:31.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 -- 172.21.15.94:0/3147717657 >> 172.21.15.94:0/3147717657 conn(0x7f1c200f8550 msgr2=0x7f1c200fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 -- 172.21.15.94:0/3147717657 shutdown_connections 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 -- 172.21.15.94:0/3147717657 wait complete. 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 Processor -- start 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.138+0000 7f1c26dd8700 1 -- start start 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c26dd8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fce70 0x7f1c2010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c26dd8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c200fdfe0 0x7f1c2010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c26dd8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2010e510 con 0x7f1c200fce70 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c26dd8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c20112eb0 con 0x7f1c200fdfe0 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c24b74700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fce70 0x7f1c2010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c24b74700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fce70 0x7f1c2010d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36430/0 (socket says 172.21.15.94:36430) 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c24b74700 1 -- 172.21.15.94:0/134003534 learned_addr learned my addr 172.21.15.94:0/134003534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c17fff700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c200fdfe0 0x7f1c2010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c24b74700 1 -- 172.21.15.94:0/134003534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c200fdfe0 msgr2=0x7f1c2010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c24b74700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c200fdfe0 0x7f1c2010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c24b74700 1 -- 172.21.15.94:0/134003534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c10005040 con 0x7f1c200fce70 2024-04-06T03:38:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c24b74700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fce70 0x7f1c2010d940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00bbb0 tx=0x7f1c1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.139+0000 7f1c157fa700 1 -- 172.21.15.94:0/134003534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c1c0092e0 con 0x7f1c200fce70 2024-04-06T03:38:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.140+0000 7f1c157fa700 1 -- 172.21.15.94:0/134003534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c1c00cbd0 con 0x7f1c200fce70 2024-04-06T03:38:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.140+0000 7f1c157fa700 1 -- 172.21.15.94:0/134003534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c1c007400 con 0x7f1c200fce70 2024-04-06T03:38:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.140+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c20113110 con 0x7f1c200fce70 2024-04-06T03:38:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.140+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c20113680 con 0x7f1c200fce70 2024-04-06T03:38:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.140+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2004c710 con 0x7f1c200fce70 2024-04-06T03:38:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.141+0000 7f1c157fa700 1 -- 172.21.15.94:0/134003534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1c1c012070 con 0x7f1c200fce70 2024-04-06T03:38:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.141+0000 7f1c157fa700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1c080692a0 0x7f1c0806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.141+0000 7f1c157fa700 1 -- 172.21.15.94:0/134003534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1c1c086d30 con 0x7f1c200fce70 2024-04-06T03:38:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.141+0000 7f1c17fff700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1c080692a0 0x7f1c0806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.142+0000 7f1c17fff700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1c080692a0 0x7f1c0806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000b970 tx=0x7f1c10009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:31.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.146+0000 7f1c157fa700 1 -- 172.21.15.94:0/134003534 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1c1c056af0 con 0x7f1c200fce70 2024-04-06T03:38:31.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.331+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f1c20105950 con 0x7f1c200fce70 2024-04-06T03:38:31.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.331+0000 7f1c157fa700 1 -- 172.21.15.94:0/134003534 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f1c1c056680 con 0x7f1c200fce70 2024-04-06T03:38:31.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.333+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1c080692a0 msgr2=0x7f1c0806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:31.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.333+0000 7f1c26dd8700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1c080692a0 0x7f1c0806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000b970 tx=0x7f1c10009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:31.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.333+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fce70 msgr2=0x7f1c2010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:31.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.333+0000 7f1c26dd8700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fce70 0x7f1c2010d940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00bbb0 tx=0x7f1c1c009840 comp rx=0 tx=0).stop 2024-04-06T03:38:31.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.333+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 shutdown_connections 2024-04-06T03:38:31.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.333+0000 7f1c26dd8700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1c080692a0 0x7f1c0806b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:31.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.333+0000 7f1c26dd8700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c200fce70 0x7f1c2010d940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:31.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.333+0000 7f1c26dd8700 1 --2- 172.21.15.94:0/134003534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c200fdfe0 0x7f1c2010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:31.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.333+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 >> 172.21.15.94:0/134003534 conn(0x7f1c200f8550 msgr2=0x7f1c200f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:31.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.334+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 shutdown_connections 2024-04-06T03:38:31.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:31.334+0000 7f1c26dd8700 1 -- 172.21.15.94:0/134003534 wait complete. 2024-04-06T03:38:31.989 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-06T03:38:32.258 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:32.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:32 smithi136 ceph-mon[36436]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-06T03:38:32.379 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:32 smithi094 ceph-mon[29713]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-06T03:38:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.817+0000 7f09730e7700 1 -- 172.21.15.94:0/3238812368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fcf80 msgr2=0x7f096c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.817+0000 7f09730e7700 1 --2- 172.21.15.94:0/3238812368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fcf80 0x7f096c0fd3f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0958004940 tx=0x7f0958009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.817+0000 7f09730e7700 1 -- 172.21.15.94:0/3238812368 shutdown_connections 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.817+0000 7f09730e7700 1 --2- 172.21.15.94:0/3238812368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fcf80 0x7f096c0fd3f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.817+0000 7f09730e7700 1 --2- 172.21.15.94:0/3238812368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f096c0fbcd0 0x7f096c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.817+0000 7f09730e7700 1 -- 172.21.15.94:0/3238812368 >> 172.21.15.94:0/3238812368 conn(0x7f096c0f7450 msgr2=0x7f096c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.817+0000 7f09730e7700 1 -- 172.21.15.94:0/3238812368 shutdown_connections 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.817+0000 7f09730e7700 1 -- 172.21.15.94:0/3238812368 wait complete. 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f09730e7700 1 Processor -- start 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f09730e7700 1 -- start start 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f09730e7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fbcd0 0x7f096c0ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f09730e7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f096c0fcf80 0x7f096c0fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f09730e7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c1001c0 con 0x7f096c0fbcd0 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f09730e7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0fe800 con 0x7f096c0fcf80 2024-04-06T03:38:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f0970e83700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fbcd0 0x7f096c0ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f0970e83700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fbcd0 0x7f096c0ffbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36442/0 (socket says 172.21.15.94:36442) 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f0970e83700 1 -- 172.21.15.94:0/423965403 learned_addr learned my addr 172.21.15.94:0/423965403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.818+0000 7f096bfff700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f096c0fcf80 0x7f096c0fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.819+0000 7f0970e83700 1 -- 172.21.15.94:0/423965403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f096c0fcf80 msgr2=0x7f096c0fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.819+0000 7f0970e83700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f096c0fcf80 0x7f096c0fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.819+0000 7f0970e83700 1 -- 172.21.15.94:0/423965403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958005040 con 0x7f096c0fbcd0 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.819+0000 7f0970e83700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fbcd0 0x7f096c0ffbc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f096000c830 tx=0x7f0960004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.819+0000 7f09697fa700 1 -- 172.21.15.94:0/423965403 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f096000a760 con 0x7f096c0fbcd0 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.819+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c0feae0 con 0x7f096c0fbcd0 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.819+0000 7f09697fa700 1 -- 172.21.15.94:0/423965403 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f096000ad90 con 0x7f096c0fbcd0 2024-04-06T03:38:33.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.819+0000 7f09697fa700 1 -- 172.21.15.94:0/423965403 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09600113a0 con 0x7f096c0fbcd0 2024-04-06T03:38:33.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.819+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c0ff020 con 0x7f096c0fbcd0 2024-04-06T03:38:33.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.820+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c101b10 con 0x7f096c0fbcd0 2024-04-06T03:38:33.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.821+0000 7f09697fa700 1 -- 172.21.15.94:0/423965403 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f096000a8c0 con 0x7f096c0fbcd0 2024-04-06T03:38:33.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.821+0000 7f09697fa700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f095c0692a0 0x7f095c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:33.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.821+0000 7f09697fa700 1 -- 172.21.15.94:0/423965403 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0960087230 con 0x7f096c0fbcd0 2024-04-06T03:38:33.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.821+0000 7f096bfff700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f095c0692a0 0x7f095c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:33.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.821+0000 7f096bfff700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f095c0692a0 0x7f095c06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f095800b970 tx=0x7f0958009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:33.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:33.825+0000 7f09697fa700 1 -- 172.21.15.94:0/423965403 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0960056ff0 con 0x7f096c0fbcd0 2024-04-06T03:38:34.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.010+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f096c04c710 con 0x7f096c0fbcd0 2024-04-06T03:38:34.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.011+0000 7f09697fa700 1 -- 172.21.15.94:0/423965403 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0960056b80 con 0x7f096c0fbcd0 2024-04-06T03:38:34.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.012+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f095c0692a0 msgr2=0x7f095c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:34.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f095c0692a0 0x7f095c06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f095800b970 tx=0x7f0958009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:34.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fbcd0 msgr2=0x7f096c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:34.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fbcd0 0x7f096c0ffbc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f096000c830 tx=0x7f0960004470 comp rx=0 tx=0).stop 2024-04-06T03:38:34.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 shutdown_connections 2024-04-06T03:38:34.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f095c0692a0 0x7f095c06b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:34.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f096c0fbcd0 0x7f096c0ffbc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:34.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 --2- 172.21.15.94:0/423965403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f096c0fcf80 0x7f096c0fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:34.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 >> 172.21.15.94:0/423965403 conn(0x7f096c0f7450 msgr2=0x7f096c0668e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:34.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 shutdown_connections 2024-04-06T03:38:34.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:34.013+0000 7f09730e7700 1 -- 172.21.15.94:0/423965403 wait complete. 2024-04-06T03:38:34.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:34 smithi094 ceph-mon[29713]: pgmap v193: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-06T03:38:34.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:34 smithi136 ceph-mon[36436]: pgmap v193: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-06T03:38:34.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-06T03:38:34.915 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:36.285 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:36 smithi094 ceph-mon[29713]: pgmap v194: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-06T03:38:36.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:36 smithi136 ceph-mon[36436]: pgmap v194: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-06T03:38:36.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.527+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/1548108749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 msgr2=0x7f0adc070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:36.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.527+0000 7f0ae1fbb700 1 --2- 172.21.15.94:0/1548108749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 0x7f0adc070ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0004940 tx=0x7f0ad0009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:36.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.527+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/1548108749 shutdown_connections 2024-04-06T03:38:36.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.527+0000 7f0ae1fbb700 1 --2- 172.21.15.94:0/1548108749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 0x7f0adc070ce0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:36.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.527+0000 7f0ae1fbb700 1 --2- 172.21.15.94:0/1548108749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0adc071e20 0x7f0adc070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:36.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.527+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/1548108749 >> 172.21.15.94:0/1548108749 conn(0x7f0adc0f8550 msgr2=0x7f0adc0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:36.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.528+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/1548108749 shutdown_connections 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.528+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/1548108749 wait complete. 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.528+0000 7f0ae1fbb700 1 Processor -- start 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.528+0000 7f0ae1fbb700 1 -- start start 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.528+0000 7f0ae1fbb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 0x7f0adc10fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.528+0000 7f0ae1fbb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0adc071e20 0x7f0adc110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.528+0000 7f0ae1fbb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc110670 con 0x7f0adc070870 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.528+0000 7f0ae1fbb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc1107e0 con 0x7f0adc071e20 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0adb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 0x7f0adc10fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0adb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 0x7f0adc10fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36460/0 (socket says 172.21.15.94:36460) 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0adb7fe700 1 -- 172.21.15.94:0/554535329 learned_addr learned my addr 172.21.15.94:0/554535329 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0adaffd700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0adc071e20 0x7f0adc110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0adb7fe700 1 -- 172.21.15.94:0/554535329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0adc071e20 msgr2=0x7f0adc110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0adb7fe700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0adc071e20 0x7f0adc110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:36.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0adb7fe700 1 -- 172.21.15.94:0/554535329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad0005040 con 0x7f0adc070870 2024-04-06T03:38:36.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0adaffd700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0adc071e20 0x7f0adc110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:38:36.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0adb7fe700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 0x7f0adc10fb30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c8e0 tx=0x7f0acc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:36.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0ac3fff700 1 -- 172.21.15.94:0/554535329 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc00a940 con 0x7f0adc070870 2024-04-06T03:38:36.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc115240 con 0x7f0adc070870 2024-04-06T03:38:36.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0ac3fff700 1 -- 172.21.15.94:0/554535329 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0acc00ce80 con 0x7f0adc070870 2024-04-06T03:38:36.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0ac3fff700 1 -- 172.21.15.94:0/554535329 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc0113a0 con 0x7f0adc070870 2024-04-06T03:38:36.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.529+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc115800 con 0x7f0adc070870 2024-04-06T03:38:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.530+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc04c710 con 0x7f0adc070870 2024-04-06T03:38:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.531+0000 7f0ac3fff700 1 -- 172.21.15.94:0/554535329 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0acc012400 con 0x7f0adc070870 2024-04-06T03:38:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.531+0000 7f0ac3fff700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0ac40692a0 0x7f0ac406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.531+0000 7f0ac3fff700 1 -- 172.21.15.94:0/554535329 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0acc087290 con 0x7f0adc070870 2024-04-06T03:38:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.531+0000 7f0adaffd700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0ac40692a0 0x7f0ac406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:36.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.531+0000 7f0adaffd700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0ac40692a0 0x7f0ac406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c00 tx=0x7f0ad0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:36.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.535+0000 7f0ac3fff700 1 -- 172.21.15.94:0/554535329 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0acc057050 con 0x7f0adc070870 2024-04-06T03:38:36.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.721+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f0adc107ad0 con 0x7f0adc070870 2024-04-06T03:38:36.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.722+0000 7f0ac3fff700 1 -- 172.21.15.94:0/554535329 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f0acc056be0 con 0x7f0adc070870 2024-04-06T03:38:36.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.723+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0ac40692a0 msgr2=0x7f0ac406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:36.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.723+0000 7f0ae1fbb700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0ac40692a0 0x7f0ac406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c00 tx=0x7f0ad0009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:36.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.723+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 msgr2=0x7f0adc10fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:36.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.723+0000 7f0ae1fbb700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 0x7f0adc10fb30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c8e0 tx=0x7f0acc0044c0 comp rx=0 tx=0).stop 2024-04-06T03:38:36.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.724+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 shutdown_connections 2024-04-06T03:38:36.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.724+0000 7f0ae1fbb700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0ac40692a0 0x7f0ac406b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:36.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.724+0000 7f0ae1fbb700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0adc070870 0x7f0adc10fb30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:36.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.724+0000 7f0ae1fbb700 1 --2- 172.21.15.94:0/554535329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0adc071e20 0x7f0adc110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:36.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.724+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 >> 172.21.15.94:0/554535329 conn(0x7f0adc0f8550 msgr2=0x7f0adc0f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:36.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.724+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 shutdown_connections 2024-04-06T03:38:36.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:36.724+0000 7f0ae1fbb700 1 -- 172.21.15.94:0/554535329 wait complete. 2024-04-06T03:38:37.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-06T03:38:37.554 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:38.345 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:38 smithi094 ceph-mon[29713]: pgmap v195: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-06T03:38:38.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:38 smithi136 ceph-mon[36436]: pgmap v195: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-06T03:38:39.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.157+0000 7f309ccc4700 1 -- 172.21.15.94:0/2936316507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fdfe0 msgr2=0x7f30980fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:39.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.157+0000 7f309ccc4700 1 --2- 172.21.15.94:0/2936316507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fdfe0 0x7f30980fe450 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f308c004940 tx=0x7f308c009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:39.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.157+0000 7f309ccc4700 1 -- 172.21.15.94:0/2936316507 shutdown_connections 2024-04-06T03:38:39.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.157+0000 7f309ccc4700 1 --2- 172.21.15.94:0/2936316507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fdfe0 0x7f30980fe450 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:39.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.157+0000 7f309ccc4700 1 --2- 172.21.15.94:0/2936316507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30980fce70 0x7f30980fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.157+0000 7f309ccc4700 1 -- 172.21.15.94:0/2936316507 >> 172.21.15.94:0/2936316507 conn(0x7f30980f8550 msgr2=0x7f30980fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.157+0000 7f309ccc4700 1 -- 172.21.15.94:0/2936316507 shutdown_connections 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.157+0000 7f309ccc4700 1 -- 172.21.15.94:0/2936316507 wait complete. 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309ccc4700 1 Processor -- start 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309ccc4700 1 -- start start 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309ccc4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fce70 0x7f309810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309ccc4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30980fdfe0 0x7f309810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309ccc4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309810e500 con 0x7f30980fce70 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309ccc4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3098112ea0 con 0x7f30980fdfe0 2024-04-06T03:38:39.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fce70 0x7f309810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fce70 0x7f309810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49570/0 (socket says 172.21.15.94:49570) 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309659c700 1 -- 172.21.15.94:0/3970140295 learned_addr learned my addr 172.21.15.94:0/3970140295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f3095d9b700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30980fdfe0 0x7f309810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309659c700 1 -- 172.21.15.94:0/3970140295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30980fdfe0 msgr2=0x7f309810de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309659c700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30980fdfe0 0x7f309810de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309659c700 1 -- 172.21.15.94:0/3970140295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c005040 con 0x7f30980fce70 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f3095d9b700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30980fdfe0 0x7f309810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f309659c700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fce70 0x7f309810d930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f308800b790 tx=0x7f3088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f3086ffd700 1 -- 172.21.15.94:0/3970140295 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f308800f770 con 0x7f30980fce70 2024-04-06T03:38:39.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.158+0000 7f3086ffd700 1 -- 172.21.15.94:0/3970140295 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f308800fda0 con 0x7f30980fce70 2024-04-06T03:38:39.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.159+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3098113100 con 0x7f30980fce70 2024-04-06T03:38:39.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.159+0000 7f3086ffd700 1 -- 172.21.15.94:0/3970140295 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f308800d310 con 0x7f30980fce70 2024-04-06T03:38:39.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.159+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3098113670 con 0x7f30980fce70 2024-04-06T03:38:39.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.159+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309804c710 con 0x7f30980fce70 2024-04-06T03:38:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.160+0000 7f3086ffd700 1 -- 172.21.15.94:0/3970140295 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3088010420 con 0x7f30980fce70 2024-04-06T03:38:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.160+0000 7f3086ffd700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f30800692a0 0x7f308006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.160+0000 7f3086ffd700 1 -- 172.21.15.94:0/3970140295 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f30880871b0 con 0x7f30980fce70 2024-04-06T03:38:39.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.160+0000 7f3095d9b700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f30800692a0 0x7f308006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:39.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.161+0000 7f3095d9b700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f30800692a0 0x7f308006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f308c00b7a0 tx=0x7f308c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:39.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.165+0000 7f3086ffd700 1 -- 172.21.15.94:0/3970140295 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3088056f70 con 0x7f30980fce70 2024-04-06T03:38:39.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.366+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6", "target": ["mon-mgr", ""]}) v1 -- 0x7f3098102300 con 0x7f30800692a0 2024-04-06T03:38:39.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.372+0000 7f3086ffd700 1 -- 172.21.15.94:0/3970140295 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f3098102300 con 0x7f30800692a0 2024-04-06T03:38:39.375 INFO:teuthology.orchestra.run.smithi094.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f30800692a0 msgr2=0x7f308006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f30800692a0 0x7f308006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f308c00b7a0 tx=0x7f308c009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fce70 msgr2=0x7f309810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fce70 0x7f309810d930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f308800b790 tx=0x7f3088009250 comp rx=0 tx=0).stop 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 shutdown_connections 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f30800692a0 0x7f308006b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30980fce70 0x7f309810d930 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 --2- 172.21.15.94:0/3970140295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30980fdfe0 0x7f309810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 >> 172.21.15.94:0/3970140295 conn(0x7f30980f8550 msgr2=0x7f30980f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:39.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 shutdown_connections 2024-04-06T03:38:39.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:39.375+0000 7f309ccc4700 1 -- 172.21.15.94:0/3970140295 wait complete. 2024-04-06T03:38:40.046 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-06T03:38:40.047 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:38:40.047 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-06T03:38:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:38:40.646 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:40 smithi094 ceph-mon[29713]: pgmap v196: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-06T03:38:40.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:40 smithi094 ceph-mon[29713]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:40.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:40 smithi094 ceph-mon[29713]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:38:40.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:40 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:40.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:40 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:38:40.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:40 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:38:40.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:40 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:38:40.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:40 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:40.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:40 smithi094 ceph-mon[29713]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:38:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:40 smithi136 ceph-mon[36436]: pgmap v196: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-06T03:38:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:40 smithi136 ceph-mon[36436]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:40 smithi136 ceph-mon[36436]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:38:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:40 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:40 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:38:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:40 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:38:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:40 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:38:40.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:40 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:38:40.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:40 smithi136 ceph-mon[36436]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 2024-04-06T03:38:42.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.131+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3739093332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 msgr2=0x7f73d00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.131+0000 7f73d4eb0700 1 --2- 172.21.15.94:0/3739093332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 0x7f73d00fd3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f73c4004940 tx=0x7f73c4009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:42.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.131+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3739093332 shutdown_connections 2024-04-06T03:38:42.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.131+0000 7f73d4eb0700 1 --2- 172.21.15.94:0/3739093332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 0x7f73d00fd3f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.131+0000 7f73d4eb0700 1 --2- 172.21.15.94:0/3739093332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73d00fbcd0 0x7f73d00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.131+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3739093332 >> 172.21.15.94:0/3739093332 conn(0x7f73d00f7430 msgr2=0x7f73d00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.131+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3739093332 shutdown_connections 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.131+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3739093332 wait complete. 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73d4eb0700 1 Processor -- start 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73d4eb0700 1 -- start start 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73d4eb0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73d00fbcd0 0x7f73d010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73d4eb0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 0x7f73d010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73d4eb0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d010e4c0 con 0x7f73d00fcf80 2024-04-06T03:38:42.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73d4eb0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d010ed10 con 0x7f73d00fbcd0 2024-04-06T03:38:42.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73cdd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 0x7f73d010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:42.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73cdd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 0x7f73d010dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49584/0 (socket says 172.21.15.94:49584) 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73cdd9b700 1 -- 172.21.15.94:0/3337024424 learned_addr learned my addr 172.21.15.94:0/3337024424 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.132+0000 7f73ce59c700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73d00fbcd0 0x7f73d010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.133+0000 7f73cdd9b700 1 -- 172.21.15.94:0/3337024424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73d00fbcd0 msgr2=0x7f73d010d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.133+0000 7f73cdd9b700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73d00fbcd0 0x7f73d010d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.133+0000 7f73cdd9b700 1 -- 172.21.15.94:0/3337024424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c4005040 con 0x7f73d00fcf80 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.133+0000 7f73cdd9b700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 0x7f73d010dec0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f73c4013870 tx=0x7f73c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.133+0000 7f73beffd700 1 -- 172.21.15.94:0/3337024424 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c4021070 con 0x7f73d00fcf80 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.133+0000 7f73beffd700 1 -- 172.21.15.94:0/3337024424 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73c4009430 con 0x7f73d00fcf80 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.133+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d010ef90 con 0x7f73d00fcf80 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.133+0000 7f73beffd700 1 -- 172.21.15.94:0/3337024424 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c400e070 con 0x7f73d00fcf80 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.133+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d010f4d0 con 0x7f73d00fcf80 2024-04-06T03:38:42.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.134+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d0101b10 con 0x7f73d00fcf80 2024-04-06T03:38:42.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.135+0000 7f73beffd700 1 -- 172.21.15.94:0/3337024424 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f73c401c750 con 0x7f73d00fcf80 2024-04-06T03:38:42.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.135+0000 7f73beffd700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f73b80692a0 0x7f73b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:42.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.135+0000 7f73beffd700 1 -- 172.21.15.94:0/3337024424 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f73c4088390 con 0x7f73d00fcf80 2024-04-06T03:38:42.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.135+0000 7f73ce59c700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f73b80692a0 0x7f73b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:42.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.135+0000 7f73ce59c700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f73b80692a0 0x7f73b806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009b80 tx=0x7f73c000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:42.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.140+0000 7f73beffd700 1 -- 172.21.15.94:0/3337024424 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f73c40581f0 con 0x7f73d00fcf80 2024-04-06T03:38:42.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.353+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73d0104950 con 0x7f73b80692a0 2024-04-06T03:38:42.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.354+0000 7f73beffd700 1 -- 172.21.15.94:0/3337024424 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f73d0104950 con 0x7f73b80692a0 2024-04-06T03:38:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f73b80692a0 msgr2=0x7f73b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f73b80692a0 0x7f73b806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009b80 tx=0x7f73c000b040 comp rx=0 tx=0).stop 2024-04-06T03:38:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 msgr2=0x7f73d010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 0x7f73d010dec0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f73c4013870 tx=0x7f73c4009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 shutdown_connections 2024-04-06T03:38:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f73b80692a0 0x7f73b806b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73d00fbcd0 0x7f73d010d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 --2- 172.21.15.94:0/3337024424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73d00fcf80 0x7f73d010dec0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 >> 172.21.15.94:0/3337024424 conn(0x7f73d00f7430 msgr2=0x7f73d00f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 shutdown_connections 2024-04-06T03:38:42.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.356+0000 7f73d4eb0700 1 -- 172.21.15.94:0/3337024424 wait complete. 2024-04-06T03:38:42.368 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:38:42.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:42 smithi094 ceph-mon[29713]: pgmap v197: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-06T03:38:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.463+0000 7ff89dac9700 1 -- 172.21.15.94:0/1373090790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 msgr2=0x7ff8980fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.463+0000 7ff89dac9700 1 --2- 172.21.15.94:0/1373090790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 0x7ff8980fc100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff888004940 tx=0x7ff888009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.463+0000 7ff89dac9700 1 -- 172.21.15.94:0/1373090790 shutdown_connections 2024-04-06T03:38:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.463+0000 7ff89dac9700 1 --2- 172.21.15.94:0/1373090790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8980fcfa0 0x7ff8980fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.463+0000 7ff89dac9700 1 --2- 172.21.15.94:0/1373090790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 0x7ff8980fc100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.463+0000 7ff89dac9700 1 -- 172.21.15.94:0/1373090790 >> 172.21.15.94:0/1373090790 conn(0x7ff8980f7450 msgr2=0x7ff8980f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.463+0000 7ff89dac9700 1 -- 172.21.15.94:0/1373090790 shutdown_connections 2024-04-06T03:38:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.463+0000 7ff89dac9700 1 -- 172.21.15.94:0/1373090790 wait complete. 2024-04-06T03:38:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.463+0000 7ff89dac9700 1 Processor -- start 2024-04-06T03:38:42.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff89dac9700 1 -- start start 2024-04-06T03:38:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff89dac9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 0x7ff8980ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff89dac9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8980fcfa0 0x7ff8980fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff89dac9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980fe890 con 0x7ff8980fbcf0 2024-04-06T03:38:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff89dac9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980fea00 con 0x7ff8980fcfa0 2024-04-06T03:38:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff8977fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 0x7ff8980ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff8977fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 0x7ff8980ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49604/0 (socket says 172.21.15.94:49604) 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff8977fe700 1 -- 172.21.15.94:0/959716618 learned_addr learned my addr 172.21.15.94:0/959716618 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff8977fe700 1 -- 172.21.15.94:0/959716618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8980fcfa0 msgr2=0x7ff8980fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff8977fe700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8980fcfa0 0x7ff8980fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff8977fe700 1 -- 172.21.15.94:0/959716618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff888005040 con 0x7ff8980fbcf0 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.464+0000 7ff8977fe700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 0x7ff8980ffe10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff88800b3b0 tx=0x7ff888009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.465+0000 7ff87ffff700 1 -- 172.21.15.94:0/959716618 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff88800e040 con 0x7ff8980fbcf0 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.465+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8980fec80 con 0x7ff8980fbcf0 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.465+0000 7ff87ffff700 1 -- 172.21.15.94:0/959716618 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff88800bcc0 con 0x7ff8980fbcf0 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.465+0000 7ff87ffff700 1 -- 172.21.15.94:0/959716618 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff88801c630 con 0x7ff8980fbcf0 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.465+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff898115a60 con 0x7ff8980fbcf0 2024-04-06T03:38:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.466+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89805f5f0 con 0x7ff8980fbcf0 2024-04-06T03:38:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.466+0000 7ff87ffff700 1 -- 172.21.15.94:0/959716618 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff888013980 con 0x7ff8980fbcf0 2024-04-06T03:38:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.467+0000 7ff87ffff700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff8800692a0 0x7ff88006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.467+0000 7ff87ffff700 1 -- 172.21.15.94:0/959716618 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff888087f00 con 0x7ff8980fbcf0 2024-04-06T03:38:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.467+0000 7ff896ffd700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff8800692a0 0x7ff88006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.467+0000 7ff896ffd700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff8800692a0 0x7ff88006b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009fd0 tx=0x7ff88c009e10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.471+0000 7ff87ffff700 1 -- 172.21.15.94:0/959716618 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff888054230 con 0x7ff8980fbcf0 2024-04-06T03:38:42.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.695+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff89805e400 con 0x7ff8800692a0 2024-04-06T03:38:42.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.696+0000 7ff87ffff700 1 -- 172.21.15.94:0/959716618 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7ff89805e400 con 0x7ff8800692a0 2024-04-06T03:38:42.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.698+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff8800692a0 msgr2=0x7ff88006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.698+0000 7ff89dac9700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff8800692a0 0x7ff88006b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009fd0 tx=0x7ff88c009e10 comp rx=0 tx=0).stop 2024-04-06T03:38:42.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.698+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 msgr2=0x7ff8980ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.698+0000 7ff89dac9700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 0x7ff8980ffe10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff88800b3b0 tx=0x7ff888009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:42.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.698+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 shutdown_connections 2024-04-06T03:38:42.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.698+0000 7ff89dac9700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff8800692a0 0x7ff88006b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.698+0000 7ff89dac9700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980fbcf0 0x7ff8980ffe10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.698+0000 7ff89dac9700 1 --2- 172.21.15.94:0/959716618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8980fcfa0 0x7ff8980fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.698+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 >> 172.21.15.94:0/959716618 conn(0x7ff8980f7450 msgr2=0x7ff898066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:42.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.699+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 shutdown_connections 2024-04-06T03:38:42.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.699+0000 7ff89dac9700 1 -- 172.21.15.94:0/959716618 wait complete. 2024-04-06T03:38:42.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.804+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/589784354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740662e0 msgr2=0x7f7c740ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.804+0000 7f7c7bdb3700 1 --2- 172.21.15.94:0/589784354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740662e0 0x7f7c740ff9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c70004940 tx=0x7f7c70009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.804+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/589784354 shutdown_connections 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.804+0000 7f7c7bdb3700 1 --2- 172.21.15.94:0/589784354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c740fff00 0x7f7c74102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.804+0000 7f7c7bdb3700 1 --2- 172.21.15.94:0/589784354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740662e0 0x7f7c740ff9c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.804+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/589784354 >> 172.21.15.94:0/589784354 conn(0x7f7c740f7430 msgr2=0x7f7c740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.804+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/589784354 shutdown_connections 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.804+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/589784354 wait complete. 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7bdb3700 1 Processor -- start 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7bdb3700 1 -- start start 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7bdb3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c740662e0 0x7f7c740fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7bdb3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740fff00 0x7f7c740fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:42.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7bdb3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740fbe70 con 0x7f7c740fff00 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7bdb3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740fbfe0 con 0x7f7c740662e0 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7934e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740fff00 0x7f7c740fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7934e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740fff00 0x7f7c740fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49626/0 (socket says 172.21.15.94:49626) 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7934e700 1 -- 172.21.15.94:0/2306728619 learned_addr learned my addr 172.21.15.94:0/2306728619 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c79b4f700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c740662e0 0x7f7c740fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7934e700 1 -- 172.21.15.94:0/2306728619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c740662e0 msgr2=0x7f7c740fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.805+0000 7f7c7934e700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c740662e0 0x7f7c740fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.806+0000 7f7c7934e700 1 -- 172.21.15.94:0/2306728619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70005040 con 0x7f7c740fff00 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.806+0000 7f7c79b4f700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c740662e0 0x7f7c740fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.806+0000 7f7c7934e700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740fff00 0x7f7c740fb8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7c64002a50 tx=0x7f7c640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.806+0000 7f7c6a7fc700 1 -- 172.21.15.94:0/2306728619 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c64007cc0 con 0x7f7c740fff00 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.806+0000 7f7c6a7fc700 1 -- 172.21.15.94:0/2306728619 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c6400f040 con 0x7f7c740fff00 2024-04-06T03:38:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.806+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c740fc2c0 con 0x7f7c740fff00 2024-04-06T03:38:42.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.807+0000 7f7c6a7fc700 1 -- 172.21.15.94:0/2306728619 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c6400db70 con 0x7f7c740fff00 2024-04-06T03:38:42.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.807+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c74115a80 con 0x7f7c740fff00 2024-04-06T03:38:42.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.807+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7405f5f0 con 0x7f7c740fff00 2024-04-06T03:38:42.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.808+0000 7f7c6a7fc700 1 -- 172.21.15.94:0/2306728619 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7c6400dcd0 con 0x7f7c740fff00 2024-04-06T03:38:42.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.808+0000 7f7c6a7fc700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f7c600690a0 0x7f7c6006b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:42.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.808+0000 7f7c79b4f700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f7c600690a0 0x7f7c6006b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:42.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.808+0000 7f7c6a7fc700 1 -- 172.21.15.94:0/2306728619 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f7c64086960 con 0x7f7c740fff00 2024-04-06T03:38:42.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.809+0000 7f7c79b4f700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f7c600690a0 0x7f7c6006b550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009fd0 tx=0x7f7c70009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:42.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:42.813+0000 7f7c6a7fc700 1 -- 172.21.15.94:0/2306728619 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7c64052c90 con 0x7f7c740fff00 2024-04-06T03:38:42.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:42 smithi136 ceph-mon[36436]: pgmap v197: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-06T03:38:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.023+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c740f8f70 con 0x7f7c600690a0 2024-04-06T03:38:43.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.034+0000 7f7c6a7fc700 1 -- 172.21.15.94:0/2306728619 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f7c740f8f70 con 0x7f7c600690a0 2024-04-06T03:38:43.036 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (4m) 24s ago 6m 26.0M - 0.25.0 c8568f914cd2 b63c91af7786 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (6m) 24s ago 6m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (5m) 29s ago 5m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (4m) 24s ago 5m 86.1M - 9.4.7 954c08fa6188 78ad0d4cf52b 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (44s) 24s ago 44s 11.8M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (50s) 24s ago 49s 14.7M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (47s) 29s ago 46s 12.0M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (42s) 29s ago 41s 14.4M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:9283 running (7m) 24s ago 7m 480M - 17.2.7-819-g6699495c ec300eaa768b d83dfb730a39 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283 running (5m) 29s ago 5m 434M - 17.2.7-819-g6699495c ec300eaa768b 9e1727db1762 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (7m) 24s ago 7m 45.4M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (5m) 29s ago 5m 35.6M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (6m) 24s ago 6m 16.1M - 1.5.0 0da6a335fe13 f8f823e89f23 2024-04-06T03:38:43.037 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (5m) 29s ago 5m 17.1M - 1.5.0 0da6a335fe13 e5324db1ea82 2024-04-06T03:38:43.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (4m) 24s ago 4m 45.0M 4288M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:38:43.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (3m) 24s ago 3m 46.0M 4288M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:38:43.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (3m) 24s ago 3m 44.2M 4288M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:38:43.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (3m) 29s ago 3m 43.4M 5312M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:38:43.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (2m) 29s ago 2m 41.1M 5312M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:38:43.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (2m) 29s ago 2m 41.8M 5312M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:38:43.038 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (4m) 24s ago 5m 55.6M - 2.43.0 a07b618ecd1d 82a078c9e43c 2024-04-06T03:38:43.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f7c600690a0 msgr2=0x7f7c6006b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f7c600690a0 0x7f7c6006b550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009fd0 tx=0x7f7c70009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:43.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740fff00 msgr2=0x7f7c740fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740fff00 0x7f7c740fb8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7c64002a50 tx=0x7f7c640046b0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 shutdown_connections 2024-04-06T03:38:43.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f7c600690a0 0x7f7c6006b550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c740662e0 0x7f7c740fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 --2- 172.21.15.94:0/2306728619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c740fff00 0x7f7c740fb8c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 >> 172.21.15.94:0/2306728619 conn(0x7f7c740f7430 msgr2=0x7f7c740f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:43.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 shutdown_connections 2024-04-06T03:38:43.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.036+0000 7f7c7bdb3700 1 -- 172.21.15.94:0/2306728619 wait complete. 2024-04-06T03:38:43.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.147+0000 7fc674f1d700 1 -- 172.21.15.94:0/4031729612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fbcd0 msgr2=0x7fc6700fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.147+0000 7fc674f1d700 1 --2- 172.21.15.94:0/4031729612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fbcd0 0x7fc6700fc0e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc660004940 tx=0x7fc660009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:43.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.147+0000 7fc674f1d700 1 -- 172.21.15.94:0/4031729612 shutdown_connections 2024-04-06T03:38:43.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.147+0000 7fc674f1d700 1 --2- 172.21.15.94:0/4031729612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6700fcf80 0x7fc6700fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.147+0000 7fc674f1d700 1 --2- 172.21.15.94:0/4031729612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fbcd0 0x7fc6700fc0e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.147+0000 7fc674f1d700 1 -- 172.21.15.94:0/4031729612 >> 172.21.15.94:0/4031729612 conn(0x7fc6700f7430 msgr2=0x7fc6700f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:43.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.147+0000 7fc674f1d700 1 -- 172.21.15.94:0/4031729612 shutdown_connections 2024-04-06T03:38:43.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.147+0000 7fc674f1d700 1 -- 172.21.15.94:0/4031729612 wait complete. 2024-04-06T03:38:43.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc674f1d700 1 Processor -- start 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc674f1d700 1 -- start start 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc674f1d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6700fbcd0 0x7fc670110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc674f1d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fcf80 0x7fc670112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc674f1d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6700663e0 con 0x7fc6700fcf80 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc674f1d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc670066520 con 0x7fc6700fbcd0 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc667fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fcf80 0x7fc670112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc667fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fcf80 0x7fc670112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49646/0 (socket says 172.21.15.94:49646) 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc667fff700 1 -- 172.21.15.94:0/3540569679 learned_addr learned my addr 172.21.15.94:0/3540569679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.148+0000 7fc66e59c700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6700fbcd0 0x7fc670110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.149+0000 7fc667fff700 1 -- 172.21.15.94:0/3540569679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6700fbcd0 msgr2=0x7fc670110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.149+0000 7fc667fff700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6700fbcd0 0x7fc670110730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.149+0000 7fc667fff700 1 -- 172.21.15.94:0/3540569679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660005040 con 0x7fc6700fcf80 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.149+0000 7fc667fff700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fcf80 0x7fc670112c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc65800b790 tx=0x7fc658009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:43.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.149+0000 7fc666ffd700 1 -- 172.21.15.94:0/3540569679 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc65800f6f0 con 0x7fc6700fcf80 2024-04-06T03:38:43.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.149+0000 7fc666ffd700 1 -- 172.21.15.94:0/3540569679 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc65800fd20 con 0x7fc6700fcf80 2024-04-06T03:38:43.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.149+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6701132e0 con 0x7fc6700fcf80 2024-04-06T03:38:43.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.149+0000 7fc666ffd700 1 -- 172.21.15.94:0/3540569679 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6580103a0 con 0x7fc6700fcf80 2024-04-06T03:38:43.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.149+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc670113870 con 0x7fc6700fcf80 2024-04-06T03:38:43.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.150+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67005f5f0 con 0x7fc6700fcf80 2024-04-06T03:38:43.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.151+0000 7fc666ffd700 1 -- 172.21.15.94:0/3540569679 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc658011400 con 0x7fc6700fcf80 2024-04-06T03:38:43.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.151+0000 7fc666ffd700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc65c0692a0 0x7fc65c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:43.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.151+0000 7fc666ffd700 1 -- 172.21.15.94:0/3540569679 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc658086f70 con 0x7fc6700fcf80 2024-04-06T03:38:43.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.151+0000 7fc66e59c700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc65c0692a0 0x7fc65c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:43.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.152+0000 7fc66e59c700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc65c0692a0 0x7fc65c06b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc66000b970 tx=0x7fc660009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:43.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.156+0000 7fc666ffd700 1 -- 172.21.15.94:0/3540569679 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc6580532a0 con 0x7fc6700fcf80 2024-04-06T03:38:43.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:43 smithi094 ceph-mon[29713]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:43.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:38:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.446+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc67004bed0 con 0x7fc6700fcf80 2024-04-06T03:38:43.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.447+0000 7fc666ffd700 1 -- 172.21.15.94:0/3540569679 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fc6580568c0 con 0x7fc6700fcf80 2024-04-06T03:38:43.449 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:38:43.450 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:38:43.451 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 14 2024-04-06T03:38:43.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:38:43.451 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:38:43.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc65c0692a0 msgr2=0x7fc65c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc65c0692a0 0x7fc65c06b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc66000b970 tx=0x7fc660009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fcf80 msgr2=0x7fc670112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fcf80 0x7fc670112c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc65800b790 tx=0x7fc658009250 comp rx=0 tx=0).stop 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 shutdown_connections 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fc65c0692a0 0x7fc65c06b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6700fbcd0 0x7fc670110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 --2- 172.21.15.94:0/3540569679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6700fcf80 0x7fc670112c80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 >> 172.21.15.94:0/3540569679 conn(0x7fc6700f7430 msgr2=0x7fc6700f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 shutdown_connections 2024-04-06T03:38:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.449+0000 7fc674f1d700 1 -- 172.21.15.94:0/3540569679 wait complete. 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.555+0000 7f58f738e700 1 -- 172.21.15.94:0/2751588243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 msgr2=0x7f58f0100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.555+0000 7f58f738e700 1 --2- 172.21.15.94:0/2751588243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 0x7f58f0100710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f58e0004940 tx=0x7f58e0009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.555+0000 7f58f738e700 1 -- 172.21.15.94:0/2751588243 shutdown_connections 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.555+0000 7f58f738e700 1 --2- 172.21.15.94:0/2751588243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 0x7f58f0100710 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.555+0000 7f58f738e700 1 --2- 172.21.15.94:0/2751588243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58f00feff0 0x7f58f00ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.555+0000 7f58f738e700 1 -- 172.21.15.94:0/2751588243 >> 172.21.15.94:0/2751588243 conn(0x7f58f00fa950 msgr2=0x7f58f00fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.555+0000 7f58f738e700 1 -- 172.21.15.94:0/2751588243 shutdown_connections 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f738e700 1 -- 172.21.15.94:0/2751588243 wait complete. 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f738e700 1 Processor -- start 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f738e700 1 -- start start 2024-04-06T03:38:43.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f738e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58f00feff0 0x7f58f010fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f738e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 0x7f58f0110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f738e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f0110800 con 0x7f58f01002a0 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f738e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f0110ff0 con 0x7f58f00feff0 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f4929700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 0x7f58f0110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f4929700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 0x7f58f0110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49660/0 (socket says 172.21.15.94:49660) 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.556+0000 7f58f4929700 1 -- 172.21.15.94:0/3362960574 learned_addr learned my addr 172.21.15.94:0/3362960574 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58f512a700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58f00feff0 0x7f58f010fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58f4929700 1 -- 172.21.15.94:0/3362960574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58f00feff0 msgr2=0x7f58f010fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58f4929700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58f00feff0 0x7f58f010fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58f4929700 1 -- 172.21.15.94:0/3362960574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e0005040 con 0x7f58f01002a0 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58f512a700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58f00feff0 0x7f58f010fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58f4929700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 0x7f58f0110170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b3b0 tx=0x7f58e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58e5ffb700 1 -- 172.21.15.94:0/3362960574 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e000e040 con 0x7f58f01002a0 2024-04-06T03:38:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58e5ffb700 1 -- 172.21.15.94:0/3362960574 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58e000bcc0 con 0x7f58f01002a0 2024-04-06T03:38:43.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58e5ffb700 1 -- 172.21.15.94:0/3362960574 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e001c630 con 0x7f58f01002a0 2024-04-06T03:38:43.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f0111270 con 0x7f58f01002a0 2024-04-06T03:38:43.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.557+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f0111830 con 0x7f58f01002a0 2024-04-06T03:38:43.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.558+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f005f5f0 con 0x7f58f01002a0 2024-04-06T03:38:43.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.559+0000 7f58e5ffb700 1 -- 172.21.15.94:0/3362960574 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f58e0013980 con 0x7f58f01002a0 2024-04-06T03:38:43.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.559+0000 7f58e5ffb700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f58dc0692a0 0x7f58dc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:43.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.560+0000 7f58e5ffb700 1 -- 172.21.15.94:0/3362960574 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f58e0087ee0 con 0x7f58f01002a0 2024-04-06T03:38:43.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.560+0000 7f58f512a700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f58dc0692a0 0x7f58dc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:43.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.560+0000 7f58f512a700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f58dc0692a0 0x7f58dc06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009b10 tx=0x7f58ec008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:43.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.564+0000 7f58e5ffb700 1 -- 172.21.15.94:0/3362960574 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f58e0057ca0 con 0x7f58f01002a0 2024-04-06T03:38:43.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.808+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f58f004bed0 con 0x7f58f01002a0 2024-04-06T03:38:43.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.808+0000 7f58e5ffb700 1 -- 172.21.15.94:0/3362960574 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7f58e005be60 con 0x7f58f01002a0 2024-04-06T03:38:43.811 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:38:43.811 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:38:43.811 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:38:43.812 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:38:43.813 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:38:43.814 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:38:43.814 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:43.814 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:43.814 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:38:43.814 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:38:43.814 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:38:43.814 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:38:43.814 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f58dc0692a0 msgr2=0x7f58dc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f58dc0692a0 0x7f58dc06b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009b10 tx=0x7f58ec008040 comp rx=0 tx=0).stop 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 msgr2=0x7f58f0110170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 0x7f58f0110170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b3b0 tx=0x7f58e0009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 shutdown_connections 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f58dc0692a0 0x7f58dc06b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58f00feff0 0x7f58f010fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 --2- 172.21.15.94:0/3362960574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f01002a0 0x7f58f0110170 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 >> 172.21.15.94:0/3362960574 conn(0x7f58f00fa950 msgr2=0x7f58f00fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:43.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 shutdown_connections 2024-04-06T03:38:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.811+0000 7f58f738e700 1 -- 172.21.15.94:0/3362960574 wait complete. 2024-04-06T03:38:43.816 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:38:43.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:43 smithi136 ceph-mon[36436]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:43.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:38:43.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.918+0000 7f1db12e5700 1 -- 172.21.15.94:0/1607121968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 msgr2=0x7f1dac0fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.918+0000 7f1db12e5700 1 --2- 172.21.15.94:0/1607121968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 0x7f1dac0fc210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c004940 tx=0x7f1d9c009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:43.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.918+0000 7f1db12e5700 1 -- 172.21.15.94:0/1607121968 shutdown_connections 2024-04-06T03:38:43.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.918+0000 7f1db12e5700 1 --2- 172.21.15.94:0/1607121968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dac0fcf70 0x7f1dac0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.918+0000 7f1db12e5700 1 --2- 172.21.15.94:0/1607121968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 0x7f1dac0fc210 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.918+0000 7f1db12e5700 1 -- 172.21.15.94:0/1607121968 >> 172.21.15.94:0/1607121968 conn(0x7f1dac0f74e0 msgr2=0x7f1dac0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:43.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.918+0000 7f1db12e5700 1 -- 172.21.15.94:0/1607121968 shutdown_connections 2024-04-06T03:38:43.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.918+0000 7f1db12e5700 1 -- 172.21.15.94:0/1607121968 wait complete. 2024-04-06T03:38:43.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1db12e5700 1 Processor -- start 2024-04-06T03:38:43.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1db12e5700 1 -- start start 2024-04-06T03:38:43.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1db12e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 0x7f1dac10c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:43.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1db12e5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dac0fcf70 0x7f1dac10ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:43.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1db12e5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac10d4e0 con 0x7f1dac0fbe00 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1db12e5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac10dcd0 con 0x7f1dac0fcf70 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1daaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 0x7f1dac10c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1daaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 0x7f1dac10c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49676/0 (socket says 172.21.15.94:49676) 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1daaffd700 1 -- 172.21.15.94:0/1958852893 learned_addr learned my addr 172.21.15.94:0/1958852893 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1daa7fc700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dac0fcf70 0x7f1dac10ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1daaffd700 1 -- 172.21.15.94:0/1958852893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dac0fcf70 msgr2=0x7f1dac10ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1daaffd700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dac0fcf70 0x7f1dac10ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1daaffd700 1 -- 172.21.15.94:0/1958852893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c005040 con 0x7f1dac0fbe00 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.919+0000 7f1daaffd700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 0x7f1dac10c910 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00b3b0 tx=0x7f1d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.920+0000 7f1d937fe700 1 -- 172.21.15.94:0/1958852893 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9c00e040 con 0x7f1dac0fbe00 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.920+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac10df50 con 0x7f1dac0fbe00 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.920+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac10e510 con 0x7f1dac0fbe00 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.920+0000 7f1d937fe700 1 -- 172.21.15.94:0/1958852893 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d9c00bcc0 con 0x7f1dac0fbe00 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.920+0000 7f1d937fe700 1 -- 172.21.15.94:0/1958852893 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9c01c630 con 0x7f1dac0fbe00 2024-04-06T03:38:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.921+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d980052e0 con 0x7f1dac0fbe00 2024-04-06T03:38:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.921+0000 7f1d937fe700 1 -- 172.21.15.94:0/1958852893 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1d9c013420 con 0x7f1dac0fbe00 2024-04-06T03:38:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.922+0000 7f1d937fe700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1d940692a0 0x7f1d9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.922+0000 7f1d937fe700 1 -- 172.21.15.94:0/1958852893 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1d9c087e60 con 0x7f1dac0fbe00 2024-04-06T03:38:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.922+0000 7f1daa7fc700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1d940692a0 0x7f1d9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.922+0000 7f1daa7fc700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1d940692a0 0x7f1d9406b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1da0006fd0 tx=0x7f1da0008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:43.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:43.926+0000 7f1d937fe700 1 -- 172.21.15.94:0/1958852893 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1d9c057c20 con 0x7f1dac0fbe00 2024-04-06T03:38:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.151+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d98000bf0 con 0x7f1d940692a0 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.152+0000 7f1d937fe700 1 -- 172.21.15.94:0/1958852893 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f1d98000bf0 con 0x7f1d940692a0 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6", 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "", 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 image", 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:38:44.155 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:38:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1d940692a0 msgr2=0x7f1d9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1d940692a0 0x7f1d9406b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1da0006fd0 tx=0x7f1da0008040 comp rx=0 tx=0).stop 2024-04-06T03:38:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 msgr2=0x7f1dac10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 0x7f1dac10c910 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00b3b0 tx=0x7f1d9c009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 shutdown_connections 2024-04-06T03:38:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1d940692a0 0x7f1d9406b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dac0fbe00 0x7f1dac10c910 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 --2- 172.21.15.94:0/1958852893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1dac0fcf70 0x7f1dac10ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 >> 172.21.15.94:0/1958852893 conn(0x7f1dac0f74e0 msgr2=0x7f1dac0f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.154+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 shutdown_connections 2024-04-06T03:38:44.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.155+0000 7f1db12e5700 1 -- 172.21.15.94:0/1958852893 wait complete. 2024-04-06T03:38:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.258+0000 7f3bba1e8700 1 -- 172.21.15.94:0/898542485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 msgr2=0x7f3bb40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.258+0000 7f3bba1e8700 1 --2- 172.21.15.94:0/898542485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 0x7f3bb40fc0e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4004940 tx=0x7f3ba4009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.258+0000 7f3bba1e8700 1 -- 172.21.15.94:0/898542485 shutdown_connections 2024-04-06T03:38:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.258+0000 7f3bba1e8700 1 --2- 172.21.15.94:0/898542485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb40fcf80 0x7f3bb40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.258+0000 7f3bba1e8700 1 --2- 172.21.15.94:0/898542485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 0x7f3bb40fc0e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.258+0000 7f3bba1e8700 1 -- 172.21.15.94:0/898542485 >> 172.21.15.94:0/898542485 conn(0x7f3bb40f7430 msgr2=0x7f3bb40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.258+0000 7f3bba1e8700 1 -- 172.21.15.94:0/898542485 shutdown_connections 2024-04-06T03:38:44.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.258+0000 7f3bba1e8700 1 -- 172.21.15.94:0/898542485 wait complete. 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bba1e8700 1 Processor -- start 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bba1e8700 1 -- start start 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bba1e8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 0x7f3bb4110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bba1e8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb40fcf80 0x7f3bb4112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bba1e8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb4066370 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bba1e8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb40664b0 con 0x7f3bb40fcf80 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 0x7f3bb4110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 0x7f3bb4110700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49698/0 (socket says 172.21.15.94:49698) 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bb37fe700 1 -- 172.21.15.94:0/3473629633 learned_addr learned my addr 172.21.15.94:0/3473629633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bb2ffd700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb40fcf80 0x7f3bb4112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bb37fe700 1 -- 172.21.15.94:0/3473629633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb40fcf80 msgr2=0x7f3bb4112c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bb37fe700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb40fcf80 0x7f3bb4112c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.259+0000 7f3bb37fe700 1 -- 172.21.15.94:0/3473629633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4005040 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.260+0000 7f3bb37fe700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 0x7f3bb4110700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b3b0 tx=0x7f3ba4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.260+0000 7f3b9bfff700 1 -- 172.21.15.94:0/3473629633 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba400e040 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.260+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb41131f0 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.260+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb4113780 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.260+0000 7f3b9bfff700 1 -- 172.21.15.94:0/3473629633 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ba400bcc0 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.260+0000 7f3b9bfff700 1 -- 172.21.15.94:0/3473629633 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba401c630 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.261+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba00052e0 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.262+0000 7f3b9bfff700 1 -- 172.21.15.94:0/3473629633 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3ba4013980 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.262+0000 7f3b9bfff700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3b9c0692a0 0x7f3b9c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.262+0000 7f3b9bfff700 1 -- 172.21.15.94:0/3473629633 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3ba4087f20 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.262+0000 7f3bb2ffd700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3b9c0692a0 0x7f3b9c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:38:44.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.263+0000 7f3bb2ffd700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3b9c0692a0 0x7f3b9c06b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8004990 tx=0x7f3ba8009ee0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:38:44.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.268+0000 7f3b9bfff700 1 -- 172.21.15.94:0/3473629633 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3ba40ba190 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:44 smithi094 ceph-mon[29713]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:44 smithi094 ceph-mon[29713]: pgmap v198: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-06T03:38:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:44 smithi094 ceph-mon[29713]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:44 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3540569679' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:38:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:44 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3362960574' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:38:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.569+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3ba0005590 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.569+0000 7f3b9bfff700 1 -- 172.21.15.94:0/3473629633 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3ba40541d0 con 0x7f3bb40fbcd0 2024-04-06T03:38:44.572 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-04-06T03:38:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.571+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3b9c0692a0 msgr2=0x7f3b9c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.571+0000 7f3bba1e8700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3b9c0692a0 0x7f3b9c06b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8004990 tx=0x7f3ba8009ee0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.571+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 msgr2=0x7f3bb4110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:38:44.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.571+0000 7f3bba1e8700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 0x7f3bb4110700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b3b0 tx=0x7f3ba4009e30 comp rx=0 tx=0).stop 2024-04-06T03:38:44.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.571+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 shutdown_connections 2024-04-06T03:38:44.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.571+0000 7f3bba1e8700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f3b9c0692a0 0x7f3b9c06b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.571+0000 7f3bba1e8700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bb40fbcd0 0x7f3bb4110700 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.571+0000 7f3bba1e8700 1 --2- 172.21.15.94:0/3473629633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb40fcf80 0x7f3bb4112c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:38:44.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.571+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 >> 172.21.15.94:0/3473629633 conn(0x7f3bb40f7430 msgr2=0x7f3bb40f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:38:44.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.572+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 shutdown_connections 2024-04-06T03:38:44.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:38:44.572+0000 7f3bba1e8700 1 -- 172.21.15.94:0/3473629633 wait complete. 2024-04-06T03:38:44.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:44 smithi136 ceph-mon[36436]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:44.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:44 smithi136 ceph-mon[36436]: pgmap v198: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-06T03:38:44.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:44 smithi136 ceph-mon[36436]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:44.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:44 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3540569679' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:38:44.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:44 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3362960574' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:38:45.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:45 smithi094 ceph-mon[29713]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:45.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:45 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3473629633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:38:45.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:45 smithi136 ceph-mon[36436]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:38:45.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:45 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3473629633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:38:46.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:46 smithi094 ceph-mon[29713]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:46.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:46 smithi136 ceph-mon[36436]: pgmap v199: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:48.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:48 smithi094 ceph-mon[29713]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:48.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:48 smithi136 ceph-mon[36436]: pgmap v200: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:50.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:50 smithi094 ceph-mon[29713]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:50.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:50 smithi136 ceph-mon[36436]: pgmap v201: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:52.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:52 smithi094 ceph-mon[29713]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:52.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:52 smithi136 ceph-mon[36436]: pgmap v202: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:54.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:54 smithi094 ceph-mon[29713]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:54.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:54 smithi136 ceph-mon[36436]: pgmap v203: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:56.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:56 smithi094 ceph-mon[29713]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:56 smithi136 ceph-mon[36436]: pgmap v204: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:58.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:58 smithi094 ceph-mon[29713]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:58.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:38:58 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:38:58.738 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:58 smithi136 ceph-mon[36436]: pgmap v205: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:38:58.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:38:58 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:39:00.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:00 smithi094 ceph-mon[29713]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:00.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:00 smithi136 ceph-mon[36436]: pgmap v206: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:02.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:02 smithi094 ceph-mon[29713]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:02.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:02 smithi136 ceph-mon[36436]: pgmap v207: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:04.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:04 smithi094 ceph-mon[29713]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-06T03:39:04.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:04 smithi136 ceph-mon[36436]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-06T03:39:06.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:06 smithi094 ceph-mon[29713]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-06T03:39:06.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:06 smithi136 ceph-mon[36436]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-06T03:39:08.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:08 smithi094 ceph-mon[29713]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-06T03:39:08.738 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:08 smithi136 ceph-mon[36436]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-06T03:39:10.649 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:10 smithi094 ceph-mon[29713]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-06T03:39:10.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:10 smithi136 ceph-mon[36436]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-06T03:39:12.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:12 smithi094 ceph-mon[29713]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-06T03:39:12.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:12.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:39:12.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:12 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:12.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:12 smithi136 ceph-mon[36436]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-06T03:39:12.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:12.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:39:12.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:12 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:13.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:13 smithi094 ceph-mon[29713]: Upgrade: Target is version 19.0.0-2027-g9f192350 (unknown) 2024-04-06T03:39:13.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:13 smithi094 ceph-mon[29713]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c'] 2024-04-06T03:39:13.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:13 smithi094 ceph-mon[29713]: Upgrade: Need to upgrade myself (mgr.smithi094.purlzp) 2024-04-06T03:39:13.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:13 smithi094 ceph-mon[29713]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c on smithi136 2024-04-06T03:39:13.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:13 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:39:13.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:13 smithi136 ceph-mon[36436]: Upgrade: Target is version 19.0.0-2027-g9f192350 (unknown) 2024-04-06T03:39:13.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:13 smithi136 ceph-mon[36436]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c'] 2024-04-06T03:39:13.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:13 smithi136 ceph-mon[36436]: Upgrade: Need to upgrade myself (mgr.smithi094.purlzp) 2024-04-06T03:39:13.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:13 smithi136 ceph-mon[36436]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c on smithi136 2024-04-06T03:39:13.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:13 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:39:14.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:14 smithi094 ceph-mon[29713]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-06T03:39:14.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.732+0000 7fdb73703700 1 -- 172.21.15.94:0/3560689384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c0fd300 msgr2=0x7fdb6c0fd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:14.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.732+0000 7fdb73703700 1 --2- 172.21.15.94:0/3560689384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c0fd300 0x7fdb6c0fd710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdb68004940 tx=0x7fdb68009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:14.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.732+0000 7fdb73703700 1 -- 172.21.15.94:0/3560689384 shutdown_connections 2024-04-06T03:39:14.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.732+0000 7fdb73703700 1 --2- 172.21.15.94:0/3560689384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb6c0fdc50 0x7fdb6c0fc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:14.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.732+0000 7fdb73703700 1 --2- 172.21.15.94:0/3560689384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c0fd300 0x7fdb6c0fd710 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:14.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.732+0000 7fdb73703700 1 -- 172.21.15.94:0/3560689384 >> 172.21.15.94:0/3560689384 conn(0x7fdb6c0f6be0 msgr2=0x7fdb6c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:14.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.732+0000 7fdb73703700 1 -- 172.21.15.94:0/3560689384 shutdown_connections 2024-04-06T03:39:14.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.732+0000 7fdb73703700 1 -- 172.21.15.94:0/3560689384 wait complete. 2024-04-06T03:39:14.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.733+0000 7fdb73703700 1 Processor -- start 2024-04-06T03:39:14.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.733+0000 7fdb73703700 1 -- start start 2024-04-06T03:39:14.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.733+0000 7fdb73703700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb6c0fdc50 0x7fdb6c10da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:14.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.733+0000 7fdb73703700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c10dfb0 0x7fdb6c112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.733+0000 7fdb7149f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb6c0fdc50 0x7fdb6c10da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb70c9e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c10dfb0 0x7fdb6c112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb70c9e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c10dfb0 0x7fdb6c112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53544/0 (socket says 172.21.15.94:53544) 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb7149f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb6c0fdc50 0x7fdb6c10da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:34336/0 (socket says 172.21.15.94:34336) 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb70c9e700 1 -- 172.21.15.94:0/4175818755 learned_addr learned my addr 172.21.15.94:0/4175818755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c10e470 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c10e5b0 con 0x7fdb6c0fdc50 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb70c9e700 1 -- 172.21.15.94:0/4175818755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb6c0fdc50 msgr2=0x7fdb6c10da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb70c9e700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb6c0fdc50 0x7fdb6c10da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb70c9e700 1 -- 172.21.15.94:0/4175818755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb68005040 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.734+0000 7fdb70c9e700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c10dfb0 0x7fdb6c112fe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00eb40 tx=0x7fdb5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.735+0000 7fdb61ffb700 1 -- 172.21.15.94:0/4175818755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5c007cc0 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.735+0000 7fdb61ffb700 1 -- 172.21.15.94:0/4175818755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb5c00f040 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.735+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6c113520 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.735+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6c113950 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.735+0000 7fdb61ffb700 1 -- 172.21.15.94:0/4175818755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5c00c9d0 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.737+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb500052e0 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.737+0000 7fdb61ffb700 1 -- 172.21.15.94:0/4175818755 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdb5c00cb30 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.738+0000 7fdb61ffb700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdb580692a0 0x7fdb5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:14.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.738+0000 7fdb61ffb700 1 -- 172.21.15.94:0/4175818755 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fdb5c086a50 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.738+0000 7fdb7149f700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdb580692a0 0x7fdb5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:14.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.739+0000 7fdb7149f700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdb580692a0 0x7fdb5806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdb68000c00 tx=0x7fdb68009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:14.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:14.746+0000 7fdb61ffb700 1 -- 172.21.15.94:0/4175818755 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdb5c04e190 con 0x7fdb6c10dfb0 2024-04-06T03:39:14.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:14 smithi136 ceph-mon[36436]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-06T03:39:15.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.160+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb50000bf0 con 0x7fdb580692a0 2024-04-06T03:39:15.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.162+0000 7fdb61ffb700 1 -- 172.21.15.94:0/4175818755 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fdb50000bf0 con 0x7fdb580692a0 2024-04-06T03:39:15.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.164+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdb580692a0 msgr2=0x7fdb5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:15.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.164+0000 7fdb73703700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdb580692a0 0x7fdb5806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdb68000c00 tx=0x7fdb68009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:15.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.164+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c10dfb0 msgr2=0x7fdb6c112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:15.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.164+0000 7fdb73703700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c10dfb0 0x7fdb6c112fe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00eb40 tx=0x7fdb5c00b040 comp rx=0 tx=0).stop 2024-04-06T03:39:15.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.164+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 shutdown_connections 2024-04-06T03:39:15.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.164+0000 7fdb73703700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fdb580692a0 0x7fdb5806b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.164+0000 7fdb73703700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb6c0fdc50 0x7fdb6c10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.164+0000 7fdb73703700 1 --2- 172.21.15.94:0/4175818755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb6c10dfb0 0x7fdb6c112fe0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.164+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 >> 172.21.15.94:0/4175818755 conn(0x7fdb6c0f6be0 msgr2=0x7fdb6c0f8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:15.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.165+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 shutdown_connections 2024-04-06T03:39:15.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.165+0000 7fdb73703700 1 -- 172.21.15.94:0/4175818755 wait complete. 2024-04-06T03:39:15.182 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:39:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.314+0000 7f174d942700 1 -- 172.21.15.94:0/3634361313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748102360 msgr2=0x7f1748104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.314+0000 7f174d942700 1 --2- 172.21.15.94:0/3634361313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748102360 0x7f1748104740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f173c004940 tx=0x7f173c009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.314+0000 7f174d942700 1 -- 172.21.15.94:0/3634361313 shutdown_connections 2024-04-06T03:39:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.314+0000 7f174d942700 1 --2- 172.21.15.94:0/3634361313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748102360 0x7f1748104740 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.314+0000 7f174d942700 1 --2- 172.21.15.94:0/3634361313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748066280 0x7f1748101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.314+0000 7f174d942700 1 -- 172.21.15.94:0/3634361313 >> 172.21.15.94:0/3634361313 conn(0x7f17480f7430 msgr2=0x7f17480f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:15.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.314+0000 7f174d942700 1 -- 172.21.15.94:0/3634361313 shutdown_connections 2024-04-06T03:39:15.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.314+0000 7f174d942700 1 -- 172.21.15.94:0/3634361313 wait complete. 2024-04-06T03:39:15.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.315+0000 7f174d942700 1 Processor -- start 2024-04-06T03:39:15.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.315+0000 7f174d942700 1 -- start start 2024-04-06T03:39:15.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.315+0000 7f174d942700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748066280 0x7f174810fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:15.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.315+0000 7f174d942700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748110350 0x7f1748111070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:15.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.315+0000 7f174d942700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748110810 con 0x7f1748066280 2024-04-06T03:39:15.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.315+0000 7f174d942700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748110980 con 0x7f1748110350 2024-04-06T03:39:15.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f17467fc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748110350 0x7f1748111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f17467fc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748110350 0x7f1748111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:34350/0 (socket says 172.21.15.94:34350) 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f17467fc700 1 -- 172.21.15.94:0/1798917902 learned_addr learned my addr 172.21.15.94:0/1798917902 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f1746ffd700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748066280 0x7f174810fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f17467fc700 1 -- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748066280 msgr2=0x7f174810fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f17467fc700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748066280 0x7f174810fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f17467fc700 1 -- 172.21.15.94:0/1798917902 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c005040 con 0x7f1748110350 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f17467fc700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748110350 0x7f1748111070 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f173c013870 tx=0x7f173c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f172f7fe700 1 -- 172.21.15.94:0/1798917902 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173c00e030 con 0x7f1748110350 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17481116f0 con 0x7f1748110350 2024-04-06T03:39:15.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f172f7fe700 1 -- 172.21.15.94:0/1798917902 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f173c013de0 con 0x7f1748110350 2024-04-06T03:39:15.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.316+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748111c30 con 0x7f1748110350 2024-04-06T03:39:15.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.317+0000 7f172f7fe700 1 -- 172.21.15.94:0/1798917902 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173c00b410 con 0x7f1748110350 2024-04-06T03:39:15.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.318+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17340052e0 con 0x7f1748110350 2024-04-06T03:39:15.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.318+0000 7f172f7fe700 1 -- 172.21.15.94:0/1798917902 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f173c013910 con 0x7f1748110350 2024-04-06T03:39:15.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.319+0000 7f172f7fe700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f17300692a0 0x7f173006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:15.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.319+0000 7f172f7fe700 1 -- 172.21.15.94:0/1798917902 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f173c088ae0 con 0x7f1748110350 2024-04-06T03:39:15.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.319+0000 7f1746ffd700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f17300692a0 0x7f173006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:15.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.319+0000 7f1746ffd700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f17300692a0 0x7f173006b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f174810fb80 tx=0x7f173800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:15.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.327+0000 7f172f7fe700 1 -- 172.21.15.94:0/1798917902 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f173c054d70 con 0x7f1748110350 2024-04-06T03:39:15.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.742+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1734000bf0 con 0x7f17300692a0 2024-04-06T03:39:15.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.743+0000 7f172f7fe700 1 -- 172.21.15.94:0/1798917902 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f1734000bf0 con 0x7f17300692a0 2024-04-06T03:39:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f17300692a0 msgr2=0x7f173006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f17300692a0 0x7f173006b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f174810fb80 tx=0x7f173800b040 comp rx=0 tx=0).stop 2024-04-06T03:39:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748110350 msgr2=0x7f1748111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748110350 0x7f1748111070 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f173c013870 tx=0x7f173c009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 shutdown_connections 2024-04-06T03:39:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f17300692a0 0x7f173006b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1748066280 0x7f174810fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 --2- 172.21.15.94:0/1798917902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748110350 0x7f1748111070 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 >> 172.21.15.94:0/1798917902 conn(0x7f17480f7430 msgr2=0x7f17480f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:15.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 shutdown_connections 2024-04-06T03:39:15.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.746+0000 7f174d942700 1 -- 172.21.15.94:0/1798917902 wait complete. 2024-04-06T03:39:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.889+0000 7fe0295c1700 1 -- 172.21.15.94:0/1514428448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240662e0 msgr2=0x7fe0240ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.889+0000 7fe0295c1700 1 --2- 172.21.15.94:0/1514428448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240662e0 0x7fe0240ff9c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe014004940 tx=0x7fe014009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.889+0000 7fe0295c1700 1 -- 172.21.15.94:0/1514428448 shutdown_connections 2024-04-06T03:39:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.889+0000 7fe0295c1700 1 --2- 172.21.15.94:0/1514428448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0240fff00 0x7fe024102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.889+0000 7fe0295c1700 1 --2- 172.21.15.94:0/1514428448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240662e0 0x7fe0240ff9c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.889+0000 7fe0295c1700 1 -- 172.21.15.94:0/1514428448 >> 172.21.15.94:0/1514428448 conn(0x7fe0240f7430 msgr2=0x7fe0240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.889+0000 7fe0295c1700 1 -- 172.21.15.94:0/1514428448 shutdown_connections 2024-04-06T03:39:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.890+0000 7fe0295c1700 1 -- 172.21.15.94:0/1514428448 wait complete. 2024-04-06T03:39:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.890+0000 7fe0295c1700 1 Processor -- start 2024-04-06T03:39:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.890+0000 7fe0295c1700 1 -- start start 2024-04-06T03:39:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.890+0000 7fe0295c1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0240662e0 0x7fe02410b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.890+0000 7fe0295c1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240fff00 0x7fe02410bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.890+0000 7fe0295c1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02410c340 con 0x7fe0240fff00 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.890+0000 7fe0295c1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02410cb40 con 0x7fe0240662e0 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe0227fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240fff00 0x7fe02410bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe022ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0240662e0 0x7fe02410b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe0227fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240fff00 0x7fe02410bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53578/0 (socket says 172.21.15.94:53578) 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe0227fc700 1 -- 172.21.15.94:0/2234709383 learned_addr learned my addr 172.21.15.94:0/2234709383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe022ffd700 1 -- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240fff00 msgr2=0x7fe02410bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe022ffd700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240fff00 0x7fe02410bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe022ffd700 1 -- 172.21.15.94:0/2234709383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe014005040 con 0x7fe0240662e0 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe0227fc700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240fff00 0x7fe02410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe022ffd700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0240662e0 0x7fe02410b7b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe014009fd0 tx=0x7fe014009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe00b7fe700 1 -- 172.21.15.94:0/2234709383 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe014021070 con 0x7fe0240662e0 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.891+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02410cdc0 con 0x7fe0240662e0 2024-04-06T03:39:15.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.892+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02410d300 con 0x7fe0240662e0 2024-04-06T03:39:15.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.892+0000 7fe00b7fe700 1 -- 172.21.15.94:0/2234709383 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe014026410 con 0x7fe0240662e0 2024-04-06T03:39:15.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.892+0000 7fe00b7fe700 1 -- 172.21.15.94:0/2234709383 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe01400e070 con 0x7fe0240662e0 2024-04-06T03:39:15.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.893+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02405f5f0 con 0x7fe0240662e0 2024-04-06T03:39:15.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.893+0000 7fe00b7fe700 1 -- 172.21.15.94:0/2234709383 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe0140269f0 con 0x7fe0240662e0 2024-04-06T03:39:15.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.894+0000 7fe00b7fe700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe00c0692b0 0x7fe00c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:15.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.894+0000 7fe00b7fe700 1 -- 172.21.15.94:0/2234709383 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe0140882b0 con 0x7fe0240662e0 2024-04-06T03:39:15.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.894+0000 7fe0227fc700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe00c0692b0 0x7fe00c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:15.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.894+0000 7fe0227fc700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe00c0692b0 0x7fe00c06b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe0180041d0 tx=0x7fe01800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:15.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:15.899+0000 7fe00b7fe700 1 -- 172.21.15.94:0/2234709383 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe0140545e0 con 0x7fe0240662e0 2024-04-06T03:39:16.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.116+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe02410d540 con 0x7fe00c0692b0 2024-04-06T03:39:16.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.133+0000 7fe00b7fe700 1 -- 172.21.15.94:0/2234709383 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fe02410d540 con 0x7fe00c0692b0 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (5m) 57s ago 6m 26.0M - 0.25.0 c8568f914cd2 b63c91af7786 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (6m) 57s ago 6m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (5m) 63s ago 5m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (5m) 57s ago 6m 86.1M - 9.4.7 954c08fa6188 78ad0d4cf52b 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (77s) 57s ago 77s 11.8M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (83s) 57s ago 82s 14.7M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (80s) 63s ago 80s 12.0M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (75s) 63s ago 74s 14.4M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:9283 running (7m) 57s ago 7m 480M - 17.2.7-819-g6699495c ec300eaa768b d83dfb730a39 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283 running (5m) 63s ago 5m 434M - 17.2.7-819-g6699495c ec300eaa768b 9e1727db1762 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (7m) 57s ago 8m 45.4M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (5m) 63s ago 5m 35.6M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:39:16.136 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (6m) 57s ago 6m 16.1M - 1.5.0 0da6a335fe13 f8f823e89f23 2024-04-06T03:39:16.137 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (5m) 63s ago 5m 17.1M - 1.5.0 0da6a335fe13 e5324db1ea82 2024-04-06T03:39:16.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (4m) 57s ago 4m 45.0M 4288M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:39:16.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (4m) 57s ago 4m 46.0M 4288M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:39:16.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (4m) 57s ago 4m 44.2M 4288M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:39:16.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (3m) 63s ago 3m 43.4M 5312M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:39:16.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (3m) 63s ago 3m 41.1M 5312M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:39:16.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (2m) 63s ago 2m 41.8M 5312M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:39:16.137 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (5m) 57s ago 6m 55.6M - 2.43.0 a07b618ecd1d 82a078c9e43c 2024-04-06T03:39:16.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.135+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe00c0692b0 msgr2=0x7fe00c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.135+0000 7fe0295c1700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe00c0692b0 0x7fe00c06b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe0180041d0 tx=0x7fe01800b040 comp rx=0 tx=0).stop 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.135+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0240662e0 msgr2=0x7fe02410b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.135+0000 7fe0295c1700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0240662e0 0x7fe02410b7b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe014009fd0 tx=0x7fe014009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.135+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 shutdown_connections 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.135+0000 7fe0295c1700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe00c0692b0 0x7fe00c06b760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.135+0000 7fe0295c1700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0240662e0 0x7fe02410b7b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.135+0000 7fe0295c1700 1 --2- 172.21.15.94:0/2234709383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0240fff00 0x7fe02410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.135+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 >> 172.21.15.94:0/2234709383 conn(0x7fe0240f7430 msgr2=0x7fe0240fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.136+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 shutdown_connections 2024-04-06T03:39:16.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.136+0000 7fe0295c1700 1 -- 172.21.15.94:0/2234709383 wait complete. 2024-04-06T03:39:16.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.282+0000 7fe2837cd700 1 -- 172.21.15.94:0/3383493239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 msgr2=0x7fe27c0fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.282+0000 7fe2837cd700 1 --2- 172.21.15.94:0/3383493239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 0x7fe27c0fe4b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe26c004940 tx=0x7fe26c009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:16.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.282+0000 7fe2837cd700 1 -- 172.21.15.94:0/3383493239 shutdown_connections 2024-04-06T03:39:16.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.282+0000 7fe2837cd700 1 --2- 172.21.15.94:0/3383493239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 0x7fe27c0fe4b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.282+0000 7fe2837cd700 1 --2- 172.21.15.94:0/3383493239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe27c0fce40 0x7fe27c0fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.282+0000 7fe2837cd700 1 -- 172.21.15.94:0/3383493239 >> 172.21.15.94:0/3383493239 conn(0x7fe27c0f8550 msgr2=0x7fe27c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:16.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.282+0000 7fe2837cd700 1 -- 172.21.15.94:0/3383493239 shutdown_connections 2024-04-06T03:39:16.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.282+0000 7fe2837cd700 1 -- 172.21.15.94:0/3383493239 wait complete. 2024-04-06T03:39:16.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe2837cd700 1 Processor -- start 2024-04-06T03:39:16.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe2837cd700 1 -- start start 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe2837cd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe27c0fce40 0x7fe27c10d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe2837cd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 0x7fe27c10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe2837cd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c10e510 con 0x7fe27c0fe040 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe2837cd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c10ed60 con 0x7fe27c0fce40 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe280d68700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 0x7fe27c10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe280d68700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 0x7fe27c10df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53588/0 (socket says 172.21.15.94:53588) 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe280d68700 1 -- 172.21.15.94:0/3672356614 learned_addr learned my addr 172.21.15.94:0/3672356614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.283+0000 7fe281569700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe27c0fce40 0x7fe27c10d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.284+0000 7fe280d68700 1 -- 172.21.15.94:0/3672356614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe27c0fce40 msgr2=0x7fe27c10d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.284+0000 7fe280d68700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe27c0fce40 0x7fe27c10d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.284+0000 7fe280d68700 1 -- 172.21.15.94:0/3672356614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c005040 con 0x7fe27c0fe040 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.284+0000 7fe281569700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe27c0fce40 0x7fe27c10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.284+0000 7fe280d68700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 0x7fe27c10df10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe26c013870 tx=0x7fe26c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:16.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.284+0000 7fe271ffb700 1 -- 172.21.15.94:0/3672356614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe26c00e070 con 0x7fe27c0fe040 2024-04-06T03:39:16.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.284+0000 7fe271ffb700 1 -- 172.21.15.94:0/3672356614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe26c01c400 con 0x7fe27c0fe040 2024-04-06T03:39:16.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.284+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c10efe0 con 0x7fe27c0fe040 2024-04-06T03:39:16.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.284+0000 7fe271ffb700 1 -- 172.21.15.94:0/3672356614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe26c00b630 con 0x7fe27c0fe040 2024-04-06T03:39:16.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.285+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c10f470 con 0x7fe27c0fe040 2024-04-06T03:39:16.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.286+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c05f5f0 con 0x7fe27c0fe040 2024-04-06T03:39:16.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.286+0000 7fe271ffb700 1 -- 172.21.15.94:0/3672356614 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe26c01c570 con 0x7fe27c0fe040 2024-04-06T03:39:16.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.287+0000 7fe271ffb700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe2680692b0 0x7fe26806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:16.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.287+0000 7fe271ffb700 1 -- 172.21.15.94:0/3672356614 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe26c088950 con 0x7fe27c0fe040 2024-04-06T03:39:16.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.287+0000 7fe281569700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe2680692b0 0x7fe26806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:16.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.287+0000 7fe281569700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe2680692b0 0x7fe26806b760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe278004600 tx=0x7fe27800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:16.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.292+0000 7fe271ffb700 1 -- 172.21.15.94:0/3672356614 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe26c058710 con 0x7fe27c0fe040 2024-04-06T03:39:16.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.574+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe27c04bed0 con 0x7fe27c0fe040 2024-04-06T03:39:16.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.574+0000 7fe271ffb700 1 -- 172.21.15.94:0/3672356614 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fe26c058710 con 0x7fe27c0fe040 2024-04-06T03:39:16.577 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:39:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:39:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:39:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:39:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:39:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:39:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 14 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:39:16.578 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:39:16.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.576+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe2680692b0 msgr2=0x7fe26806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.576+0000 7fe2837cd700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe2680692b0 0x7fe26806b760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe278004600 tx=0x7fe27800b040 comp rx=0 tx=0).stop 2024-04-06T03:39:16.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.576+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 msgr2=0x7fe27c10df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.576+0000 7fe2837cd700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 0x7fe27c10df10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe26c013870 tx=0x7fe26c009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:16.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.576+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 shutdown_connections 2024-04-06T03:39:16.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.576+0000 7fe2837cd700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fe2680692b0 0x7fe26806b760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.576+0000 7fe2837cd700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe27c0fce40 0x7fe27c10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.576+0000 7fe2837cd700 1 --2- 172.21.15.94:0/3672356614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe27c0fe040 0x7fe27c10df10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.576+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 >> 172.21.15.94:0/3672356614 conn(0x7fe27c0f8550 msgr2=0x7fe27c0f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:16.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.577+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 shutdown_connections 2024-04-06T03:39:16.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.577+0000 7fe2837cd700 1 -- 172.21.15.94:0/3672356614 wait complete. 2024-04-06T03:39:16.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.679+0000 7ff45dedf700 1 -- 172.21.15.94:0/1825631068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff458066b70 msgr2=0x7ff4580ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.679+0000 7ff45dedf700 1 --2- 172.21.15.94:0/1825631068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff458066b70 0x7ff4580ff9e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff448004880 tx=0x7ff448009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:16.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.679+0000 7ff45dedf700 1 -- 172.21.15.94:0/1825631068 shutdown_connections 2024-04-06T03:39:16.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.679+0000 7ff45dedf700 1 --2- 172.21.15.94:0/1825631068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4580fff20 0x7ff458102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.679+0000 7ff45dedf700 1 --2- 172.21.15.94:0/1825631068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff458066b70 0x7ff4580ff9e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.679+0000 7ff45dedf700 1 -- 172.21.15.94:0/1825631068 >> 172.21.15.94:0/1825631068 conn(0x7ff4580f7450 msgr2=0x7ff4580f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:16.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.681+0000 7ff45dedf700 1 -- 172.21.15.94:0/1825631068 shutdown_connections 2024-04-06T03:39:16.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.681+0000 7ff45dedf700 1 -- 172.21.15.94:0/1825631068 wait complete. 2024-04-06T03:39:16.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.682+0000 7ff45dedf700 1 Processor -- start 2024-04-06T03:39:16.684 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:16 smithi094 ceph-mon[29713]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-06T03:39:16.685 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:16 smithi094 ceph-mon[29713]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:16.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.682+0000 7ff45dedf700 1 -- start start 2024-04-06T03:39:16.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.682+0000 7ff45dedf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4580fff20 0x7ff4580fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:16.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.682+0000 7ff45dedf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4580fb8e0 0x7ff4580fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:16.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.682+0000 7ff45dedf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580fc290 con 0x7ff4580fb8e0 2024-04-06T03:39:16.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.682+0000 7ff45dedf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4580fc400 con 0x7ff4580fff20 2024-04-06T03:39:16.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.683+0000 7ff456ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4580fb8e0 0x7ff4580fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:16.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.683+0000 7ff456ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4580fb8e0 0x7ff4580fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53614/0 (socket says 172.21.15.94:53614) 2024-04-06T03:39:16.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.683+0000 7ff456ffd700 1 -- 172.21.15.94:0/3794449216 learned_addr learned my addr 172.21.15.94:0/3794449216 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:16.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.683+0000 7ff4577fe700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4580fff20 0x7ff4580fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:16.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.683+0000 7ff456ffd700 1 -- 172.21.15.94:0/3794449216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4580fff20 msgr2=0x7ff4580fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.683+0000 7ff456ffd700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4580fff20 0x7ff4580fd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.683+0000 7ff456ffd700 1 -- 172.21.15.94:0/3794449216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff448005040 con 0x7ff4580fb8e0 2024-04-06T03:39:16.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.684+0000 7ff456ffd700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4580fb8e0 0x7ff4580fbd50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0041d0 tx=0x7ff44c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:16.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.684+0000 7ff43ffff700 1 -- 172.21.15.94:0/3794449216 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff44c007cc0 con 0x7ff4580fb8e0 2024-04-06T03:39:16.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.684+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4580fc680 con 0x7ff4580fb8e0 2024-04-06T03:39:16.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.684+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff458115f10 con 0x7ff4580fb8e0 2024-04-06T03:39:16.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.684+0000 7ff43ffff700 1 -- 172.21.15.94:0/3794449216 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff44c00f040 con 0x7ff4580fb8e0 2024-04-06T03:39:16.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.685+0000 7ff43ffff700 1 -- 172.21.15.94:0/3794449216 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff44c00c990 con 0x7ff4580fb8e0 2024-04-06T03:39:16.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.686+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4440052e0 con 0x7ff4580fb8e0 2024-04-06T03:39:16.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.686+0000 7ff43ffff700 1 -- 172.21.15.94:0/3794449216 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff44c00caf0 con 0x7ff4580fb8e0 2024-04-06T03:39:16.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.687+0000 7ff43ffff700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff4400692a0 0x7ff44006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:16.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.687+0000 7ff43ffff700 1 -- 172.21.15.94:0/3794449216 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff44c086ba0 con 0x7ff4580fb8e0 2024-04-06T03:39:16.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.687+0000 7ff4577fe700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff4400692a0 0x7ff44006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:16.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.687+0000 7ff4577fe700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff4400692a0 0x7ff44006b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff448013710 tx=0x7ff448009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:16.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.691+0000 7ff43ffff700 1 -- 172.21.15.94:0/3794449216 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff44c04e190 con 0x7ff4580fb8e0 2024-04-06T03:39:16.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:16 smithi136 ceph-mon[36436]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-06T03:39:16.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:16 smithi136 ceph-mon[36436]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:16.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.928+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff444005da0 con 0x7ff4580fb8e0 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.929+0000 7ff43ffff700 1 -- 172.21.15.94:0/3794449216 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7ff44c056990 con 0x7ff4580fb8e0 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:39:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:39:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:39:16.936 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:39:16.937 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:39:16.937 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:39:16.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.931+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff4400692a0 msgr2=0x7ff44006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.931+0000 7ff45dedf700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff4400692a0 0x7ff44006b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff448013710 tx=0x7ff448009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.931+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4580fb8e0 msgr2=0x7ff4580fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.931+0000 7ff45dedf700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4580fb8e0 0x7ff4580fbd50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0041d0 tx=0x7ff44c00b040 comp rx=0 tx=0).stop 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.931+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 shutdown_connections 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.931+0000 7ff45dedf700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff4400692a0 0x7ff44006b750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.931+0000 7ff45dedf700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4580fff20 0x7ff4580fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.931+0000 7ff45dedf700 1 --2- 172.21.15.94:0/3794449216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4580fb8e0 0x7ff4580fbd50 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0041d0 tx=0x7ff44c00b040 comp rx=0 tx=0).stop 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.931+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 >> 172.21.15.94:0/3794449216 conn(0x7ff4580f7450 msgr2=0x7ff4580f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.932+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 shutdown_connections 2024-04-06T03:39:16.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:16.935+0000 7ff45dedf700 1 -- 172.21.15.94:0/3794449216 wait complete. 2024-04-06T03:39:16.939 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:39:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.045+0000 7fb35bd4a700 1 -- 172.21.15.94:0/1482878010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 msgr2=0x7fb3540fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.045+0000 7fb35bd4a700 1 --2- 172.21.15.94:0/1482878010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 0x7fb3540fe4e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb344004940 tx=0x7fb344009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.045+0000 7fb35bd4a700 1 -- 172.21.15.94:0/1482878010 shutdown_connections 2024-04-06T03:39:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.045+0000 7fb35bd4a700 1 --2- 172.21.15.94:0/1482878010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 0x7fb3540fe4e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.045+0000 7fb35bd4a700 1 --2- 172.21.15.94:0/1482878010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3540fcf00 0x7fb3540fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.045+0000 7fb35bd4a700 1 -- 172.21.15.94:0/1482878010 >> 172.21.15.94:0/1482878010 conn(0x7fb3540f85a0 msgr2=0x7fb3540fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.045+0000 7fb35bd4a700 1 -- 172.21.15.94:0/1482878010 shutdown_connections 2024-04-06T03:39:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.045+0000 7fb35bd4a700 1 -- 172.21.15.94:0/1482878010 wait complete. 2024-04-06T03:39:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb35bd4a700 1 Processor -- start 2024-04-06T03:39:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb35bd4a700 1 -- start start 2024-04-06T03:39:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb35bd4a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3540fcf00 0x7fb354071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb35bd4a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 0x7fb354070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb35bd4a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3541116e0 con 0x7fb3540fe070 2024-04-06T03:39:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb35bd4a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb354111850 con 0x7fb3540fcf00 2024-04-06T03:39:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb3592e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 0x7fb354070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb3592e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 0x7fb354070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53638/0 (socket says 172.21.15.94:53638) 2024-04-06T03:39:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb3592e5700 1 -- 172.21.15.94:0/4221794982 learned_addr learned my addr 172.21.15.94:0/4221794982 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.046+0000 7fb359ae6700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3540fcf00 0x7fb354071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.047+0000 7fb3592e5700 1 -- 172.21.15.94:0/4221794982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3540fcf00 msgr2=0x7fb354071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.047+0000 7fb3592e5700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3540fcf00 0x7fb354071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.047+0000 7fb3592e5700 1 -- 172.21.15.94:0/4221794982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb344005040 con 0x7fb3540fe070 2024-04-06T03:39:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.047+0000 7fb359ae6700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3540fcf00 0x7fb354071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:39:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.047+0000 7fb3592e5700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 0x7fb354070310 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb344009150 tx=0x7fb344009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.047+0000 7fb34a7fc700 1 -- 172.21.15.94:0/4221794982 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb34400e040 con 0x7fb3540fe070 2024-04-06T03:39:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.047+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb354070910 con 0x7fb3540fe070 2024-04-06T03:39:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.047+0000 7fb34a7fc700 1 -- 172.21.15.94:0/4221794982 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb34400bcc0 con 0x7fb3540fe070 2024-04-06T03:39:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.047+0000 7fb34a7fc700 1 -- 172.21.15.94:0/4221794982 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb34401c610 con 0x7fb3540fe070 2024-04-06T03:39:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.048+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb354070e80 con 0x7fb3540fe070 2024-04-06T03:39:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.049+0000 7fb34a7fc700 1 -- 172.21.15.94:0/4221794982 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb344013a80 con 0x7fb3540fe070 2024-04-06T03:39:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.049+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb354106100 con 0x7fb3540fe070 2024-04-06T03:39:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.050+0000 7fb34a7fc700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb340069370 0x7fb34006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.050+0000 7fb34a7fc700 1 -- 172.21.15.94:0/4221794982 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb344087e80 con 0x7fb3540fe070 2024-04-06T03:39:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.050+0000 7fb359ae6700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb340069370 0x7fb34006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.051+0000 7fb359ae6700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb340069370 0x7fb34006b820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb350009fd0 tx=0x7fb35000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:17.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.055+0000 7fb34a7fc700 1 -- 172.21.15.94:0/4221794982 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb344058e60 con 0x7fb3540fe070 2024-04-06T03:39:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.462+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb354105aa0 con 0x7fb340069370 2024-04-06T03:39:17.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.463+0000 7fb34a7fc700 1 -- 172.21.15.94:0/4221794982 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fb35000f040 con 0x7fb340069370 2024-04-06T03:39:17.466 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:39:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:39:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:39:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:39:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-04-06T03:39:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "0/21 daemons upgraded", 2024-04-06T03:39:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c image on host smithi136", 2024-04-06T03:39:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:39:17.466 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:39:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb340069370 msgr2=0x7fb34006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb340069370 0x7fb34006b820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb350009fd0 tx=0x7fb35000b040 comp rx=0 tx=0).stop 2024-04-06T03:39:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 msgr2=0x7fb354070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:17.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 0x7fb354070310 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb344009150 tx=0x7fb344009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 shutdown_connections 2024-04-06T03:39:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fb340069370 0x7fb34006b820 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3540fcf00 0x7fb354071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 --2- 172.21.15.94:0/4221794982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3540fe070 0x7fb354070310 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 >> 172.21.15.94:0/4221794982 conn(0x7fb3540f85a0 msgr2=0x7fb3540f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 shutdown_connections 2024-04-06T03:39:17.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.466+0000 7fb35bd4a700 1 -- 172.21.15.94:0/4221794982 wait complete. 2024-04-06T03:39:17.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.583+0000 7fa7c229b700 1 -- 172.21.15.94:0/4087688290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7bc0fce40 msgr2=0x7fa7bc0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:17.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.583+0000 7fa7c229b700 1 --2- 172.21.15.94:0/4087688290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7bc0fce40 0x7fa7bc0fd250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac004a90 tx=0x7fa7ac009dd0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.583+0000 7fa7c229b700 1 -- 172.21.15.94:0/4087688290 shutdown_connections 2024-04-06T03:39:17.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.583+0000 7fa7c229b700 1 --2- 172.21.15.94:0/4087688290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc0fe040 0x7fa7bc0fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.583+0000 7fa7c229b700 1 --2- 172.21.15.94:0/4087688290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7bc0fce40 0x7fa7bc0fd250 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.583+0000 7fa7c229b700 1 -- 172.21.15.94:0/4087688290 >> 172.21.15.94:0/4087688290 conn(0x7fa7bc0f8570 msgr2=0x7fa7bc0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:17.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.583+0000 7fa7c229b700 1 -- 172.21.15.94:0/4087688290 shutdown_connections 2024-04-06T03:39:17.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.584+0000 7fa7c229b700 1 -- 172.21.15.94:0/4087688290 wait complete. 2024-04-06T03:39:17.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.584+0000 7fa7c229b700 1 Processor -- start 2024-04-06T03:39:17.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.584+0000 7fa7c229b700 1 -- start start 2024-04-06T03:39:17.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7c229b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc0fce40 0x7fa7bc10d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:17.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7c229b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7bc0fe040 0x7fa7bc10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:17.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7c229b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc10e510 con 0x7fa7bc0fe040 2024-04-06T03:39:17.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7c229b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc10ed60 con 0x7fa7bc0fce40 2024-04-06T03:39:17.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7bb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7bc0fe040 0x7fa7bc10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:17.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7bbfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc0fce40 0x7fa7bc10d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:17.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7bbfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc0fce40 0x7fa7bc10d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:53138/0 (socket says 172.21.15.94:53138) 2024-04-06T03:39:17.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7bbfff700 1 -- 172.21.15.94:0/865942992 learned_addr learned my addr 172.21.15.94:0/865942992 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:17.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7bbfff700 1 -- 172.21.15.94:0/865942992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7bc0fe040 msgr2=0x7fa7bc10df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:17.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7bbfff700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7bc0fe040 0x7fa7bc10df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:17.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.585+0000 7fa7bbfff700 1 -- 172.21.15.94:0/865942992 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac004650 con 0x7fa7bc0fce40 2024-04-06T03:39:17.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.586+0000 7fa7bbfff700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc0fce40 0x7fa7bc10d9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009a00 tx=0x7fa7ac009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:17.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.586+0000 7fa7b8ff9700 1 -- 172.21.15.94:0/865942992 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7ac019070 con 0x7fa7bc0fce40 2024-04-06T03:39:17.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.586+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc10efe0 con 0x7fa7bc0fce40 2024-04-06T03:39:17.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.586+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc10f4c0 con 0x7fa7bc0fce40 2024-04-06T03:39:17.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.586+0000 7fa7b8ff9700 1 -- 172.21.15.94:0/865942992 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7ac00bc60 con 0x7fa7bc0fce40 2024-04-06T03:39:17.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.586+0000 7fa7b8ff9700 1 -- 172.21.15.94:0/865942992 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7ac00e040 con 0x7fa7bc0fce40 2024-04-06T03:39:17.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.587+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc05f5f0 con 0x7fa7bc0fce40 2024-04-06T03:39:17.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.589+0000 7fa7b8ff9700 1 -- 172.21.15.94:0/865942992 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa7ac014430 con 0x7fa7bc0fce40 2024-04-06T03:39:17.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.589+0000 7fa7b8ff9700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7a4069310 0x7fa7a406b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:17.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.589+0000 7fa7b8ff9700 1 -- 172.21.15.94:0/865942992 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa7ac088250 con 0x7fa7bc0fce40 2024-04-06T03:39:17.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.589+0000 7fa7bb7fe700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7a4069310 0x7fa7a406b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:17.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.590+0000 7fa7bb7fe700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7a4069310 0x7fa7a406b7c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009b80 tx=0x7fa7b000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:17.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:17.595+0000 7fa7b8ff9700 1 -- 172.21.15.94:0/865942992 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa7ac057ff0 con 0x7fa7bc0fce40 2024-04-06T03:39:17.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:17 smithi136 ceph-mon[36436]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:17.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:17 smithi136 ceph-mon[36436]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:17.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:17 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3672356614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:39:17.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:17 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3794449216' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:39:17.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:17 smithi094 ceph-mon[29713]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:17.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:17 smithi094 ceph-mon[29713]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:17.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:17 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3672356614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:39:17.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:17 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3794449216' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:39:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.130+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa7bc105a60 con 0x7fa7bc0fce40 2024-04-06T03:39:18.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.130+0000 7fa7b8ff9700 1 -- 172.21.15.94:0/865942992 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa7ac022020 con 0x7fa7bc0fce40 2024-04-06T03:39:18.134 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-04-06T03:39:18.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7a4069310 msgr2=0x7fa7a406b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:18.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7a4069310 0x7fa7a406b7c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009b80 tx=0x7fa7b000b040 comp rx=0 tx=0).stop 2024-04-06T03:39:18.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc0fce40 msgr2=0x7fa7bc10d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:18.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc0fce40 0x7fa7bc10d9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009a00 tx=0x7fa7ac009dd0 comp rx=0 tx=0).stop 2024-04-06T03:39:18.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 shutdown_connections 2024-04-06T03:39:18.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fa7a4069310 0x7fa7a406b7c0 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009b80 tx=0x7fa7b000b040 comp rx=0 tx=0).stop 2024-04-06T03:39:18.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7bc0fce40 0x7fa7bc10d9d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:18.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 --2- 172.21.15.94:0/865942992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7bc0fe040 0x7fa7bc10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:18.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 >> 172.21.15.94:0/865942992 conn(0x7fa7bc0f8570 msgr2=0x7fa7bc0f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:18.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.133+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 shutdown_connections 2024-04-06T03:39:18.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:18.134+0000 7fa7c229b700 1 -- 172.21.15.94:0/865942992 wait complete. 2024-04-06T03:39:18.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:18 smithi136 ceph-mon[36436]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-06T03:39:18.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:18 smithi136 ceph-mon[36436]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:18.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:18 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/865942992' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:39:18.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:18 smithi094 ceph-mon[29713]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-06T03:39:18.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:18 smithi094 ceph-mon[29713]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:18.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:18 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/865942992' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:39:20.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:20 smithi136 ceph-mon[36436]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-06T03:39:20.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:20 smithi094 ceph-mon[29713]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-06T03:39:22.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:22 smithi136 ceph-mon[36436]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-06T03:39:22.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:22 smithi094 ceph-mon[29713]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-06T03:39:24.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:24 smithi136 ceph-mon[36436]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-06T03:39:24.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:24 smithi094 ceph-mon[29713]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-06T03:39:26.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:26 smithi136 ceph-mon[36436]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:26.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:26 smithi094 ceph-mon[29713]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:28.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:28 smithi136 ceph-mon[36436]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:28.739 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:28 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:39:28.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:28 smithi094 ceph-mon[29713]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:28.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:28 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:39:30.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:30 smithi136 ceph-mon[36436]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:30.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:30 smithi094 ceph-mon[29713]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:32.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:32 smithi136 ceph-mon[36436]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:32.899 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:32 smithi094 ceph-mon[29713]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:34.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:34 smithi136 ceph-mon[36436]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:34.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:34 smithi094 ceph-mon[29713]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:36.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:36 smithi136 ceph-mon[36436]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:36.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:36 smithi094 ceph-mon[29713]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:37.897 INFO:tasks.workunit.client.0.smithi094.stderr:Note: switching to '584a67ccb9eb224c5312dccd96a82e67b0645518'. 2024-04-06T03:39:37.897 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-06T03:39:37.897 INFO:tasks.workunit.client.0.smithi094.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr:state without impacting any branches by switching back to a branch. 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr: git switch -c 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr:Or undo this operation with: 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr: git switch - 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-04-06T03:39:37.898 INFO:tasks.workunit.client.0.smithi094.stderr:HEAD is now at 584a67ccb9e Merge pull request #56706 from afreen23/wip-65208-squid 2024-04-06T03:39:37.902 DEBUG:teuthology.orchestra.run.smithi094:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-06T03:39:38.023 INFO:tasks.workunit.client.0.smithi094.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-06T03:39:38.025 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-06T03:39:38.025 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-06T03:39:38.738 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:38 smithi136 ceph-mon[36436]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:38.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:38 smithi094 ceph-mon[29713]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:39.797 INFO:tasks.workunit.client.1.smithi136.stderr:Updating files: 87% (10934/12461) Updating files: 88% (10966/12461) Updating files: 89% (11091/12461) Updating files: 90% (11215/12461) Updating files: 91% (11340/12461) Updating files: 92% (11465/12461) Updating files: 93% (11589/12461) Updating files: 94% (11714/12461) Updating files: 95% (11838/12461) Updating files: 96% (11963/12461) Updating files: 97% (12088/12461) Updating files: 98% (12212/12461) Updating files: 99% (12337/12461) Updating files: 100% (12461/12461) Updating files: 100% (12461/12461), done. 2024-04-06T03:39:40.484 INFO:tasks.workunit.client.1.smithi136.stderr:Note: switching to '584a67ccb9eb224c5312dccd96a82e67b0645518'. 2024-04-06T03:39:40.484 INFO:tasks.workunit.client.1.smithi136.stderr: 2024-04-06T03:39:40.484 INFO:tasks.workunit.client.1.smithi136.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-06T03:39:40.484 INFO:tasks.workunit.client.1.smithi136.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-06T03:39:40.484 INFO:tasks.workunit.client.1.smithi136.stderr:state without impacting any branches by switching back to a branch. 2024-04-06T03:39:40.484 INFO:tasks.workunit.client.1.smithi136.stderr: 2024-04-06T03:39:40.484 INFO:tasks.workunit.client.1.smithi136.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-06T03:39:40.484 INFO:tasks.workunit.client.1.smithi136.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr: 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr: git switch -c 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr: 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr:Or undo this operation with: 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr: 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr: git switch - 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr: 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr: 2024-04-06T03:39:40.485 INFO:tasks.workunit.client.1.smithi136.stderr:HEAD is now at 584a67ccb9e Merge pull request #56706 from afreen23/wip-65208-squid 2024-04-06T03:39:40.487 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-06T03:39:40.566 INFO:tasks.workunit.client.1.smithi136.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-06T03:39:40.568 INFO:tasks.workunit.client.1.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-06T03:39:40.568 INFO:tasks.workunit.client.1.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-06T03:39:40.710 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:40 smithi136 ceph-mon[36436]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:40.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:40 smithi094 ceph-mon[29713]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:42.372 INFO:tasks.workunit.client.1.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-06T03:39:42.432 INFO:tasks.workunit.client.1.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-06T03:39:42.474 INFO:tasks.workunit.client.1.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-06T03:39:42.475 INFO:tasks.workunit.client.1.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-06T03:39:42.475 INFO:tasks.workunit.client.1.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-06T03:39:42.517 INFO:tasks.workunit.client.1.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-06T03:39:42.521 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:39:42.521 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-06T03:39:42.577 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-06T03:39:42.579 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-06T03:39:42.579 DEBUG:teuthology.orchestra.run.smithi136:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=584a67ccb9eb224c5312dccd96a82e67b0645518 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-06T03:39:42.638 INFO:tasks.workunit.client.1.smithi136.stderr:+ mkdir -p fsstress 2024-04-06T03:39:42.640 INFO:tasks.workunit.client.1.smithi136.stderr:+ pushd fsstress 2024-04-06T03:39:42.640 INFO:tasks.workunit.client.1.smithi136.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-06T03:39:42.640 INFO:tasks.workunit.client.1.smithi136.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-06T03:39:42.707 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:42 smithi136 ceph-mon[36436]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:42.707 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:42 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:42.707 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:42 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:39:42.707 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:42 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:39:42.707 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:42 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:39:42.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:42 smithi094 ceph-mon[29713]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:42.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:42 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:42.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:42 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:39:42.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:42 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:39:42.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:42 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:39:43.550 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:43 smithi136 ceph-mon[36436]: Upgrade: Updating mgr.smithi136.jagvwt 2024-04-06T03:39:43.551 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:43 smithi136 ceph-mon[36436]: Deploying daemon mgr.smithi136.jagvwt on smithi136 2024-04-06T03:39:43.551 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:43 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:39:43.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:43 smithi094 ceph-mon[29713]: Upgrade: Updating mgr.smithi136.jagvwt 2024-04-06T03:39:43.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:43 smithi094 ceph-mon[29713]: Deploying daemon mgr.smithi136.jagvwt on smithi136 2024-04-06T03:39:43.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:43 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:39:44.162 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-06T03:39:44.254 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-06T03:39:44.295 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-06T03:39:44.296 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-06T03:39:44.297 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-06T03:39:44.336 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-06T03:39:44.339 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:39:44.339 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-06T03:39:44.355 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-06T03:39:44.356 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-06T03:39:44.357 DEBUG:teuthology.orchestra.run.smithi094:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=584a67ccb9eb224c5312dccd96a82e67b0645518 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-06T03:39:44.415 INFO:tasks.workunit.client.0.smithi094.stderr:+ mkdir -p fsstress 2024-04-06T03:39:44.416 INFO:tasks.workunit.client.0.smithi094.stderr:+ pushd fsstress 2024-04-06T03:39:44.417 INFO:tasks.workunit.client.0.smithi094.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-06T03:39:44.417 INFO:tasks.workunit.client.0.smithi094.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-06T03:39:44.673 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:44 smithi136 ceph-mon[36436]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:44.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:44 smithi094 ceph-mon[29713]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:44.947 INFO:tasks.workunit.client.0.smithi094.stderr:+ tar xzf ltp-full.tgz 2024-04-06T03:39:46.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:46 smithi136 ceph-mon[36436]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:46.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:46 smithi094 ceph-mon[29713]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:39:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.256+0000 7f227b59e700 1 -- 172.21.15.94:0/437901624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c066280 msgr2=0x7f227c0ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.256+0000 7f227b59e700 1 --2- 172.21.15.94:0/437901624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c066280 0x7f227c0ff860 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f226c004940 tx=0x7f226c009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.256+0000 7f227b59e700 1 -- 172.21.15.94:0/437901624 shutdown_connections 2024-04-06T03:39:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.256+0000 7f227b59e700 1 --2- 172.21.15.94:0/437901624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f227c0ffda0 0x7f227c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.256+0000 7f227b59e700 1 --2- 172.21.15.94:0/437901624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c066280 0x7f227c0ff860 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.256+0000 7f227b59e700 1 -- 172.21.15.94:0/437901624 >> 172.21.15.94:0/437901624 conn(0x7f227c0f7270 msgr2=0x7f227c0f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:48.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.256+0000 7f227b59e700 1 -- 172.21.15.94:0/437901624 shutdown_connections 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.256+0000 7f227b59e700 1 -- 172.21.15.94:0/437901624 wait complete. 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f227b59e700 1 Processor -- start 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f227b59e700 1 -- start start 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f227b59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f227c066280 0x7f227c10d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f227b59e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c0ffda0 0x7f227c10de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f227b59e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c10e400 con 0x7f227c0ffda0 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f227b59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c10ec50 con 0x7f227c066280 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f2279d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c0ffda0 0x7f227c10de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f2279d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c0ffda0 0x7f227c10de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51434/0 (socket says 172.21.15.94:51434) 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f2279d9b700 1 -- 172.21.15.94:0/340626052 learned_addr learned my addr 172.21.15.94:0/340626052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f227a59c700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f227c066280 0x7f227c10d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:48.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f2279d9b700 1 -- 172.21.15.94:0/340626052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f227c066280 msgr2=0x7f227c10d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:48.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f2279d9b700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f227c066280 0x7f227c10d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.257+0000 7f2279d9b700 1 -- 172.21.15.94:0/340626052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c005040 con 0x7f227c0ffda0 2024-04-06T03:39:48.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.258+0000 7f2279d9b700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c0ffda0 0x7f227c10de00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2270002a50 tx=0x7f22700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:48.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.258+0000 7f226affd700 1 -- 172.21.15.94:0/340626052 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2270007cc0 con 0x7f227c0ffda0 2024-04-06T03:39:48.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.258+0000 7f226affd700 1 -- 172.21.15.94:0/340626052 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f227000f040 con 0x7f227c0ffda0 2024-04-06T03:39:48.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.258+0000 7f226affd700 1 -- 172.21.15.94:0/340626052 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f227000da30 con 0x7f227c0ffda0 2024-04-06T03:39:48.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.258+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c10ef30 con 0x7f227c0ffda0 2024-04-06T03:39:48.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.258+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c10f3b0 con 0x7f227c0ffda0 2024-04-06T03:39:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.259+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c05f5f0 con 0x7f227c0ffda0 2024-04-06T03:39:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.259+0000 7f226affd700 1 -- 172.21.15.94:0/340626052 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f227000db90 con 0x7f227c0ffda0 2024-04-06T03:39:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.260+0000 7f226affd700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f22640692a0 0x7f226406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.260+0000 7f226affd700 1 -- 172.21.15.94:0/340626052 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2270086f40 con 0x7f227c0ffda0 2024-04-06T03:39:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.260+0000 7f227a59c700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f22640692a0 0x7f226406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.260+0000 7f227a59c700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f22640692a0 0x7f226406b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f226c004890 tx=0x7f226c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.265+0000 7f226affd700 1 -- 172.21.15.94:0/340626052 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2270056c80 con 0x7f227c0ffda0 2024-04-06T03:39:48.333 INFO:tasks.workunit.client.1.smithi136.stderr:+ tar xzf ltp-full.tgz 2024-04-06T03:39:48.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.519+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f227c10f740 con 0x7f22640692a0 2024-04-06T03:39:48.522 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:48 smithi094 ceph-mon[29713]: pgmap v230: 65 pgs: 65 active+clean; 4.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 349 KiB/s wr, 0 op/s 2024-04-06T03:39:48.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.521+0000 7f226affd700 1 -- 172.21.15.94:0/340626052 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f227c10f740 con 0x7f22640692a0 2024-04-06T03:39:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f22640692a0 msgr2=0x7f226406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f22640692a0 0x7f226406b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f226c004890 tx=0x7f226c009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c0ffda0 msgr2=0x7f227c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c0ffda0 0x7f227c10de00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2270002a50 tx=0x7f22700046b0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 shutdown_connections 2024-04-06T03:39:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f22640692a0 0x7f226406b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f227c066280 0x7f227c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 --2- 172.21.15.94:0/340626052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f227c0ffda0 0x7f227c10de00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 >> 172.21.15.94:0/340626052 conn(0x7f227c0f7270 msgr2=0x7f227c0f7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:48.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 shutdown_connections 2024-04-06T03:39:48.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.523+0000 7f227b59e700 1 -- 172.21.15.94:0/340626052 wait complete. 2024-04-06T03:39:48.538 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:39:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.685+0000 7fcf9ec19700 1 -- 172.21.15.94:0/2575700438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 msgr2=0x7fcf980fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.685+0000 7fcf9ec19700 1 --2- 172.21.15.94:0/2575700438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 0x7fcf980fc0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c004940 tx=0x7fcf8c009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.685+0000 7fcf9ec19700 1 -- 172.21.15.94:0/2575700438 shutdown_connections 2024-04-06T03:39:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.685+0000 7fcf9ec19700 1 --2- 172.21.15.94:0/2575700438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf980fcf80 0x7fcf980fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.685+0000 7fcf9ec19700 1 --2- 172.21.15.94:0/2575700438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 0x7fcf980fc0e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.685+0000 7fcf9ec19700 1 -- 172.21.15.94:0/2575700438 >> 172.21.15.94:0/2575700438 conn(0x7fcf980f7430 msgr2=0x7fcf980f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.685+0000 7fcf9ec19700 1 -- 172.21.15.94:0/2575700438 shutdown_connections 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.686+0000 7fcf9ec19700 1 -- 172.21.15.94:0/2575700438 wait complete. 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.686+0000 7fcf9ec19700 1 Processor -- start 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.686+0000 7fcf9ec19700 1 -- start start 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.686+0000 7fcf9ec19700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 0x7fcf980ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.686+0000 7fcf9ec19700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf980fcf80 0x7fcf980fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.686+0000 7fcf9ec19700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf980fe870 con 0x7fcf980fbcd0 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.686+0000 7fcf9ec19700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf980fe9e0 con 0x7fcf980fcf80 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.686+0000 7fcf9c9b5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 0x7fcf980ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.686+0000 7fcf9c9b5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 0x7fcf980ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51456/0 (socket says 172.21.15.94:51456) 2024-04-06T03:39:48.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf9c9b5700 1 -- 172.21.15.94:0/141867590 learned_addr learned my addr 172.21.15.94:0/141867590 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf97fff700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf980fcf80 0x7fcf980fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf9c9b5700 1 -- 172.21.15.94:0/141867590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf980fcf80 msgr2=0x7fcf980fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf9c9b5700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf980fcf80 0x7fcf980fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf9c9b5700 1 -- 172.21.15.94:0/141867590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c005040 con 0x7fcf980fbcd0 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf9c9b5700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 0x7fcf980ffdf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009150 tx=0x7fcf8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf957fa700 1 -- 172.21.15.94:0/141867590 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf8c00e040 con 0x7fcf980fbcd0 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf980fec60 con 0x7fcf980fbcd0 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf957fa700 1 -- 172.21.15.94:0/141867590 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf8c004290 con 0x7fcf980fbcd0 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf957fa700 1 -- 172.21.15.94:0/141867590 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf8c01c630 con 0x7fcf980fbcd0 2024-04-06T03:39:48.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.687+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf981159b0 con 0x7fcf980fbcd0 2024-04-06T03:39:48.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.688+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9805f5f0 con 0x7fcf980fbcd0 2024-04-06T03:39:48.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.689+0000 7fcf957fa700 1 -- 172.21.15.94:0/141867590 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcf8c00bbf0 con 0x7fcf980fbcd0 2024-04-06T03:39:48.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.689+0000 7fcf957fa700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fcf880692a0 0x7fcf8806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.689+0000 7fcf957fa700 1 -- 172.21.15.94:0/141867590 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fcf8c087e50 con 0x7fcf980fbcd0 2024-04-06T03:39:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.689+0000 7fcf97fff700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fcf880692a0 0x7fcf8806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:48.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.690+0000 7fcf97fff700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fcf880692a0 0x7fcf8806b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009fd0 tx=0x7fcf8400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:48.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.697+0000 7fcf957fa700 1 -- 172.21.15.94:0/141867590 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcf8c054180 con 0x7fcf980fbcd0 2024-04-06T03:39:48.764 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:48 smithi136 ceph-mon[36436]: pgmap v230: 65 pgs: 65 active+clean; 4.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 349 KiB/s wr, 0 op/s 2024-04-06T03:39:48.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.959+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf98104930 con 0x7fcf880692a0 2024-04-06T03:39:48.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.960+0000 7fcf957fa700 1 -- 172.21.15.94:0/141867590 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7fcf98104930 con 0x7fcf880692a0 2024-04-06T03:39:48.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fcf880692a0 msgr2=0x7fcf8806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:48.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fcf880692a0 0x7fcf8806b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009fd0 tx=0x7fcf8400b040 comp rx=0 tx=0).stop 2024-04-06T03:39:48.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 msgr2=0x7fcf980ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:48.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 0x7fcf980ffdf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009150 tx=0x7fcf8c009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:48.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 shutdown_connections 2024-04-06T03:39:48.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7fcf880692a0 0x7fcf8806b750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf980fbcd0 0x7fcf980ffdf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 --2- 172.21.15.94:0/141867590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf980fcf80 0x7fcf980fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:48.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 >> 172.21.15.94:0/141867590 conn(0x7fcf980f7430 msgr2=0x7fcf980f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:48.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 shutdown_connections 2024-04-06T03:39:48.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:48.963+0000 7fcf9ec19700 1 -- 172.21.15.94:0/141867590 wait complete. 2024-04-06T03:39:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.087+0000 7ff08c158700 1 -- 172.21.15.94:0/1386537168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 msgr2=0x7ff0840fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.087+0000 7ff08c158700 1 --2- 172.21.15.94:0/1386537168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 0x7ff0840fc100 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff080004940 tx=0x7ff080009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.088+0000 7ff08c158700 1 -- 172.21.15.94:0/1386537168 shutdown_connections 2024-04-06T03:39:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.088+0000 7ff08c158700 1 --2- 172.21.15.94:0/1386537168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0840fcfa0 0x7ff0840fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.088+0000 7ff08c158700 1 --2- 172.21.15.94:0/1386537168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 0x7ff0840fc100 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.088+0000 7ff08c158700 1 -- 172.21.15.94:0/1386537168 >> 172.21.15.94:0/1386537168 conn(0x7ff0840f7450 msgr2=0x7ff0840f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:49.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.088+0000 7ff08c158700 1 -- 172.21.15.94:0/1386537168 shutdown_connections 2024-04-06T03:39:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.088+0000 7ff08c158700 1 -- 172.21.15.94:0/1386537168 wait complete. 2024-04-06T03:39:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.088+0000 7ff08c158700 1 Processor -- start 2024-04-06T03:39:49.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.088+0000 7ff08c158700 1 -- start start 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff08c158700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 0x7ff084110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff08c158700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0840fcfa0 0x7ff084112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff08c158700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084066c40 con 0x7ff0840fbcf0 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff08c158700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084066d80 con 0x7ff0840fcfa0 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff089ef4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 0x7ff084110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff089ef4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 0x7ff084110670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51476/0 (socket says 172.21.15.94:51476) 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff089ef4700 1 -- 172.21.15.94:0/4259650656 learned_addr learned my addr 172.21.15.94:0/4259650656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff0896f3700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0840fcfa0 0x7ff084112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff089ef4700 1 -- 172.21.15.94:0/4259650656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0840fcfa0 msgr2=0x7ff084112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:49.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff089ef4700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0840fcfa0 0x7ff084112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.089+0000 7ff089ef4700 1 -- 172.21.15.94:0/4259650656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff080005040 con 0x7ff0840fbcf0 2024-04-06T03:39:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.090+0000 7ff089ef4700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 0x7ff084110670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff080013870 tx=0x7ff080009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.090+0000 7ff07a7fc700 1 -- 172.21.15.94:0/4259650656 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff080021070 con 0x7ff0840fbcf0 2024-04-06T03:39:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.090+0000 7ff07a7fc700 1 -- 172.21.15.94:0/4259650656 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff080009430 con 0x7ff0840fbcf0 2024-04-06T03:39:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.090+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff084113160 con 0x7ff0840fbcf0 2024-04-06T03:39:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.090+0000 7ff07a7fc700 1 -- 172.21.15.94:0/4259650656 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff08000e070 con 0x7ff0840fbcf0 2024-04-06T03:39:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.090+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0841136a0 con 0x7ff0840fbcf0 2024-04-06T03:39:49.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.092+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08405f5f0 con 0x7ff0840fbcf0 2024-04-06T03:39:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.092+0000 7ff07a7fc700 1 -- 172.21.15.94:0/4259650656 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff080026410 con 0x7ff0840fbcf0 2024-04-06T03:39:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.092+0000 7ff07a7fc700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff070069230 0x7ff07006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.092+0000 7ff07a7fc700 1 -- 172.21.15.94:0/4259650656 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff080088450 con 0x7ff0840fbcf0 2024-04-06T03:39:49.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.092+0000 7ff0896f3700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff070069230 0x7ff07006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:49.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.093+0000 7ff0896f3700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff070069230 0x7ff07006b6e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff074009b80 tx=0x7ff07400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:49.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.098+0000 7ff07a7fc700 1 -- 172.21.15.94:0/4259650656 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff080055810 con 0x7ff0840fbcf0 2024-04-06T03:39:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.478+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff084101240 con 0x7ff070069230 2024-04-06T03:39:49.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.488+0000 7ff07a7fc700 1 -- 172.21.15.94:0/4259650656 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7ff084101240 con 0x7ff070069230 2024-04-06T03:39:49.491 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:39:49.491 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (5m) 90s ago 7m 26.0M - 0.25.0 c8568f914cd2 b63c91af7786 2024-04-06T03:39:49.491 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (7m) 90s ago 7m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:39:49.491 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (6m) 96s ago 6m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (5m) 90s ago 6m 86.1M - 9.4.7 954c08fa6188 78ad0d4cf52b 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (111s) 90s ago 110s 11.8M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (116s) 90s ago 115s 14.7M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (113s) 96s ago 113s 12.0M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (108s) 96s ago 108s 14.4M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:9283 running (8m) 90s ago 8m 480M - 17.2.7-819-g6699495c ec300eaa768b d83dfb730a39 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283 starting - - - - 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (8m) 90s ago 8m 45.4M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (6m) 96s ago 6m 35.6M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (7m) 90s ago 7m 16.1M - 1.5.0 0da6a335fe13 f8f823e89f23 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (6m) 96s ago 6m 17.1M - 1.5.0 0da6a335fe13 e5324db1ea82 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (5m) 90s ago 5m 45.0M 4288M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (5m) 90s ago 5m 46.0M 4288M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (4m) 90s ago 4m 44.2M 4288M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (4m) 96s ago 4m 43.4M 5312M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (3m) 96s ago 3m 41.1M 5312M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:39:49.492 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (3m) 96s ago 3m 41.8M 5312M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:39:49.493 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (5m) 90s ago 6m 55.6M - 2.43.0 a07b618ecd1d 82a078c9e43c 2024-04-06T03:39:49.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff070069230 msgr2=0x7ff07006b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:49.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff070069230 0x7ff07006b6e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff074009b80 tx=0x7ff07400b040 comp rx=0 tx=0).stop 2024-04-06T03:39:49.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 msgr2=0x7ff084110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:49.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 0x7ff084110670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff080013870 tx=0x7ff080009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:49.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 shutdown_connections 2024-04-06T03:39:49.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7ff070069230 0x7ff07006b6e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:49.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0840fbcf0 0x7ff084110670 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:49.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 --2- 172.21.15.94:0/4259650656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0840fcfa0 0x7ff084112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:49.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 >> 172.21.15.94:0/4259650656 conn(0x7ff0840f7450 msgr2=0x7ff0840f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:49.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 shutdown_connections 2024-04-06T03:39:49.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.491+0000 7ff08c158700 1 -- 172.21.15.94:0/4259650656 wait complete. 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 -- 172.21.15.94:0/848906553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fe400 msgr2=0x7f9a740fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 --2- 172.21.15.94:0/848906553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fe400 0x7f9a740fe810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9a64004940 tx=0x7f9a64009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 -- 172.21.15.94:0/848906553 shutdown_connections 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 --2- 172.21.15.94:0/848906553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a740fed50 0x7f9a740fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 --2- 172.21.15.94:0/848906553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fe400 0x7f9a740fe810 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 -- 172.21.15.94:0/848906553 >> 172.21.15.94:0/848906553 conn(0x7f9a740f6ba0 msgr2=0x7f9a740f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 -- 172.21.15.94:0/848906553 shutdown_connections 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 -- 172.21.15.94:0/848906553 wait complete. 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 Processor -- start 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.603+0000 7f9a78808700 1 -- start start 2024-04-06T03:39:49.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a78808700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a740fe400 0x7f9a7410d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a78808700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fed50 0x7f9a7410df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a78808700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7410e530 con 0x7f9a740fed50 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a78808700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7410ed80 con 0x7f9a740fe400 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a71d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fed50 0x7f9a7410df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a71d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fed50 0x7f9a7410df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51488/0 (socket says 172.21.15.94:51488) 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a71d9b700 1 -- 172.21.15.94:0/2441523827 learned_addr learned my addr 172.21.15.94:0/2441523827 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a7259c700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a740fe400 0x7f9a7410d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a71d9b700 1 -- 172.21.15.94:0/2441523827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a740fe400 msgr2=0x7f9a7410d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a71d9b700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a740fe400 0x7f9a7410d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:49.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a71d9b700 1 -- 172.21.15.94:0/2441523827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64005040 con 0x7f9a740fed50 2024-04-06T03:39:49.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a71d9b700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fed50 0x7f9a7410df30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9a68002a50 tx=0x7f9a680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:49.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.604+0000 7f9a62ffd700 1 -- 172.21.15.94:0/2441523827 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a68007cc0 con 0x7f9a740fed50 2024-04-06T03:39:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.605+0000 7f9a78808700 1 -- 172.21.15.94:0/2441523827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7410f060 con 0x7f9a740fed50 2024-04-06T03:39:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.605+0000 7f9a62ffd700 1 -- 172.21.15.94:0/2441523827 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a6800f040 con 0x7f9a740fed50 2024-04-06T03:39:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.605+0000 7f9a78808700 1 -- 172.21.15.94:0/2441523827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7410f5a0 con 0x7f9a740fed50 2024-04-06T03:39:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.605+0000 7f9a62ffd700 1 -- 172.21.15.94:0/2441523827 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a6800da30 con 0x7f9a740fed50 2024-04-06T03:39:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.606+0000 7f9a78808700 1 -- 172.21.15.94:0/2441523827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a540052e0 con 0x7f9a740fed50 2024-04-06T03:39:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.606+0000 7f9a62ffd700 1 -- 172.21.15.94:0/2441523827 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9a6800db90 con 0x7f9a740fed50 2024-04-06T03:39:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.607+0000 7f9a62ffd700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9a5c0692a0 0x7f9a5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.607+0000 7f9a62ffd700 1 -- 172.21.15.94:0/2441523827 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9a68086e20 con 0x7f9a740fed50 2024-04-06T03:39:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.607+0000 7f9a7259c700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9a5c0692a0 0x7f9a5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:49.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.607+0000 7f9a7259c700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9a5c0692a0 0x7f9a5c06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400b3b0 tx=0x7f9a64009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:49.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:49.613+0000 7f9a62ffd700 1 -- 172.21.15.94:0/2441523827 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9a68056b60 con 0x7f9a740fed50 2024-04-06T03:39:49.769 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:49 smithi136 ceph-mon[36436]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:49.770 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:49.770 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:49.770 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:49 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:39:49.880 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:49 smithi094 ceph-mon[29713]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:49.880 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:49.880 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:39:49.880 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:49 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:39:50.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.024+0000 7f9a78808700 1 -- 172.21.15.94:0/2441523827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9a54005da0 con 0x7f9a740fed50 2024-04-06T03:39:50.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.027+0000 7f9a62ffd700 1 -- 172.21.15.94:0/2441523827 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f9a680566f0 con 0x7f9a740fed50 2024-04-06T03:39:50.029 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:39:50.030 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:39:50.031 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 14 2024-04-06T03:39:50.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:39:50.031 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:39:50.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.032+0000 7f9a60ff9700 1 -- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9a5c0692a0 msgr2=0x7f9a5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:50.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.032+0000 7f9a60ff9700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9a5c0692a0 0x7f9a5c06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400b3b0 tx=0x7f9a64009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:50.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.032+0000 7f9a60ff9700 1 -- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fed50 msgr2=0x7f9a7410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:50.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.032+0000 7f9a60ff9700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fed50 0x7f9a7410df30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9a68002a50 tx=0x7f9a680046b0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.032+0000 7f9a60ff9700 1 -- 172.21.15.94:0/2441523827 shutdown_connections 2024-04-06T03:39:50.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.032+0000 7f9a60ff9700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f9a5c0692a0 0x7f9a5c06b750 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400b3b0 tx=0x7f9a64009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:50.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.032+0000 7f9a60ff9700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a740fe400 0x7f9a7410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.032+0000 7f9a60ff9700 1 --2- 172.21.15.94:0/2441523827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a740fed50 0x7f9a7410df30 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9a68002a50 tx=0x7f9a680046b0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.032+0000 7f9a60ff9700 1 -- 172.21.15.94:0/2441523827 >> 172.21.15.94:0/2441523827 conn(0x7f9a740f6ba0 msgr2=0x7f9a740f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:50.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.035+0000 7f9a60ff9700 1 -- 172.21.15.94:0/2441523827 shutdown_connections 2024-04-06T03:39:50.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.037+0000 7f9a60ff9700 1 -- 172.21.15.94:0/2441523827 wait complete. 2024-04-06T03:39:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.220+0000 7f1be5a9c700 1 -- 172.21.15.94:0/2129397823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1be0066ac0 msgr2=0x7f1be006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.220+0000 7f1be5a9c700 1 --2- 172.21.15.94:0/2129397823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1be0066ac0 0x7f1be006e7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009ce0 tx=0x7f1bd8004d10 comp rx=0 tx=0).stop 2024-04-06T03:39:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.220+0000 7f1be5a9c700 1 -- 172.21.15.94:0/2129397823 shutdown_connections 2024-04-06T03:39:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.220+0000 7f1be5a9c700 1 --2- 172.21.15.94:0/2129397823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be006ece0 0x7f1be006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.220+0000 7f1be5a9c700 1 --2- 172.21.15.94:0/2129397823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1be0066ac0 0x7f1be006e7a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.220+0000 7f1be5a9c700 1 -- 172.21.15.94:0/2129397823 >> 172.21.15.94:0/2129397823 conn(0x7f1be006a380 msgr2=0x7f1be006c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.225+0000 7f1be5a9c700 1 -- 172.21.15.94:0/2129397823 shutdown_connections 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.226+0000 7f1be5a9c700 1 -- 172.21.15.94:0/2129397823 wait complete. 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.227+0000 7f1be5a9c700 1 Processor -- start 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.227+0000 7f1be5a9c700 1 -- start start 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.227+0000 7f1be5a9c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be0066ac0 0x7f1be0118560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.227+0000 7f1be5a9c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1be006ece0 0x7f1be0118aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.227+0000 7f1be5a9c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0119130 con 0x7f1be0066ac0 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.227+0000 7f1be5a9c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0074210 con 0x7f1be006ece0 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.228+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1be006ece0 0x7f1be0118aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.228+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1be006ece0 0x7f1be0118aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:59222/0 (socket says 172.21.15.94:59222) 2024-04-06T03:39:50.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.228+0000 7f1bdeffd700 1 -- 172.21.15.94:0/1755908885 learned_addr learned my addr 172.21.15.94:0/1755908885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:50.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.229+0000 7f1bdf7fe700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be0066ac0 0x7f1be0118560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:50.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.230+0000 7f1bdf7fe700 1 -- 172.21.15.94:0/1755908885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1be006ece0 msgr2=0x7f1be0118aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:50.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.230+0000 7f1bdf7fe700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1be006ece0 0x7f1be0118aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.230+0000 7f1bdf7fe700 1 -- 172.21.15.94:0/1755908885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8009940 con 0x7f1be0066ac0 2024-04-06T03:39:50.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.230+0000 7f1bdf7fe700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be0066ac0 0x7f1be0118560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8000f80 tx=0x7f1bd8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:50.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.231+0000 7f1bc7fff700 1 -- 172.21.15.94:0/1755908885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bd800e040 con 0x7f1be0066ac0 2024-04-06T03:39:50.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.231+0000 7f1bc7fff700 1 -- 172.21.15.94:0/1755908885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bd8008e20 con 0x7f1be0066ac0 2024-04-06T03:39:50.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.231+0000 7f1bc7fff700 1 -- 172.21.15.94:0/1755908885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bd801c740 con 0x7f1be0066ac0 2024-04-06T03:39:50.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.232+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be0074490 con 0x7f1be0066ac0 2024-04-06T03:39:50.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.232+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be0074990 con 0x7f1be0066ac0 2024-04-06T03:39:50.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.234+0000 7f1bc7fff700 1 -- 172.21.15.94:0/1755908885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1bd8012b70 con 0x7f1be0066ac0 2024-04-06T03:39:50.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.234+0000 7f1bc7fff700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1bc8069480 0x7f1bc806b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:50.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.234+0000 7f1bc7fff700 1 -- 172.21.15.94:0/1755908885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1bd8088b80 con 0x7f1be0066ac0 2024-04-06T03:39:50.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.236+0000 7f1bdeffd700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1bc8069480 0x7f1bc806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.239+0000 7f1bdeffd700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1bc8069480 0x7f1bc806b930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009ca0 tx=0x7f1bd0009250 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.239+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be004c710 con 0x7f1be0066ac0 2024-04-06T03:39:50.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.249+0000 7f1bc7fff700 1 -- 172.21.15.94:0/1755908885 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1bd8058920 con 0x7f1be0066ac0 2024-04-06T03:39:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:50 smithi094 ceph-mon[29713]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:50 smithi094 ceph-mon[29713]: pgmap v231: 65 pgs: 65 active+clean; 4.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 349 KiB/s wr, 0 op/s 2024-04-06T03:39:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:50 smithi094 ceph-mon[29713]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:50 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2441523827' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:39:50.722 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:50 smithi136 ceph-mon[36436]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:50.723 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:50 smithi136 ceph-mon[36436]: pgmap v231: 65 pgs: 65 active+clean; 4.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 349 KiB/s wr, 0 op/s 2024-04-06T03:39:50.723 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:50 smithi136 ceph-mon[36436]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:50.723 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:50 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2441523827' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:39:50.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.750+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1be0074c60 con 0x7f1be0066ac0 2024-04-06T03:39:50.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.751+0000 7f1bc7fff700 1 -- 172.21.15.94:0/1755908885 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7f1bd8017070 con 0x7f1be0066ac0 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:39:50.755 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:39:50.756 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:39:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:39:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:39:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:39:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:39:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:39:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:39:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:39:50.757 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:39:50.758 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:39:50.758 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:39:50.758 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:39:50.758 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:39:50.758 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:39:50.758 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:39:50.758 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:39:50.758 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:39:50.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.756+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1bc8069480 msgr2=0x7f1bc806b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:50.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.756+0000 7f1be5a9c700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1bc8069480 0x7f1bc806b930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009ca0 tx=0x7f1bd0009250 comp rx=0 tx=0).stop 2024-04-06T03:39:50.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.756+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be0066ac0 msgr2=0x7f1be0118560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:50.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.756+0000 7f1be5a9c700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be0066ac0 0x7f1be0118560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8000f80 tx=0x7f1bd8004d10 comp rx=0 tx=0).stop 2024-04-06T03:39:50.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.757+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 shutdown_connections 2024-04-06T03:39:50.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.757+0000 7f1be5a9c700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f1bc8069480 0x7f1bc806b930 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.757+0000 7f1be5a9c700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be0066ac0 0x7f1be0118560 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.757+0000 7f1be5a9c700 1 --2- 172.21.15.94:0/1755908885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1be006ece0 0x7f1be0118aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.757+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 >> 172.21.15.94:0/1755908885 conn(0x7f1be006a380 msgr2=0x7f1be010a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.758+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 shutdown_connections 2024-04-06T03:39:50.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.759+0000 7f1be5a9c700 1 -- 172.21.15.94:0/1755908885 wait complete. 2024-04-06T03:39:50.764 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:39:50.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.920+0000 7f0d01ffb700 1 -- 172.21.15.94:0/3577681602 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cfc00bcb0 con 0x7f0d040fc2d0 2024-04-06T03:39:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.921+0000 7f0d0946c700 1 -- 172.21.15.94:0/3577681602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fc2d0 msgr2=0x7f0d040fc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.921+0000 7f0d0946c700 1 --2- 172.21.15.94:0/3577681602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fc2d0 0x7f0d040fc6e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004940 tx=0x7f0cfc009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.921+0000 7f0d0946c700 1 -- 172.21.15.94:0/3577681602 shutdown_connections 2024-04-06T03:39:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.921+0000 7f0d0946c700 1 --2- 172.21.15.94:0/3577681602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040fcc20 0x7f0d040fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.921+0000 7f0d0946c700 1 --2- 172.21.15.94:0/3577681602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fc2d0 0x7f0d040fc6e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.921+0000 7f0d0946c700 1 -- 172.21.15.94:0/3577681602 >> 172.21.15.94:0/3577681602 conn(0x7f0d040f6be0 msgr2=0x7f0d040f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.921+0000 7f0d0946c700 1 -- 172.21.15.94:0/3577681602 shutdown_connections 2024-04-06T03:39:50.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.921+0000 7f0d0946c700 1 -- 172.21.15.94:0/3577681602 wait complete. 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d0946c700 1 Processor -- start 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d0946c700 1 -- start start 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d0946c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040fc2d0 0x7f0d0410fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d0946c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fcc20 0x7f0d04110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d0946c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04110700 con 0x7f0d040fcc20 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d0946c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04110870 con 0x7f0d040fc2d0 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d027fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fcc20 0x7f0d04110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d027fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fcc20 0x7f0d04110100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51532/0 (socket says 172.21.15.94:51532) 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d027fc700 1 -- 172.21.15.94:0/3222873623 learned_addr learned my addr 172.21.15.94:0/3222873623 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.922+0000 7f0d02ffd700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040fc2d0 0x7f0d0410fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:50.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.923+0000 7f0d027fc700 1 -- 172.21.15.94:0/3222873623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040fc2d0 msgr2=0x7f0d0410fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:50.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.923+0000 7f0d027fc700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040fc2d0 0x7f0d0410fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:50.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.923+0000 7f0d027fc700 1 -- 172.21.15.94:0/3222873623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc005040 con 0x7f0d040fcc20 2024-04-06T03:39:50.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.923+0000 7f0d027fc700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fcc20 0x7f0d04110100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0cec002a50 tx=0x7f0cec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:50.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.923+0000 7f0cfb7fe700 1 -- 172.21.15.94:0/3222873623 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cec007cc0 con 0x7f0d040fcc20 2024-04-06T03:39:50.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.924+0000 7f0d0946c700 1 -- 172.21.15.94:0/3222873623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d04115280 con 0x7f0d040fcc20 2024-04-06T03:39:50.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.924+0000 7f0cfb7fe700 1 -- 172.21.15.94:0/3222873623 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cec00f040 con 0x7f0d040fcc20 2024-04-06T03:39:50.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.925+0000 7f0cfb7fe700 1 -- 172.21.15.94:0/3222873623 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cec009610 con 0x7f0d040fcc20 2024-04-06T03:39:50.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.925+0000 7f0d0946c700 1 -- 172.21.15.94:0/3222873623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d04105c30 con 0x7f0d040fcc20 2024-04-06T03:39:50.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.926+0000 7f0cf97fa700 1 -- 172.21.15.94:0/3222873623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0404c710 con 0x7f0d040fcc20 2024-04-06T03:39:50.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.926+0000 7f0cfb7fe700 1 -- 172.21.15.94:0/3222873623 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0cec009770 con 0x7f0d040fcc20 2024-04-06T03:39:50.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.927+0000 7f0cfb7fe700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0cf0069090 0x7f0cf006b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:50.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.927+0000 7f0d02ffd700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0cf0069090 0x7f0cf006b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:50.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.927+0000 7f0cfb7fe700 1 -- 172.21.15.94:0/3222873623 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0cec086760 con 0x7f0d040fcc20 2024-04-06T03:39:50.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.928+0000 7f0d02ffd700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0cf0069090 0x7f0cf006b540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00b3b0 tx=0x7f0cfc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:50.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:50.933+0000 7f0cfb7fe700 1 -- 172.21.15.94:0/3222873623 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0cec056490 con 0x7f0d040fcc20 2024-04-06T03:39:51.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.187+0000 7f0cf97fa700 1 -- 172.21.15.94:0/3222873623 --> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d0405ee60 con 0x7f0cf0069090 2024-04-06T03:39:51.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.188+0000 7f0cfb7fe700 1 -- 172.21.15.94:0/3222873623 <== mgr.14208 v2:172.21.15.94:6800/3627366865 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f0d0405ee60 con 0x7f0cf0069090 2024-04-06T03:39:51.190 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:39:51.191 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:39:51.191 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:39:51.191 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:39:51.191 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-04-06T03:39:51.191 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "", 2024-04-06T03:39:51.191 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mgr daemons", 2024-04-06T03:39:51.191 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:39:51.191 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:39:51.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 -- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0cf0069090 msgr2=0x7f0cf006b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0cf0069090 0x7f0cf006b540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00b3b0 tx=0x7f0cfc009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 -- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fcc20 msgr2=0x7f0d04110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fcc20 0x7f0d04110100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0cec002a50 tx=0x7f0cec0046b0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 -- 172.21.15.94:0/3222873623 shutdown_connections 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f0cf0069090 0x7f0cf006b540 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040fc2d0 0x7f0d0410fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 --2- 172.21.15.94:0/3222873623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d040fcc20 0x7f0d04110100 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 -- 172.21.15.94:0/3222873623 >> 172.21.15.94:0/3222873623 conn(0x7f0d040f6be0 msgr2=0x7f0d040fb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 -- 172.21.15.94:0/3222873623 shutdown_connections 2024-04-06T03:39:51.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.190+0000 7f0cf97fa700 1 -- 172.21.15.94:0/3222873623 wait complete. 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.293+0000 7f56d3854700 1 -- 172.21.15.94:0/2811626075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fcfb0 msgr2=0x7f56cc0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.293+0000 7f56d3854700 1 --2- 172.21.15.94:0/2811626075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fcfb0 0x7f56cc0fd420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f56bc004940 tx=0x7f56bc009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.293+0000 7f56d3854700 1 -- 172.21.15.94:0/2811626075 shutdown_connections 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.293+0000 7f56d3854700 1 --2- 172.21.15.94:0/2811626075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fcfb0 0x7f56cc0fd420 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.293+0000 7f56d3854700 1 --2- 172.21.15.94:0/2811626075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56cc0fbd00 0x7f56cc0fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.293+0000 7f56d3854700 1 -- 172.21.15.94:0/2811626075 >> 172.21.15.94:0/2811626075 conn(0x7f56cc0f7440 msgr2=0x7f56cc0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d3854700 1 -- 172.21.15.94:0/2811626075 shutdown_connections 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d3854700 1 -- 172.21.15.94:0/2811626075 wait complete. 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d3854700 1 Processor -- start 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d3854700 1 -- start start 2024-04-06T03:39:51.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d3854700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fbd00 0x7f56cc066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d3854700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56cc0fcfb0 0x7f56cc112670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d3854700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc112bb0 con 0x7f56cc0fbd00 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d3854700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc112cf0 con 0x7f56cc0fcfb0 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d15f0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fbd00 0x7f56cc066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d15f0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fbd00 0x7f56cc066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51544/0 (socket says 172.21.15.94:51544) 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d15f0700 1 -- 172.21.15.94:0/1532996563 learned_addr learned my addr 172.21.15.94:0/1532996563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.294+0000 7f56d0def700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56cc0fcfb0 0x7f56cc112670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56d15f0700 1 -- 172.21.15.94:0/1532996563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56cc0fcfb0 msgr2=0x7f56cc112670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56d15f0700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56cc0fcfb0 0x7f56cc112670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56d15f0700 1 -- 172.21.15.94:0/1532996563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc005040 con 0x7f56cc0fbd00 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56d0def700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56cc0fcfb0 0x7f56cc112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56d15f0700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fbd00 0x7f56cc066cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f56c800eb40 tx=0x7f56c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56c1ffb700 1 -- 172.21.15.94:0/1532996563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c8007cc0 con 0x7f56cc0fbd00 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56c1ffb700 1 -- 172.21.15.94:0/1532996563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56c800f040 con 0x7f56cc0fbd00 2024-04-06T03:39:51.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc112f70 con 0x7f56cc0fbd00 2024-04-06T03:39:51.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56c1ffb700 1 -- 172.21.15.94:0/1532996563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c800ca10 con 0x7f56cc0fbd00 2024-04-06T03:39:51.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.295+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc113360 con 0x7f56cc0fbd00 2024-04-06T03:39:51.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.296+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc04c710 con 0x7f56cc0fbd00 2024-04-06T03:39:51.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.297+0000 7f56c1ffb700 1 -- 172.21.15.94:0/1532996563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f56c800cb70 con 0x7f56cc0fbd00 2024-04-06T03:39:51.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.297+0000 7f56c1ffb700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f56b80692a0 0x7f56b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:39:51.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.297+0000 7f56c1ffb700 1 -- 172.21.15.94:0/1532996563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f56c8086c70 con 0x7f56cc0fbd00 2024-04-06T03:39:51.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.297+0000 7f56d0def700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f56b80692a0 0x7f56b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:39:51.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.298+0000 7f56d0def700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f56b80692a0 0x7f56b806b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009b30 tx=0x7f56bc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:39:51.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.302+0000 7f56c1ffb700 1 -- 172.21.15.94:0/1532996563 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f56c804e190 con 0x7f56cc0fbd00 2024-04-06T03:39:51.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.603+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f56cc1047f0 con 0x7f56cc0fbd00 2024-04-06T03:39:51.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.604+0000 7f56c1ffb700 1 -- 172.21.15.94:0/1532996563 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f56c8056850 con 0x7f56cc0fbd00 2024-04-06T03:39:51.607 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f56b80692a0 msgr2=0x7f56b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f56b80692a0 0x7f56b806b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009b30 tx=0x7f56bc009e30 comp rx=0 tx=0).stop 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fbd00 msgr2=0x7f56cc066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fbd00 0x7f56cc066cd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f56c800eb40 tx=0x7f56c800b040 comp rx=0 tx=0).stop 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 shutdown_connections 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:6800/3627366865,v1:172.21.15.94:6801/3627366865] conn(0x7f56b80692a0 0x7f56b806b750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56cc0fbd00 0x7f56cc066cd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 --2- 172.21.15.94:0/1532996563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56cc0fcfb0 0x7f56cc112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 >> 172.21.15.94:0/1532996563 conn(0x7f56cc0f7440 msgr2=0x7f56cc0f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:39:51.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 shutdown_connections 2024-04-06T03:39:51.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:39:51.607+0000 7f56d3854700 1 -- 172.21.15.94:0/1532996563 wait complete. 2024-04-06T03:39:51.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:51 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1755908885' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:39:51.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:51 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1755908885' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:39:52.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:52 smithi136 ceph-mon[36436]: pgmap v232: 65 pgs: 65 active+clean; 6.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 534 KiB/s wr, 41 op/s 2024-04-06T03:39:52.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:52 smithi136 ceph-mon[36436]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:52.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:52 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1532996563' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:39:52.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:52 smithi094 ceph-mon[29713]: pgmap v232: 65 pgs: 65 active+clean; 6.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 534 KiB/s wr, 41 op/s 2024-04-06T03:39:52.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:52 smithi094 ceph-mon[29713]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:39:52.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:52 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1532996563' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:39:54.735 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:54 smithi136 ceph-mon[36436]: pgmap v233: 65 pgs: 65 active+clean; 45 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.9 MiB/s wr, 301 op/s 2024-04-06T03:39:54.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:54 smithi094 ceph-mon[29713]: pgmap v233: 65 pgs: 65 active+clean; 45 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.9 MiB/s wr, 301 op/s 2024-04-06T03:39:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:56 smithi136 ceph-mon[36436]: pgmap v234: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 5.5 MiB/s wr, 437 op/s 2024-04-06T03:39:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:56 smithi136 ceph-mon[36436]: Standby manager daemon smithi136.jagvwt restarted 2024-04-06T03:39:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:56 smithi136 ceph-mon[36436]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:39:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:56 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:39:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:56 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:39:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:56 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:39:56.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:56 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:39:56.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:56 smithi094 ceph-mon[29713]: pgmap v234: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 5.5 MiB/s wr, 437 op/s 2024-04-06T03:39:56.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:56 smithi094 ceph-mon[29713]: Standby manager daemon smithi136.jagvwt restarted 2024-04-06T03:39:56.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:56 smithi094 ceph-mon[29713]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:39:56.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:56 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:39:56.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:56 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:39:56.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:56 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:39:56.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:56 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:39:57.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:57 smithi136 ceph-mon[36436]: mgrmap e20: smithi094.purlzp(active, since 6m), standbys: smithi136.jagvwt 2024-04-06T03:39:57.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:57 smithi094 ceph-mon[29713]: mgrmap e20: smithi094.purlzp(active, since 6m), standbys: smithi136.jagvwt 2024-04-06T03:39:58.740 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:58 smithi136 ceph-mon[36436]: pgmap v235: 65 pgs: 65 active+clean; 74 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 6.4 MiB/s wr, 449 op/s 2024-04-06T03:39:58.740 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:39:58 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:39:58.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:58 smithi094 ceph-mon[29713]: pgmap v235: 65 pgs: 65 active+clean; 74 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 6.4 MiB/s wr, 449 op/s 2024-04-06T03:39:58.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:39:58 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:40:00.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:00 smithi136 ceph-mon[36436]: pgmap v236: 65 pgs: 65 active+clean; 76 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.2 MiB/s wr, 464 op/s 2024-04-06T03:40:00.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:00 smithi136 ceph-mon[36436]: overall HEALTH_OK 2024-04-06T03:40:00.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:00 smithi094 ceph-mon[29713]: pgmap v236: 65 pgs: 65 active+clean; 76 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.2 MiB/s wr, 464 op/s 2024-04-06T03:40:00.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:00 smithi094 ceph-mon[29713]: overall HEALTH_OK 2024-04-06T03:40:02.779 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:02 smithi136 ceph-mon[36436]: pgmap v237: 65 pgs: 65 active+clean; 79 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.5 MiB/s wr, 496 op/s 2024-04-06T03:40:02.779 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:02.779 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:02.779 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:02.780 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:02 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:02.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:02 smithi094 ceph-mon[29713]: pgmap v237: 65 pgs: 65 active+clean; 79 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.5 MiB/s wr, 496 op/s 2024-04-06T03:40:02.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:02.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:02.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:02.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:02 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:04.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:04 smithi136 ceph-mon[36436]: pgmap v238: 65 pgs: 65 active+clean; 101 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 8.3 MiB/s wr, 659 op/s 2024-04-06T03:40:04.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:04 smithi094 ceph-mon[29713]: pgmap v238: 65 pgs: 65 active+clean; 101 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 8.3 MiB/s wr, 659 op/s 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: pgmap v239: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 5.9 MiB/s wr, 498 op/s 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr fail", "who": "smithi094.purlzp"}]: dispatch 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: osdmap e46: 6 total, 6 up, 6 in 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "mgr fail", "who": "smithi094.purlzp"}]': finished 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: mgrmap e21: smithi136.jagvwt(active, starting, since 0.015578s) 2024-04-06T03:40:06.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mgr metadata", "who": "smithi136.jagvwt", "id": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:40:06.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:40:06.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:40:06.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: Manager daemon smithi136.jagvwt is now available 2024-04-06T03:40:06.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:40:06.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi136.jagvwt/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:40:06.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:40:06.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi136.jagvwt/trash_purge_schedule"}]: dispatch 2024-04-06T03:40:06.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: pgmap v239: 65 pgs: 65 active+clean; 111 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 5.9 MiB/s wr, 498 op/s 2024-04-06T03:40:06.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:06.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr fail", "who": "smithi094.purlzp"}]: dispatch 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: osdmap e46: 6 total, 6 up, 6 in 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.14208 172.21.15.94:0/3810213661' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "mgr fail", "who": "smithi094.purlzp"}]': finished 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: mgrmap e21: smithi136.jagvwt(active, starting, since 0.015578s) 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:40:06.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mgr metadata", "who": "smithi136.jagvwt", "id": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: Manager daemon smithi136.jagvwt is now available 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi136.jagvwt/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:40:06.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:40:06.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi136.jagvwt/trash_purge_schedule"}]: dispatch 2024-04-06T03:40:08.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:08 smithi136 ceph-mon[36436]: Deploying cephadm binary to smithi136 2024-04-06T03:40:08.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:08 smithi136 ceph-mon[36436]: Deploying cephadm binary to smithi094 2024-04-06T03:40:08.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:08 smithi136 ceph-mon[36436]: mgrmap e22: smithi136.jagvwt(active, since 1.00882s) 2024-04-06T03:40:08.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:08 smithi136 ceph-mon[36436]: pgmap v3: 65 pgs: 65 active+clean; 145 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:40:08.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:08 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:08.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:08 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:08.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:08 smithi094 ceph-mon[29713]: Deploying cephadm binary to smithi136 2024-04-06T03:40:08.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:08 smithi094 ceph-mon[29713]: Deploying cephadm binary to smithi094 2024-04-06T03:40:08.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:08 smithi094 ceph-mon[29713]: mgrmap e22: smithi136.jagvwt(active, since 1.00882s) 2024-04-06T03:40:08.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:08 smithi094 ceph-mon[29713]: pgmap v3: 65 pgs: 65 active+clean; 145 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:40:08.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:08 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:08.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:08 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:09.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:09 smithi136 ceph-mon[36436]: mgrmap e23: smithi136.jagvwt(active, since 2s) 2024-04-06T03:40:09.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:09 smithi136 ceph-mon[36436]: pgmap v4: 65 pgs: 65 active+clean; 145 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:40:09.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:09 smithi094 ceph-mon[29713]: mgrmap e23: smithi136.jagvwt(active, since 2s) 2024-04-06T03:40:09.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:09 smithi094 ceph-mon[29713]: pgmap v4: 65 pgs: 65 active+clean; 145 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:40:10.342 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:10 smithi094 ceph-mon[29713]: [06/Apr/2024:03:40:08] ENGINE Bus STARTING 2024-04-06T03:40:10.342 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:10 smithi094 ceph-mon[29713]: [06/Apr/2024:03:40:08] ENGINE Serving on https://172.21.15.136:7150 2024-04-06T03:40:10.342 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:10 smithi094 ceph-mon[29713]: [06/Apr/2024:03:40:08] ENGINE Serving on http://172.21.15.136:8765 2024-04-06T03:40:10.342 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:10 smithi094 ceph-mon[29713]: [06/Apr/2024:03:40:08] ENGINE Bus STARTED 2024-04-06T03:40:10.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:10 smithi136 ceph-mon[36436]: [06/Apr/2024:03:40:08] ENGINE Bus STARTING 2024-04-06T03:40:10.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:10 smithi136 ceph-mon[36436]: [06/Apr/2024:03:40:08] ENGINE Serving on https://172.21.15.136:7150 2024-04-06T03:40:10.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:10 smithi136 ceph-mon[36436]: [06/Apr/2024:03:40:08] ENGINE Serving on http://172.21.15.136:8765 2024-04-06T03:40:10.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:10 smithi136 ceph-mon[36436]: [06/Apr/2024:03:40:08] ENGINE Bus STARTED 2024-04-06T03:40:11.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:11 smithi094 ceph-mon[29713]: mgrmap e24: smithi136.jagvwt(active, since 4s) 2024-04-06T03:40:11.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:11 smithi094 ceph-mon[29713]: pgmap v5: 65 pgs: 65 active+clean; 145 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:40:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:11 smithi136 ceph-mon[36436]: mgrmap e24: smithi136.jagvwt(active, since 4s) 2024-04-06T03:40:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:11 smithi136 ceph-mon[36436]: pgmap v5: 65 pgs: 65 active+clean; 145 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:40:12.200 INFO:tasks.workunit.client.0.smithi094.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-06T03:40:12.201 INFO:tasks.workunit.client.0.smithi094.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-06T03:40:12.201 INFO:tasks.workunit.client.0.smithi094.stderr:+ make 2024-04-06T03:40:12.652 INFO:tasks.workunit.client.0.smithi094.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-06T03:40:13.228 INFO:tasks.workunit.client.0.smithi094.stderr:++ readlink -f fsstress 2024-04-06T03:40:13.229 INFO:tasks.workunit.client.0.smithi094.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-06T03:40:13.229 INFO:tasks.workunit.client.0.smithi094.stderr:+ popd 2024-04-06T03:40:13.229 INFO:tasks.workunit.client.0.smithi094.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-06T03:40:13.230 INFO:tasks.workunit.client.0.smithi094.stderr:+ popd 2024-04-06T03:40:13.230 INFO:tasks.workunit.client.0.smithi094.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-06T03:40:13.230 INFO:tasks.workunit.client.0.smithi094.stderr:++ mktemp -d -p . 2024-04-06T03:40:13.232 INFO:tasks.workunit.client.0.smithi094.stderr:+ T=./tmp.xr8h8PEbww 2024-04-06T03:40:13.232 INFO:tasks.workunit.client.0.smithi094.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.xr8h8PEbww -l 1 -n 1000 -p 10 -v 2024-04-06T03:40:13.234 INFO:tasks.workunit.client.0.smithi094.stdout:seed = 1712540165 2024-04-06T03:40:13.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/0: dread - no filename 2024-04-06T03:40:13.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/1: dwrite - no filename 2024-04-06T03:40:13.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/2: write - no filename 2024-04-06T03:40:13.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/3: read - no filename 2024-04-06T03:40:13.239 INFO:tasks.workunit.client.0.smithi094.stdout:1/4: readlink - no filename 2024-04-06T03:40:13.240 INFO:tasks.workunit.client.0.smithi094.stdout:0/0: creat f0 x:0 0 0 2024-04-06T03:40:13.240 INFO:tasks.workunit.client.0.smithi094.stdout:0/1: dread - f0 zero size 2024-04-06T03:40:13.242 INFO:tasks.workunit.client.0.smithi094.stdout:2/0: rename - no filename 2024-04-06T03:40:13.242 INFO:tasks.workunit.client.0.smithi094.stdout:2/1: dwrite - no filename 2024-04-06T03:40:13.242 INFO:tasks.workunit.client.0.smithi094.stdout:2/2: dread - no filename 2024-04-06T03:40:13.242 INFO:tasks.workunit.client.0.smithi094.stdout:2/3: dread - no filename 2024-04-06T03:40:13.242 INFO:tasks.workunit.client.0.smithi094.stdout:1/5: mkdir d0 0 2024-04-06T03:40:13.242 INFO:tasks.workunit.client.0.smithi094.stdout:1/6: chown d0 28937 1 2024-04-06T03:40:13.243 INFO:tasks.workunit.client.0.smithi094.stdout:1/7: unlink - no file 2024-04-06T03:40:13.243 INFO:tasks.workunit.client.0.smithi094.stdout:0/2: mkdir d1 0 2024-04-06T03:40:13.244 INFO:tasks.workunit.client.0.smithi094.stdout:3/0: readlink - no filename 2024-04-06T03:40:13.244 INFO:tasks.workunit.client.0.smithi094.stdout:3/1: dwrite - no filename 2024-04-06T03:40:13.246 INFO:tasks.workunit.client.0.smithi094.stdout:2/4: mkdir d0 0 2024-04-06T03:40:13.246 INFO:tasks.workunit.client.0.smithi094.stdout:2/5: write - no filename 2024-04-06T03:40:13.247 INFO:tasks.workunit.client.0.smithi094.stdout:1/8: mkdir d0/d1 0 2024-04-06T03:40:13.247 INFO:tasks.workunit.client.0.smithi094.stdout:1/9: chown d0/d1 0 1 2024-04-06T03:40:13.251 INFO:tasks.workunit.client.0.smithi094.stdout:4/0: rmdir - no directory 2024-04-06T03:40:13.251 INFO:tasks.workunit.client.0.smithi094.stdout:4/1: rename - no filename 2024-04-06T03:40:13.251 INFO:tasks.workunit.client.0.smithi094.stdout:3/2: mkdir d0 0 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/3: dread - no filename 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/4: chown d0 2348393 1 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/5: chown d0 4429041 1 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/6: rename d0 to d0/d1 22 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/7: fsync - no filename 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/8: dwrite - no filename 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/9: fsync - no filename 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/10: dwrite - no filename 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/11: dwrite - no filename 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/12: chown d0 11011 1 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/13: rename d0 to d0/d2 22 2024-04-06T03:40:13.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/14: read - no filename 2024-04-06T03:40:13.253 INFO:tasks.workunit.client.0.smithi094.stdout:2/6: creat d0/f1 x:0 0 0 2024-04-06T03:40:13.253 INFO:tasks.workunit.client.0.smithi094.stdout:2/7: write d0/f1 [138833,34436] 0 2024-04-06T03:40:13.253 INFO:tasks.workunit.client.0.smithi094.stdout:1/10: creat d0/d1/f2 x:0 0 0 2024-04-06T03:40:13.255 INFO:tasks.workunit.client.0.smithi094.stdout:4/2: mkdir d0 0 2024-04-06T03:40:13.255 INFO:tasks.workunit.client.0.smithi094.stdout:4/3: fdatasync - no filename 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/15: rmdir d0 0 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/16: unlink - no file 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/17: rename - no filename 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/18: unlink - no file 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/19: chown . 16056 1 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/20: rename - no filename 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/21: rename - no filename 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/22: dwrite - no filename 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/23: rename - no filename 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/24: truncate - no filename 2024-04-06T03:40:13.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/25: dwrite - no filename 2024-04-06T03:40:13.258 INFO:tasks.workunit.client.0.smithi094.stdout:3/26: truncate - no filename 2024-04-06T03:40:13.258 INFO:tasks.workunit.client.0.smithi094.stdout:0/3: dwrite f0 [0,4194304] 0 2024-04-06T03:40:13.258 INFO:tasks.workunit.client.0.smithi094.stdout:0/4: chown f0 56 1 2024-04-06T03:40:13.258 INFO:tasks.workunit.client.0.smithi094.stdout:2/8: dread d0/f1 [0,4194304] 0 2024-04-06T03:40:13.259 INFO:tasks.workunit.client.0.smithi094.stdout:1/11: unlink d0/d1/f2 0 2024-04-06T03:40:13.259 INFO:tasks.workunit.client.0.smithi094.stdout:7/0: write - no filename 2024-04-06T03:40:13.259 INFO:tasks.workunit.client.0.smithi094.stdout:7/1: stat - no entries 2024-04-06T03:40:13.259 INFO:tasks.workunit.client.0.smithi094.stdout:7/2: dread - no filename 2024-04-06T03:40:13.260 INFO:tasks.workunit.client.0.smithi094.stdout:9/0: mkdir d0 0 2024-04-06T03:40:13.260 INFO:tasks.workunit.client.0.smithi094.stdout:9/1: dwrite - no filename 2024-04-06T03:40:13.261 INFO:tasks.workunit.client.0.smithi094.stdout:9/2: dwrite - no filename 2024-04-06T03:40:13.261 INFO:tasks.workunit.client.0.smithi094.stdout:9/3: chown d0 59440 1 2024-04-06T03:40:13.261 INFO:tasks.workunit.client.0.smithi094.stdout:9/4: read - no filename 2024-04-06T03:40:13.261 INFO:tasks.workunit.client.0.smithi094.stdout:9/5: dwrite - no filename 2024-04-06T03:40:13.261 INFO:tasks.workunit.client.0.smithi094.stdout:9/6: unlink - no file 2024-04-06T03:40:13.261 INFO:tasks.workunit.client.0.smithi094.stdout:9/7: rename d0 to d0/d1 22 2024-04-06T03:40:13.261 INFO:tasks.workunit.client.0.smithi094.stdout:9/8: read - no filename 2024-04-06T03:40:13.261 INFO:tasks.workunit.client.0.smithi094.stdout:4/4: symlink d0/l1 0 2024-04-06T03:40:13.261 INFO:tasks.workunit.client.0.smithi094.stdout:4/5: chown d0 151 1 2024-04-06T03:40:13.262 INFO:tasks.workunit.client.0.smithi094.stdout:3/27: symlink l3 0 2024-04-06T03:40:13.262 INFO:tasks.workunit.client.0.smithi094.stdout:3/28: chown l3 1816 1 2024-04-06T03:40:13.265 INFO:tasks.workunit.client.0.smithi094.stdout:0/5: creat d1/f2 x:0 0 0 2024-04-06T03:40:13.266 INFO:tasks.workunit.client.0.smithi094.stdout:2/9: mknod d0/c2 0 2024-04-06T03:40:13.268 INFO:tasks.workunit.client.0.smithi094.stdout:1/12: symlink d0/l3 0 2024-04-06T03:40:13.268 INFO:tasks.workunit.client.0.smithi094.stdout:1/13: write - no filename 2024-04-06T03:40:13.268 INFO:tasks.workunit.client.0.smithi094.stdout:1/14: dwrite - no filename 2024-04-06T03:40:13.268 INFO:tasks.workunit.client.0.smithi094.stdout:1/15: dwrite - no filename 2024-04-06T03:40:13.271 INFO:tasks.workunit.client.0.smithi094.stdout:7/3: mknod c0 0 2024-04-06T03:40:13.271 INFO:tasks.workunit.client.0.smithi094.stdout:7/4: dwrite - no filename 2024-04-06T03:40:13.272 INFO:tasks.workunit.client.0.smithi094.stdout:7/5: fdatasync - no filename 2024-04-06T03:40:13.272 INFO:tasks.workunit.client.0.smithi094.stdout:7/6: dwrite - no filename 2024-04-06T03:40:13.276 INFO:tasks.workunit.client.0.smithi094.stdout:9/9: rmdir d0 0 2024-04-06T03:40:13.276 INFO:tasks.workunit.client.0.smithi094.stdout:9/10: chown . 235595 1 2024-04-06T03:40:13.277 INFO:tasks.workunit.client.0.smithi094.stdout:4/6: creat d0/f2 x:0 0 0 2024-04-06T03:40:13.280 INFO:tasks.workunit.client.0.smithi094.stdout:3/29: creat f4 x:0 0 0 2024-04-06T03:40:13.282 INFO:tasks.workunit.client.0.smithi094.stdout:2/10: symlink d0/l3 0 2024-04-06T03:40:13.282 INFO:tasks.workunit.client.0.smithi094.stdout:2/11: fdatasync d0/f1 0 2024-04-06T03:40:13.294 INFO:tasks.workunit.client.0.smithi094.stdout:1/16: unlink d0/l3 0 2024-04-06T03:40:13.294 INFO:tasks.workunit.client.0.smithi094.stdout:1/17: dwrite - no filename 2024-04-06T03:40:13.310 INFO:tasks.workunit.client.0.smithi094.stdout:8/0: mknod c0 0 2024-04-06T03:40:13.316 INFO:tasks.workunit.client.0.smithi094.stdout:9/11: mkdir d2 0 2024-04-06T03:40:13.316 INFO:tasks.workunit.client.0.smithi094.stdout:9/12: getdents d2 0 2024-04-06T03:40:13.319 INFO:tasks.workunit.client.0.smithi094.stdout:2/12: link d0/l3 d0/l4 0 2024-04-06T03:40:13.323 INFO:tasks.workunit.client.0.smithi094.stdout:1/18: mkdir d0/d4 0 2024-04-06T03:40:13.323 INFO:tasks.workunit.client.0.smithi094.stdout:1/19: dread - no filename 2024-04-06T03:40:13.332 INFO:tasks.workunit.client.0.smithi094.stdout:9/13: symlink d2/l3 0 2024-04-06T03:40:13.342 INFO:tasks.workunit.client.0.smithi094.stdout:2/13: link d0/f1 d0/f5 0 2024-04-06T03:40:13.342 INFO:tasks.workunit.client.0.smithi094.stdout:2/14: fdatasync d0/f5 0 2024-04-06T03:40:13.344 INFO:tasks.workunit.client.0.smithi094.stdout:1/20: creat d0/f5 x:0 0 0 2024-04-06T03:40:13.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/14: creat d2/f4 x:0 0 0 2024-04-06T03:40:13.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/15: write d2/f4 [926050,14902] 0 2024-04-06T03:40:13.372 INFO:tasks.workunit.client.0.smithi094.stdout:0/6: dwrite d1/f2 [0,4194304] 0 2024-04-06T03:40:13.411 INFO:tasks.workunit.client.0.smithi094.stdout:1/21: mknod d0/d1/c6 0 2024-04-06T03:40:13.413 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:13 smithi136 ceph-mon[36436]: pgmap v6: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.8 MiB/s wr, 51 op/s 2024-04-06T03:40:13.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:13 smithi094 ceph-mon[29713]: pgmap v6: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.8 MiB/s wr, 51 op/s 2024-04-06T03:40:13.429 INFO:tasks.workunit.client.0.smithi094.stdout:2/15: dwrite d0/f5 [0,4194304] 0 2024-04-06T03:40:13.456 INFO:tasks.workunit.client.0.smithi094.stdout:8/1: write - no filename 2024-04-06T03:40:13.476 INFO:tasks.workunit.client.0.smithi094.stdout:1/22: dwrite d0/f5 [0,4194304] 0 2024-04-06T03:40:13.525 INFO:tasks.workunit.client.0.smithi094.stdout:4/7: dwrite d0/f2 [0,4194304] 0 2024-04-06T03:40:13.525 INFO:tasks.workunit.client.0.smithi094.stdout:4/8: creat d0/f3 x:0 0 0 2024-04-06T03:40:13.525 INFO:tasks.workunit.client.0.smithi094.stdout:4/9: truncate d0/f3 788649 0 2024-04-06T03:40:13.541 INFO:tasks.workunit.client.0.smithi094.stdout:4/10: dwrite d0/f2 [0,4194304] 0 2024-04-06T03:40:13.541 INFO:tasks.workunit.client.0.smithi094.stdout:4/11: chown d0/l1 2604 1 2024-04-06T03:40:13.606 INFO:tasks.workunit.client.0.smithi094.stdout:9/16: fsync d2/f4 0 2024-04-06T03:40:13.618 INFO:tasks.workunit.client.0.smithi094.stdout:9/17: dwrite d2/f4 [0,4194304] 0 2024-04-06T03:40:14.555 INFO:tasks.workunit.client.0.smithi094.stdout:6/0: symlink l0 0 2024-04-06T03:40:14.555 INFO:tasks.workunit.client.0.smithi094.stdout:6/1: dwrite - no filename 2024-04-06T03:40:14.556 INFO:tasks.workunit.client.0.smithi094.stdout:6/2: dread - no filename 2024-04-06T03:40:14.556 INFO:tasks.workunit.client.0.smithi094.stdout:6/3: dread - no filename 2024-04-06T03:40:14.556 INFO:tasks.workunit.client.0.smithi094.stdout:8/2: creat f1 x:0 0 0 2024-04-06T03:40:14.558 INFO:tasks.workunit.client.0.smithi094.stdout:5/0: symlink l0 0 2024-04-06T03:40:14.568 INFO:tasks.workunit.client.0.smithi094.stdout:8/3: dwrite f1 [0,4194304] 0 2024-04-06T03:40:14.575 INFO:tasks.workunit.client.0.smithi094.stdout:8/4: dread f1 [0,4194304] 0 2024-04-06T03:40:14.575 INFO:tasks.workunit.client.0.smithi094.stdout:8/5: creat f2 x:0 0 0 2024-04-06T03:40:14.610 INFO:tasks.workunit.client.0.smithi094.stdout:2/16: dwrite d0/f5 [4194304,4194304] 0 2024-04-06T03:40:14.610 INFO:tasks.workunit.client.0.smithi094.stdout:2/17: fdatasync d0/f5 0 2024-04-06T03:40:15.130 INFO:tasks.workunit.client.0.smithi094.stdout:0/7: unlink d1/f2 0 2024-04-06T03:40:15.134 INFO:tasks.workunit.client.0.smithi094.stdout:6/4: link l0 l1 0 2024-04-06T03:40:15.134 INFO:tasks.workunit.client.0.smithi094.stdout:6/5: rmdir - no directory 2024-04-06T03:40:15.134 INFO:tasks.workunit.client.0.smithi094.stdout:6/6: chown l1 327902790 1 2024-04-06T03:40:15.134 INFO:tasks.workunit.client.0.smithi094.stdout:5/1: creat f1 x:0 0 0 2024-04-06T03:40:15.142 INFO:tasks.workunit.client.0.smithi094.stdout:2/18: symlink d0/l6 0 2024-04-06T03:40:15.143 INFO:tasks.workunit.client.0.smithi094.stdout:1/23: unlink d0/f5 0 2024-04-06T03:40:15.143 INFO:tasks.workunit.client.0.smithi094.stdout:1/24: readlink - no filename 2024-04-06T03:40:15.145 INFO:tasks.workunit.client.0.smithi094.stdout:0/8: mknod d1/c3 0 2024-04-06T03:40:15.147 INFO:tasks.workunit.client.0.smithi094.stdout:6/7: creat f2 x:0 0 0 2024-04-06T03:40:15.151 INFO:tasks.workunit.client.0.smithi094.stdout:8/6: unlink f2 0 2024-04-06T03:40:15.151 INFO:tasks.workunit.client.0.smithi094.stdout:8/7: chown c0 5 1 2024-04-06T03:40:15.154 INFO:tasks.workunit.client.0.smithi094.stdout:8/8: write f1 [2682282,114668] 0 2024-04-06T03:40:15.155 INFO:tasks.workunit.client.0.smithi094.stdout:1/25: mkdir d0/d7 0 2024-04-06T03:40:15.159 INFO:tasks.workunit.client.0.smithi094.stdout:0/9: rename d1/c3 to d1/c4 0 2024-04-06T03:40:15.159 INFO:tasks.workunit.client.0.smithi094.stdout:0/10: creat d1/f5 x:0 0 0 2024-04-06T03:40:15.165 INFO:tasks.workunit.client.0.smithi094.stdout:6/8: dwrite f2 [0,4194304] 0 2024-04-06T03:40:15.167 INFO:tasks.workunit.client.0.smithi094.stdout:1/26: symlink d0/d1/l8 0 2024-04-06T03:40:15.169 INFO:tasks.workunit.client.0.smithi094.stdout:0/11: symlink d1/l6 0 2024-04-06T03:40:15.173 INFO:tasks.workunit.client.0.smithi094.stdout:8/9: dwrite f1 [0,4194304] 0 2024-04-06T03:40:15.173 INFO:tasks.workunit.client.0.smithi094.stdout:8/10: creat f3 x:0 0 0 2024-04-06T03:40:15.173 INFO:tasks.workunit.client.0.smithi094.stdout:6/9: mknod c3 0 2024-04-06T03:40:15.176 INFO:tasks.workunit.client.0.smithi094.stdout:1/27: creat d0/d4/f9 x:0 0 0 2024-04-06T03:40:15.176 INFO:tasks.workunit.client.0.smithi094.stdout:1/28: fdatasync d0/d4/f9 0 2024-04-06T03:40:15.180 INFO:tasks.workunit.client.0.smithi094.stdout:8/11: rename c0 to c4 0 2024-04-06T03:40:15.184 INFO:tasks.workunit.client.0.smithi094.stdout:0/12: dwrite f0 [4194304,4194304] 0 2024-04-06T03:40:15.185 INFO:tasks.workunit.client.0.smithi094.stdout:6/10: dread f2 [0,4194304] 0 2024-04-06T03:40:15.201 INFO:tasks.workunit.client.0.smithi094.stdout:6/11: dwrite f2 [0,4194304] 0 2024-04-06T03:40:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:15 smithi094 ceph-mon[29713]: pgmap v7: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.0 MiB/s wr, 36 op/s 2024-04-06T03:40:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:15 smithi094 ceph-mon[29713]: Standby manager daemon smithi094.purlzp started 2024-04-06T03:40:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:15 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.94:0/2124958319' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.purlzp/crt"}]: dispatch 2024-04-06T03:40:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:15 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.94:0/2124958319' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:40:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:15 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.94:0/2124958319' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.purlzp/key"}]: dispatch 2024-04-06T03:40:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:15 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.94:0/2124958319' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:40:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:15 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:15 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:15.574 INFO:tasks.workunit.client.0.smithi094.stdout:4/12: rmdir d0 39 2024-04-06T03:40:15.592 INFO:tasks.workunit.client.0.smithi094.stdout:4/13: dwrite d0/f2 [4194304,4194304] 0 2024-04-06T03:40:15.592 INFO:tasks.workunit.client.0.smithi094.stdout:4/14: chown d0/l1 41666 1 2024-04-06T03:40:15.596 INFO:tasks.workunit.client.0.smithi094.stdout:4/15: mkdir d0/d4 0 2024-04-06T03:40:15.596 INFO:tasks.workunit.client.0.smithi094.stdout:4/16: rename d0 to d0/d5 22 2024-04-06T03:40:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:15 smithi136 ceph-mon[36436]: pgmap v7: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.0 MiB/s wr, 36 op/s 2024-04-06T03:40:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:15 smithi136 ceph-mon[36436]: Standby manager daemon smithi094.purlzp started 2024-04-06T03:40:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:15 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.94:0/2124958319' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.purlzp/crt"}]: dispatch 2024-04-06T03:40:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:15 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.94:0/2124958319' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:40:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:15 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.94:0/2124958319' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.purlzp/key"}]: dispatch 2024-04-06T03:40:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:15 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.94:0/2124958319' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:40:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:15 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:15 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:15.600 INFO:tasks.workunit.client.0.smithi094.stdout:4/17: link d0/f2 d0/d4/f6 0 2024-04-06T03:40:15.600 INFO:tasks.workunit.client.0.smithi094.stdout:5/2: getdents . 0 2024-04-06T03:40:15.600 INFO:tasks.workunit.client.0.smithi094.stdout:5/3: chown f1 172377 1 2024-04-06T03:40:15.603 INFO:tasks.workunit.client.0.smithi094.stdout:4/18: mkdir d0/d7 0 2024-04-06T03:40:15.610 INFO:tasks.workunit.client.0.smithi094.stdout:2/19: rmdir d0 39 2024-04-06T03:40:15.622 INFO:tasks.workunit.client.0.smithi094.stdout:5/4: dwrite f1 [0,4194304] 0 2024-04-06T03:40:15.623 INFO:tasks.workunit.client.0.smithi094.stdout:4/19: dwrite d0/f2 [4194304,4194304] 0 2024-04-06T03:40:15.623 INFO:tasks.workunit.client.0.smithi094.stdout:2/20: write d0/f1 [3820875,68578] 0 2024-04-06T03:40:15.626 INFO:tasks.workunit.client.0.smithi094.stdout:4/20: rmdir d0/d7 0 2024-04-06T03:40:15.629 INFO:tasks.workunit.client.0.smithi094.stdout:2/21: mknod d0/c7 0 2024-04-06T03:40:15.631 INFO:tasks.workunit.client.0.smithi094.stdout:5/5: read f1 [3992409,59896] 0 2024-04-06T03:40:15.631 INFO:tasks.workunit.client.0.smithi094.stdout:5/6: write f1 [105183,61360] 0 2024-04-06T03:40:15.631 INFO:tasks.workunit.client.0.smithi094.stdout:5/7: chown f1 925964453 1 2024-04-06T03:40:15.633 INFO:tasks.workunit.client.0.smithi094.stdout:4/21: write d0/f2 [3294263,76900] 0 2024-04-06T03:40:15.639 INFO:tasks.workunit.client.0.smithi094.stdout:5/8: dread f1 [0,4194304] 0 2024-04-06T03:40:15.643 INFO:tasks.workunit.client.0.smithi094.stdout:8/12: truncate f1 689428 0 2024-04-06T03:40:15.643 INFO:tasks.workunit.client.0.smithi094.stdout:8/13: dread - f3 zero size 2024-04-06T03:40:15.643 INFO:tasks.workunit.client.0.smithi094.stdout:2/22: dread d0/f1 [4194304,4194304] 0 2024-04-06T03:40:15.644 INFO:tasks.workunit.client.0.smithi094.stdout:8/14: mknod c5 0 2024-04-06T03:40:15.645 INFO:tasks.workunit.client.0.smithi094.stdout:2/23: mkdir d0/d8 0 2024-04-06T03:40:15.645 INFO:tasks.workunit.client.0.smithi094.stdout:2/24: write d0/f5 [481460,2002] 0 2024-04-06T03:40:15.647 INFO:tasks.workunit.client.0.smithi094.stdout:2/25: fsync d0/f5 0 2024-04-06T03:40:15.652 INFO:tasks.workunit.client.0.smithi094.stdout:2/26: link d0/l6 d0/d8/l9 0 2024-04-06T03:40:15.654 INFO:tasks.workunit.client.0.smithi094.stdout:4/22: dwrite d0/f3 [0,4194304] 0 2024-04-06T03:40:15.660 INFO:tasks.workunit.client.0.smithi094.stdout:5/9: dwrite f1 [0,4194304] 0 2024-04-06T03:40:15.664 INFO:tasks.workunit.client.0.smithi094.stdout:5/10: creat f2 x:0 0 0 2024-04-06T03:40:15.666 INFO:tasks.workunit.client.0.smithi094.stdout:5/11: rename l0 to l3 0 2024-04-06T03:40:15.678 INFO:tasks.workunit.client.0.smithi094.stdout:5/12: dwrite f2 [0,4194304] 0 2024-04-06T03:40:15.683 INFO:tasks.workunit.client.0.smithi094.stdout:5/13: mknod c4 0 2024-04-06T03:40:15.685 INFO:tasks.workunit.client.0.smithi094.stdout:5/14: symlink l5 0 2024-04-06T03:40:15.685 INFO:tasks.workunit.client.0.smithi094.stdout:5/15: truncate f1 5161277 0 2024-04-06T03:40:16.028 INFO:tasks.workunit.client.0.smithi094.stdout:2/27: truncate d0/f5 6014830 0 2024-04-06T03:40:16.029 INFO:tasks.workunit.client.0.smithi094.stdout:2/28: rmdir d0/d8 39 2024-04-06T03:40:16.031 INFO:tasks.workunit.client.0.smithi094.stdout:2/29: link d0/l3 d0/d8/la 0 2024-04-06T03:40:16.033 INFO:tasks.workunit.client.0.smithi094.stdout:4/23: truncate d0/f3 672775 0 2024-04-06T03:40:16.033 INFO:tasks.workunit.client.0.smithi094.stdout:4/24: chown d0/f3 1 1 2024-04-06T03:40:16.034 INFO:tasks.workunit.client.0.smithi094.stdout:4/25: unlink d0/l1 0 2024-04-06T03:40:16.036 INFO:tasks.workunit.client.0.smithi094.stdout:8/15: dwrite f1 [0,4194304] 0 2024-04-06T03:40:16.036 INFO:tasks.workunit.client.0.smithi094.stdout:8/16: fdatasync f3 0 2024-04-06T03:40:16.041 INFO:tasks.workunit.client.0.smithi094.stdout:8/17: dread f1 [0,4194304] 0 2024-04-06T03:40:16.042 INFO:tasks.workunit.client.0.smithi094.stdout:8/18: chown f3 26 1 2024-04-06T03:40:16.042 INFO:tasks.workunit.client.0.smithi094.stdout:8/19: rename c4 to c6 0 2024-04-06T03:40:16.043 INFO:tasks.workunit.client.0.smithi094.stdout:8/20: dread - f3 zero size 2024-04-06T03:40:16.059 INFO:tasks.workunit.client.0.smithi094.stdout:8/21: dwrite f3 [0,4194304] 0 2024-04-06T03:40:16.068 INFO:tasks.workunit.client.0.smithi094.stdout:8/22: dread f3 [0,4194304] 0 2024-04-06T03:40:16.068 INFO:tasks.workunit.client.0.smithi094.stdout:8/23: rmdir - no directory 2024-04-06T03:40:16.069 INFO:tasks.workunit.client.0.smithi094.stdout:8/24: mknod c7 0 2024-04-06T03:40:16.070 INFO:tasks.workunit.client.0.smithi094.stdout:8/25: symlink l8 0 2024-04-06T03:40:16.071 INFO:tasks.workunit.client.0.smithi094.stdout:8/26: symlink l9 0 2024-04-06T03:40:16.071 INFO:tasks.workunit.client.0.smithi094.stdout:8/27: fsync f1 0 2024-04-06T03:40:16.140 INFO:tasks.workunit.client.0.smithi094.stdout:2/30: write d0/f5 [622508,119369] 0 2024-04-06T03:40:16.140 INFO:tasks.workunit.client.0.smithi094.stdout:2/31: write d0/f1 [2768539,111151] 0 2024-04-06T03:40:16.147 INFO:tasks.workunit.client.0.smithi094.stdout:2/32: rename d0/d8/l9 to d0/lb 0 2024-04-06T03:40:16.153 INFO:tasks.workunit.client.0.smithi094.stdout:8/28: truncate f1 2915693 0 2024-04-06T03:40:16.154 INFO:tasks.workunit.client.0.smithi094.stdout:8/29: mknod ca 0 2024-04-06T03:40:16.157 INFO:tasks.workunit.client.0.smithi094.stdout:8/30: link c5 cb 0 2024-04-06T03:40:16.160 INFO:tasks.workunit.client.0.smithi094.stdout:8/31: symlink lc 0 2024-04-06T03:40:16.160 INFO:tasks.workunit.client.0.smithi094.stdout:8/32: creat fd x:0 0 0 2024-04-06T03:40:16.164 INFO:tasks.workunit.client.0.smithi094.stdout:8/33: mknod ce 0 2024-04-06T03:40:16.168 INFO:tasks.workunit.client.0.smithi094.stdout:8/34: symlink lf 0 2024-04-06T03:40:16.169 INFO:tasks.workunit.client.0.smithi094.stdout:2/33: dwrite d0/f1 [0,4194304] 0 2024-04-06T03:40:16.171 INFO:tasks.workunit.client.0.smithi094.stdout:8/35: link lf l10 0 2024-04-06T03:40:16.174 INFO:tasks.workunit.client.0.smithi094.stdout:2/34: mknod d0/d8/cc 0 2024-04-06T03:40:16.174 INFO:tasks.workunit.client.0.smithi094.stdout:2/35: chown d0/f5 485 1 2024-04-06T03:40:16.177 INFO:tasks.workunit.client.0.smithi094.stdout:2/36: mkdir d0/dd 0 2024-04-06T03:40:16.180 INFO:tasks.workunit.client.0.smithi094.stdout:2/37: creat d0/d8/fe x:0 0 0 2024-04-06T03:40:16.183 INFO:tasks.workunit.client.0.smithi094.stdout:8/36: dwrite fd [0,4194304] 0 2024-04-06T03:40:16.189 INFO:tasks.workunit.client.0.smithi094.stdout:8/37: link f1 f11 0 2024-04-06T03:40:16.192 INFO:tasks.workunit.client.0.smithi094.stdout:8/38: link l10 l12 0 2024-04-06T03:40:16.192 INFO:tasks.workunit.client.0.smithi094.stdout:8/39: creat f13 x:0 0 0 2024-04-06T03:40:16.197 INFO:tasks.workunit.client.0.smithi094.stdout:2/38: dwrite d0/f1 [0,4194304] 0 2024-04-06T03:40:16.204 INFO:tasks.workunit.client.0.smithi094.stdout:8/40: dwrite fd [0,4194304] 0 2024-04-06T03:40:16.204 INFO:tasks.workunit.client.0.smithi094.stdout:8/41: truncate f11 3168735 0 2024-04-06T03:40:16.204 INFO:tasks.workunit.client.0.smithi094.stdout:8/42: readlink l9 0 2024-04-06T03:40:16.209 INFO:tasks.workunit.client.0.smithi094.stdout:2/39: dread d0/f1 [0,4194304] 0 2024-04-06T03:40:16.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/40: write d0/f5 [559974,69872] 0 2024-04-06T03:40:16.215 INFO:tasks.workunit.client.0.smithi094.stdout:2/41: chown d0/c2 15 1 2024-04-06T03:40:16.215 INFO:tasks.workunit.client.0.smithi094.stdout:2/42: stat d0/f5 0 2024-04-06T03:40:16.215 INFO:tasks.workunit.client.0.smithi094.stdout:2/43: chown d0/d8/la 714 1 2024-04-06T03:40:16.215 INFO:tasks.workunit.client.0.smithi094.stdout:2/44: chown d0/d8/fe 29227982 1 2024-04-06T03:40:16.217 INFO:tasks.workunit.client.0.smithi094.stdout:8/43: dwrite f11 [0,4194304] 0 2024-04-06T03:40:16.230 INFO:tasks.workunit.client.0.smithi094.stdout:8/44: dwrite f11 [0,4194304] 0 2024-04-06T03:40:16.231 INFO:tasks.workunit.client.0.smithi094.stdout:8/45: link f11 f14 0 2024-04-06T03:40:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:16 smithi094 ceph-mon[29713]: mgrmap e25: smithi136.jagvwt(active, since 9s), standbys: smithi094.purlzp 2024-04-06T03:40:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:16 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:40:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:16 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:16 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:16.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:16 smithi094 ceph-mon[29713]: pgmap v8: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-06T03:40:16.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:16 smithi136 ceph-mon[36436]: mgrmap e25: smithi136.jagvwt(active, since 9s), standbys: smithi094.purlzp 2024-04-06T03:40:16.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:16 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:40:16.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:16 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:16.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:16 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:16.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:16 smithi136 ceph-mon[36436]: pgmap v8: 65 pgs: 65 active+clean; 159 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.6 MiB/s wr, 28 op/s 2024-04-06T03:40:17.003 INFO:tasks.workunit.client.1.smithi136.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-06T03:40:17.008 INFO:tasks.workunit.client.1.smithi136.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-06T03:40:17.008 INFO:tasks.workunit.client.1.smithi136.stderr:+ make 2024-04-06T03:40:17.093 INFO:tasks.workunit.client.1.smithi136.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-06T03:40:19.369 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:19 smithi136 ceph-mon[36436]: pgmap v9: 65 pgs: 65 active+clean; 238 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 131 op/s 2024-04-06T03:40:19.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:19 smithi094 ceph-mon[29713]: pgmap v9: 65 pgs: 65 active+clean; 238 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 131 op/s 2024-04-06T03:40:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:20 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:20 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:20 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:20 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:20 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:20 smithi136 ceph-mon[36436]: Adjusting osd_memory_target on smithi136 to 2581M 2024-04-06T03:40:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:20 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:21.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:20 smithi136 ceph-mon[36436]: pgmap v10: 65 pgs: 65 active+clean; 238 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 120 op/s 2024-04-06T03:40:21.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:20 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:21.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:20 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:21.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:20 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:21.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:20 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:21.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:20 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:21.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:20 smithi094 ceph-mon[29713]: Adjusting osd_memory_target on smithi136 to 2581M 2024-04-06T03:40:21.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:20 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:21.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:20 smithi094 ceph-mon[29713]: pgmap v10: 65 pgs: 65 active+clean; 238 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 12 MiB/s wr, 120 op/s 2024-04-06T03:40:21.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.732+0000 7fad288ff700 1 -- 172.21.15.94:0/2436520944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 msgr2=0x7fad240ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:21.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.732+0000 7fad288ff700 1 --2- 172.21.15.94:0/2436520944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 0x7fad240ffa50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fad14004940 tx=0x7fad14009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.732+0000 7fad288ff700 1 -- 172.21.15.94:0/2436520944 shutdown_connections 2024-04-06T03:40:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.732+0000 7fad288ff700 1 --2- 172.21.15.94:0/2436520944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad240fff90 0x7fad24102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.732+0000 7fad288ff700 1 --2- 172.21.15.94:0/2436520944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 0x7fad240ffa50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.732+0000 7fad288ff700 1 -- 172.21.15.94:0/2436520944 >> 172.21.15.94:0/2436520944 conn(0x7fad240f7480 msgr2=0x7fad240f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.732+0000 7fad288ff700 1 -- 172.21.15.94:0/2436520944 shutdown_connections 2024-04-06T03:40:21.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.733+0000 7fad288ff700 1 -- 172.21.15.94:0/2436520944 wait complete. 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.733+0000 7fad288ff700 1 Processor -- start 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.733+0000 7fad288ff700 1 -- start start 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.733+0000 7fad288ff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 0x7fad2410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.733+0000 7fad288ff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad240fff90 0x7fad2410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.733+0000 7fad288ff700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2410e600 con 0x7fad24066af0 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.733+0000 7fad288ff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2410edf0 con 0x7fad240fff90 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad2259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 0x7fad2410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad2259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 0x7fad2410da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40048/0 (socket says 172.21.15.94:40048) 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad2259c700 1 -- 172.21.15.94:0/3466914718 learned_addr learned my addr 172.21.15.94:0/3466914718 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad21d9b700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad240fff90 0x7fad2410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:21.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad2259c700 1 -- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad240fff90 msgr2=0x7fad2410df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad2259c700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad240fff90 0x7fad2410df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad2259c700 1 -- 172.21.15.94:0/3466914718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14005040 con 0x7fad24066af0 2024-04-06T03:40:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad2259c700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 0x7fad2410da30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fad1400b970 tx=0x7fad14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad12ffd700 1 -- 172.21.15.94:0/3466914718 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad1400e040 con 0x7fad24066af0 2024-04-06T03:40:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad12ffd700 1 -- 172.21.15.94:0/3466914718 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad14004290 con 0x7fad24066af0 2024-04-06T03:40:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.734+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2410f070 con 0x7fad24066af0 2024-04-06T03:40:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.735+0000 7fad12ffd700 1 -- 172.21.15.94:0/3466914718 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad1401c3f0 con 0x7fad24066af0 2024-04-06T03:40:21.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.735+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2410f630 con 0x7fad24066af0 2024-04-06T03:40:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.736+0000 7fad12ffd700 1 -- 172.21.15.94:0/3466914718 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7fad1400bbf0 con 0x7fad24066af0 2024-04-06T03:40:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.736+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2405f5f0 con 0x7fad24066af0 2024-04-06T03:40:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.736+0000 7fad12ffd700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fad0c06b540 0x7fad0c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.737+0000 7fad12ffd700 1 -- 172.21.15.94:0/3466914718 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fad1408b3e0 con 0x7fad24066af0 2024-04-06T03:40:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.737+0000 7fad21d9b700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fad0c06b540 0x7fad0c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:21.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.737+0000 7fad21d9b700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fad0c06b540 0x7fad0c06d9f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad18009bf0 tx=0x7fad180091a0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:21.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.746+0000 7fad12ffd700 1 -- 172.21.15.94:0/3466914718 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fad14058d00 con 0x7fad24066af0 2024-04-06T03:40:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:21.999+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad2410f910 con 0x7fad0c06b540 2024-04-06T03:40:22.002 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:21 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:40:22.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.001+0000 7fad12ffd700 1 -- 172.21.15.94:0/3466914718 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fad2410f910 con 0x7fad0c06b540 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.003+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fad0c06b540 msgr2=0x7fad0c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fad0c06b540 0x7fad0c06d9f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad18009bf0 tx=0x7fad180091a0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 msgr2=0x7fad2410da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 0x7fad2410da30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fad1400b970 tx=0x7fad14009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 shutdown_connections 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24066af0 0x7fad2410da30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fad0c06b540 0x7fad0c06d9f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 --2- 172.21.15.94:0/3466914718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad240fff90 0x7fad2410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 >> 172.21.15.94:0/3466914718 conn(0x7fad240f7480 msgr2=0x7fad240f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 shutdown_connections 2024-04-06T03:40:22.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.004+0000 7fad288ff700 1 -- 172.21.15.94:0/3466914718 wait complete. 2024-04-06T03:40:22.018 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:40:22.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:21 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:40:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.139+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3079655763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c000fff70 msgr2=0x7f7c00102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.139+0000 7f7c04a6a700 1 --2- 172.21.15.94:0/3079655763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c000fff70 0x7f7c00102350 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4004940 tx=0x7f7bf4009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3079655763 shutdown_connections 2024-04-06T03:40:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 --2- 172.21.15.94:0/3079655763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c000fff70 0x7f7c00102350 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 --2- 172.21.15.94:0/3079655763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c00066b10 0x7f7c000ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3079655763 >> 172.21.15.94:0/3079655763 conn(0x7f7c000f7440 msgr2=0x7f7c000f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3079655763 shutdown_connections 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3079655763 wait complete. 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 Processor -- start 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 -- start start 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c00066b10 0x7f7c0010da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c000fff70 0x7f7c0010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0010e560 con 0x7f7c00066b10 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.140+0000 7f7c04a6a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0010ed60 con 0x7f7c000fff70 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7bfe59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c00066b10 0x7f7c0010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7bfe59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c00066b10 0x7f7c0010da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40068/0 (socket says 172.21.15.94:40068) 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7bfe59c700 1 -- 172.21.15.94:0/3736078986 learned_addr learned my addr 172.21.15.94:0/3736078986 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7bfdd9b700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c000fff70 0x7f7c0010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7bfe59c700 1 -- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c000fff70 msgr2=0x7f7c0010df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7bfe59c700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c000fff70 0x7f7c0010df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7bfe59c700 1 -- 172.21.15.94:0/3736078986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4005040 con 0x7f7c00066b10 2024-04-06T03:40:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7bfe59c700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c00066b10 0x7f7c0010da20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000eb70 tx=0x7f7bf000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7beeffd700 1 -- 172.21.15.94:0/3736078986 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf0007cc0 con 0x7f7c00066b10 2024-04-06T03:40:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0010efe0 con 0x7f7c00066b10 2024-04-06T03:40:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7beeffd700 1 -- 172.21.15.94:0/3736078986 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bf000f040 con 0x7f7c00066b10 2024-04-06T03:40:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0010f520 con 0x7f7c00066b10 2024-04-06T03:40:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.141+0000 7f7beeffd700 1 -- 172.21.15.94:0/3736078986 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bf000cbd0 con 0x7f7c00066b10 2024-04-06T03:40:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.142+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be00052e0 con 0x7f7c00066b10 2024-04-06T03:40:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.143+0000 7f7beeffd700 1 -- 172.21.15.94:0/3736078986 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f7bf0009830 con 0x7f7c00066b10 2024-04-06T03:40:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.143+0000 7f7beeffd700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7be806f950 0x7f7be8071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.143+0000 7f7beeffd700 1 -- 172.21.15.94:0/3736078986 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f7bf0089ad0 con 0x7f7c00066b10 2024-04-06T03:40:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.144+0000 7f7bfdd9b700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7be806f950 0x7f7be8071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.144+0000 7f7bfdd9b700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7be806f950 0x7f7be8071e00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009b30 tx=0x7f7bf4009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.148+0000 7f7beeffd700 1 -- 172.21.15.94:0/3736078986 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f7bf004fb80 con 0x7f7c00066b10 2024-04-06T03:40:22.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.440+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7be0000bf0 con 0x7f7be806f950 2024-04-06T03:40:22.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.441+0000 7f7beeffd700 1 -- 172.21.15.94:0/3736078986 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f7be0000bf0 con 0x7f7be806f950 2024-04-06T03:40:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7be806f950 msgr2=0x7f7be8071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7be806f950 0x7f7be8071e00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009b30 tx=0x7f7bf4009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c00066b10 msgr2=0x7f7c0010da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c00066b10 0x7f7c0010da20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000eb70 tx=0x7f7bf000b040 comp rx=0 tx=0).stop 2024-04-06T03:40:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 shutdown_connections 2024-04-06T03:40:22.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c00066b10 0x7f7c0010da20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7be806f950 0x7f7be8071e00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 --2- 172.21.15.94:0/3736078986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c000fff70 0x7f7c0010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 >> 172.21.15.94:0/3736078986 conn(0x7f7c000f7440 msgr2=0x7f7c000f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 shutdown_connections 2024-04-06T03:40:22.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.445+0000 7f7c04a6a700 1 -- 172.21.15.94:0/3736078986 wait complete. 2024-04-06T03:40:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.564+0000 7faf3ebe9700 1 -- 172.21.15.94:0/4102454561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 msgr2=0x7faf380fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.564+0000 7faf3ebe9700 1 --2- 172.21.15.94:0/4102454561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 0x7faf380fd3f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faf24004940 tx=0x7faf24009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.564+0000 7faf3ebe9700 1 -- 172.21.15.94:0/4102454561 shutdown_connections 2024-04-06T03:40:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.564+0000 7faf3ebe9700 1 --2- 172.21.15.94:0/4102454561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 0x7faf380fd3f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.564+0000 7faf3ebe9700 1 --2- 172.21.15.94:0/4102454561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf380fbcd0 0x7faf380fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.564+0000 7faf3ebe9700 1 -- 172.21.15.94:0/4102454561 >> 172.21.15.94:0/4102454561 conn(0x7faf380f7430 msgr2=0x7faf380f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.564+0000 7faf3ebe9700 1 -- 172.21.15.94:0/4102454561 shutdown_connections 2024-04-06T03:40:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.564+0000 7faf3ebe9700 1 -- 172.21.15.94:0/4102454561 wait complete. 2024-04-06T03:40:22.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf3ebe9700 1 Processor -- start 2024-04-06T03:40:22.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf3ebe9700 1 -- start start 2024-04-06T03:40:22.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf3ebe9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf380fbcd0 0x7faf3810daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:22.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf3ebe9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 0x7faf3810dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:22.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf3ebe9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3810e5e0 con 0x7faf380fcf80 2024-04-06T03:40:22.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf3ebe9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3810ed90 con 0x7faf380fbcd0 2024-04-06T03:40:22.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf37fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 0x7faf3810dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:22.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf37fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 0x7faf3810dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40096/0 (socket says 172.21.15.94:40096) 2024-04-06T03:40:22.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf37fff700 1 -- 172.21.15.94:0/665818752 learned_addr learned my addr 172.21.15.94:0/665818752 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.565+0000 7faf3c985700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf380fbcd0 0x7faf3810daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.566+0000 7faf37fff700 1 -- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf380fbcd0 msgr2=0x7faf3810daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.566+0000 7faf37fff700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf380fbcd0 0x7faf3810daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.566+0000 7faf37fff700 1 -- 172.21.15.94:0/665818752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf24005040 con 0x7faf380fcf80 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.566+0000 7faf37fff700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 0x7faf3810dfe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf24009150 tx=0x7faf24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.566+0000 7faf357fa700 1 -- 172.21.15.94:0/665818752 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf2400e040 con 0x7faf380fcf80 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.566+0000 7faf357fa700 1 -- 172.21.15.94:0/665818752 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf24004290 con 0x7faf380fcf80 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.566+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3810f010 con 0x7faf380fcf80 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.566+0000 7faf357fa700 1 -- 172.21.15.94:0/665818752 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf2401c630 con 0x7faf380fcf80 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.566+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3810f550 con 0x7faf380fcf80 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.567+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf38101b10 con 0x7faf380fcf80 2024-04-06T03:40:22.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.568+0000 7faf357fa700 1 -- 172.21.15.94:0/665818752 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7faf2400bbf0 con 0x7faf380fcf80 2024-04-06T03:40:22.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.568+0000 7faf357fa700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7faf2806b540 0x7faf2806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:22.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.568+0000 7faf357fa700 1 -- 172.21.15.94:0/665818752 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7faf2408abb0 con 0x7faf380fcf80 2024-04-06T03:40:22.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.569+0000 7faf3c985700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7faf2806b540 0x7faf2806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:22.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.569+0000 7faf3c985700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7faf2806b540 0x7faf2806d9f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faf2c007fd0 tx=0x7faf2c009040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:22.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.573+0000 7faf357fa700 1 -- 172.21.15.94:0/665818752 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7faf24091080 con 0x7faf380fcf80 2024-04-06T03:40:22.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.834+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faf38104980 con 0x7faf2806b540 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.846+0000 7faf357fa700 1 -- 172.21.15.94:0/665818752 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7faf38104980 con 0x7faf2806b540 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (6m) 2m ago 7m 26.0M - 0.25.0 c8568f914cd2 b63c91af7786 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (7m) 2m ago 7m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (6m) 8s ago 6m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (6m) 2m ago 7m 86.1M - 9.4.7 954c08fa6188 78ad0d4cf52b 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (2m) 2m ago 2m 11.8M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (2m) 2m ago 2m 14.7M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (2m) 8s ago 2m 13.0M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (2m) 8s ago 2m 15.4M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:40:22.849 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:9283 running (9m) 2m ago 9m 480M - 17.2.7-819-g6699495c ec300eaa768b d83dfb730a39 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283 running (37s) 8s ago 6m 488M - 19.0.0-2027-g9f192350 a5511c7deba9 7f3bea4f4172 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (9m) 2m ago 9m 45.4M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (6m) 8s ago 6m 38.5M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (7m) 2m ago 7m 16.1M - 1.5.0 0da6a335fe13 f8f823e89f23 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (6m) 8s ago 6m 17.2M - 1.5.0 0da6a335fe13 e5324db1ea82 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (6m) 2m ago 6m 45.0M 4288M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (5m) 2m ago 5m 46.0M 4288M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (5m) 2m ago 5m 44.2M 4288M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (4m) 8s ago 4m 111M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (4m) 8s ago 4m 106M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (3m) 8s ago 3m 104M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:40:22.850 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (6m) 2m ago 7m 55.6M - 2.43.0 a07b618ecd1d 82a078c9e43c 2024-04-06T03:40:22.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7faf2806b540 msgr2=0x7faf2806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7faf2806b540 0x7faf2806d9f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faf2c007fd0 tx=0x7faf2c009040 comp rx=0 tx=0).stop 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 msgr2=0x7faf3810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 0x7faf3810dfe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf24009150 tx=0x7faf24009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 shutdown_connections 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7faf2806b540 0x7faf2806d9f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf380fbcd0 0x7faf3810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 --2- 172.21.15.94:0/665818752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf380fcf80 0x7faf3810dfe0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 >> 172.21.15.94:0/665818752 conn(0x7faf380f7430 msgr2=0x7faf38065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 shutdown_connections 2024-04-06T03:40:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.849+0000 7faf3ebe9700 1 -- 172.21.15.94:0/665818752 wait complete. 2024-04-06T03:40:22.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.957+0000 7f0bd1b56700 1 -- 172.21.15.94:0/1448105310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fed50 msgr2=0x7f0bcc0fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.957+0000 7f0bd1b56700 1 --2- 172.21.15.94:0/1448105310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fed50 0x7f0bcc0fae90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0004940 tx=0x7f0bc0009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.957+0000 7f0bd1b56700 1 -- 172.21.15.94:0/1448105310 shutdown_connections 2024-04-06T03:40:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.957+0000 7f0bd1b56700 1 --2- 172.21.15.94:0/1448105310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fed50 0x7f0bcc0fae90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.957+0000 7f0bd1b56700 1 --2- 172.21.15.94:0/1448105310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc0fe400 0x7f0bcc0fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.957+0000 7f0bd1b56700 1 -- 172.21.15.94:0/1448105310 >> 172.21.15.94:0/1448105310 conn(0x7f0bcc0f6ba0 msgr2=0x7f0bcc0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.957+0000 7f0bd1b56700 1 -- 172.21.15.94:0/1448105310 shutdown_connections 2024-04-06T03:40:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.957+0000 7f0bd1b56700 1 -- 172.21.15.94:0/1448105310 wait complete. 2024-04-06T03:40:22.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.958+0000 7f0bd1b56700 1 Processor -- start 2024-04-06T03:40:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.958+0000 7f0bd1b56700 1 -- start start 2024-04-06T03:40:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.958+0000 7f0bd1b56700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fe400 0x7f0bcc0fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.958+0000 7f0bd1b56700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc0fed50 0x7f0bcc0fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.958+0000 7f0bd1b56700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0fc670 con 0x7f0bcc0fe400 2024-04-06T03:40:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.958+0000 7f0bd1b56700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0fc7e0 con 0x7f0bcc0fed50 2024-04-06T03:40:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.958+0000 7f0bcb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fe400 0x7f0bcc0fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:22.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.958+0000 7f0bcb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fe400 0x7f0bcc0fdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40104/0 (socket says 172.21.15.94:40104) 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.958+0000 7f0bcb7fe700 1 -- 172.21.15.94:0/146184605 learned_addr learned my addr 172.21.15.94:0/146184605 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bcaffd700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc0fed50 0x7f0bcc0fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bcb7fe700 1 -- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc0fed50 msgr2=0x7f0bcc0fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bcb7fe700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc0fed50 0x7f0bcc0fc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bcb7fe700 1 -- 172.21.15.94:0/146184605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc0005040 con 0x7f0bcc0fe400 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bcaffd700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc0fed50 0x7f0bcc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bcb7fe700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fe400 0x7f0bcc0fdbf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc002a50 tx=0x7f0bbc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bb3fff700 1 -- 172.21.15.94:0/146184605 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bbc007cc0 con 0x7f0bcc0fe400 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bb3fff700 1 -- 172.21.15.94:0/146184605 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bbc00f040 con 0x7f0bcc0fe400 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc0fcac0 con 0x7f0bcc0fe400 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bb3fff700 1 -- 172.21.15.94:0/146184605 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bbc009640 con 0x7f0bcc0fe400 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.959+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc1159f0 con 0x7f0bcc0fe400 2024-04-06T03:40:22.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.960+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc05f5f0 con 0x7f0bcc0fe400 2024-04-06T03:40:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.961+0000 7f0bb3fff700 1 -- 172.21.15.94:0/146184605 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f0bbc0097a0 con 0x7f0bcc0fe400 2024-04-06T03:40:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.961+0000 7f0bb3fff700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0bb406b540 0x7f0bb406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.962+0000 7f0bb3fff700 1 -- 172.21.15.94:0/146184605 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f0bbc089e10 con 0x7f0bcc0fe400 2024-04-06T03:40:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.962+0000 7f0bcaffd700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0bb406b540 0x7f0bb406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:22.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.962+0000 7f0bcaffd700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0bb406b540 0x7f0bb406d9f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000b3b0 tx=0x7f0bc0009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:22.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:22.966+0000 7f0bb3fff700 1 -- 172.21.15.94:0/146184605 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f0bbc053c00 con 0x7f0bcc0fe400 2024-04-06T03:40:23.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:22 smithi136 ceph-mon[36436]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:23.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:22 smithi136 ceph-mon[36436]: pgmap v11: 65 pgs: 65 active+clean; 276 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 16 MiB/s wr, 675 op/s 2024-04-06T03:40:23.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:22 smithi094 ceph-mon[29713]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:23.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:22 smithi094 ceph-mon[29713]: pgmap v11: 65 pgs: 65 active+clean; 276 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 16 MiB/s wr, 675 op/s 2024-04-06T03:40:23.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.307+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0bcc1058b0 con 0x7f0bcc0fe400 2024-04-06T03:40:23.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.308+0000 7f0bb3fff700 1 -- 172.21.15.94:0/146184605 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f0bbc0572c0 con 0x7f0bcc0fe400 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 1, 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 1 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:40:23.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:40:23.312 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:40:23.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:40:23.312 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:40:23.312 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 13, 2024-04-06T03:40:23.312 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 1 2024-04-06T03:40:23.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:40:23.312 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:40:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0bb406b540 msgr2=0x7f0bb406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0bb406b540 0x7f0bb406d9f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000b3b0 tx=0x7f0bc0009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fe400 msgr2=0x7f0bcc0fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fe400 0x7f0bcc0fdbf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc002a50 tx=0x7f0bbc0046b0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 shutdown_connections 2024-04-06T03:40:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0bcc0fe400 0x7f0bcc0fdbf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0bb406b540 0x7f0bb406d9f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 --2- 172.21.15.94:0/146184605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc0fed50 0x7f0bcc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 >> 172.21.15.94:0/146184605 conn(0x7f0bcc0f6ba0 msgr2=0x7f0bcc0f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 shutdown_connections 2024-04-06T03:40:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.311+0000 7f0bd1b56700 1 -- 172.21.15.94:0/146184605 wait complete. 2024-04-06T03:40:23.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.422+0000 7fea47ebc700 1 -- 172.21.15.94:0/2182557419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea400fff60 msgr2=0x7fea40102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:23.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.422+0000 7fea47ebc700 1 --2- 172.21.15.94:0/2182557419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea400fff60 0x7fea40102340 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fea30004940 tx=0x7fea30009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:23.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.422+0000 7fea47ebc700 1 -- 172.21.15.94:0/2182557419 shutdown_connections 2024-04-06T03:40:23.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.422+0000 7fea47ebc700 1 --2- 172.21.15.94:0/2182557419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea400fff60 0x7fea40102340 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.422+0000 7fea47ebc700 1 --2- 172.21.15.94:0/2182557419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea40066280 0x7fea400ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.422+0000 7fea47ebc700 1 -- 172.21.15.94:0/2182557419 >> 172.21.15.94:0/2182557419 conn(0x7fea400f7430 msgr2=0x7fea400f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:23.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.422+0000 7fea47ebc700 1 -- 172.21.15.94:0/2182557419 shutdown_connections 2024-04-06T03:40:23.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.422+0000 7fea47ebc700 1 -- 172.21.15.94:0/2182557419 wait complete. 2024-04-06T03:40:23.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea47ebc700 1 Processor -- start 2024-04-06T03:40:23.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea47ebc700 1 -- start start 2024-04-06T03:40:23.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea47ebc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea40066280 0x7fea400fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:23.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea47ebc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea400fff60 0x7fea400fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:23.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea47ebc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea400fbed0 con 0x7fea40066280 2024-04-06T03:40:23.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea47ebc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea400fc040 con 0x7fea400fff60 2024-04-06T03:40:23.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea45c58700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea40066280 0x7fea400fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:23.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea45c58700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea40066280 0x7fea400fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40108/0 (socket says 172.21.15.94:40108) 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea45c58700 1 -- 172.21.15.94:0/1129231022 learned_addr learned my addr 172.21.15.94:0/1129231022 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea45457700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea400fff60 0x7fea400fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea45c58700 1 -- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea400fff60 msgr2=0x7fea400fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea45c58700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea400fff60 0x7fea400fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.423+0000 7fea45c58700 1 -- 172.21.15.94:0/1129231022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea30005040 con 0x7fea40066280 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.424+0000 7fea45c58700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea40066280 0x7fea400fd450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fea3800ca90 tx=0x7fea38007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.424+0000 7fea367fc700 1 -- 172.21.15.94:0/1129231022 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea3800aa50 con 0x7fea40066280 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.424+0000 7fea367fc700 1 -- 172.21.15.94:0/1129231022 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea3800abb0 con 0x7fea40066280 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.424+0000 7fea367fc700 1 -- 172.21.15.94:0/1129231022 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea380115e0 con 0x7fea40066280 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.424+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea400fc320 con 0x7fea40066280 2024-04-06T03:40:23.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.424+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea40115a60 con 0x7fea40066280 2024-04-06T03:40:23.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.425+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4005f5f0 con 0x7fea40066280 2024-04-06T03:40:23.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.426+0000 7fea367fc700 1 -- 172.21.15.94:0/1129231022 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7fea3800ad20 con 0x7fea40066280 2024-04-06T03:40:23.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.426+0000 7fea367fc700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fea2c06b610 0x7fea2c06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:23.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.426+0000 7fea367fc700 1 -- 172.21.15.94:0/1129231022 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fea3808a720 con 0x7fea40066280 2024-04-06T03:40:23.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.427+0000 7fea45457700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fea2c06b610 0x7fea2c06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:23.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.427+0000 7fea45457700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fea2c06b610 0x7fea2c06dac0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea3000b3b0 tx=0x7fea30009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:23.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.434+0000 7fea367fc700 1 -- 172.21.15.94:0/1129231022 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fea38054580 con 0x7fea40066280 2024-04-06T03:40:23.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.746+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fea401058b0 con 0x7fea40066280 2024-04-06T03:40:23.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.747+0000 7fea367fc700 1 -- 172.21.15.94:0/1129231022 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7fea38057c40 con 0x7fea40066280 2024-04-06T03:40:23.750 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:40:23.751 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:40:23.751 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:40:23.751 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:40:23.751 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:40:23.751 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:40:23.751 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:40:23.751 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:40:23.751 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:40:23.752 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:40:23.752 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:40:23.752 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:40:23.752 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:40:23.752 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:40:23.752 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:40:23.752 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:40:23.752 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:40:23.753 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:40:23.753 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:40:23.753 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:40:23.753 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:40:23.753 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:40:23.753 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:40:23.753 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:40:23.753 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:40:23.753 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:40:23.754 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:40:23.754 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:40:23.754 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:40:23.754 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:40:23.754 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:40:23.754 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:40:23.754 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:40:23.754 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:40:23.755 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:40:23.755 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:40:23.755 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:40:23.755 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:40:23.755 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:40:23.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fea2c06b610 msgr2=0x7fea2c06dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:23.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fea2c06b610 0x7fea2c06dac0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea3000b3b0 tx=0x7fea30009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea40066280 msgr2=0x7fea400fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea40066280 0x7fea400fd450 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fea3800ca90 tx=0x7fea38007f00 comp rx=0 tx=0).stop 2024-04-06T03:40:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 shutdown_connections 2024-04-06T03:40:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea40066280 0x7fea400fd450 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fea2c06b610 0x7fea2c06dac0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 --2- 172.21.15.94:0/1129231022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea400fff60 0x7fea400fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 >> 172.21.15.94:0/1129231022 conn(0x7fea400f7430 msgr2=0x7fea400f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:23.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 shutdown_connections 2024-04-06T03:40:23.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.749+0000 7fea47ebc700 1 -- 172.21.15.94:0/1129231022 wait complete. 2024-04-06T03:40:23.758 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:40:23.768 INFO:tasks.workunit.client.1.smithi136.stderr:++ readlink -f fsstress 2024-04-06T03:40:23.768 INFO:tasks.workunit.client.1.smithi136.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-06T03:40:23.769 INFO:tasks.workunit.client.1.smithi136.stderr:+ popd 2024-04-06T03:40:23.769 INFO:tasks.workunit.client.1.smithi136.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-06T03:40:23.769 INFO:tasks.workunit.client.1.smithi136.stderr:+ popd 2024-04-06T03:40:23.769 INFO:tasks.workunit.client.1.smithi136.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-06T03:40:23.770 INFO:tasks.workunit.client.1.smithi136.stderr:++ mktemp -d -p . 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:2/45: sync 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:0/13: sync 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/7: sync 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/8: dwrite - no filename 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/9: write - no filename 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:1/29: sync 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/10: chown c0 452988 1 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/11: dwrite - no filename 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/12: chown c0 36 1 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:9/18: sync 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:4/26: sync 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:1/30: creat d0/d1/fa x:0 0 0 2024-04-06T03:40:23.792 INFO:tasks.workunit.client.0.smithi094.stdout:0/14: creat d1/f7 x:0 0 0 2024-04-06T03:40:23.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/19: readlink d2/l3 0 2024-04-06T03:40:23.793 INFO:tasks.workunit.client.0.smithi094.stdout:0/15: truncate d1/f7 33706 0 2024-04-06T03:40:23.793 INFO:tasks.workunit.client.0.smithi094.stdout:5/16: sync 2024-04-06T03:40:23.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/20: creat d2/f5 x:0 0 0 2024-04-06T03:40:23.793 INFO:tasks.workunit.client.0.smithi094.stdout:5/17: creat f6 x:0 0 0 2024-04-06T03:40:23.793 INFO:tasks.workunit.client.0.smithi094.stdout:8/46: sync 2024-04-06T03:40:23.793 INFO:tasks.workunit.client.0.smithi094.stdout:6/12: sync 2024-04-06T03:40:23.793 INFO:tasks.workunit.client.0.smithi094.stdout:3/30: sync 2024-04-06T03:40:23.793 INFO:tasks.workunit.client.0.smithi094.stdout:2/46: creat d0/dd/ff x:0 0 0 2024-04-06T03:40:23.796 INFO:tasks.workunit.client.0.smithi094.stdout:7/13: symlink l1 0 2024-04-06T03:40:23.798 INFO:tasks.workunit.client.0.smithi094.stdout:1/31: link d0/d1/l8 d0/d1/lb 0 2024-04-06T03:40:23.802 INFO:tasks.workunit.client.0.smithi094.stdout:5/18: mknod c7 0 2024-04-06T03:40:23.803 INFO:tasks.workunit.client.0.smithi094.stdout:5/19: fdatasync f2 0 2024-04-06T03:40:23.804 INFO:tasks.workunit.client.0.smithi094.stdout:8/47: symlink l15 0 2024-04-06T03:40:23.805 INFO:tasks.workunit.client.0.smithi094.stdout:2/47: truncate d0/f1 727935 0 2024-04-06T03:40:23.806 INFO:tasks.workunit.client.0.smithi094.stdout:7/14: mkdir d2 0 2024-04-06T03:40:23.806 INFO:tasks.workunit.client.0.smithi094.stdout:7/15: truncate - no filename 2024-04-06T03:40:23.809 INFO:tasks.workunit.client.0.smithi094.stdout:1/32: rename d0/d1/fa to d0/d4/fc 0 2024-04-06T03:40:23.809 INFO:tasks.workunit.client.0.smithi094.stdout:0/16: dread f0 [0,4194304] 0 2024-04-06T03:40:23.809 INFO:tasks.workunit.client.0.smithi094.stdout:3/31: dwrite f4 [0,4194304] 0 2024-04-06T03:40:23.809 INFO:tasks.workunit.client.0.smithi094.stdout:9/21: dread d2/f4 [0,4194304] 0 2024-04-06T03:40:23.809 INFO:tasks.workunit.client.0.smithi094.stdout:3/32: creat f5 x:0 0 0 2024-04-06T03:40:23.810 INFO:tasks.workunit.client.0.smithi094.stdout:5/20: mknod c8 0 2024-04-06T03:40:23.810 INFO:tasks.workunit.client.0.smithi094.stdout:8/48: truncate f1 615066 0 2024-04-06T03:40:23.812 INFO:tasks.workunit.client.0.smithi094.stdout:2/48: mknod d0/dd/c10 0 2024-04-06T03:40:23.812 INFO:tasks.workunit.client.0.smithi094.stdout:2/49: creat d0/d8/f11 x:0 0 0 2024-04-06T03:40:23.813 INFO:tasks.workunit.client.0.smithi094.stdout:2/50: fdatasync d0/d8/fe 0 2024-04-06T03:40:23.813 INFO:tasks.workunit.client.0.smithi094.stdout:2/51: write d0/d8/fe [620413,38157] 0 2024-04-06T03:40:23.814 INFO:tasks.workunit.client.0.smithi094.stdout:7/16: rename c0 to d2/c3 0 2024-04-06T03:40:23.815 INFO:tasks.workunit.client.0.smithi094.stdout:1/33: mknod d0/d1/cd 0 2024-04-06T03:40:23.816 INFO:tasks.workunit.client.0.smithi094.stdout:0/17: truncate f0 8537232 0 2024-04-06T03:40:23.822 INFO:tasks.workunit.client.0.smithi094.stdout:3/33: rename f4 to f6 0 2024-04-06T03:40:23.827 INFO:tasks.workunit.client.0.smithi094.stdout:7/17: creat d2/f4 x:0 0 0 2024-04-06T03:40:23.828 INFO:tasks.workunit.client.0.smithi094.stdout:0/18: mkdir d1/d8 0 2024-04-06T03:40:23.828 INFO:tasks.workunit.client.0.smithi094.stdout:0/19: fdatasync d1/f5 0 2024-04-06T03:40:23.830 INFO:tasks.workunit.client.0.smithi094.stdout:1/34: dwrite d0/d4/fc [0,4194304] 0 2024-04-06T03:40:23.830 INFO:tasks.workunit.client.0.smithi094.stdout:1/35: write d0/d4/f9 [311288,9861] 0 2024-04-06T03:40:23.831 INFO:tasks.workunit.client.0.smithi094.stdout:9/22: dwrite d2/f5 [0,4194304] 0 2024-04-06T03:40:23.836 INFO:tasks.workunit.client.0.smithi094.stdout:9/23: link d2/l3 d2/l6 0 2024-04-06T03:40:23.836 INFO:tasks.workunit.client.0.smithi094.stdout:9/24: creat d2/f7 x:0 0 0 2024-04-06T03:40:23.839 INFO:tasks.workunit.client.0.smithi094.stdout:1/36: dread d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:23.845 INFO:tasks.workunit.client.0.smithi094.stdout:1/37: write d0/d4/fc [1564287,112965] 0 2024-04-06T03:40:23.845 INFO:tasks.workunit.client.0.smithi094.stdout:1/38: truncate d0/d4/f9 935488 0 2024-04-06T03:40:23.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/18: dwrite d2/f4 [0,4194304] 0 2024-04-06T03:40:23.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/19: creat d2/f5 x:0 0 0 2024-04-06T03:40:23.860 INFO:tasks.workunit.client.0.smithi094.stdout:1/39: dwrite d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:23.860 INFO:tasks.workunit.client.0.smithi094.stdout:1/40: write d0/d4/fc [3800405,19172] 0 2024-04-06T03:40:23.865 INFO:tasks.workunit.client.0.smithi094.stdout:1/41: dread d0/d4/fc [0,4194304] 0 2024-04-06T03:40:23.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/42: symlink d0/le 0 2024-04-06T03:40:23.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/43: fsync d0/d4/f9 0 2024-04-06T03:40:23.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/44: chown d0/le 1079730363 1 2024-04-06T03:40:23.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/45: chown d0 19876 1 2024-04-06T03:40:23.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.886+0000 7fdffab34700 1 -- 172.21.15.94:0/711582339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff4066280 msgr2=0x7fdff40ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:23.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.886+0000 7fdffab34700 1 --2- 172.21.15.94:0/711582339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff4066280 0x7fdff40ffa20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8004940 tx=0x7fdfe8009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:23.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.886+0000 7fdffab34700 1 -- 172.21.15.94:0/711582339 shutdown_connections 2024-04-06T03:40:23.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.886+0000 7fdffab34700 1 --2- 172.21.15.94:0/711582339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdff40fff60 0x7fdff4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.886+0000 7fdffab34700 1 --2- 172.21.15.94:0/711582339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff4066280 0x7fdff40ffa20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.886+0000 7fdffab34700 1 -- 172.21.15.94:0/711582339 >> 172.21.15.94:0/711582339 conn(0x7fdff40f7430 msgr2=0x7fdff40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:23.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.886+0000 7fdffab34700 1 -- 172.21.15.94:0/711582339 shutdown_connections 2024-04-06T03:40:23.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.886+0000 7fdffab34700 1 -- 172.21.15.94:0/711582339 wait complete. 2024-04-06T03:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdffab34700 1 Processor -- start 2024-04-06T03:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdffab34700 1 -- start start 2024-04-06T03:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdffab34700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdff4066280 0x7fdff410da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdffab34700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fff60 0x7fdff410dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdffab34700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff410e5c0 con 0x7fdff40fff60 2024-04-06T03:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdffab34700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff410edc0 con 0x7fdff4066280 2024-04-06T03:40:23.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdff3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fff60 0x7fdff410dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdff3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fff60 0x7fdff410dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40130/0 (socket says 172.21.15.94:40130) 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdff3fff700 1 -- 172.21.15.94:0/1563244528 learned_addr learned my addr 172.21.15.94:0/1563244528 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.887+0000 7fdff88d0700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdff4066280 0x7fdff410da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.888+0000 7fdff88d0700 1 -- 172.21.15.94:0/1563244528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fff60 msgr2=0x7fdff410dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.888+0000 7fdff88d0700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fff60 0x7fdff410dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.888+0000 7fdff88d0700 1 -- 172.21.15.94:0/1563244528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8005040 con 0x7fdff4066280 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.888+0000 7fdff88d0700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdff4066280 0x7fdff410da80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8000c00 tx=0x7fdfe8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.888+0000 7fdff17fa700 1 -- 172.21.15.94:0/1563244528 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfe800e030 con 0x7fdff4066280 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.888+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff410f040 con 0x7fdff4066280 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.888+0000 7fdff17fa700 1 -- 172.21.15.94:0/1563244528 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfe8013de0 con 0x7fdff4066280 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.888+0000 7fdff17fa700 1 -- 172.21.15.94:0/1563244528 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfe801cdc0 con 0x7fdff4066280 2024-04-06T03:40:23.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.888+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff410f580 con 0x7fdff4066280 2024-04-06T03:40:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.889+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff405f5f0 con 0x7fdff4066280 2024-04-06T03:40:23.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.891+0000 7fdff17fa700 1 -- 172.21.15.94:0/1563244528 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7fdfe801c650 con 0x7fdff4066280 2024-04-06T03:40:23.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.891+0000 7fdff17fa700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fdfe406b540 0x7fdfe406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:23.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.892+0000 7fdff17fa700 1 -- 172.21.15.94:0/1563244528 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fdfe808bcb0 con 0x7fdff4066280 2024-04-06T03:40:23.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.892+0000 7fdff3fff700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fdfe406b540 0x7fdfe406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:23.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.892+0000 7fdff3fff700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fdfe406b540 0x7fdfe406d9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0008fd0 tx=0x7fdfe000b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:23.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:23.895+0000 7fdff17fa700 1 -- 172.21.15.94:0/1563244528 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fdfe80596d0 con 0x7fdff4066280 2024-04-06T03:40:24.049 INFO:tasks.workunit.client.0.smithi094.stdout:1/46: rmdir d0/d1 39 2024-04-06T03:40:24.050 INFO:tasks.workunit.client.0.smithi094.stdout:8/49: write f14 [282462,112136] 0 2024-04-06T03:40:24.051 INFO:tasks.workunit.client.0.smithi094.stdout:2/52: dread d0/f5 [0,4194304] 0 2024-04-06T03:40:24.051 INFO:tasks.workunit.client.0.smithi094.stdout:8/50: truncate fd 1818227 0 2024-04-06T03:40:24.053 INFO:tasks.workunit.client.0.smithi094.stdout:3/34: rename f6 to f7 0 2024-04-06T03:40:24.053 INFO:tasks.workunit.client.0.smithi094.stdout:3/35: write f5 [595652,11525] 0 2024-04-06T03:40:24.053 INFO:tasks.workunit.client.0.smithi094.stdout:3/36: creat f8 x:0 0 0 2024-04-06T03:40:24.053 INFO:tasks.workunit.client.0.smithi094.stdout:3/37: fdatasync f8 0 2024-04-06T03:40:24.053 INFO:tasks.workunit.client.0.smithi094.stdout:3/38: chown f5 50 1 2024-04-06T03:40:24.053 INFO:tasks.workunit.client.0.smithi094.stdout:3/39: read - f8 zero size 2024-04-06T03:40:24.054 INFO:tasks.workunit.client.0.smithi094.stdout:0/20: dread f0 [0,4194304] 0 2024-04-06T03:40:24.055 INFO:tasks.workunit.client.0.smithi094.stdout:2/53: symlink d0/dd/l12 0 2024-04-06T03:40:24.055 INFO:tasks.workunit.client.0.smithi094.stdout:2/54: truncate d0/dd/ff 95368 0 2024-04-06T03:40:24.056 INFO:tasks.workunit.client.0.smithi094.stdout:8/51: link c7 c16 0 2024-04-06T03:40:24.056 INFO:tasks.workunit.client.0.smithi094.stdout:8/52: chown l15 244775 1 2024-04-06T03:40:24.059 INFO:tasks.workunit.client.0.smithi094.stdout:1/47: write d0/d4/f9 [2089004,14565] 0 2024-04-06T03:40:24.059 INFO:tasks.workunit.client.0.smithi094.stdout:3/40: dread f5 [0,4194304] 0 2024-04-06T03:40:24.059 INFO:tasks.workunit.client.0.smithi094.stdout:3/41: chown f8 5 1 2024-04-06T03:40:24.060 INFO:tasks.workunit.client.0.smithi094.stdout:3/42: dread - f8 zero size 2024-04-06T03:40:24.060 INFO:tasks.workunit.client.0.smithi094.stdout:3/43: write f8 [626530,99127] 0 2024-04-06T03:40:24.060 INFO:tasks.workunit.client.0.smithi094.stdout:3/44: rmdir - no directory 2024-04-06T03:40:24.061 INFO:tasks.workunit.client.0.smithi094.stdout:0/21: rmdir d1/d8 0 2024-04-06T03:40:24.061 INFO:tasks.workunit.client.0.smithi094.stdout:0/22: dread - d1/f5 zero size 2024-04-06T03:40:24.063 INFO:tasks.workunit.client.0.smithi094.stdout:1/48: mknod d0/d4/cf 0 2024-04-06T03:40:24.065 INFO:tasks.workunit.client.0.smithi094.stdout:0/23: mkdir d1/d9 0 2024-04-06T03:40:24.065 INFO:tasks.workunit.client.0.smithi094.stdout:0/24: fsync d1/f7 0 2024-04-06T03:40:24.065 INFO:tasks.workunit.client.0.smithi094.stdout:0/25: readlink d1/l6 0 2024-04-06T03:40:24.065 INFO:tasks.workunit.client.0.smithi094.stdout:0/26: chown f0 924292628 1 2024-04-06T03:40:24.065 INFO:tasks.workunit.client.0.smithi094.stdout:1/49: creat d0/d1/f10 x:0 0 0 2024-04-06T03:40:24.067 INFO:tasks.workunit.client.0.smithi094.stdout:9/25: rmdir d2 39 2024-04-06T03:40:24.067 INFO:tasks.workunit.client.0.smithi094.stdout:5/21: write f1 [1597138,115396] 0 2024-04-06T03:40:24.067 INFO:tasks.workunit.client.0.smithi094.stdout:5/22: fsync f2 0 2024-04-06T03:40:24.069 INFO:tasks.workunit.client.0.smithi094.stdout:8/53: dread f14 [0,4194304] 0 2024-04-06T03:40:24.069 INFO:tasks.workunit.client.0.smithi094.stdout:8/54: creat f17 x:0 0 0 2024-04-06T03:40:24.069 INFO:tasks.workunit.client.0.smithi094.stdout:0/27: rename f0 to d1/fa 0 2024-04-06T03:40:24.070 INFO:tasks.workunit.client.0.smithi094.stdout:1/50: symlink d0/d1/l11 0 2024-04-06T03:40:24.070 INFO:tasks.workunit.client.0.smithi094.stdout:1/51: rename d0 to d0/d12 22 2024-04-06T03:40:24.070 INFO:tasks.workunit.client.0.smithi094.stdout:7/20: fsync d2/f5 0 2024-04-06T03:40:24.071 INFO:tasks.workunit.client.0.smithi094.stdout:9/26: symlink d2/l8 0 2024-04-06T03:40:24.071 INFO:tasks.workunit.client.0.smithi094.stdout:9/27: chown d2/l8 137164 1 2024-04-06T03:40:24.071 INFO:tasks.workunit.client.0.smithi094.stdout:2/55: dwrite d0/d8/f11 [0,4194304] 0 2024-04-06T03:40:24.071 INFO:tasks.workunit.client.0.smithi094.stdout:2/56: readlink d0/lb 0 2024-04-06T03:40:24.072 INFO:tasks.workunit.client.0.smithi094.stdout:2/57: fsync d0/dd/ff 0 2024-04-06T03:40:24.078 INFO:tasks.workunit.client.0.smithi094.stdout:0/28: mknod d1/cb 0 2024-04-06T03:40:24.080 INFO:tasks.workunit.client.0.smithi094.stdout:1/52: mknod d0/d7/c13 0 2024-04-06T03:40:24.081 INFO:tasks.workunit.client.0.smithi094.stdout:9/28: dread d2/f5 [0,4194304] 0 2024-04-06T03:40:24.082 INFO:tasks.workunit.client.0.smithi094.stdout:7/21: mkdir d2/d6 0 2024-04-06T03:40:24.086 INFO:tasks.workunit.client.0.smithi094.stdout:2/58: link d0/lb d0/d8/l13 0 2024-04-06T03:40:24.086 INFO:tasks.workunit.client.0.smithi094.stdout:5/23: dwrite f2 [4194304,4194304] 0 2024-04-06T03:40:24.087 INFO:tasks.workunit.client.0.smithi094.stdout:2/59: read d0/dd/ff [81602,99993] 0 2024-04-06T03:40:24.087 INFO:tasks.workunit.client.0.smithi094.stdout:2/60: truncate d0/d8/fe 1306506 0 2024-04-06T03:40:24.087 INFO:tasks.workunit.client.0.smithi094.stdout:2/61: write d0/dd/ff [521220,27657] 0 2024-04-06T03:40:24.087 INFO:tasks.workunit.client.0.smithi094.stdout:2/62: chown d0/l6 3801 1 2024-04-06T03:40:24.088 INFO:tasks.workunit.client.0.smithi094.stdout:3/45: dwrite f8 [0,4194304] 0 2024-04-06T03:40:24.089 INFO:tasks.workunit.client.0.smithi094.stdout:0/29: creat d1/d9/fc x:0 0 0 2024-04-06T03:40:24.094 INFO:tasks.workunit.client.0.smithi094.stdout:2/63: write d0/f5 [183791,37055] 0 2024-04-06T03:40:24.094 INFO:tasks.workunit.client.0.smithi094.stdout:5/24: dread f2 [0,4194304] 0 2024-04-06T03:40:24.098 INFO:tasks.workunit.client.0.smithi094.stdout:3/46: mkdir d9 0 2024-04-06T03:40:24.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:23 smithi136 ceph-mon[36436]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:24.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:23 smithi136 ceph-mon[36436]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:24.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:23 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/146184605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:40:24.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:23 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1129231022' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:40:24.103 INFO:tasks.workunit.client.0.smithi094.stdout:7/22: dwrite d2/f4 [0,4194304] 0 2024-04-06T03:40:24.106 INFO:tasks.workunit.client.0.smithi094.stdout:5/25: rename f2 to f9 0 2024-04-06T03:40:24.112 INFO:tasks.workunit.client.0.smithi094.stdout:5/26: dread f1 [0,4194304] 0 2024-04-06T03:40:24.112 INFO:tasks.workunit.client.0.smithi094.stdout:5/27: dread - f6 zero size 2024-04-06T03:40:24.115 INFO:tasks.workunit.client.0.smithi094.stdout:7/23: symlink d2/l7 0 2024-04-06T03:40:24.116 INFO:tasks.workunit.client.0.smithi094.stdout:5/28: dread f1 [0,4194304] 0 2024-04-06T03:40:24.117 INFO:tasks.workunit.client.0.smithi094.stdout:5/29: creat fa x:0 0 0 2024-04-06T03:40:24.117 INFO:tasks.workunit.client.0.smithi094.stdout:5/30: dread - fa zero size 2024-04-06T03:40:24.119 INFO:tasks.workunit.client.0.smithi094.stdout:5/31: mkdir db 0 2024-04-06T03:40:24.119 INFO:tasks.workunit.client.0.smithi094.stdout:5/32: getdents db 0 2024-04-06T03:40:24.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/33: chown l3 1017 1 2024-04-06T03:40:24.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/34: fdatasync f9 0 2024-04-06T03:40:24.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/35: readlink l3 0 2024-04-06T03:40:24.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/36: stat f9 0 2024-04-06T03:40:24.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/37: dread - fa zero size 2024-04-06T03:40:24.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/38: creat db/fc x:0 0 0 2024-04-06T03:40:24.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/39: chown l5 708297164 1 2024-04-06T03:40:24.120 INFO:tasks.workunit.client.0.smithi094.stdout:5/40: chown c7 112705396 1 2024-04-06T03:40:24.122 INFO:tasks.workunit.client.0.smithi094.stdout:7/24: dread d2/f4 [0,4194304] 0 2024-04-06T03:40:24.122 INFO:tasks.workunit.client.0.smithi094.stdout:7/25: fdatasync d2/f5 0 2024-04-06T03:40:24.122 INFO:tasks.workunit.client.0.smithi094.stdout:5/41: rename fa to db/fd 0 2024-04-06T03:40:24.123 INFO:tasks.workunit.client.0.smithi094.stdout:5/42: truncate db/fd 809027 0 2024-04-06T03:40:24.124 INFO:tasks.workunit.client.0.smithi094.stdout:7/26: creat d2/d6/f8 x:0 0 0 2024-04-06T03:40:24.126 INFO:tasks.workunit.client.0.smithi094.stdout:7/27: rename d2/f5 to d2/d6/f9 0 2024-04-06T03:40:24.126 INFO:tasks.workunit.client.0.smithi094.stdout:7/28: chown d2/f4 397 1 2024-04-06T03:40:24.127 INFO:tasks.workunit.client.0.smithi094.stdout:5/43: dread f1 [0,4194304] 0 2024-04-06T03:40:24.128 INFO:tasks.workunit.client.0.smithi094.stdout:5/44: mknod db/ce 0 2024-04-06T03:40:24.128 INFO:tasks.workunit.client.0.smithi094.stdout:5/45: creat db/ff x:0 0 0 2024-04-06T03:40:24.128 INFO:tasks.workunit.client.0.smithi094.stdout:5/46: chown l5 184774579 1 2024-04-06T03:40:24.129 INFO:tasks.workunit.client.0.smithi094.stdout:5/47: mknod db/c10 0 2024-04-06T03:40:24.131 INFO:tasks.workunit.client.0.smithi094.stdout:4/27: rename d0/f3 to d0/f8 0 2024-04-06T03:40:24.133 INFO:tasks.workunit.client.1.smithi136.stderr:+ T=./tmp.FkHTAOF2Rv 2024-04-06T03:40:24.133 INFO:tasks.workunit.client.1.smithi136.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.FkHTAOF2Rv -l 1 -n 1000 -p 10 -v 2024-04-06T03:40:24.135 INFO:tasks.workunit.client.0.smithi094.stdout:4/28: dread d0/d4/f6 [0,4194304] 0 2024-04-06T03:40:24.135 INFO:tasks.workunit.client.1.smithi136.stdout:seed = 1712508778 2024-04-06T03:40:24.141 INFO:tasks.workunit.client.1.smithi136.stdout:0/0: fsync - no filename 2024-04-06T03:40:24.141 INFO:tasks.workunit.client.1.smithi136.stdout:0/1: chown . 1445 1 2024-04-06T03:40:24.143 INFO:tasks.workunit.client.1.smithi136.stdout:1/0: dwrite - no filename 2024-04-06T03:40:24.148 INFO:tasks.workunit.client.1.smithi136.stdout:1/1: mknod c0 0 2024-04-06T03:40:24.149 INFO:tasks.workunit.client.1.smithi136.stdout:3/0: rename - no filename 2024-04-06T03:40:24.149 INFO:tasks.workunit.client.1.smithi136.stdout:3/1: fsync - no filename 2024-04-06T03:40:24.149 INFO:tasks.workunit.client.0.smithi094.stdout:4/29: dwrite d0/d4/f6 [0,4194304] 0 2024-04-06T03:40:24.150 INFO:tasks.workunit.client.1.smithi136.stdout:2/0: mknod c0 0 2024-04-06T03:40:24.150 INFO:tasks.workunit.client.1.smithi136.stdout:2/1: fdatasync - no filename 2024-04-06T03:40:24.150 INFO:tasks.workunit.client.1.smithi136.stdout:2/2: write - no filename 2024-04-06T03:40:24.150 INFO:tasks.workunit.client.1.smithi136.stdout:2/3: chown c0 199104 1 2024-04-06T03:40:24.150 INFO:tasks.workunit.client.1.smithi136.stdout:2/4: truncate - no filename 2024-04-06T03:40:24.150 INFO:tasks.workunit.client.1.smithi136.stdout:2/5: truncate - no filename 2024-04-06T03:40:24.151 INFO:tasks.workunit.client.1.smithi136.stdout:1/2: link c0 c1 0 2024-04-06T03:40:24.151 INFO:tasks.workunit.client.1.smithi136.stdout:1/3: truncate - no filename 2024-04-06T03:40:24.151 INFO:tasks.workunit.client.1.smithi136.stdout:1/4: dwrite - no filename 2024-04-06T03:40:24.151 INFO:tasks.workunit.client.1.smithi136.stdout:1/5: truncate - no filename 2024-04-06T03:40:24.151 INFO:tasks.workunit.client.1.smithi136.stdout:1/6: write - no filename 2024-04-06T03:40:24.151 INFO:tasks.workunit.client.1.smithi136.stdout:1/7: chown c0 1032144 1 2024-04-06T03:40:24.151 INFO:tasks.workunit.client.1.smithi136.stdout:1/8: dread - no filename 2024-04-06T03:40:24.153 INFO:tasks.workunit.client.1.smithi136.stdout:3/2: mkdir d0 0 2024-04-06T03:40:24.153 INFO:tasks.workunit.client.1.smithi136.stdout:3/3: write - no filename 2024-04-06T03:40:24.153 INFO:tasks.workunit.client.1.smithi136.stdout:3/4: dwrite - no filename 2024-04-06T03:40:24.153 INFO:tasks.workunit.client.1.smithi136.stdout:3/5: fdatasync - no filename 2024-04-06T03:40:24.154 INFO:tasks.workunit.client.1.smithi136.stdout:2/6: symlink l1 0 2024-04-06T03:40:24.154 INFO:tasks.workunit.client.1.smithi136.stdout:2/7: write - no filename 2024-04-06T03:40:24.154 INFO:tasks.workunit.client.1.smithi136.stdout:2/8: fsync - no filename 2024-04-06T03:40:24.155 INFO:tasks.workunit.client.1.smithi136.stdout:1/9: rename c1 to c2 0 2024-04-06T03:40:24.157 INFO:tasks.workunit.client.0.smithi094.stdout:4/30: read d0/f2 [4591894,9674] 0 2024-04-06T03:40:24.157 INFO:tasks.workunit.client.1.smithi136.stdout:4/0: creat f0 x:0 0 0 2024-04-06T03:40:24.159 INFO:tasks.workunit.client.1.smithi136.stdout:3/6: creat d0/f1 x:0 0 0 2024-04-06T03:40:24.161 INFO:tasks.workunit.client.0.smithi094.stdout:4/31: link d0/d4/f6 d0/d4/f9 0 2024-04-06T03:40:24.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/9: link l1 l2 0 2024-04-06T03:40:24.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/10: write - no filename 2024-04-06T03:40:24.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/11: chown l1 7744 1 2024-04-06T03:40:24.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/12: write - no filename 2024-04-06T03:40:24.164 INFO:tasks.workunit.client.1.smithi136.stdout:2/13: dread - no filename 2024-04-06T03:40:24.165 INFO:tasks.workunit.client.1.smithi136.stdout:1/10: creat f3 x:0 0 0 2024-04-06T03:40:24.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:23 smithi094 ceph-mon[29713]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:24.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:23 smithi094 ceph-mon[29713]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:24.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:23 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/146184605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:40:24.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:23 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1129231022' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:40:24.169 INFO:tasks.workunit.client.1.smithi136.stdout:5/0: symlink l0 0 2024-04-06T03:40:24.169 INFO:tasks.workunit.client.0.smithi094.stdout:4/32: write d0/f2 [43230,64934] 0 2024-04-06T03:40:24.171 INFO:tasks.workunit.client.1.smithi136.stdout:2/14: mkdir d3 0 2024-04-06T03:40:24.173 INFO:tasks.workunit.client.1.smithi136.stdout:1/11: link c0 c4 0 2024-04-06T03:40:24.173 INFO:tasks.workunit.client.1.smithi136.stdout:1/12: read - f3 zero size 2024-04-06T03:40:24.175 INFO:tasks.workunit.client.1.smithi136.stdout:7/0: dread - no filename 2024-04-06T03:40:24.175 INFO:tasks.workunit.client.1.smithi136.stdout:7/1: dread - no filename 2024-04-06T03:40:24.175 INFO:tasks.workunit.client.1.smithi136.stdout:7/2: fsync - no filename 2024-04-06T03:40:24.176 INFO:tasks.workunit.client.1.smithi136.stdout:6/0: symlink l0 0 2024-04-06T03:40:24.176 INFO:tasks.workunit.client.1.smithi136.stdout:6/1: dwrite - no filename 2024-04-06T03:40:24.176 INFO:tasks.workunit.client.1.smithi136.stdout:6/2: dwrite - no filename 2024-04-06T03:40:24.176 INFO:tasks.workunit.client.1.smithi136.stdout:6/3: fsync - no filename 2024-04-06T03:40:24.176 INFO:tasks.workunit.client.1.smithi136.stdout:6/4: write - no filename 2024-04-06T03:40:24.178 INFO:tasks.workunit.client.0.smithi094.stdout:0/30: rmdir d1 39 2024-04-06T03:40:24.178 INFO:tasks.workunit.client.0.smithi094.stdout:0/31: write d1/f7 [546554,115101] 0 2024-04-06T03:40:24.178 INFO:tasks.workunit.client.0.smithi094.stdout:0/32: readlink d1/l6 0 2024-04-06T03:40:24.180 INFO:tasks.workunit.client.1.smithi136.stdout:2/15: link c0 d3/c4 0 2024-04-06T03:40:24.181 INFO:tasks.workunit.client.1.smithi136.stdout:3/7: dwrite d0/f1 [0,4194304] 0 2024-04-06T03:40:24.181 INFO:tasks.workunit.client.1.smithi136.stdout:3/8: creat d0/f2 x:0 0 0 2024-04-06T03:40:24.182 INFO:tasks.workunit.client.1.smithi136.stdout:1/13: mknod c5 0 2024-04-06T03:40:24.183 INFO:tasks.workunit.client.0.smithi094.stdout:9/29: write d2/f4 [839043,123370] 0 2024-04-06T03:40:24.184 INFO:tasks.workunit.client.1.smithi136.stdout:8/0: dread - no filename 2024-04-06T03:40:24.184 INFO:tasks.workunit.client.1.smithi136.stdout:8/1: chown . 657530 1 2024-04-06T03:40:24.184 INFO:tasks.workunit.client.1.smithi136.stdout:8/2: dwrite - no filename 2024-04-06T03:40:24.185 INFO:tasks.workunit.client.1.smithi136.stdout:7/3: mknod c0 0 2024-04-06T03:40:24.185 INFO:tasks.workunit.client.1.smithi136.stdout:7/4: dwrite - no filename 2024-04-06T03:40:24.186 INFO:tasks.workunit.client.1.smithi136.stdout:6/5: creat f1 x:0 0 0 2024-04-06T03:40:24.186 INFO:tasks.workunit.client.1.smithi136.stdout:6/6: rmdir - no directory 2024-04-06T03:40:24.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/30: unlink d2/l8 0 2024-04-06T03:40:24.192 INFO:tasks.workunit.client.0.smithi094.stdout:1/53: truncate d0/d4/f9 1675129 0 2024-04-06T03:40:24.194 INFO:tasks.workunit.client.1.smithi136.stdout:2/16: link l1 d3/l5 0 2024-04-06T03:40:24.196 INFO:tasks.workunit.client.0.smithi094.stdout:1/54: creat d0/f14 x:0 0 0 2024-04-06T03:40:24.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/0: dwrite - no filename 2024-04-06T03:40:24.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/1: rename - no filename 2024-04-06T03:40:24.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/2: unlink - no file 2024-04-06T03:40:24.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/3: dwrite - no filename 2024-04-06T03:40:24.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/4: truncate - no filename 2024-04-06T03:40:24.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/5: fsync - no filename 2024-04-06T03:40:24.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/6: write - no filename 2024-04-06T03:40:24.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/7: unlink - no file 2024-04-06T03:40:24.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/8: write - no filename 2024-04-06T03:40:24.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.195+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdff410f920 con 0x7fdfe406b540 2024-04-06T03:40:24.199 INFO:tasks.workunit.client.1.smithi136.stdout:8/3: creat f0 x:0 0 0 2024-04-06T03:40:24.199 INFO:tasks.workunit.client.0.smithi094.stdout:1/55: symlink d0/d1/l15 0 2024-04-06T03:40:24.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.197+0000 7fdff17fa700 1 -- 172.21.15.94:0/1563244528 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fdff410f920 con 0x7fdfe406b540 2024-04-06T03:40:24.200 INFO:tasks.workunit.client.0.smithi094.stdout:0/33: dwrite d1/f7 [0,4194304] 0 2024-04-06T03:40:24.200 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:40:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:40:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:40:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:40:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-04-06T03:40:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "1/21 daemons upgraded", 2024-04-06T03:40:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "", 2024-04-06T03:40:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:40:24.201 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:40:24.201 INFO:tasks.workunit.client.0.smithi094.stdout:0/34: fsync d1/d9/fc 0 2024-04-06T03:40:24.202 INFO:tasks.workunit.client.1.smithi136.stdout:2/17: mknod d3/c6 0 2024-04-06T03:40:24.202 INFO:tasks.workunit.client.0.smithi094.stdout:1/56: mkdir d0/d1/d16 0 2024-04-06T03:40:24.202 INFO:tasks.workunit.client.0.smithi094.stdout:1/57: chown d0/d1/c6 0 1 2024-04-06T03:40:24.202 INFO:tasks.workunit.client.0.smithi094.stdout:1/58: read - d0/d1/f10 zero size 2024-04-06T03:40:24.203 INFO:tasks.workunit.client.1.smithi136.stdout:9/9: creat f0 x:0 0 0 2024-04-06T03:40:24.203 INFO:tasks.workunit.client.1.smithi136.stdout:9/10: chown f0 16310 1 2024-04-06T03:40:24.203 INFO:tasks.workunit.client.1.smithi136.stdout:9/11: creat f1 x:0 0 0 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fdfe406b540 msgr2=0x7fdfe406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fdfe406b540 0x7fdfe406d9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0008fd0 tx=0x7fdfe000b040 comp rx=0 tx=0).stop 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdff4066280 msgr2=0x7fdff410da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdff4066280 0x7fdff410da80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8000c00 tx=0x7fdfe8009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 shutdown_connections 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fdfe406b540 0x7fdfe406d9f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdff4066280 0x7fdff410da80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 --2- 172.21.15.94:0/1563244528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdff40fff60 0x7fdff410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 >> 172.21.15.94:0/1563244528 conn(0x7fdff40f7430 msgr2=0x7fdff40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 shutdown_connections 2024-04-06T03:40:24.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.200+0000 7fdffab34700 1 -- 172.21.15.94:0/1563244528 wait complete. 2024-04-06T03:40:24.205 INFO:tasks.workunit.client.1.smithi136.stdout:8/4: link f0 f1 0 2024-04-06T03:40:24.205 INFO:tasks.workunit.client.1.smithi136.stdout:8/5: read - f1 zero size 2024-04-06T03:40:24.205 INFO:tasks.workunit.client.1.smithi136.stdout:8/6: creat f2 x:0 0 0 2024-04-06T03:40:24.205 INFO:tasks.workunit.client.0.smithi094.stdout:0/35: mkdir d1/d9/dd 0 2024-04-06T03:40:24.206 INFO:tasks.workunit.client.1.smithi136.stdout:2/18: link d3/c6 d3/c7 0 2024-04-06T03:40:24.206 INFO:tasks.workunit.client.1.smithi136.stdout:2/19: fdatasync - no filename 2024-04-06T03:40:24.206 INFO:tasks.workunit.client.1.smithi136.stdout:2/20: write - no filename 2024-04-06T03:40:24.206 INFO:tasks.workunit.client.1.smithi136.stdout:2/21: dread - no filename 2024-04-06T03:40:24.206 INFO:tasks.workunit.client.1.smithi136.stdout:2/22: write - no filename 2024-04-06T03:40:24.210 INFO:tasks.workunit.client.1.smithi136.stdout:8/7: mkdir d3 0 2024-04-06T03:40:24.210 INFO:tasks.workunit.client.1.smithi136.stdout:8/8: readlink - no filename 2024-04-06T03:40:24.210 INFO:tasks.workunit.client.1.smithi136.stdout:8/9: truncate f0 986534 0 2024-04-06T03:40:24.210 INFO:tasks.workunit.client.1.smithi136.stdout:8/10: read - f2 zero size 2024-04-06T03:40:24.210 INFO:tasks.workunit.client.1.smithi136.stdout:2/23: stat d3/c4 0 2024-04-06T03:40:24.211 INFO:tasks.workunit.client.1.smithi136.stdout:2/24: truncate - no filename 2024-04-06T03:40:24.212 INFO:tasks.workunit.client.1.smithi136.stdout:2/25: mknod d3/c8 0 2024-04-06T03:40:24.212 INFO:tasks.workunit.client.1.smithi136.stdout:2/26: stat d3/c4 0 2024-04-06T03:40:24.214 INFO:tasks.workunit.client.1.smithi136.stdout:2/27: mkdir d3/d9 0 2024-04-06T03:40:24.214 INFO:tasks.workunit.client.1.smithi136.stdout:2/28: write - no filename 2024-04-06T03:40:24.214 INFO:tasks.workunit.client.1.smithi136.stdout:2/29: chown d3/d9 4081809 1 2024-04-06T03:40:24.215 INFO:tasks.workunit.client.1.smithi136.stdout:1/14: dwrite f3 [0,4194304] 0 2024-04-06T03:40:24.215 INFO:tasks.workunit.client.1.smithi136.stdout:2/30: rename d3/c7 to d3/d9/ca 0 2024-04-06T03:40:24.215 INFO:tasks.workunit.client.1.smithi136.stdout:2/31: write - no filename 2024-04-06T03:40:24.216 INFO:tasks.workunit.client.1.smithi136.stdout:2/32: stat d3/d9 0 2024-04-06T03:40:24.216 INFO:tasks.workunit.client.1.smithi136.stdout:2/33: dwrite - no filename 2024-04-06T03:40:24.216 INFO:tasks.workunit.client.1.smithi136.stdout:8/11: dread f0 [0,4194304] 0 2024-04-06T03:40:24.217 INFO:tasks.workunit.client.0.smithi094.stdout:9/31: dwrite d2/f5 [0,4194304] 0 2024-04-06T03:40:24.219 INFO:tasks.workunit.client.1.smithi136.stdout:2/34: link d3/c4 d3/cb 0 2024-04-06T03:40:24.220 INFO:tasks.workunit.client.0.smithi094.stdout:9/32: creat d2/f9 x:0 0 0 2024-04-06T03:40:24.221 INFO:tasks.workunit.client.1.smithi136.stdout:8/12: creat d3/f4 x:0 0 0 2024-04-06T03:40:24.221 INFO:tasks.workunit.client.0.smithi094.stdout:0/36: dwrite d1/f7 [0,4194304] 0 2024-04-06T03:40:24.222 INFO:tasks.workunit.client.1.smithi136.stdout:2/35: mknod d3/d9/cc 0 2024-04-06T03:40:24.223 INFO:tasks.workunit.client.0.smithi094.stdout:9/33: mknod d2/ca 0 2024-04-06T03:40:24.224 INFO:tasks.workunit.client.1.smithi136.stdout:8/13: mkdir d3/d5 0 2024-04-06T03:40:24.225 INFO:tasks.workunit.client.1.smithi136.stdout:8/14: dread - f2 zero size 2024-04-06T03:40:24.225 INFO:tasks.workunit.client.1.smithi136.stdout:8/15: chown d3/f4 15288183 1 2024-04-06T03:40:24.225 INFO:tasks.workunit.client.0.smithi094.stdout:0/37: rmdir d1 39 2024-04-06T03:40:24.226 INFO:tasks.workunit.client.1.smithi136.stdout:2/36: symlink d3/d9/ld 0 2024-04-06T03:40:24.226 INFO:tasks.workunit.client.1.smithi136.stdout:2/37: dwrite - no filename 2024-04-06T03:40:24.226 INFO:tasks.workunit.client.1.smithi136.stdout:2/38: chown d3/c8 9921702 1 2024-04-06T03:40:24.228 INFO:tasks.workunit.client.1.smithi136.stdout:8/16: symlink d3/l6 0 2024-04-06T03:40:24.229 INFO:tasks.workunit.client.1.smithi136.stdout:8/17: dread f0 [0,4194304] 0 2024-04-06T03:40:24.229 INFO:tasks.workunit.client.0.smithi094.stdout:0/38: mknod d1/d9/ce 0 2024-04-06T03:40:24.230 INFO:tasks.workunit.client.1.smithi136.stdout:2/39: creat d3/d9/fe x:0 0 0 2024-04-06T03:40:24.231 INFO:tasks.workunit.client.1.smithi136.stdout:8/18: symlink d3/l7 0 2024-04-06T03:40:24.232 INFO:tasks.workunit.client.0.smithi094.stdout:0/39: mknod d1/d9/cf 0 2024-04-06T03:40:24.233 INFO:tasks.workunit.client.1.smithi136.stdout:2/40: link d3/d9/cc d3/d9/cf 0 2024-04-06T03:40:24.234 INFO:tasks.workunit.client.1.smithi136.stdout:8/19: unlink f2 0 2024-04-06T03:40:24.235 INFO:tasks.workunit.client.1.smithi136.stdout:8/20: write f1 [1658961,73799] 0 2024-04-06T03:40:24.235 INFO:tasks.workunit.client.0.smithi094.stdout:0/40: mkdir d1/d10 0 2024-04-06T03:40:24.235 INFO:tasks.workunit.client.0.smithi094.stdout:0/41: read - d1/d9/fc zero size 2024-04-06T03:40:24.238 INFO:tasks.workunit.client.1.smithi136.stdout:8/21: rmdir d3/d5 0 2024-04-06T03:40:24.238 INFO:tasks.workunit.client.1.smithi136.stdout:8/22: creat d3/f8 x:0 0 0 2024-04-06T03:40:24.238 INFO:tasks.workunit.client.1.smithi136.stdout:8/23: chown d3/l7 516281988 1 2024-04-06T03:40:24.238 INFO:tasks.workunit.client.1.smithi136.stdout:8/24: write d3/f8 [136307,8012] 0 2024-04-06T03:40:24.238 INFO:tasks.workunit.client.1.smithi136.stdout:8/25: write f1 [1865565,58450] 0 2024-04-06T03:40:24.238 INFO:tasks.workunit.client.1.smithi136.stdout:8/26: creat d3/f9 x:0 0 0 2024-04-06T03:40:24.240 INFO:tasks.workunit.client.0.smithi094.stdout:8/55: truncate fd 477283 0 2024-04-06T03:40:24.240 INFO:tasks.workunit.client.1.smithi136.stdout:4/1: getdents . 0 2024-04-06T03:40:24.241 INFO:tasks.workunit.client.0.smithi094.stdout:8/56: mkdir d18 0 2024-04-06T03:40:24.243 INFO:tasks.workunit.client.0.smithi094.stdout:8/57: creat d18/f19 x:0 0 0 2024-04-06T03:40:24.243 INFO:tasks.workunit.client.0.smithi094.stdout:8/58: read f1 [485255,47880] 0 2024-04-06T03:40:24.246 INFO:tasks.workunit.client.0.smithi094.stdout:0/42: dread d1/f7 [0,4194304] 0 2024-04-06T03:40:24.248 INFO:tasks.workunit.client.1.smithi136.stdout:5/1: getdents . 0 2024-04-06T03:40:24.248 INFO:tasks.workunit.client.1.smithi136.stdout:5/2: truncate - no filename 2024-04-06T03:40:24.248 INFO:tasks.workunit.client.1.smithi136.stdout:5/3: chown l0 2497 1 2024-04-06T03:40:24.251 INFO:tasks.workunit.client.0.smithi094.stdout:0/43: mknod d1/c11 0 2024-04-06T03:40:24.251 INFO:tasks.workunit.client.0.smithi094.stdout:0/44: read - d1/f5 zero size 2024-04-06T03:40:24.253 INFO:tasks.workunit.client.1.smithi136.stdout:5/4: mkdir d1 0 2024-04-06T03:40:24.253 INFO:tasks.workunit.client.1.smithi136.stdout:5/5: stat d1 0 2024-04-06T03:40:24.254 INFO:tasks.workunit.client.0.smithi094.stdout:4/33: write d0/f8 [1450995,34277] 0 2024-04-06T03:40:24.255 INFO:tasks.workunit.client.0.smithi094.stdout:0/45: rename d1/cb to d1/d9/c12 0 2024-04-06T03:40:24.257 INFO:tasks.workunit.client.1.smithi136.stdout:5/6: creat d1/f2 x:0 0 0 2024-04-06T03:40:24.257 INFO:tasks.workunit.client.1.smithi136.stdout:5/7: read - d1/f2 zero size 2024-04-06T03:40:24.258 INFO:tasks.workunit.client.0.smithi094.stdout:8/59: dwrite d18/f19 [0,4194304] 0 2024-04-06T03:40:24.258 INFO:tasks.workunit.client.0.smithi094.stdout:8/60: creat d18/f1a x:0 0 0 2024-04-06T03:40:24.259 INFO:tasks.workunit.client.0.smithi094.stdout:8/61: creat d18/f1b x:0 0 0 2024-04-06T03:40:24.259 INFO:tasks.workunit.client.0.smithi094.stdout:4/34: creat d0/fa x:0 0 0 2024-04-06T03:40:24.260 INFO:tasks.workunit.client.0.smithi094.stdout:0/46: creat d1/d10/f13 x:0 0 0 2024-04-06T03:40:24.261 INFO:tasks.workunit.client.1.smithi136.stdout:5/8: rename d1/f2 to d1/f3 0 2024-04-06T03:40:24.261 INFO:tasks.workunit.client.1.smithi136.stdout:5/9: write d1/f3 [817294,106941] 0 2024-04-06T03:40:24.261 INFO:tasks.workunit.client.1.smithi136.stdout:5/10: write d1/f3 [1795905,119265] 0 2024-04-06T03:40:24.263 INFO:tasks.workunit.client.1.smithi136.stdout:4/2: dwrite f0 [0,4194304] 0 2024-04-06T03:40:24.263 INFO:tasks.workunit.client.1.smithi136.stdout:4/3: fdatasync f0 0 2024-04-06T03:40:24.263 INFO:tasks.workunit.client.1.smithi136.stdout:4/4: fdatasync f0 0 2024-04-06T03:40:24.265 INFO:tasks.workunit.client.0.smithi094.stdout:4/35: dread d0/d4/f6 [4194304,4194304] 0 2024-04-06T03:40:24.266 INFO:tasks.workunit.client.1.smithi136.stdout:6/7: fsync f1 0 2024-04-06T03:40:24.266 INFO:tasks.workunit.client.1.smithi136.stdout:6/8: write f1 [298245,99324] 0 2024-04-06T03:40:24.268 INFO:tasks.workunit.client.0.smithi094.stdout:0/47: creat d1/d9/f14 x:0 0 0 2024-04-06T03:40:24.268 INFO:tasks.workunit.client.0.smithi094.stdout:0/48: write d1/d9/f14 [786127,56518] 0 2024-04-06T03:40:24.271 INFO:tasks.workunit.client.1.smithi136.stdout:6/9: fsync f1 0 2024-04-06T03:40:24.272 INFO:tasks.workunit.client.0.smithi094.stdout:4/36: creat d0/d4/fb x:0 0 0 2024-04-06T03:40:24.274 INFO:tasks.workunit.client.1.smithi136.stdout:4/5: symlink l1 0 2024-04-06T03:40:24.274 INFO:tasks.workunit.client.1.smithi136.stdout:4/6: rmdir - no directory 2024-04-06T03:40:24.275 INFO:tasks.workunit.client.0.smithi094.stdout:4/37: dread d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:24.275 INFO:tasks.workunit.client.0.smithi094.stdout:4/38: write d0/d4/fb [794900,23797] 0 2024-04-06T03:40:24.276 INFO:tasks.workunit.client.0.smithi094.stdout:8/62: dwrite d18/f19 [0,4194304] 0 2024-04-06T03:40:24.276 INFO:tasks.workunit.client.0.smithi094.stdout:1/59: getdents d0 0 2024-04-06T03:40:24.276 INFO:tasks.workunit.client.0.smithi094.stdout:1/60: write d0/f14 [345720,32255] 0 2024-04-06T03:40:24.276 INFO:tasks.workunit.client.0.smithi094.stdout:1/61: chown d0/d4/cf 0 1 2024-04-06T03:40:24.279 INFO:tasks.workunit.client.1.smithi136.stdout:6/10: mkdir d2 0 2024-04-06T03:40:24.280 INFO:tasks.workunit.client.0.smithi094.stdout:8/63: dread f3 [0,4194304] 0 2024-04-06T03:40:24.281 INFO:tasks.workunit.client.1.smithi136.stdout:5/11: dwrite d1/f3 [0,4194304] 0 2024-04-06T03:40:24.281 INFO:tasks.workunit.client.1.smithi136.stdout:5/12: creat d1/f4 x:0 0 0 2024-04-06T03:40:24.281 INFO:tasks.workunit.client.1.smithi136.stdout:5/13: chown d1/f3 53023082 1 2024-04-06T03:40:24.282 INFO:tasks.workunit.client.1.smithi136.stdout:4/7: unlink f0 0 2024-04-06T03:40:24.282 INFO:tasks.workunit.client.1.smithi136.stdout:4/8: chown l1 1708 1 2024-04-06T03:40:24.284 INFO:tasks.workunit.client.1.smithi136.stdout:6/11: symlink d2/l3 0 2024-04-06T03:40:24.284 INFO:tasks.workunit.client.1.smithi136.stdout:8/27: fsync f0 0 2024-04-06T03:40:24.285 INFO:tasks.workunit.client.0.smithi094.stdout:1/62: mkdir d0/d7/d17 0 2024-04-06T03:40:24.285 INFO:tasks.workunit.client.0.smithi094.stdout:1/63: read d0/d4/f9 [1323832,69958] 0 2024-04-06T03:40:24.290 INFO:tasks.workunit.client.1.smithi136.stdout:4/9: rename l1 to l2 0 2024-04-06T03:40:24.292 INFO:tasks.workunit.client.1.smithi136.stdout:6/12: creat d2/f4 x:0 0 0 2024-04-06T03:40:24.292 INFO:tasks.workunit.client.1.smithi136.stdout:6/13: stat l0 0 2024-04-06T03:40:24.294 INFO:tasks.workunit.client.0.smithi094.stdout:1/64: link d0/d4/fc d0/d1/f18 0 2024-04-06T03:40:24.294 INFO:tasks.workunit.client.0.smithi094.stdout:1/65: stat d0/d4 0 2024-04-06T03:40:24.294 INFO:tasks.workunit.client.1.smithi136.stdout:1/15: write f3 [5097699,86941] 0 2024-04-06T03:40:24.295 INFO:tasks.workunit.client.0.smithi094.stdout:1/66: creat d0/f19 x:0 0 0 2024-04-06T03:40:24.296 INFO:tasks.workunit.client.1.smithi136.stdout:8/28: dwrite d3/f4 [0,4194304] 0 2024-04-06T03:40:24.296 INFO:tasks.workunit.client.1.smithi136.stdout:8/29: fdatasync d3/f4 0 2024-04-06T03:40:24.296 INFO:tasks.workunit.client.1.smithi136.stdout:8/30: dread - d3/f9 zero size 2024-04-06T03:40:24.298 INFO:tasks.workunit.client.1.smithi136.stdout:6/14: unlink l0 0 2024-04-06T03:40:24.298 INFO:tasks.workunit.client.1.smithi136.stdout:6/15: creat d2/f5 x:0 0 0 2024-04-06T03:40:24.298 INFO:tasks.workunit.client.1.smithi136.stdout:1/16: stat c2 0 2024-04-06T03:40:24.300 INFO:tasks.workunit.client.0.smithi094.stdout:1/67: mkdir d0/d7/d1a 0 2024-04-06T03:40:24.304 INFO:tasks.workunit.client.1.smithi136.stdout:6/16: mknod d2/c6 0 2024-04-06T03:40:24.307 INFO:tasks.workunit.client.1.smithi136.stdout:2/41: fsync d3/d9/fe 0 2024-04-06T03:40:24.307 INFO:tasks.workunit.client.1.smithi136.stdout:2/42: read - d3/d9/fe zero size 2024-04-06T03:40:24.308 INFO:tasks.workunit.client.1.smithi136.stdout:1/17: mkdir d6 0 2024-04-06T03:40:24.310 INFO:tasks.workunit.client.1.smithi136.stdout:6/17: mkdir d2/d7 0 2024-04-06T03:40:24.310 INFO:tasks.workunit.client.1.smithi136.stdout:2/43: stat c0 0 2024-04-06T03:40:24.313 INFO:tasks.workunit.client.1.smithi136.stdout:2/44: mkdir d3/d9/d10 0 2024-04-06T03:40:24.313 INFO:tasks.workunit.client.1.smithi136.stdout:2/45: chown d3/c4 10 1 2024-04-06T03:40:24.314 INFO:tasks.workunit.client.1.smithi136.stdout:2/46: mkdir d3/d9/d10/d11 0 2024-04-06T03:40:24.315 INFO:tasks.workunit.client.1.smithi136.stdout:2/47: creat d3/d9/f12 x:0 0 0 2024-04-06T03:40:24.315 INFO:tasks.workunit.client.1.smithi136.stdout:2/48: read - d3/d9/fe zero size 2024-04-06T03:40:24.321 INFO:tasks.workunit.client.1.smithi136.stdout:2/49: symlink d3/d9/d10/d11/l13 0 2024-04-06T03:40:24.321 INFO:tasks.workunit.client.1.smithi136.stdout:2/50: dread - d3/d9/f12 zero size 2024-04-06T03:40:24.323 INFO:tasks.workunit.client.1.smithi136.stdout:2/51: link d3/d9/ca d3/d9/d10/d11/c14 0 2024-04-06T03:40:24.323 INFO:tasks.workunit.client.1.smithi136.stdout:2/52: dread - d3/d9/fe zero size 2024-04-06T03:40:24.323 INFO:tasks.workunit.client.1.smithi136.stdout:2/53: truncate d3/d9/f12 230839 0 2024-04-06T03:40:24.330 INFO:tasks.workunit.client.0.smithi094.stdout:8/64: write d18/f19 [4275214,50473] 0 2024-04-06T03:40:24.332 INFO:tasks.workunit.client.0.smithi094.stdout:8/65: rmdir d18 39 2024-04-06T03:40:24.335 INFO:tasks.workunit.client.0.smithi094.stdout:8/66: link d18/f19 d18/f1c 0 2024-04-06T03:40:24.335 INFO:tasks.workunit.client.1.smithi136.stdout:6/18: dwrite f1 [0,4194304] 0 2024-04-06T03:40:24.336 INFO:tasks.workunit.client.1.smithi136.stdout:1/18: dwrite f3 [0,4194304] 0 2024-04-06T03:40:24.336 INFO:tasks.workunit.client.1.smithi136.stdout:1/19: chown d6 505790925 1 2024-04-06T03:40:24.336 INFO:tasks.workunit.client.1.smithi136.stdout:1/20: getdents d6 0 2024-04-06T03:40:24.336 INFO:tasks.workunit.client.1.smithi136.stdout:1/21: readlink - no filename 2024-04-06T03:40:24.337 INFO:tasks.workunit.client.0.smithi094.stdout:8/67: dread f1 [0,4194304] 0 2024-04-06T03:40:24.337 INFO:tasks.workunit.client.0.smithi094.stdout:4/39: rmdir d0 39 2024-04-06T03:40:24.337 INFO:tasks.workunit.client.0.smithi094.stdout:4/40: dread - d0/fa zero size 2024-04-06T03:40:24.340 INFO:tasks.workunit.client.0.smithi094.stdout:8/68: rename c16 to d18/c1d 0 2024-04-06T03:40:24.341 INFO:tasks.workunit.client.0.smithi094.stdout:4/41: link d0/fa d0/d4/fc 0 2024-04-06T03:40:24.346 INFO:tasks.workunit.client.1.smithi136.stdout:1/22: rmdir d6 0 2024-04-06T03:40:24.346 INFO:tasks.workunit.client.1.smithi136.stdout:1/23: rmdir - no directory 2024-04-06T03:40:24.348 INFO:tasks.workunit.client.1.smithi136.stdout:1/24: symlink l7 0 2024-04-06T03:40:24.348 INFO:tasks.workunit.client.1.smithi136.stdout:1/25: chown l7 113066712 1 2024-04-06T03:40:24.349 INFO:tasks.workunit.client.1.smithi136.stdout:5/14: fsync d1/f4 0 2024-04-06T03:40:24.349 INFO:tasks.workunit.client.1.smithi136.stdout:5/15: write d1/f4 [956190,98030] 0 2024-04-06T03:40:24.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.347+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/394986513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c40fff60 msgr2=0x7fc0c4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:24.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.347+0000 7fc0cb6a5700 1 --2- 172.21.15.94:0/394986513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c40fff60 0x7fc0c4102340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4004940 tx=0x7fc0b4009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:24.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.347+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/394986513 shutdown_connections 2024-04-06T03:40:24.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.347+0000 7fc0cb6a5700 1 --2- 172.21.15.94:0/394986513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c40fff60 0x7fc0c4102340 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:24.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.347+0000 7fc0cb6a5700 1 --2- 172.21.15.94:0/394986513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0c4066e90 0x7fc0c40ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:24.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.347+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/394986513 >> 172.21.15.94:0/394986513 conn(0x7fc0c40f7430 msgr2=0x7fc0c40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:24.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.348+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/394986513 shutdown_connections 2024-04-06T03:40:24.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.348+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/394986513 wait complete. 2024-04-06T03:40:24.352 INFO:tasks.workunit.client.1.smithi136.stdout:4/10: rename l2 to l3 0 2024-04-06T03:40:24.352 INFO:tasks.workunit.client.1.smithi136.stdout:4/11: dwrite - no filename 2024-04-06T03:40:24.352 INFO:tasks.workunit.client.1.smithi136.stdout:5/16: mknod d1/c5 0 2024-04-06T03:40:24.352 INFO:tasks.workunit.client.1.smithi136.stdout:6/19: dwrite d2/f4 [0,4194304] 0 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.348+0000 7fc0cb6a5700 1 Processor -- start 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.348+0000 7fc0cb6a5700 1 -- start start 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.348+0000 7fc0cb6a5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c4066e90 0x7fc0c40fd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.348+0000 7fc0cb6a5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0c40fff60 0x7fc0c40fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0cb6a5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40fd820 con 0x7fc0c4066e90 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0cb6a5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c40fbe60 con 0x7fc0c40fff60 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0c9441700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c4066e90 0x7fc0c40fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0c9441700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c4066e90 0x7fc0c40fd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40148/0 (socket says 172.21.15.94:40148) 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0c9441700 1 -- 172.21.15.94:0/3570089268 learned_addr learned my addr 172.21.15.94:0/3570089268 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0c8c40700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0c40fff60 0x7fc0c40fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0c9441700 1 -- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0c40fff60 msgr2=0x7fc0c40fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0c9441700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0c40fff60 0x7fc0c40fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:24.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0c9441700 1 -- 172.21.15.94:0/3570089268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b4005040 con 0x7fc0c4066e90 2024-04-06T03:40:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.349+0000 7fc0c9441700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c4066e90 0x7fc0c40fd220 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000eb70 tx=0x7fc0c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.350+0000 7fc0b9ffb700 1 -- 172.21.15.94:0/3570089268 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0c0007cc0 con 0x7fc0c4066e90 2024-04-06T03:40:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.350+0000 7fc0b9ffb700 1 -- 172.21.15.94:0/3570089268 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0c000f040 con 0x7fc0c4066e90 2024-04-06T03:40:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.350+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c40fc0e0 con 0x7fc0c4066e90 2024-04-06T03:40:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.350+0000 7fc0b9ffb700 1 -- 172.21.15.94:0/3570089268 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0c000cb50 con 0x7fc0c4066e90 2024-04-06T03:40:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.350+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c40fc620 con 0x7fc0c4066e90 2024-04-06T03:40:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.351+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c404c710 con 0x7fc0c4066e90 2024-04-06T03:40:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.353+0000 7fc0b9ffb700 1 -- 172.21.15.94:0/3570089268 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7fc0c000ccb0 con 0x7fc0c4066e90 2024-04-06T03:40:24.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.354+0000 7fc0b9ffb700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc0b006b580 0x7fc0b006da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:24.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.354+0000 7fc0b9ffb700 1 -- 172.21.15.94:0/3570089268 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc0c0089a60 con 0x7fc0c4066e90 2024-04-06T03:40:24.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.354+0000 7fc0c8c40700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc0b006b580 0x7fc0b006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:24.358 INFO:tasks.workunit.client.1.smithi136.stdout:6/20: unlink d2/f4 0 2024-04-06T03:40:24.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.356+0000 7fc0c8c40700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc0b006b580 0x7fc0b006da30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4013870 tx=0x7fc0b4009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:24.363 INFO:tasks.workunit.client.1.smithi136.stdout:6/21: unlink f1 0 2024-04-06T03:40:24.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.360+0000 7fc0b9ffb700 1 -- 172.21.15.94:0/3570089268 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fc0c004fb80 con 0x7fc0c4066e90 2024-04-06T03:40:24.365 INFO:tasks.workunit.client.1.smithi136.stdout:6/22: symlink d2/l8 0 2024-04-06T03:40:24.366 INFO:tasks.workunit.client.1.smithi136.stdout:6/23: creat d2/f9 x:0 0 0 2024-04-06T03:40:24.373 INFO:tasks.workunit.client.1.smithi136.stdout:1/26: dwrite f3 [0,4194304] 0 2024-04-06T03:40:24.374 INFO:tasks.workunit.client.1.smithi136.stdout:1/27: fsync f3 0 2024-04-06T03:40:24.374 INFO:tasks.workunit.client.1.smithi136.stdout:2/54: rmdir d3/d9 39 2024-04-06T03:40:24.374 INFO:tasks.workunit.client.1.smithi136.stdout:5/17: dwrite d1/f4 [0,4194304] 0 2024-04-06T03:40:24.376 INFO:tasks.workunit.client.0.smithi094.stdout:8/69: dread fd [0,4194304] 0 2024-04-06T03:40:24.376 INFO:tasks.workunit.client.1.smithi136.stdout:1/28: mknod c8 0 2024-04-06T03:40:24.378 INFO:tasks.workunit.client.1.smithi136.stdout:2/55: mkdir d3/d15 0 2024-04-06T03:40:24.379 INFO:tasks.workunit.client.1.smithi136.stdout:2/56: truncate d3/d9/f12 690189 0 2024-04-06T03:40:24.380 INFO:tasks.workunit.client.1.smithi136.stdout:6/24: dwrite d2/f9 [0,4194304] 0 2024-04-06T03:40:24.382 INFO:tasks.workunit.client.0.smithi094.stdout:4/42: fsync d0/d4/fc 0 2024-04-06T03:40:24.382 INFO:tasks.workunit.client.0.smithi094.stdout:4/43: readlink - no filename 2024-04-06T03:40:24.383 INFO:tasks.workunit.client.1.smithi136.stdout:1/29: mknod c9 0 2024-04-06T03:40:24.383 INFO:tasks.workunit.client.1.smithi136.stdout:1/30: chown c4 23741315 1 2024-04-06T03:40:24.384 INFO:tasks.workunit.client.1.smithi136.stdout:2/57: symlink d3/d9/d10/l16 0 2024-04-06T03:40:24.384 INFO:tasks.workunit.client.1.smithi136.stdout:2/58: chown d3/d9/d10/l16 15 1 2024-04-06T03:40:24.384 INFO:tasks.workunit.client.1.smithi136.stdout:2/59: fsync d3/d9/f12 0 2024-04-06T03:40:24.384 INFO:tasks.workunit.client.1.smithi136.stdout:2/60: chown d3/d9/ld 13440645 1 2024-04-06T03:40:24.386 INFO:tasks.workunit.client.1.smithi136.stdout:5/18: read d1/f4 [2389174,25590] 0 2024-04-06T03:40:24.387 INFO:tasks.workunit.client.1.smithi136.stdout:1/31: link c5 ca 0 2024-04-06T03:40:24.387 INFO:tasks.workunit.client.1.smithi136.stdout:1/32: creat fb x:0 0 0 2024-04-06T03:40:24.388 INFO:tasks.workunit.client.1.smithi136.stdout:4/12: rename l3 to l4 0 2024-04-06T03:40:24.392 INFO:tasks.workunit.client.1.smithi136.stdout:2/61: creat d3/d9/d10/f17 x:0 0 0 2024-04-06T03:40:24.396 INFO:tasks.workunit.client.0.smithi094.stdout:8/70: dwrite d18/f1b [0,4194304] 0 2024-04-06T03:40:24.397 INFO:tasks.workunit.client.1.smithi136.stdout:1/33: link l7 lc 0 2024-04-06T03:40:24.397 INFO:tasks.workunit.client.1.smithi136.stdout:1/34: chown l7 25298956 1 2024-04-06T03:40:24.397 INFO:tasks.workunit.client.1.smithi136.stdout:1/35: stat c2 0 2024-04-06T03:40:24.398 INFO:tasks.workunit.client.1.smithi136.stdout:4/13: stat l4 0 2024-04-06T03:40:24.398 INFO:tasks.workunit.client.1.smithi136.stdout:4/14: dwrite - no filename 2024-04-06T03:40:24.399 INFO:tasks.workunit.client.1.smithi136.stdout:1/36: stat l7 0 2024-04-06T03:40:24.399 INFO:tasks.workunit.client.1.smithi136.stdout:1/37: read - fb zero size 2024-04-06T03:40:24.401 INFO:tasks.workunit.client.0.smithi094.stdout:8/71: rename c7 to d18/c1e 0 2024-04-06T03:40:24.402 INFO:tasks.workunit.client.1.smithi136.stdout:1/38: write f3 [4877528,79240] 0 2024-04-06T03:40:24.402 INFO:tasks.workunit.client.1.smithi136.stdout:1/39: readlink l7 0 2024-04-06T03:40:24.402 INFO:tasks.workunit.client.1.smithi136.stdout:1/40: creat fd x:0 0 0 2024-04-06T03:40:24.402 INFO:tasks.workunit.client.1.smithi136.stdout:5/19: write d1/f4 [4752230,17589] 0 2024-04-06T03:40:24.403 INFO:tasks.workunit.client.1.smithi136.stdout:1/41: symlink le 0 2024-04-06T03:40:24.405 INFO:tasks.workunit.client.1.smithi136.stdout:5/20: unlink d1/c5 0 2024-04-06T03:40:24.405 INFO:tasks.workunit.client.1.smithi136.stdout:5/21: write d1/f3 [4867836,104015] 0 2024-04-06T03:40:24.406 INFO:tasks.workunit.client.1.smithi136.stdout:2/62: dwrite d3/d9/fe [0,4194304] 0 2024-04-06T03:40:24.407 INFO:tasks.workunit.client.1.smithi136.stdout:5/22: mknod d1/c6 0 2024-04-06T03:40:24.407 INFO:tasks.workunit.client.1.smithi136.stdout:5/23: truncate d1/f4 5540049 0 2024-04-06T03:40:24.409 INFO:tasks.workunit.client.1.smithi136.stdout:4/15: rename l4 to l5 0 2024-04-06T03:40:24.409 INFO:tasks.workunit.client.1.smithi136.stdout:4/16: chown l5 16015108 1 2024-04-06T03:40:24.411 INFO:tasks.workunit.client.1.smithi136.stdout:5/24: mknod d1/c7 0 2024-04-06T03:40:24.414 INFO:tasks.workunit.client.1.smithi136.stdout:1/42: getdents . 0 2024-04-06T03:40:24.415 INFO:tasks.workunit.client.1.smithi136.stdout:5/25: truncate d1/f3 4194435 0 2024-04-06T03:40:24.417 INFO:tasks.workunit.client.1.smithi136.stdout:4/17: rename l5 to l6 0 2024-04-06T03:40:24.417 INFO:tasks.workunit.client.1.smithi136.stdout:4/18: dread - no filename 2024-04-06T03:40:24.419 INFO:tasks.workunit.client.1.smithi136.stdout:4/19: creat f7 x:0 0 0 2024-04-06T03:40:24.419 INFO:tasks.workunit.client.1.smithi136.stdout:4/20: rmdir - no directory 2024-04-06T03:40:24.420 INFO:tasks.workunit.client.1.smithi136.stdout:4/21: fdatasync f7 0 2024-04-06T03:40:24.420 INFO:tasks.workunit.client.1.smithi136.stdout:5/26: getdents d1 0 2024-04-06T03:40:24.498 INFO:tasks.workunit.client.1.smithi136.stdout:4/22: dwrite f7 [0,4194304] 0 2024-04-06T03:40:24.498 INFO:tasks.workunit.client.1.smithi136.stdout:4/23: chown l6 5 1 2024-04-06T03:40:24.498 INFO:tasks.workunit.client.1.smithi136.stdout:4/24: creat f8 x:0 0 0 2024-04-06T03:40:24.498 INFO:tasks.workunit.client.1.smithi136.stdout:4/25: fdatasync f8 0 2024-04-06T03:40:24.504 INFO:tasks.workunit.client.1.smithi136.stdout:6/25: dwrite d2/f9 [0,4194304] 0 2024-04-06T03:40:24.504 INFO:tasks.workunit.client.1.smithi136.stdout:6/26: creat d2/fa x:0 0 0 2024-04-06T03:40:24.507 INFO:tasks.workunit.client.0.smithi094.stdout:4/44: dwrite d0/d4/fc [0,4194304] 0 2024-04-06T03:40:24.512 INFO:tasks.workunit.client.1.smithi136.stdout:6/27: link d2/l8 d2/d7/lb 0 2024-04-06T03:40:24.519 INFO:tasks.workunit.client.1.smithi136.stdout:2/63: dwrite d3/d9/f12 [0,4194304] 0 2024-04-06T03:40:24.521 INFO:tasks.workunit.client.0.smithi094.stdout:8/72: dwrite f11 [0,4194304] 0 2024-04-06T03:40:24.521 INFO:tasks.workunit.client.1.smithi136.stdout:5/27: dwrite d1/f4 [0,4194304] 0 2024-04-06T03:40:24.522 INFO:tasks.workunit.client.1.smithi136.stdout:4/26: dwrite f8 [0,4194304] 0 2024-04-06T03:40:24.522 INFO:tasks.workunit.client.1.smithi136.stdout:1/43: dwrite fd [0,4194304] 0 2024-04-06T03:40:24.522 INFO:tasks.workunit.client.1.smithi136.stdout:2/64: mkdir d3/d9/d10/d18 0 2024-04-06T03:40:24.524 INFO:tasks.workunit.client.1.smithi136.stdout:4/27: mknod c9 0 2024-04-06T03:40:24.524 INFO:tasks.workunit.client.1.smithi136.stdout:4/28: readlink l6 0 2024-04-06T03:40:24.524 INFO:tasks.workunit.client.1.smithi136.stdout:4/29: truncate f8 4700687 0 2024-04-06T03:40:24.525 INFO:tasks.workunit.client.1.smithi136.stdout:6/28: dread d2/f9 [0,4194304] 0 2024-04-06T03:40:24.525 INFO:tasks.workunit.client.1.smithi136.stdout:6/29: write d2/fa [613107,99693] 0 2024-04-06T03:40:24.527 INFO:tasks.workunit.client.0.smithi094.stdout:4/45: truncate d0/fa 785592 0 2024-04-06T03:40:24.528 INFO:tasks.workunit.client.0.smithi094.stdout:4/46: creat d0/d4/fd x:0 0 0 2024-04-06T03:40:24.533 INFO:tasks.workunit.client.1.smithi136.stdout:5/28: dread d1/f3 [0,4194304] 0 2024-04-06T03:40:24.533 INFO:tasks.workunit.client.1.smithi136.stdout:1/44: dread f3 [4194304,4194304] 0 2024-04-06T03:40:24.533 INFO:tasks.workunit.client.0.smithi094.stdout:8/73: write d18/f1b [768638,126287] 0 2024-04-06T03:40:24.533 INFO:tasks.workunit.client.0.smithi094.stdout:8/74: chown f1 8473033 1 2024-04-06T03:40:24.534 INFO:tasks.workunit.client.1.smithi136.stdout:1/45: unlink ca 0 2024-04-06T03:40:24.536 INFO:tasks.workunit.client.1.smithi136.stdout:1/46: mknod cf 0 2024-04-06T03:40:24.541 INFO:tasks.workunit.client.1.smithi136.stdout:4/30: read f8 [1512638,105462] 0 2024-04-06T03:40:24.541 INFO:tasks.workunit.client.1.smithi136.stdout:2/65: dwrite d3/d9/fe [0,4194304] 0 2024-04-06T03:40:24.541 INFO:tasks.workunit.client.0.smithi094.stdout:8/75: write f14 [3473600,117275] 0 2024-04-06T03:40:24.547 INFO:tasks.workunit.client.0.smithi094.stdout:8/76: fsync f14 0 2024-04-06T03:40:24.547 INFO:tasks.workunit.client.1.smithi136.stdout:1/47: write f3 [1802085,5123] 0 2024-04-06T03:40:24.548 INFO:tasks.workunit.client.1.smithi136.stdout:4/31: truncate f8 3226455 0 2024-04-06T03:40:24.550 INFO:tasks.workunit.client.0.smithi094.stdout:8/77: rename ca to d18/c1f 0 2024-04-06T03:40:24.550 INFO:tasks.workunit.client.0.smithi094.stdout:8/78: read - f17 zero size 2024-04-06T03:40:24.550 INFO:tasks.workunit.client.0.smithi094.stdout:8/79: chown l8 628579 1 2024-04-06T03:40:24.550 INFO:tasks.workunit.client.0.smithi094.stdout:8/80: write f1 [5230409,8950] 0 2024-04-06T03:40:24.550 INFO:tasks.workunit.client.0.smithi094.stdout:8/81: readlink lf 0 2024-04-06T03:40:24.551 INFO:tasks.workunit.client.1.smithi136.stdout:4/32: mknod ca 0 2024-04-06T03:40:24.555 INFO:tasks.workunit.client.0.smithi094.stdout:8/82: write f3 [3506159,2048] 0 2024-04-06T03:40:24.556 INFO:tasks.workunit.client.1.smithi136.stdout:2/66: write d3/d9/f12 [3956526,20591] 0 2024-04-06T03:40:24.556 INFO:tasks.workunit.client.1.smithi136.stdout:5/29: dwrite d1/f4 [0,4194304] 0 2024-04-06T03:40:24.556 INFO:tasks.workunit.client.1.smithi136.stdout:5/30: getdents d1 0 2024-04-06T03:40:24.556 INFO:tasks.workunit.client.1.smithi136.stdout:2/67: write d3/d9/d10/f17 [253238,91848] 0 2024-04-06T03:40:24.556 INFO:tasks.workunit.client.0.smithi094.stdout:8/83: creat d18/f20 x:0 0 0 2024-04-06T03:40:24.557 INFO:tasks.workunit.client.1.smithi136.stdout:1/48: dread f3 [0,4194304] 0 2024-04-06T03:40:24.558 INFO:tasks.workunit.client.1.smithi136.stdout:4/33: dread f8 [0,4194304] 0 2024-04-06T03:40:24.559 INFO:tasks.workunit.client.1.smithi136.stdout:2/68: dread d3/d9/d10/f17 [0,4194304] 0 2024-04-06T03:40:24.559 INFO:tasks.workunit.client.1.smithi136.stdout:5/31: unlink d1/f3 0 2024-04-06T03:40:24.561 INFO:tasks.workunit.client.1.smithi136.stdout:4/34: rename c9 to cb 0 2024-04-06T03:40:24.561 INFO:tasks.workunit.client.1.smithi136.stdout:2/69: fsync d3/d9/f12 0 2024-04-06T03:40:24.561 INFO:tasks.workunit.client.1.smithi136.stdout:2/70: write d3/d9/d10/f17 [305198,14165] 0 2024-04-06T03:40:24.561 INFO:tasks.workunit.client.1.smithi136.stdout:2/71: write d3/d9/fe [4833593,15472] 0 2024-04-06T03:40:24.561 INFO:tasks.workunit.client.1.smithi136.stdout:2/72: readlink d3/d9/d10/l16 0 2024-04-06T03:40:24.564 INFO:tasks.workunit.client.1.smithi136.stdout:2/73: rmdir d3/d9/d10 39 2024-04-06T03:40:24.564 INFO:tasks.workunit.client.1.smithi136.stdout:2/74: fsync d3/d9/f12 0 2024-04-06T03:40:24.564 INFO:tasks.workunit.client.1.smithi136.stdout:2/75: truncate d3/d9/d10/f17 1248541 0 2024-04-06T03:40:24.565 INFO:tasks.workunit.client.1.smithi136.stdout:1/49: write fd [1751086,98395] 0 2024-04-06T03:40:24.566 INFO:tasks.workunit.client.1.smithi136.stdout:2/76: mkdir d3/d15/d19 0 2024-04-06T03:40:24.566 INFO:tasks.workunit.client.1.smithi136.stdout:4/35: dread f8 [0,4194304] 0 2024-04-06T03:40:24.566 INFO:tasks.workunit.client.1.smithi136.stdout:4/36: rmdir - no directory 2024-04-06T03:40:24.567 INFO:tasks.workunit.client.1.smithi136.stdout:4/37: truncate f7 2436065 0 2024-04-06T03:40:24.570 INFO:tasks.workunit.client.1.smithi136.stdout:4/38: truncate f8 2850185 0 2024-04-06T03:40:24.570 INFO:tasks.workunit.client.1.smithi136.stdout:4/39: fdatasync f7 0 2024-04-06T03:40:24.574 INFO:tasks.workunit.client.1.smithi136.stdout:2/77: dread d3/d9/d10/f17 [0,4194304] 0 2024-04-06T03:40:24.576 INFO:tasks.workunit.client.1.smithi136.stdout:2/78: mknod d3/d9/c1a 0 2024-04-06T03:40:24.576 INFO:tasks.workunit.client.1.smithi136.stdout:5/32: dwrite d1/f4 [4194304,4194304] 0 2024-04-06T03:40:24.577 INFO:tasks.workunit.client.1.smithi136.stdout:2/79: creat d3/d15/d19/f1b x:0 0 0 2024-04-06T03:40:24.577 INFO:tasks.workunit.client.1.smithi136.stdout:2/80: chown d3/d15/d19/f1b 1527050621 1 2024-04-06T03:40:24.577 INFO:tasks.workunit.client.1.smithi136.stdout:1/50: dread fd [0,4194304] 0 2024-04-06T03:40:24.578 INFO:tasks.workunit.client.1.smithi136.stdout:5/33: unlink d1/c6 0 2024-04-06T03:40:24.579 INFO:tasks.workunit.client.1.smithi136.stdout:2/81: fsync d3/d9/fe 0 2024-04-06T03:40:24.582 INFO:tasks.workunit.client.1.smithi136.stdout:2/82: symlink d3/d9/d10/d18/l1c 0 2024-04-06T03:40:24.583 INFO:tasks.workunit.client.1.smithi136.stdout:4/40: write f7 [2112402,8615] 0 2024-04-06T03:40:24.583 INFO:tasks.workunit.client.1.smithi136.stdout:4/41: creat fc x:0 0 0 2024-04-06T03:40:24.583 INFO:tasks.workunit.client.1.smithi136.stdout:4/42: write fc [940048,24556] 0 2024-04-06T03:40:24.584 INFO:tasks.workunit.client.0.smithi094.stdout:8/84: dwrite fd [0,4194304] 0 2024-04-06T03:40:24.585 INFO:tasks.workunit.client.1.smithi136.stdout:4/43: symlink ld 0 2024-04-06T03:40:24.586 INFO:tasks.workunit.client.1.smithi136.stdout:4/44: symlink le 0 2024-04-06T03:40:24.586 INFO:tasks.workunit.client.1.smithi136.stdout:4/45: chown l6 24783649 1 2024-04-06T03:40:24.588 INFO:tasks.workunit.client.0.smithi094.stdout:8/85: rmdir d18 39 2024-04-06T03:40:24.590 INFO:tasks.workunit.client.1.smithi136.stdout:2/83: truncate d3/d9/fe 4533458 0 2024-04-06T03:40:24.591 INFO:tasks.workunit.client.0.smithi094.stdout:8/86: creat d18/f21 x:0 0 0 2024-04-06T03:40:24.591 INFO:tasks.workunit.client.0.smithi094.stdout:8/87: readlink l9 0 2024-04-06T03:40:24.593 INFO:tasks.workunit.client.1.smithi136.stdout:2/84: mkdir d3/d9/d10/d18/d1d 0 2024-04-06T03:40:24.599 INFO:tasks.workunit.client.1.smithi136.stdout:5/34: dwrite d1/f4 [4194304,4194304] 0 2024-04-06T03:40:24.599 INFO:tasks.workunit.client.0.smithi094.stdout:8/88: rmdir d18 39 2024-04-06T03:40:24.607 INFO:tasks.workunit.client.0.smithi094.stdout:8/89: fsync f1 0 2024-04-06T03:40:24.608 INFO:tasks.workunit.client.1.smithi136.stdout:1/51: dwrite f3 [0,4194304] 0 2024-04-06T03:40:24.609 INFO:tasks.workunit.client.0.smithi094.stdout:8/90: stat d18/c1f 0 2024-04-06T03:40:24.609 INFO:tasks.workunit.client.1.smithi136.stdout:2/85: dwrite d3/d9/d10/f17 [0,4194304] 0 2024-04-06T03:40:24.611 INFO:tasks.workunit.client.1.smithi136.stdout:5/35: dread d1/f4 [4194304,4194304] 0 2024-04-06T03:40:24.611 INFO:tasks.workunit.client.1.smithi136.stdout:1/52: creat f10 x:0 0 0 2024-04-06T03:40:24.611 INFO:tasks.workunit.client.1.smithi136.stdout:1/53: creat f11 x:0 0 0 2024-04-06T03:40:24.615 INFO:tasks.workunit.client.1.smithi136.stdout:2/86: rename d3/d9/d10/d11 to d3/d1e 0 2024-04-06T03:40:24.616 INFO:tasks.workunit.client.1.smithi136.stdout:5/36: symlink d1/l8 0 2024-04-06T03:40:24.616 INFO:tasks.workunit.client.1.smithi136.stdout:5/37: fdatasync d1/f4 0 2024-04-06T03:40:24.617 INFO:tasks.workunit.client.1.smithi136.stdout:1/54: mkdir d12 0 2024-04-06T03:40:24.617 INFO:tasks.workunit.client.1.smithi136.stdout:1/55: fsync f3 0 2024-04-06T03:40:24.618 INFO:tasks.workunit.client.1.smithi136.stdout:2/87: rmdir d3/d9 39 2024-04-06T03:40:24.619 INFO:tasks.workunit.client.1.smithi136.stdout:5/38: write d1/f4 [5129929,78573] 0 2024-04-06T03:40:24.619 INFO:tasks.workunit.client.1.smithi136.stdout:2/88: symlink d3/d15/l1f 0 2024-04-06T03:40:24.619 INFO:tasks.workunit.client.1.smithi136.stdout:5/39: write d1/f4 [4583573,1788] 0 2024-04-06T03:40:24.620 INFO:tasks.workunit.client.1.smithi136.stdout:2/89: creat d3/d9/d10/d18/d1d/f20 x:0 0 0 2024-04-06T03:40:24.622 INFO:tasks.workunit.client.1.smithi136.stdout:5/40: fdatasync d1/f4 0 2024-04-06T03:40:24.624 INFO:tasks.workunit.client.1.smithi136.stdout:5/41: rename d1/l8 to d1/l9 0 2024-04-06T03:40:24.628 INFO:tasks.workunit.client.0.smithi094.stdout:8/91: dwrite d18/f1c [0,4194304] 0 2024-04-06T03:40:24.631 INFO:tasks.workunit.client.1.smithi136.stdout:5/42: getdents d1 0 2024-04-06T03:40:24.631 INFO:tasks.workunit.client.1.smithi136.stdout:5/43: getdents d1 0 2024-04-06T03:40:24.631 INFO:tasks.workunit.client.1.smithi136.stdout:5/44: getdents d1 0 2024-04-06T03:40:24.635 INFO:tasks.workunit.client.1.smithi136.stdout:2/90: dread d3/d9/d10/f17 [0,4194304] 0 2024-04-06T03:40:24.637 INFO:tasks.workunit.client.1.smithi136.stdout:2/91: rename c0 to d3/d9/d10/d18/d1d/c21 0 2024-04-06T03:40:24.637 INFO:tasks.workunit.client.1.smithi136.stdout:2/92: getdents d3 0 2024-04-06T03:40:24.638 INFO:tasks.workunit.client.1.smithi136.stdout:2/93: creat d3/d9/d10/f22 x:0 0 0 2024-04-06T03:40:24.639 INFO:tasks.workunit.client.1.smithi136.stdout:2/94: dread - d3/d9/d10/f22 zero size 2024-04-06T03:40:24.639 INFO:tasks.workunit.client.1.smithi136.stdout:2/95: creat d3/d9/d10/d18/f23 x:0 0 0 2024-04-06T03:40:24.642 INFO:tasks.workunit.client.1.smithi136.stdout:1/56: dwrite f3 [4194304,4194304] 0 2024-04-06T03:40:24.642 INFO:tasks.workunit.client.1.smithi136.stdout:1/57: fsync f10 0 2024-04-06T03:40:24.646 INFO:tasks.workunit.client.1.smithi136.stdout:2/96: rmdir d3/d9/d10/d18/d1d 39 2024-04-06T03:40:24.646 INFO:tasks.workunit.client.1.smithi136.stdout:2/97: rename d3/d9 to d3/d9/d10/d18/d1d/d24 22 2024-04-06T03:40:24.646 INFO:tasks.workunit.client.1.smithi136.stdout:2/98: chown d3/d9/d10/d18/d1d/c21 9683 1 2024-04-06T03:40:24.646 INFO:tasks.workunit.client.1.smithi136.stdout:2/99: dread - d3/d9/d10/d18/f23 zero size 2024-04-06T03:40:24.646 INFO:tasks.workunit.client.1.smithi136.stdout:2/100: mkdir d3/d15/d19/d25 0 2024-04-06T03:40:24.648 INFO:tasks.workunit.client.1.smithi136.stdout:2/101: mknod d3/c26 0 2024-04-06T03:40:24.654 INFO:tasks.workunit.client.0.smithi094.stdout:8/92: dwrite f11 [4194304,4194304] 0 2024-04-06T03:40:24.664 INFO:tasks.workunit.client.1.smithi136.stdout:2/102: dwrite d3/d9/d10/f17 [0,4194304] 0 2024-04-06T03:40:24.664 INFO:tasks.workunit.client.1.smithi136.stdout:2/103: chown d3/d9/cf 374 1 2024-04-06T03:40:24.664 INFO:tasks.workunit.client.1.smithi136.stdout:2/104: chown d3/d15/l1f 517268 1 2024-04-06T03:40:24.664 INFO:tasks.workunit.client.1.smithi136.stdout:2/105: dread - d3/d9/d10/f22 zero size 2024-04-06T03:40:24.665 INFO:tasks.workunit.client.1.smithi136.stdout:2/106: dread - d3/d9/d10/f22 zero size 2024-04-06T03:40:24.665 INFO:tasks.workunit.client.1.smithi136.stdout:2/107: read - d3/d9/d10/d18/f23 zero size 2024-04-06T03:40:24.665 INFO:tasks.workunit.client.1.smithi136.stdout:2/108: readlink l1 0 2024-04-06T03:40:24.665 INFO:tasks.workunit.client.1.smithi136.stdout:2/109: mknod d3/d9/d10/c27 0 2024-04-06T03:40:24.672 INFO:tasks.workunit.client.0.smithi094.stdout:8/93: dwrite d18/f1b [0,4194304] 0 2024-04-06T03:40:24.677 INFO:tasks.workunit.client.0.smithi094.stdout:8/94: truncate d18/f1c 2307107 0 2024-04-06T03:40:24.677 INFO:tasks.workunit.client.0.smithi094.stdout:8/95: readlink lc 0 2024-04-06T03:40:24.688 INFO:tasks.workunit.client.0.smithi094.stdout:8/96: dwrite fd [0,4194304] 0 2024-04-06T03:40:24.689 INFO:tasks.workunit.client.0.smithi094.stdout:8/97: creat d18/f22 x:0 0 0 2024-04-06T03:40:24.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.724+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc0c41058b0 con 0x7fc0c4066e90 2024-04-06T03:40:24.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.725+0000 7fc0b9ffb700 1 -- 172.21.15.94:0/3570089268 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc0c00575f0 con 0x7fc0c4066e90 2024-04-06T03:40:24.728 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc0b006b580 msgr2=0x7fc0b006da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc0b006b580 0x7fc0b006da30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4013870 tx=0x7fc0b4009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c4066e90 msgr2=0x7fc0c40fd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c4066e90 0x7fc0c40fd220 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000eb70 tx=0x7fc0c000b040 comp rx=0 tx=0).stop 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 shutdown_connections 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0c4066e90 0x7fc0c40fd220 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc0b006b580 0x7fc0b006da30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 --2- 172.21.15.94:0/3570089268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0c40fff60 0x7fc0c40fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 >> 172.21.15.94:0/3570089268 conn(0x7fc0c40f7430 msgr2=0x7fc0c40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 shutdown_connections 2024-04-06T03:40:24.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:24.727+0000 7fc0cb6a5700 1 -- 172.21.15.94:0/3570089268 wait complete. 2024-04-06T03:40:25.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:24 smithi136 ceph-mon[36436]: pgmap v12: 65 pgs: 65 active+clean; 276 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 654 op/s 2024-04-06T03:40:25.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:24 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3570089268' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:40:25.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:24 smithi094 ceph-mon[29713]: pgmap v12: 65 pgs: 65 active+clean; 276 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 654 op/s 2024-04-06T03:40:25.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:24 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3570089268' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:40:25.798 INFO:tasks.workunit.client.0.smithi094.stdout:5/48: sync 2024-04-06T03:40:25.798 INFO:tasks.workunit.client.0.smithi094.stdout:9/34: sync 2024-04-06T03:40:25.798 INFO:tasks.workunit.client.0.smithi094.stdout:2/64: sync 2024-04-06T03:40:25.798 INFO:tasks.workunit.client.0.smithi094.stdout:8/98: sync 2024-04-06T03:40:25.798 INFO:tasks.workunit.client.0.smithi094.stdout:4/47: sync 2024-04-06T03:40:25.798 INFO:tasks.workunit.client.0.smithi094.stdout:2/65: chown d0/c7 105276561 1 2024-04-06T03:40:25.798 INFO:tasks.workunit.client.0.smithi094.stdout:9/35: dread - d2/f9 zero size 2024-04-06T03:40:25.798 INFO:tasks.workunit.client.0.smithi094.stdout:4/48: write d0/d4/f6 [478183,5803] 0 2024-04-06T03:40:25.798 INFO:tasks.workunit.client.0.smithi094.stdout:4/49: chown d0/d4/fd 5623496 1 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:6/13: sync 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:9/36: creat d2/fb x:0 0 0 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:0/49: sync 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:0/50: truncate d1/d9/fc 922482 0 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:1/68: sync 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:3/47: sync 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:7/29: sync 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:3/48: fsync f8 0 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:7/30: fsync d2/f4 0 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:7/31: write d2/d6/f9 [786475,62054] 0 2024-04-06T03:40:25.799 INFO:tasks.workunit.client.0.smithi094.stdout:7/32: stat d2/d6/f8 0 2024-04-06T03:40:25.800 INFO:tasks.workunit.client.0.smithi094.stdout:7/33: read d2/d6/f9 [448640,44544] 0 2024-04-06T03:40:25.800 INFO:tasks.workunit.client.0.smithi094.stdout:5/49: rename f1 to db/f11 0 2024-04-06T03:40:25.800 INFO:tasks.workunit.client.0.smithi094.stdout:5/50: fdatasync f9 0 2024-04-06T03:40:25.800 INFO:tasks.workunit.client.0.smithi094.stdout:5/51: dread - db/fc zero size 2024-04-06T03:40:25.800 INFO:tasks.workunit.client.0.smithi094.stdout:5/52: dread - db/ff zero size 2024-04-06T03:40:25.800 INFO:tasks.workunit.client.0.smithi094.stdout:5/53: dread - f6 zero size 2024-04-06T03:40:25.800 INFO:tasks.workunit.client.0.smithi094.stdout:2/66: dread d0/f5 [0,4194304] 0 2024-04-06T03:40:25.804 INFO:tasks.workunit.client.0.smithi094.stdout:4/50: mknod d0/ce 0 2024-04-06T03:40:25.806 INFO:tasks.workunit.client.0.smithi094.stdout:6/14: rename l0 to l4 0 2024-04-06T03:40:25.809 INFO:tasks.workunit.client.0.smithi094.stdout:3/49: dread f7 [0,4194304] 0 2024-04-06T03:40:25.812 INFO:tasks.workunit.client.0.smithi094.stdout:7/34: rename d2/f4 to d2/d6/fa 0 2024-04-06T03:40:25.813 INFO:tasks.workunit.client.0.smithi094.stdout:7/35: fdatasync d2/d6/f8 0 2024-04-06T03:40:25.813 INFO:tasks.workunit.client.0.smithi094.stdout:7/36: write d2/d6/f9 [1579805,21650] 0 2024-04-06T03:40:25.814 INFO:tasks.workunit.client.0.smithi094.stdout:2/67: mknod d0/dd/c14 0 2024-04-06T03:40:25.818 INFO:tasks.workunit.client.0.smithi094.stdout:6/15: link f2 f5 0 2024-04-06T03:40:25.820 INFO:tasks.workunit.client.0.smithi094.stdout:0/51: rmdir d1/d9 39 2024-04-06T03:40:25.822 INFO:tasks.workunit.client.0.smithi094.stdout:2/68: truncate d0/d8/f11 3317465 0 2024-04-06T03:40:25.827 INFO:tasks.workunit.client.0.smithi094.stdout:1/69: dwrite d0/f19 [0,4194304] 0 2024-04-06T03:40:25.830 INFO:tasks.workunit.client.0.smithi094.stdout:2/69: rename d0/l6 to d0/dd/l15 0 2024-04-06T03:40:25.830 INFO:tasks.workunit.client.0.smithi094.stdout:2/70: fsync d0/d8/fe 0 2024-04-06T03:40:25.831 INFO:tasks.workunit.client.0.smithi094.stdout:1/70: dread d0/d1/f18 [0,4194304] 0 2024-04-06T03:40:25.833 INFO:tasks.workunit.client.0.smithi094.stdout:3/50: dwrite f8 [0,4194304] 0 2024-04-06T03:40:25.836 INFO:tasks.workunit.client.0.smithi094.stdout:2/71: mknod d0/c16 0 2024-04-06T03:40:25.838 INFO:tasks.workunit.client.0.smithi094.stdout:1/71: dread d0/f19 [0,4194304] 0 2024-04-06T03:40:25.838 INFO:tasks.workunit.client.0.smithi094.stdout:1/72: chown d0/d1/f18 182156927 1 2024-04-06T03:40:25.838 INFO:tasks.workunit.client.0.smithi094.stdout:9/37: dwrite d2/f5 [0,4194304] 0 2024-04-06T03:40:25.839 INFO:tasks.workunit.client.0.smithi094.stdout:5/54: dwrite db/fc [0,4194304] 0 2024-04-06T03:40:25.840 INFO:tasks.workunit.client.0.smithi094.stdout:5/55: chown c4 43534 1 2024-04-06T03:40:25.841 INFO:tasks.workunit.client.0.smithi094.stdout:3/51: unlink f7 0 2024-04-06T03:40:25.843 INFO:tasks.workunit.client.0.smithi094.stdout:1/73: dread d0/f19 [0,4194304] 0 2024-04-06T03:40:25.843 INFO:tasks.workunit.client.0.smithi094.stdout:1/74: chown d0/d1 828069528 1 2024-04-06T03:40:25.843 INFO:tasks.workunit.client.0.smithi094.stdout:1/75: readlink d0/d1/l8 0 2024-04-06T03:40:25.848 INFO:tasks.workunit.client.0.smithi094.stdout:3/52: symlink d9/la 0 2024-04-06T03:40:25.850 INFO:tasks.workunit.client.0.smithi094.stdout:1/76: mkdir d0/d4/d1b 0 2024-04-06T03:40:25.851 INFO:tasks.workunit.client.0.smithi094.stdout:6/16: dwrite f2 [0,4194304] 0 2024-04-06T03:40:25.852 INFO:tasks.workunit.client.0.smithi094.stdout:7/37: dwrite d2/d6/f9 [0,4194304] 0 2024-04-06T03:40:25.852 INFO:tasks.workunit.client.0.smithi094.stdout:7/38: readlink d2/l7 0 2024-04-06T03:40:25.852 INFO:tasks.workunit.client.0.smithi094.stdout:7/39: creat d2/fb x:0 0 0 2024-04-06T03:40:25.855 INFO:tasks.workunit.client.0.smithi094.stdout:3/53: link l3 d9/lb 0 2024-04-06T03:40:25.857 INFO:tasks.workunit.client.0.smithi094.stdout:1/77: creat d0/d1/d16/f1c x:0 0 0 2024-04-06T03:40:25.857 INFO:tasks.workunit.client.0.smithi094.stdout:0/52: dwrite d1/d10/f13 [0,4194304] 0 2024-04-06T03:40:25.861 INFO:tasks.workunit.client.0.smithi094.stdout:2/72: dwrite d0/f5 [0,4194304] 0 2024-04-06T03:40:25.862 INFO:tasks.workunit.client.0.smithi094.stdout:2/73: readlink d0/l4 0 2024-04-06T03:40:25.862 INFO:tasks.workunit.client.0.smithi094.stdout:7/40: mkdir d2/d6/dc 0 2024-04-06T03:40:25.862 INFO:tasks.workunit.client.0.smithi094.stdout:7/41: fsync d2/d6/f8 0 2024-04-06T03:40:25.863 INFO:tasks.workunit.client.0.smithi094.stdout:3/54: creat d9/fc x:0 0 0 2024-04-06T03:40:25.865 INFO:tasks.workunit.client.0.smithi094.stdout:1/78: mkdir d0/d7/d1d 0 2024-04-06T03:40:25.865 INFO:tasks.workunit.client.0.smithi094.stdout:1/79: truncate d0/d4/f9 2269404 0 2024-04-06T03:40:25.868 INFO:tasks.workunit.client.0.smithi094.stdout:0/53: dread d1/f7 [0,4194304] 0 2024-04-06T03:40:25.869 INFO:tasks.workunit.client.0.smithi094.stdout:6/17: dwrite f5 [0,4194304] 0 2024-04-06T03:40:25.869 INFO:tasks.workunit.client.0.smithi094.stdout:6/18: chown c3 0 1 2024-04-06T03:40:25.870 INFO:tasks.workunit.client.0.smithi094.stdout:3/55: read f8 [1952397,55702] 0 2024-04-06T03:40:25.870 INFO:tasks.workunit.client.0.smithi094.stdout:3/56: write f5 [931520,88646] 0 2024-04-06T03:40:25.871 INFO:tasks.workunit.client.0.smithi094.stdout:3/57: write f5 [68672,103904] 0 2024-04-06T03:40:25.872 INFO:tasks.workunit.client.0.smithi094.stdout:7/42: mkdir d2/d6/dc/dd 0 2024-04-06T03:40:25.874 INFO:tasks.workunit.client.0.smithi094.stdout:1/80: unlink d0/d1/f18 0 2024-04-06T03:40:25.874 INFO:tasks.workunit.client.0.smithi094.stdout:1/81: dread - d0/d1/d16/f1c zero size 2024-04-06T03:40:25.876 INFO:tasks.workunit.client.0.smithi094.stdout:0/54: getdents d1 0 2024-04-06T03:40:25.876 INFO:tasks.workunit.client.0.smithi094.stdout:0/55: rename d1/d9 to d1/d9/d15 22 2024-04-06T03:40:25.876 INFO:tasks.workunit.client.0.smithi094.stdout:0/56: write d1/d9/f14 [1132747,49563] 0 2024-04-06T03:40:25.876 INFO:tasks.workunit.client.0.smithi094.stdout:0/57: write d1/f7 [5224505,47924] 0 2024-04-06T03:40:25.877 INFO:tasks.workunit.client.0.smithi094.stdout:7/43: dread d2/d6/fa [0,4194304] 0 2024-04-06T03:40:25.877 INFO:tasks.workunit.client.0.smithi094.stdout:6/19: mknod c6 0 2024-04-06T03:40:25.882 INFO:tasks.workunit.client.0.smithi094.stdout:3/58: unlink d9/la 0 2024-04-06T03:40:25.882 INFO:tasks.workunit.client.0.smithi094.stdout:3/59: write d9/fc [674776,49067] 0 2024-04-06T03:40:25.882 INFO:tasks.workunit.client.0.smithi094.stdout:3/60: chown d9/lb 10 1 2024-04-06T03:40:25.882 INFO:tasks.workunit.client.0.smithi094.stdout:3/61: read f5 [158675,64278] 0 2024-04-06T03:40:25.883 INFO:tasks.workunit.client.0.smithi094.stdout:1/82: write d0/d4/fc [473786,34075] 0 2024-04-06T03:40:25.883 INFO:tasks.workunit.client.0.smithi094.stdout:1/83: stat d0 0 2024-04-06T03:40:25.885 INFO:tasks.workunit.client.0.smithi094.stdout:0/58: link d1/d9/f14 d1/d9/dd/f16 0 2024-04-06T03:40:25.888 INFO:tasks.workunit.client.0.smithi094.stdout:3/62: link d9/fc d9/fd 0 2024-04-06T03:40:25.888 INFO:tasks.workunit.client.0.smithi094.stdout:3/63: write d9/fc [982267,11388] 0 2024-04-06T03:40:25.891 INFO:tasks.workunit.client.0.smithi094.stdout:1/84: creat d0/d7/d1a/f1e x:0 0 0 2024-04-06T03:40:25.893 INFO:tasks.workunit.client.0.smithi094.stdout:0/59: dread d1/f7 [0,4194304] 0 2024-04-06T03:40:25.893 INFO:tasks.workunit.client.0.smithi094.stdout:0/60: fdatasync d1/f7 0 2024-04-06T03:40:25.893 INFO:tasks.workunit.client.0.smithi094.stdout:0/61: write d1/f5 [903025,20735] 0 2024-04-06T03:40:25.895 INFO:tasks.workunit.client.0.smithi094.stdout:7/44: dwrite d2/d6/fa [0,4194304] 0 2024-04-06T03:40:25.900 INFO:tasks.workunit.client.0.smithi094.stdout:0/62: creat d1/d9/f17 x:0 0 0 2024-04-06T03:40:25.900 INFO:tasks.workunit.client.0.smithi094.stdout:0/63: fdatasync d1/d9/fc 0 2024-04-06T03:40:25.910 INFO:tasks.workunit.client.0.smithi094.stdout:7/45: symlink d2/d6/le 0 2024-04-06T03:40:25.912 INFO:tasks.workunit.client.0.smithi094.stdout:0/64: dwrite d1/d9/fc [0,4194304] 0 2024-04-06T03:40:25.923 INFO:tasks.workunit.client.0.smithi094.stdout:0/65: rmdir d1/d9/dd 39 2024-04-06T03:40:25.924 INFO:tasks.workunit.client.0.smithi094.stdout:0/66: write d1/d9/f17 [1005749,103803] 0 2024-04-06T03:40:25.926 INFO:tasks.workunit.client.0.smithi094.stdout:0/67: mkdir d1/d9/dd/d18 0 2024-04-06T03:40:25.929 INFO:tasks.workunit.client.0.smithi094.stdout:0/68: symlink d1/d9/l19 0 2024-04-06T03:40:25.930 INFO:tasks.workunit.client.0.smithi094.stdout:0/69: rmdir d1/d9/dd 39 2024-04-06T03:40:25.930 INFO:tasks.workunit.client.0.smithi094.stdout:0/70: readlink d1/d9/l19 0 2024-04-06T03:40:25.933 INFO:tasks.workunit.client.0.smithi094.stdout:0/71: mkdir d1/d9/dd/d18/d1a 0 2024-04-06T03:40:25.933 INFO:tasks.workunit.client.0.smithi094.stdout:0/72: fsync d1/fa 0 2024-04-06T03:40:25.933 INFO:tasks.workunit.client.0.smithi094.stdout:0/73: chown d1/d9 7280710 1 2024-04-06T03:40:25.933 INFO:tasks.workunit.client.0.smithi094.stdout:0/74: truncate d1/f5 1929451 0 2024-04-06T03:40:25.941 INFO:tasks.workunit.client.0.smithi094.stdout:0/75: dread d1/d10/f13 [0,4194304] 0 2024-04-06T03:40:25.944 INFO:tasks.workunit.client.0.smithi094.stdout:0/76: link d1/d9/c12 d1/d9/dd/c1b 0 2024-04-06T03:40:25.944 INFO:tasks.workunit.client.0.smithi094.stdout:0/77: truncate d1/f5 2139693 0 2024-04-06T03:40:25.944 INFO:tasks.workunit.client.0.smithi094.stdout:0/78: read d1/d9/f14 [790135,92270] 0 2024-04-06T03:40:26.088 INFO:tasks.workunit.client.0.smithi094.stdout:5/56: getdents db 0 2024-04-06T03:40:26.091 INFO:tasks.workunit.client.0.smithi094.stdout:5/57: creat db/f12 x:0 0 0 2024-04-06T03:40:26.092 INFO:tasks.workunit.client.0.smithi094.stdout:4/51: getdents d0 0 2024-04-06T03:40:26.092 INFO:tasks.workunit.client.0.smithi094.stdout:4/52: write d0/d4/fd [896150,85277] 0 2024-04-06T03:40:26.094 INFO:tasks.workunit.client.0.smithi094.stdout:5/58: truncate db/fc 5230439 0 2024-04-06T03:40:26.094 INFO:tasks.workunit.client.0.smithi094.stdout:4/53: dread d0/f8 [0,4194304] 0 2024-04-06T03:40:26.094 INFO:tasks.workunit.client.0.smithi094.stdout:4/54: chown d0/fa 2700874 1 2024-04-06T03:40:26.101 INFO:tasks.workunit.client.0.smithi094.stdout:5/59: mkdir db/d13 0 2024-04-06T03:40:26.102 INFO:tasks.workunit.client.0.smithi094.stdout:4/55: rmdir d0/d4 39 2024-04-06T03:40:26.106 INFO:tasks.workunit.client.0.smithi094.stdout:4/56: mkdir d0/df 0 2024-04-06T03:40:26.111 INFO:tasks.workunit.client.0.smithi094.stdout:4/57: symlink d0/l10 0 2024-04-06T03:40:26.111 INFO:tasks.workunit.client.0.smithi094.stdout:2/74: write d0/d8/f11 [2078489,76744] 0 2024-04-06T03:40:26.113 INFO:tasks.workunit.client.0.smithi094.stdout:5/60: link db/ff db/f14 0 2024-04-06T03:40:26.116 INFO:tasks.workunit.client.0.smithi094.stdout:2/75: dread d0/d8/f11 [0,4194304] 0 2024-04-06T03:40:26.116 INFO:tasks.workunit.client.0.smithi094.stdout:2/76: truncate d0/f5 5045954 0 2024-04-06T03:40:26.116 INFO:tasks.workunit.client.0.smithi094.stdout:2/77: write d0/d8/f11 [1832195,37954] 0 2024-04-06T03:40:26.116 INFO:tasks.workunit.client.0.smithi094.stdout:2/78: chown d0/f5 3 1 2024-04-06T03:40:26.117 INFO:tasks.workunit.client.0.smithi094.stdout:4/58: mknod d0/d4/c11 0 2024-04-06T03:40:26.121 INFO:tasks.workunit.client.0.smithi094.stdout:4/59: dread d0/d4/f6 [0,4194304] 0 2024-04-06T03:40:26.124 INFO:tasks.workunit.client.0.smithi094.stdout:4/60: link d0/f8 d0/f12 0 2024-04-06T03:40:26.128 INFO:tasks.workunit.client.0.smithi094.stdout:5/61: link db/c10 db/c15 0 2024-04-06T03:40:26.128 INFO:tasks.workunit.client.0.smithi094.stdout:5/62: rename db to db/d13/d16 22 2024-04-06T03:40:26.128 INFO:tasks.workunit.client.0.smithi094.stdout:5/63: truncate db/f12 356870 0 2024-04-06T03:40:26.131 INFO:tasks.workunit.client.0.smithi094.stdout:5/64: mknod db/d13/c17 0 2024-04-06T03:40:26.131 INFO:tasks.workunit.client.0.smithi094.stdout:5/65: fdatasync db/fc 0 2024-04-06T03:40:26.134 INFO:tasks.workunit.client.0.smithi094.stdout:5/66: symlink db/d13/l18 0 2024-04-06T03:40:26.134 INFO:tasks.workunit.client.0.smithi094.stdout:5/67: chown c8 295349 1 2024-04-06T03:40:26.147 INFO:tasks.workunit.client.0.smithi094.stdout:5/68: dwrite db/f14 [0,4194304] 0 2024-04-06T03:40:26.154 INFO:tasks.workunit.client.0.smithi094.stdout:1/85: getdents d0/d1 0 2024-04-06T03:40:26.157 INFO:tasks.workunit.client.0.smithi094.stdout:1/86: mknod d0/d7/c1f 0 2024-04-06T03:40:26.160 INFO:tasks.workunit.client.0.smithi094.stdout:1/87: symlink d0/d1/l20 0 2024-04-06T03:40:26.160 INFO:tasks.workunit.client.0.smithi094.stdout:5/69: dwrite db/ff [0,4194304] 0 2024-04-06T03:40:26.163 INFO:tasks.workunit.client.0.smithi094.stdout:1/88: rename d0/d1/l20 to d0/d1/l21 0 2024-04-06T03:40:26.166 INFO:tasks.workunit.client.0.smithi094.stdout:1/89: symlink d0/d7/d17/l22 0 2024-04-06T03:40:26.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:25 smithi094 ceph-mon[29713]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:26.168 INFO:tasks.workunit.client.0.smithi094.stdout:5/70: mkdir db/d19 0 2024-04-06T03:40:26.171 INFO:tasks.workunit.client.0.smithi094.stdout:6/20: truncate f5 953239 0 2024-04-06T03:40:26.171 INFO:tasks.workunit.client.0.smithi094.stdout:6/21: creat f7 x:0 0 0 2024-04-06T03:40:26.171 INFO:tasks.workunit.client.0.smithi094.stdout:6/22: chown c6 0 1 2024-04-06T03:40:26.171 INFO:tasks.workunit.client.0.smithi094.stdout:5/71: unlink db/f14 0 2024-04-06T03:40:26.178 INFO:tasks.workunit.client.0.smithi094.stdout:5/72: link c4 db/d13/c1a 0 2024-04-06T03:40:26.194 INFO:tasks.workunit.client.0.smithi094.stdout:5/73: dwrite db/fc [0,4194304] 0 2024-04-06T03:40:26.198 INFO:tasks.workunit.client.0.smithi094.stdout:5/74: write db/fd [1130336,118132] 0 2024-04-06T03:40:26.201 INFO:tasks.workunit.client.0.smithi094.stdout:5/75: rename db/d13/c17 to db/d19/c1b 0 2024-04-06T03:40:26.203 INFO:tasks.workunit.client.0.smithi094.stdout:5/76: mknod db/d13/c1c 0 2024-04-06T03:40:26.209 INFO:tasks.workunit.client.0.smithi094.stdout:0/79: getdents d1/d9/dd/d18 0 2024-04-06T03:40:26.210 INFO:tasks.workunit.client.0.smithi094.stdout:0/80: read d1/f5 [1385050,65744] 0 2024-04-06T03:40:26.210 INFO:tasks.workunit.client.0.smithi094.stdout:0/81: write d1/f7 [5821061,76299] 0 2024-04-06T03:40:26.217 INFO:tasks.workunit.client.0.smithi094.stdout:0/82: read d1/d9/fc [2470363,71333] 0 2024-04-06T03:40:26.218 INFO:tasks.workunit.client.0.smithi094.stdout:5/77: dwrite db/f12 [0,4194304] 0 2024-04-06T03:40:26.220 INFO:tasks.workunit.client.0.smithi094.stdout:0/83: creat d1/d9/dd/f1c x:0 0 0 2024-04-06T03:40:26.224 INFO:tasks.workunit.client.0.smithi094.stdout:5/78: creat db/f1d x:0 0 0 2024-04-06T03:40:26.226 INFO:tasks.workunit.client.0.smithi094.stdout:0/84: unlink d1/c4 0 2024-04-06T03:40:26.227 INFO:tasks.workunit.client.0.smithi094.stdout:5/79: rmdir db/d13 39 2024-04-06T03:40:26.227 INFO:tasks.workunit.client.0.smithi094.stdout:5/80: dread - f6 zero size 2024-04-06T03:40:26.228 INFO:tasks.workunit.client.0.smithi094.stdout:0/85: truncate d1/d10/f13 3109271 0 2024-04-06T03:40:26.230 INFO:tasks.workunit.client.0.smithi094.stdout:5/81: creat db/f1e x:0 0 0 2024-04-06T03:40:26.232 INFO:tasks.workunit.client.0.smithi094.stdout:5/82: creat db/d19/f1f x:0 0 0 2024-04-06T03:40:26.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:25 smithi136 ceph-mon[36436]: from='client.24417 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:26.367 INFO:tasks.workunit.client.0.smithi094.stdout:4/61: getdents d0 0 2024-04-06T03:40:26.370 INFO:tasks.workunit.client.0.smithi094.stdout:2/79: truncate d0/d8/f11 675263 0 2024-04-06T03:40:26.370 INFO:tasks.workunit.client.0.smithi094.stdout:4/62: mkdir d0/df/d13 0 2024-04-06T03:40:26.372 INFO:tasks.workunit.client.0.smithi094.stdout:2/80: symlink d0/l17 0 2024-04-06T03:40:26.374 INFO:tasks.workunit.client.0.smithi094.stdout:4/63: getdents d0/d4 0 2024-04-06T03:40:26.375 INFO:tasks.workunit.client.0.smithi094.stdout:4/64: chown d0/f12 6326 1 2024-04-06T03:40:26.376 INFO:tasks.workunit.client.0.smithi094.stdout:4/65: write d0/f8 [47055,89482] 0 2024-04-06T03:40:26.376 INFO:tasks.workunit.client.0.smithi094.stdout:4/66: fsync d0/d4/fd 0 2024-04-06T03:40:26.376 INFO:tasks.workunit.client.0.smithi094.stdout:4/67: chown d0/df/d13 694208 1 2024-04-06T03:40:26.376 INFO:tasks.workunit.client.0.smithi094.stdout:4/68: getdents d0 0 2024-04-06T03:40:26.385 INFO:tasks.workunit.client.0.smithi094.stdout:4/69: dread d0/f12 [0,4194304] 0 2024-04-06T03:40:26.387 INFO:tasks.workunit.client.0.smithi094.stdout:4/70: dread d0/f2 [0,4194304] 0 2024-04-06T03:40:26.388 INFO:tasks.workunit.client.0.smithi094.stdout:2/81: dwrite d0/d8/fe [0,4194304] 0 2024-04-06T03:40:26.388 INFO:tasks.workunit.client.0.smithi094.stdout:2/82: readlink d0/dd/l15 0 2024-04-06T03:40:26.389 INFO:tasks.workunit.client.0.smithi094.stdout:2/83: dread d0/dd/ff [0,4194304] 0 2024-04-06T03:40:26.389 INFO:tasks.workunit.client.0.smithi094.stdout:2/84: fdatasync d0/f5 0 2024-04-06T03:40:26.389 INFO:tasks.workunit.client.0.smithi094.stdout:2/85: chown d0/d8/l13 60 1 2024-04-06T03:40:26.390 INFO:tasks.workunit.client.0.smithi094.stdout:4/71: creat d0/d4/f14 x:0 0 0 2024-04-06T03:40:26.390 INFO:tasks.workunit.client.0.smithi094.stdout:4/72: write d0/d4/f9 [2277578,77846] 0 2024-04-06T03:40:26.394 INFO:tasks.workunit.client.0.smithi094.stdout:4/73: mkdir d0/d15 0 2024-04-06T03:40:26.395 INFO:tasks.workunit.client.0.smithi094.stdout:1/90: getdents d0/d7 0 2024-04-06T03:40:26.397 INFO:tasks.workunit.client.0.smithi094.stdout:4/74: rename d0/l10 to d0/d4/l16 0 2024-04-06T03:40:26.398 INFO:tasks.workunit.client.0.smithi094.stdout:4/75: write d0/d4/fb [492830,37545] 0 2024-04-06T03:40:26.400 INFO:tasks.workunit.client.0.smithi094.stdout:1/91: symlink d0/d7/d17/l23 0 2024-04-06T03:40:26.403 INFO:tasks.workunit.client.0.smithi094.stdout:4/76: mknod d0/d15/c17 0 2024-04-06T03:40:26.406 INFO:tasks.workunit.client.0.smithi094.stdout:4/77: write d0/d4/fd [27265,6520] 0 2024-04-06T03:40:26.406 INFO:tasks.workunit.client.0.smithi094.stdout:1/92: rename d0/d7/d1d to d0/d1/d16/d24 0 2024-04-06T03:40:26.406 INFO:tasks.workunit.client.0.smithi094.stdout:1/93: write d0/d1/d16/f1c [443513,44181] 0 2024-04-06T03:40:26.407 INFO:tasks.workunit.client.0.smithi094.stdout:2/86: dwrite d0/f5 [0,4194304] 0 2024-04-06T03:40:26.411 INFO:tasks.workunit.client.0.smithi094.stdout:8/99: sync 2024-04-06T03:40:26.411 INFO:tasks.workunit.client.0.smithi094.stdout:9/38: sync 2024-04-06T03:40:26.411 INFO:tasks.workunit.client.0.smithi094.stdout:7/46: sync 2024-04-06T03:40:26.411 INFO:tasks.workunit.client.0.smithi094.stdout:3/64: sync 2024-04-06T03:40:26.412 INFO:tasks.workunit.client.0.smithi094.stdout:7/47: chown d2/d6/dc/dd 152461632 1 2024-04-06T03:40:26.413 INFO:tasks.workunit.client.0.smithi094.stdout:4/78: creat d0/df/d13/f18 x:0 0 0 2024-04-06T03:40:26.423 INFO:tasks.workunit.client.0.smithi094.stdout:1/94: dwrite d0/f14 [0,4194304] 0 2024-04-06T03:40:26.426 INFO:tasks.workunit.client.0.smithi094.stdout:2/87: mkdir d0/d18 0 2024-04-06T03:40:26.427 INFO:tasks.workunit.client.0.smithi094.stdout:8/100: mknod d18/c23 0 2024-04-06T03:40:26.429 INFO:tasks.workunit.client.0.smithi094.stdout:3/65: mknod d9/ce 0 2024-04-06T03:40:26.433 INFO:tasks.workunit.client.0.smithi094.stdout:9/39: mkdir d2/dc 0 2024-04-06T03:40:26.434 INFO:tasks.workunit.client.0.smithi094.stdout:7/48: truncate d2/d6/fa 1818517 0 2024-04-06T03:40:26.436 INFO:tasks.workunit.client.0.smithi094.stdout:6/23: rename f2 to f8 0 2024-04-06T03:40:26.437 INFO:tasks.workunit.client.0.smithi094.stdout:6/24: stat f7 0 2024-04-06T03:40:26.438 INFO:tasks.workunit.client.0.smithi094.stdout:1/95: dwrite d0/d7/d1a/f1e [0,4194304] 0 2024-04-06T03:40:26.438 INFO:tasks.workunit.client.0.smithi094.stdout:4/79: unlink d0/f2 0 2024-04-06T03:40:26.445 INFO:tasks.workunit.client.0.smithi094.stdout:3/66: dwrite d9/fc [0,4194304] 0 2024-04-06T03:40:26.445 INFO:tasks.workunit.client.0.smithi094.stdout:3/67: chown f5 778 1 2024-04-06T03:40:26.445 INFO:tasks.workunit.client.0.smithi094.stdout:3/68: creat d9/ff x:0 0 0 2024-04-06T03:40:26.447 INFO:tasks.workunit.client.0.smithi094.stdout:8/101: rename l8 to d18/l24 0 2024-04-06T03:40:26.447 INFO:tasks.workunit.client.0.smithi094.stdout:8/102: creat d18/f25 x:0 0 0 2024-04-06T03:40:26.448 INFO:tasks.workunit.client.0.smithi094.stdout:9/40: mkdir d2/dc/dd 0 2024-04-06T03:40:26.451 INFO:tasks.workunit.client.0.smithi094.stdout:7/49: mkdir d2/d6/dc/dd/df 0 2024-04-06T03:40:26.454 INFO:tasks.workunit.client.0.smithi094.stdout:6/25: mknod c9 0 2024-04-06T03:40:26.454 INFO:tasks.workunit.client.0.smithi094.stdout:6/26: creat fa x:0 0 0 2024-04-06T03:40:26.454 INFO:tasks.workunit.client.0.smithi094.stdout:6/27: chown c3 97 1 2024-04-06T03:40:26.460 INFO:tasks.workunit.client.0.smithi094.stdout:7/50: write d2/d6/f9 [3043867,109554] 0 2024-04-06T03:40:26.462 INFO:tasks.workunit.client.0.smithi094.stdout:9/41: dwrite d2/fb [0,4194304] 0 2024-04-06T03:40:26.463 INFO:tasks.workunit.client.0.smithi094.stdout:4/80: write d0/d4/f9 [2449236,88779] 0 2024-04-06T03:40:26.463 INFO:tasks.workunit.client.0.smithi094.stdout:4/81: dread - d0/d4/f14 zero size 2024-04-06T03:40:26.463 INFO:tasks.workunit.client.0.smithi094.stdout:4/82: chown d0/d4/fb 73 1 2024-04-06T03:40:26.468 INFO:tasks.workunit.client.0.smithi094.stdout:3/69: mkdir d9/d10 0 2024-04-06T03:40:26.470 INFO:tasks.workunit.client.0.smithi094.stdout:3/70: dread f5 [0,4194304] 0 2024-04-06T03:40:26.470 INFO:tasks.workunit.client.0.smithi094.stdout:3/71: chown l3 48832 1 2024-04-06T03:40:26.471 INFO:tasks.workunit.client.0.smithi094.stdout:3/72: write d9/ff [248785,14054] 0 2024-04-06T03:40:26.471 INFO:tasks.workunit.client.0.smithi094.stdout:3/73: fdatasync f8 0 2024-04-06T03:40:26.472 INFO:tasks.workunit.client.0.smithi094.stdout:8/103: symlink d18/l26 0 2024-04-06T03:40:26.476 INFO:tasks.workunit.client.0.smithi094.stdout:6/28: mknod cb 0 2024-04-06T03:40:26.485 INFO:tasks.workunit.client.0.smithi094.stdout:9/42: unlink d2/l6 0 2024-04-06T03:40:26.485 INFO:tasks.workunit.client.0.smithi094.stdout:9/43: creat d2/fe x:0 0 0 2024-04-06T03:40:26.489 INFO:tasks.workunit.client.0.smithi094.stdout:9/44: creat d2/dc/dd/ff x:0 0 0 2024-04-06T03:40:26.489 INFO:tasks.workunit.client.0.smithi094.stdout:9/45: chown d2 1017147 1 2024-04-06T03:40:26.500 INFO:tasks.workunit.client.0.smithi094.stdout:0/86: dwrite d1/d10/f13 [0,4194304] 0 2024-04-06T03:40:26.501 INFO:tasks.workunit.client.0.smithi094.stdout:0/87: truncate d1/d10/f13 4944785 0 2024-04-06T03:40:26.501 INFO:tasks.workunit.client.0.smithi094.stdout:3/74: dwrite d9/fc [0,4194304] 0 2024-04-06T03:40:26.502 INFO:tasks.workunit.client.0.smithi094.stdout:3/75: fsync d9/fc 0 2024-04-06T03:40:26.502 INFO:tasks.workunit.client.0.smithi094.stdout:0/88: dread d1/d9/f17 [0,4194304] 0 2024-04-06T03:40:26.503 INFO:tasks.workunit.client.0.smithi094.stdout:6/29: dwrite f5 [0,4194304] 0 2024-04-06T03:40:26.507 INFO:tasks.workunit.client.0.smithi094.stdout:8/104: dwrite d18/f1a [0,4194304] 0 2024-04-06T03:40:26.509 INFO:tasks.workunit.client.0.smithi094.stdout:6/30: mkdir dc 0 2024-04-06T03:40:26.509 INFO:tasks.workunit.client.0.smithi094.stdout:6/31: chown f7 49 1 2024-04-06T03:40:26.514 INFO:tasks.workunit.client.0.smithi094.stdout:8/105: read fd [898712,113241] 0 2024-04-06T03:40:26.514 INFO:tasks.workunit.client.0.smithi094.stdout:3/76: read d9/fd [4037494,64937] 0 2024-04-06T03:40:26.525 INFO:tasks.workunit.client.0.smithi094.stdout:6/32: dread f5 [0,4194304] 0 2024-04-06T03:40:26.526 INFO:tasks.workunit.client.0.smithi094.stdout:6/33: creat dc/fd x:0 0 0 2024-04-06T03:40:26.533 INFO:tasks.workunit.client.0.smithi094.stdout:0/89: dwrite d1/f5 [0,4194304] 0 2024-04-06T03:40:26.535 INFO:tasks.workunit.client.0.smithi094.stdout:3/77: dwrite d9/fd [0,4194304] 0 2024-04-06T03:40:26.541 INFO:tasks.workunit.client.0.smithi094.stdout:3/78: mknod d9/d10/c11 0 2024-04-06T03:40:26.546 INFO:tasks.workunit.client.0.smithi094.stdout:0/90: dwrite d1/fa [0,4194304] 0 2024-04-06T03:40:26.548 INFO:tasks.workunit.client.0.smithi094.stdout:0/91: unlink d1/d10/f13 0 2024-04-06T03:40:26.551 INFO:tasks.workunit.client.0.smithi094.stdout:0/92: unlink d1/d9/ce 0 2024-04-06T03:40:26.551 INFO:tasks.workunit.client.0.smithi094.stdout:0/93: stat d1/d9/dd/d18 0 2024-04-06T03:40:26.551 INFO:tasks.workunit.client.0.smithi094.stdout:3/79: write d9/fd [3836051,93083] 0 2024-04-06T03:40:26.553 INFO:tasks.workunit.client.0.smithi094.stdout:0/94: mkdir d1/d9/dd/d18/d1d 0 2024-04-06T03:40:26.553 INFO:tasks.workunit.client.0.smithi094.stdout:0/95: dread - d1/d9/dd/f1c zero size 2024-04-06T03:40:26.554 INFO:tasks.workunit.client.0.smithi094.stdout:3/80: dread f5 [0,4194304] 0 2024-04-06T03:40:26.557 INFO:tasks.workunit.client.0.smithi094.stdout:0/96: unlink d1/fa 0 2024-04-06T03:40:26.557 INFO:tasks.workunit.client.0.smithi094.stdout:3/81: creat d9/d10/f12 x:0 0 0 2024-04-06T03:40:26.560 INFO:tasks.workunit.client.0.smithi094.stdout:0/97: mkdir d1/d9/dd/d18/d1e 0 2024-04-06T03:40:26.561 INFO:tasks.workunit.client.0.smithi094.stdout:3/82: stat d9/lb 0 2024-04-06T03:40:26.562 INFO:tasks.workunit.client.0.smithi094.stdout:3/83: dread d9/ff [0,4194304] 0 2024-04-06T03:40:26.563 INFO:tasks.workunit.client.0.smithi094.stdout:0/98: unlink d1/d9/f17 0 2024-04-06T03:40:26.564 INFO:tasks.workunit.client.0.smithi094.stdout:3/84: symlink d9/l13 0 2024-04-06T03:40:26.567 INFO:tasks.workunit.client.0.smithi094.stdout:3/85: link d9/d10/c11 d9/c14 0 2024-04-06T03:40:26.567 INFO:tasks.workunit.client.0.smithi094.stdout:3/86: write d9/fc [39311,83682] 0 2024-04-06T03:40:26.567 INFO:tasks.workunit.client.0.smithi094.stdout:3/87: write d9/ff [1243525,60406] 0 2024-04-06T03:40:26.570 INFO:tasks.workunit.client.0.smithi094.stdout:3/88: rename f8 to d9/f15 0 2024-04-06T03:40:26.572 INFO:tasks.workunit.client.0.smithi094.stdout:3/89: rename f5 to d9/d10/f16 0 2024-04-06T03:40:26.574 INFO:tasks.workunit.client.0.smithi094.stdout:3/90: symlink d9/d10/l17 0 2024-04-06T03:40:26.575 INFO:tasks.workunit.client.0.smithi094.stdout:3/91: creat d9/f18 x:0 0 0 2024-04-06T03:40:26.575 INFO:tasks.workunit.client.0.smithi094.stdout:3/92: stat d9/d10/f16 0 2024-04-06T03:40:26.575 INFO:tasks.workunit.client.0.smithi094.stdout:3/93: creat d9/f19 x:0 0 0 2024-04-06T03:40:26.579 INFO:tasks.workunit.client.0.smithi094.stdout:3/94: symlink d9/d10/l1a 0 2024-04-06T03:40:26.587 INFO:tasks.workunit.client.0.smithi094.stdout:3/95: dread d9/fd [0,4194304] 0 2024-04-06T03:40:26.592 INFO:tasks.workunit.client.0.smithi094.stdout:3/96: dread d9/fc [0,4194304] 0 2024-04-06T03:40:26.594 INFO:tasks.workunit.client.0.smithi094.stdout:3/97: rename d9/d10/f12 to d9/f1b 0 2024-04-06T03:40:26.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/98: dwrite d9/fc [0,4194304] 0 2024-04-06T03:40:26.612 INFO:tasks.workunit.client.0.smithi094.stdout:3/99: symlink d9/l1c 0 2024-04-06T03:40:26.614 INFO:tasks.workunit.client.0.smithi094.stdout:3/100: mkdir d9/d1d 0 2024-04-06T03:40:26.616 INFO:tasks.workunit.client.0.smithi094.stdout:3/101: mkdir d9/d1e 0 2024-04-06T03:40:26.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/102: mknod d9/d1e/c1f 0 2024-04-06T03:40:26.626 INFO:tasks.workunit.client.0.smithi094.stdout:3/103: dread d9/fd [0,4194304] 0 2024-04-06T03:40:26.637 INFO:tasks.workunit.client.0.smithi094.stdout:2/88: fsync d0/d8/f11 0 2024-04-06T03:40:26.641 INFO:tasks.workunit.client.0.smithi094.stdout:2/89: mkdir d0/dd/d19 0 2024-04-06T03:40:26.641 INFO:tasks.workunit.client.0.smithi094.stdout:2/90: chown d0/dd/l12 54611 1 2024-04-06T03:40:26.653 INFO:tasks.workunit.client.0.smithi094.stdout:2/91: dwrite d0/dd/ff [0,4194304] 0 2024-04-06T03:40:26.662 INFO:tasks.workunit.client.0.smithi094.stdout:2/92: write d0/f1 [2509415,102385] 0 2024-04-06T03:40:26.664 INFO:tasks.workunit.client.0.smithi094.stdout:2/93: mkdir d0/dd/d1a 0 2024-04-06T03:40:26.666 INFO:tasks.workunit.client.0.smithi094.stdout:2/94: mkdir d0/dd/d1b 0 2024-04-06T03:40:26.668 INFO:tasks.workunit.client.0.smithi094.stdout:1/96: rmdir d0/d1/d16 39 2024-04-06T03:40:26.668 INFO:tasks.workunit.client.0.smithi094.stdout:1/97: readlink d0/d7/d17/l22 0 2024-04-06T03:40:26.669 INFO:tasks.workunit.client.0.smithi094.stdout:1/98: dread - d0/d1/f10 zero size 2024-04-06T03:40:26.669 INFO:tasks.workunit.client.0.smithi094.stdout:2/95: creat d0/dd/d1b/f1c x:0 0 0 2024-04-06T03:40:26.672 INFO:tasks.workunit.client.0.smithi094.stdout:2/96: mkdir d0/d8/d1d 0 2024-04-06T03:40:26.673 INFO:tasks.workunit.client.0.smithi094.stdout:4/83: fsync d0/df/d13/f18 0 2024-04-06T03:40:26.676 INFO:tasks.workunit.client.0.smithi094.stdout:2/97: rmdir d0/dd/d1a 0 2024-04-06T03:40:26.676 INFO:tasks.workunit.client.0.smithi094.stdout:2/98: dread - d0/dd/d1b/f1c zero size 2024-04-06T03:40:26.677 INFO:tasks.workunit.client.0.smithi094.stdout:4/84: getdents d0/d15 0 2024-04-06T03:40:26.677 INFO:tasks.workunit.client.0.smithi094.stdout:4/85: write d0/d4/f6 [5986029,2470] 0 2024-04-06T03:40:26.677 INFO:tasks.workunit.client.0.smithi094.stdout:4/86: dread - d0/d4/f14 zero size 2024-04-06T03:40:26.683 INFO:tasks.workunit.client.0.smithi094.stdout:2/99: dread d0/dd/ff [0,4194304] 0 2024-04-06T03:40:26.683 INFO:tasks.workunit.client.0.smithi094.stdout:2/100: truncate d0/f1 6085104 0 2024-04-06T03:40:26.684 INFO:tasks.workunit.client.0.smithi094.stdout:1/99: dwrite d0/d1/f10 [0,4194304] 0 2024-04-06T03:40:26.684 INFO:tasks.workunit.client.0.smithi094.stdout:1/100: creat d0/d4/f25 x:0 0 0 2024-04-06T03:40:26.685 INFO:tasks.workunit.client.0.smithi094.stdout:2/101: link d0/dd/l12 d0/dd/d1b/l1e 0 2024-04-06T03:40:26.697 INFO:tasks.workunit.client.0.smithi094.stdout:6/34: fsync f5 0 2024-04-06T03:40:26.700 INFO:tasks.workunit.client.0.smithi094.stdout:6/35: symlink dc/le 0 2024-04-06T03:40:26.709 INFO:tasks.workunit.client.0.smithi094.stdout:2/102: dwrite d0/d8/fe [0,4194304] 0 2024-04-06T03:40:26.709 INFO:tasks.workunit.client.0.smithi094.stdout:2/103: chown d0/dd/d1b 11 1 2024-04-06T03:40:26.709 INFO:tasks.workunit.client.0.smithi094.stdout:1/101: dwrite d0/f14 [0,4194304] 0 2024-04-06T03:40:26.709 INFO:tasks.workunit.client.0.smithi094.stdout:1/102: write d0/d1/d16/f1c [1347238,104917] 0 2024-04-06T03:40:26.712 INFO:tasks.workunit.client.0.smithi094.stdout:2/104: link d0/d8/la d0/d8/l1f 0 2024-04-06T03:40:26.712 INFO:tasks.workunit.client.0.smithi094.stdout:2/105: write d0/f1 [6738916,78400] 0 2024-04-06T03:40:26.716 INFO:tasks.workunit.client.0.smithi094.stdout:2/106: creat d0/f20 x:0 0 0 2024-04-06T03:40:26.717 INFO:tasks.workunit.client.0.smithi094.stdout:2/107: link d0/dd/l12 d0/l21 0 2024-04-06T03:40:26.719 INFO:tasks.workunit.client.0.smithi094.stdout:2/108: unlink d0/dd/c10 0 2024-04-06T03:40:26.719 INFO:tasks.workunit.client.0.smithi094.stdout:2/109: readlink d0/dd/l12 0 2024-04-06T03:40:26.720 INFO:tasks.workunit.client.0.smithi094.stdout:2/110: creat d0/d18/f22 x:0 0 0 2024-04-06T03:40:26.720 INFO:tasks.workunit.client.0.smithi094.stdout:2/111: mknod d0/d18/c23 0 2024-04-06T03:40:26.721 INFO:tasks.workunit.client.0.smithi094.stdout:3/104: write d9/fc [5034388,82146] 0 2024-04-06T03:40:26.724 INFO:tasks.workunit.client.0.smithi094.stdout:3/105: mkdir d9/d20 0 2024-04-06T03:40:26.724 INFO:tasks.workunit.client.0.smithi094.stdout:3/106: read - d9/f19 zero size 2024-04-06T03:40:26.725 INFO:tasks.workunit.client.0.smithi094.stdout:3/107: creat d9/d20/f21 x:0 0 0 2024-04-06T03:40:26.726 INFO:tasks.workunit.client.0.smithi094.stdout:1/103: dwrite d0/d4/f25 [0,4194304] 0 2024-04-06T03:40:26.736 INFO:tasks.workunit.client.0.smithi094.stdout:2/112: dwrite d0/dd/ff [0,4194304] 0 2024-04-06T03:40:26.736 INFO:tasks.workunit.client.0.smithi094.stdout:2/113: creat d0/d18/f24 x:0 0 0 2024-04-06T03:40:26.740 INFO:tasks.workunit.client.0.smithi094.stdout:2/114: creat d0/d8/d1d/f25 x:0 0 0 2024-04-06T03:40:26.741 INFO:tasks.workunit.client.0.smithi094.stdout:9/46: rename d2/dc/dd to d2/dc/d10 0 2024-04-06T03:40:26.742 INFO:tasks.workunit.client.0.smithi094.stdout:1/104: dwrite d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:26.742 INFO:tasks.workunit.client.0.smithi094.stdout:2/115: unlink d0/dd/l15 0 2024-04-06T03:40:26.744 INFO:tasks.workunit.client.0.smithi094.stdout:9/47: link d2/l3 d2/dc/d10/l11 0 2024-04-06T03:40:26.745 INFO:tasks.workunit.client.0.smithi094.stdout:1/105: creat d0/d7/f26 x:0 0 0 2024-04-06T03:40:26.746 INFO:tasks.workunit.client.0.smithi094.stdout:2/116: link d0/lb d0/d18/l26 0 2024-04-06T03:40:26.747 INFO:tasks.workunit.client.0.smithi094.stdout:9/48: link d2/dc/d10/ff d2/dc/d10/f12 0 2024-04-06T03:40:26.747 INFO:tasks.workunit.client.0.smithi094.stdout:9/49: fdatasync d2/f9 0 2024-04-06T03:40:26.748 INFO:tasks.workunit.client.0.smithi094.stdout:2/117: mkdir d0/d8/d27 0 2024-04-06T03:40:26.749 INFO:tasks.workunit.client.0.smithi094.stdout:2/118: mknod d0/dd/c28 0 2024-04-06T03:40:26.750 INFO:tasks.workunit.client.0.smithi094.stdout:2/119: rename d0/dd/d1b/l1e to d0/d18/l29 0 2024-04-06T03:40:26.755 INFO:tasks.workunit.client.0.smithi094.stdout:6/36: truncate f8 865221 0 2024-04-06T03:40:26.762 INFO:tasks.workunit.client.0.smithi094.stdout:9/50: dwrite d2/fe [0,4194304] 0 2024-04-06T03:40:26.762 INFO:tasks.workunit.client.0.smithi094.stdout:9/51: write d2/f9 [362169,47685] 0 2024-04-06T03:40:26.762 INFO:tasks.workunit.client.0.smithi094.stdout:9/52: dread - d2/dc/d10/f12 zero size 2024-04-06T03:40:26.762 INFO:tasks.workunit.client.0.smithi094.stdout:9/53: dread - d2/f7 zero size 2024-04-06T03:40:26.762 INFO:tasks.workunit.client.0.smithi094.stdout:9/54: readlink d2/dc/d10/l11 0 2024-04-06T03:40:26.763 INFO:tasks.workunit.client.0.smithi094.stdout:0/99: getdents d1/d10 0 2024-04-06T03:40:26.785 INFO:tasks.workunit.client.0.smithi094.stdout:9/55: dwrite d2/dc/d10/ff [0,4194304] 0 2024-04-06T03:40:26.785 INFO:tasks.workunit.client.0.smithi094.stdout:0/100: dwrite d1/f7 [4194304,4194304] 0 2024-04-06T03:40:26.785 INFO:tasks.workunit.client.0.smithi094.stdout:0/101: fdatasync d1/f7 0 2024-04-06T03:40:26.842 INFO:tasks.workunit.client.1.smithi136.stdout:2/110: sync 2024-04-06T03:40:26.842 INFO:tasks.workunit.client.1.smithi136.stdout:4/46: sync 2024-04-06T03:40:26.842 INFO:tasks.workunit.client.1.smithi136.stdout:0/2: sync 2024-04-06T03:40:26.842 INFO:tasks.workunit.client.1.smithi136.stdout:8/31: sync 2024-04-06T03:40:26.842 INFO:tasks.workunit.client.1.smithi136.stdout:7/5: sync 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:3/9: sync 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:5/45: sync 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:8/32: readlink d3/l6 0 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:8/33: stat d3/l7 0 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:3/10: fdatasync d0/f2 0 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:8/34: chown d3 437750315 1 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:3/11: chown d0 18840 1 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:4/47: write f8 [1860707,67142] 0 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:6/30: sync 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:4/48: creat ff x:0 0 0 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:1/58: sync 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:9/12: sync 2024-04-06T03:40:26.843 INFO:tasks.workunit.client.1.smithi136.stdout:9/13: truncate f1 480174 0 2024-04-06T03:40:26.844 INFO:tasks.workunit.client.1.smithi136.stdout:2/111: symlink d3/d9/d10/l28 0 2024-04-06T03:40:26.845 INFO:tasks.workunit.client.0.smithi094.stdout:5/83: sync 2024-04-06T03:40:26.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/51: sync 2024-04-06T03:40:26.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/52: fdatasync d2/d6/f9 0 2024-04-06T03:40:26.845 INFO:tasks.workunit.client.0.smithi094.stdout:8/106: sync 2024-04-06T03:40:26.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/53: write d2/d6/f8 [784774,71054] 0 2024-04-06T03:40:26.845 INFO:tasks.workunit.client.1.smithi136.stdout:0/3: mknod c0 0 2024-04-06T03:40:26.845 INFO:tasks.workunit.client.1.smithi136.stdout:0/4: dwrite - no filename 2024-04-06T03:40:26.846 INFO:tasks.workunit.client.1.smithi136.stdout:0/5: dread - no filename 2024-04-06T03:40:26.846 INFO:tasks.workunit.client.1.smithi136.stdout:7/6: creat f1 x:0 0 0 2024-04-06T03:40:26.846 INFO:tasks.workunit.client.1.smithi136.stdout:7/7: readlink - no filename 2024-04-06T03:40:26.846 INFO:tasks.workunit.client.1.smithi136.stdout:7/8: write f1 [1020169,62087] 0 2024-04-06T03:40:26.848 INFO:tasks.workunit.client.0.smithi094.stdout:8/107: dread f1 [0,4194304] 0 2024-04-06T03:40:26.849 INFO:tasks.workunit.client.0.smithi094.stdout:8/108: readlink lc 0 2024-04-06T03:40:26.849 INFO:tasks.workunit.client.0.smithi094.stdout:5/84: chown db/ce 6224 1 2024-04-06T03:40:26.852 INFO:tasks.workunit.client.1.smithi136.stdout:8/35: unlink f0 0 2024-04-06T03:40:26.852 INFO:tasks.workunit.client.1.smithi136.stdout:8/36: write d3/f8 [204519,35473] 0 2024-04-06T03:40:26.856 INFO:tasks.workunit.client.0.smithi094.stdout:7/54: truncate d2/d6/f9 2466440 0 2024-04-06T03:40:26.857 INFO:tasks.workunit.client.1.smithi136.stdout:5/46: dwrite d1/f4 [0,4194304] 0 2024-04-06T03:40:26.862 INFO:tasks.workunit.client.1.smithi136.stdout:8/37: write d3/f4 [230740,41576] 0 2024-04-06T03:40:26.865 INFO:tasks.workunit.client.1.smithi136.stdout:4/49: symlink l10 0 2024-04-06T03:40:26.865 INFO:tasks.workunit.client.0.smithi094.stdout:9/56: truncate d2/fe 2591006 0 2024-04-06T03:40:26.866 INFO:tasks.workunit.client.0.smithi094.stdout:5/85: dwrite db/f1e [0,4194304] 0 2024-04-06T03:40:26.866 INFO:tasks.workunit.client.0.smithi094.stdout:5/86: readlink l5 0 2024-04-06T03:40:26.867 INFO:tasks.workunit.client.1.smithi136.stdout:4/50: dread fc [0,4194304] 0 2024-04-06T03:40:26.868 INFO:tasks.workunit.client.1.smithi136.stdout:6/31: rename d2/l3 to d2/d7/lc 0 2024-04-06T03:40:26.868 INFO:tasks.workunit.client.1.smithi136.stdout:6/32: dread - d2/f5 zero size 2024-04-06T03:40:26.868 INFO:tasks.workunit.client.1.smithi136.stdout:6/33: creat d2/fd x:0 0 0 2024-04-06T03:40:26.871 INFO:tasks.workunit.client.0.smithi094.stdout:7/55: rename d2/d6/le to d2/d6/l10 0 2024-04-06T03:40:26.871 INFO:tasks.workunit.client.0.smithi094.stdout:7/56: fsync d2/d6/fa 0 2024-04-06T03:40:26.871 INFO:tasks.workunit.client.0.smithi094.stdout:7/57: creat d2/f11 x:0 0 0 2024-04-06T03:40:26.873 INFO:tasks.workunit.client.1.smithi136.stdout:1/59: link c9 d12/c13 0 2024-04-06T03:40:26.873 INFO:tasks.workunit.client.1.smithi136.stdout:8/38: dread d3/f4 [0,4194304] 0 2024-04-06T03:40:26.874 INFO:tasks.workunit.client.0.smithi094.stdout:0/102: truncate d1/f7 1505315 0 2024-04-06T03:40:26.877 INFO:tasks.workunit.client.1.smithi136.stdout:9/14: rename f0 to f2 0 2024-04-06T03:40:26.878 INFO:tasks.workunit.client.1.smithi136.stdout:5/47: dwrite d1/f4 [4194304,4194304] 0 2024-04-06T03:40:26.878 INFO:tasks.workunit.client.1.smithi136.stdout:9/15: dread f1 [0,4194304] 0 2024-04-06T03:40:26.879 INFO:tasks.workunit.client.1.smithi136.stdout:2/112: creat d3/f29 x:0 0 0 2024-04-06T03:40:26.881 INFO:tasks.workunit.client.1.smithi136.stdout:0/6: mkdir d1 0 2024-04-06T03:40:26.881 INFO:tasks.workunit.client.1.smithi136.stdout:0/7: getdents d1 0 2024-04-06T03:40:26.881 INFO:tasks.workunit.client.1.smithi136.stdout:0/8: dread - no filename 2024-04-06T03:40:26.885 INFO:tasks.workunit.client.0.smithi094.stdout:7/58: dwrite d2/fb [0,4194304] 0 2024-04-06T03:40:26.886 INFO:tasks.workunit.client.0.smithi094.stdout:6/37: dwrite f8 [0,4194304] 0 2024-04-06T03:40:26.886 INFO:tasks.workunit.client.0.smithi094.stdout:6/38: creat dc/ff x:0 0 0 2024-04-06T03:40:26.886 INFO:tasks.workunit.client.0.smithi094.stdout:6/39: creat dc/f10 x:0 0 0 2024-04-06T03:40:26.888 INFO:tasks.workunit.client.1.smithi136.stdout:5/48: dread d1/f4 [0,4194304] 0 2024-04-06T03:40:26.949 INFO:tasks.workunit.client.1.smithi136.stdout:7/9: mknod c2 0 2024-04-06T03:40:26.949 INFO:tasks.workunit.client.1.smithi136.stdout:7/10: fdatasync f1 0 2024-04-06T03:40:26.954 INFO:tasks.workunit.client.1.smithi136.stdout:4/51: rename ca to c11 0 2024-04-06T03:40:26.956 INFO:tasks.workunit.client.0.smithi094.stdout:7/59: mkdir d2/d6/dc/d12 0 2024-04-06T03:40:26.959 INFO:tasks.workunit.client.1.smithi136.stdout:6/34: mkdir d2/de 0 2024-04-06T03:40:26.959 INFO:tasks.workunit.client.1.smithi136.stdout:6/35: truncate d2/fa 1249588 0 2024-04-06T03:40:26.961 INFO:tasks.workunit.client.0.smithi094.stdout:6/40: mkdir dc/d11 0 2024-04-06T03:40:26.962 INFO:tasks.workunit.client.1.smithi136.stdout:6/36: dread d2/f9 [0,4194304] 0 2024-04-06T03:40:26.963 INFO:tasks.workunit.client.1.smithi136.stdout:1/60: unlink le 0 2024-04-06T03:40:26.965 INFO:tasks.workunit.client.1.smithi136.stdout:8/39: unlink d3/l7 0 2024-04-06T03:40:26.966 INFO:tasks.workunit.client.0.smithi094.stdout:5/87: rmdir db 39 2024-04-06T03:40:26.970 INFO:tasks.workunit.client.1.smithi136.stdout:9/16: unlink f1 0 2024-04-06T03:40:26.970 INFO:tasks.workunit.client.1.smithi136.stdout:9/17: truncate f2 153111 0 2024-04-06T03:40:26.971 INFO:tasks.workunit.client.1.smithi136.stdout:2/113: rmdir d3/d15/d19/d25 0 2024-04-06T03:40:26.972 INFO:tasks.workunit.client.1.smithi136.stdout:2/114: write d3/d9/d10/d18/d1d/f20 [189759,81892] 0 2024-04-06T03:40:26.972 INFO:tasks.workunit.client.1.smithi136.stdout:2/115: creat d3/d9/d10/d18/f2a x:0 0 0 2024-04-06T03:40:26.972 INFO:tasks.workunit.client.1.smithi136.stdout:0/9: mknod d1/c2 0 2024-04-06T03:40:26.976 INFO:tasks.workunit.client.1.smithi136.stdout:2/116: fdatasync d3/d9/d10/d18/d1d/f20 0 2024-04-06T03:40:26.977 INFO:tasks.workunit.client.0.smithi094.stdout:7/60: truncate d2/fb 3096103 0 2024-04-06T03:40:26.978 INFO:tasks.workunit.client.1.smithi136.stdout:5/49: mknod d1/ca 0 2024-04-06T03:40:26.978 INFO:tasks.workunit.client.1.smithi136.stdout:5/50: chown d1 252996584 1 2024-04-06T03:40:26.979 INFO:tasks.workunit.client.0.smithi094.stdout:5/88: symlink db/l20 0 2024-04-06T03:40:26.982 INFO:tasks.workunit.client.0.smithi094.stdout:6/41: dwrite dc/ff [0,4194304] 0 2024-04-06T03:40:26.984 INFO:tasks.workunit.client.1.smithi136.stdout:7/11: unlink c0 0 2024-04-06T03:40:26.987 INFO:tasks.workunit.client.1.smithi136.stdout:7/12: dread f1 [0,4194304] 0 2024-04-06T03:40:26.987 INFO:tasks.workunit.client.0.smithi094.stdout:5/89: mkdir db/d19/d21 0 2024-04-06T03:40:26.990 INFO:tasks.workunit.client.1.smithi136.stdout:4/52: mkdir d12 0 2024-04-06T03:40:26.991 INFO:tasks.workunit.client.1.smithi136.stdout:7/13: dread f1 [0,4194304] 0 2024-04-06T03:40:26.992 INFO:tasks.workunit.client.1.smithi136.stdout:6/37: mknod d2/cf 0 2024-04-06T03:40:26.992 INFO:tasks.workunit.client.0.smithi094.stdout:6/42: write dc/ff [4063164,52286] 0 2024-04-06T03:40:26.992 INFO:tasks.workunit.client.0.smithi094.stdout:6/43: stat fa 0 2024-04-06T03:40:26.993 INFO:tasks.workunit.client.0.smithi094.stdout:6/44: getdents dc/d11 0 2024-04-06T03:40:26.994 INFO:tasks.workunit.client.1.smithi136.stdout:1/61: symlink d12/l14 0 2024-04-06T03:40:26.994 INFO:tasks.workunit.client.1.smithi136.stdout:7/14: read f1 [219324,101880] 0 2024-04-06T03:40:26.994 INFO:tasks.workunit.client.1.smithi136.stdout:7/15: fsync f1 0 2024-04-06T03:40:26.994 INFO:tasks.workunit.client.1.smithi136.stdout:1/62: write f10 [625154,61560] 0 2024-04-06T03:40:26.995 INFO:tasks.workunit.client.1.smithi136.stdout:9/18: dwrite f2 [0,4194304] 0 2024-04-06T03:40:26.998 INFO:tasks.workunit.client.1.smithi136.stdout:6/38: read d2/f9 [3075372,102722] 0 2024-04-06T03:40:26.998 INFO:tasks.workunit.client.1.smithi136.stdout:7/16: dread f1 [0,4194304] 0 2024-04-06T03:40:26.998 INFO:tasks.workunit.client.1.smithi136.stdout:7/17: readlink - no filename 2024-04-06T03:40:26.998 INFO:tasks.workunit.client.1.smithi136.stdout:7/18: creat f3 x:0 0 0 2024-04-06T03:40:26.998 INFO:tasks.workunit.client.1.smithi136.stdout:7/19: readlink - no filename 2024-04-06T03:40:26.998 INFO:tasks.workunit.client.1.smithi136.stdout:0/10: getdents d1 0 2024-04-06T03:40:26.999 INFO:tasks.workunit.client.1.smithi136.stdout:5/51: mknod d1/cb 0 2024-04-06T03:40:26.999 INFO:tasks.workunit.client.1.smithi136.stdout:2/117: dwrite d3/d9/d10/d18/f2a [0,4194304] 0 2024-04-06T03:40:27.001 INFO:tasks.workunit.client.1.smithi136.stdout:4/53: rmdir d12 0 2024-04-06T03:40:27.001 INFO:tasks.workunit.client.0.smithi094.stdout:7/61: write d2/fb [3913714,56826] 0 2024-04-06T03:40:27.005 INFO:tasks.workunit.client.0.smithi094.stdout:7/62: rmdir d2/d6/dc/dd 39 2024-04-06T03:40:27.005 INFO:tasks.workunit.client.0.smithi094.stdout:7/63: write d2/f11 [540103,110245] 0 2024-04-06T03:40:27.005 INFO:tasks.workunit.client.0.smithi094.stdout:7/64: fsync d2/d6/f8 0 2024-04-06T03:40:27.012 INFO:tasks.workunit.client.0.smithi094.stdout:6/45: dread f8 [0,4194304] 0 2024-04-06T03:40:27.012 INFO:tasks.workunit.client.0.smithi094.stdout:6/46: creat dc/f12 x:0 0 0 2024-04-06T03:40:27.012 INFO:tasks.workunit.client.0.smithi094.stdout:6/47: dread - dc/fd zero size 2024-04-06T03:40:27.013 INFO:tasks.workunit.client.0.smithi094.stdout:4/87: sync 2024-04-06T03:40:27.014 INFO:tasks.workunit.client.0.smithi094.stdout:7/65: write d2/fb [3479738,119781] 0 2024-04-06T03:40:27.016 INFO:tasks.workunit.client.1.smithi136.stdout:7/20: mknod c4 0 2024-04-06T03:40:27.016 INFO:tasks.workunit.client.1.smithi136.stdout:0/11: rename c0 to d1/c3 0 2024-04-06T03:40:27.018 INFO:tasks.workunit.client.1.smithi136.stdout:7/21: dread f1 [0,4194304] 0 2024-04-06T03:40:27.019 INFO:tasks.workunit.client.1.smithi136.stdout:5/52: truncate d1/f4 3003455 0 2024-04-06T03:40:27.022 INFO:tasks.workunit.client.0.smithi094.stdout:6/48: mknod dc/c13 0 2024-04-06T03:40:27.024 INFO:tasks.workunit.client.0.smithi094.stdout:4/88: write d0/d4/f6 [529587,93506] 0 2024-04-06T03:40:27.025 INFO:tasks.workunit.client.1.smithi136.stdout:1/63: dwrite f10 [0,4194304] 0 2024-04-06T03:40:27.027 INFO:tasks.workunit.client.1.smithi136.stdout:8/40: dwrite d3/f4 [0,4194304] 0 2024-04-06T03:40:27.027 INFO:tasks.workunit.client.1.smithi136.stdout:8/41: fsync d3/f9 0 2024-04-06T03:40:27.028 INFO:tasks.workunit.client.1.smithi136.stdout:8/42: fsync d3/f9 0 2024-04-06T03:40:27.028 INFO:tasks.workunit.client.1.smithi136.stdout:9/19: write f2 [4337961,100604] 0 2024-04-06T03:40:27.029 INFO:tasks.workunit.client.1.smithi136.stdout:2/118: creat d3/d15/f2b x:0 0 0 2024-04-06T03:40:27.030 INFO:tasks.workunit.client.0.smithi094.stdout:7/66: symlink d2/d6/dc/l13 0 2024-04-06T03:40:27.030 INFO:tasks.workunit.client.0.smithi094.stdout:7/67: chown d2/d6/f8 48924 1 2024-04-06T03:40:27.030 INFO:tasks.workunit.client.1.smithi136.stdout:6/39: dwrite d2/f5 [0,4194304] 0 2024-04-06T03:40:27.031 INFO:tasks.workunit.client.1.smithi136.stdout:4/54: dwrite ff [0,4194304] 0 2024-04-06T03:40:27.031 INFO:tasks.workunit.client.1.smithi136.stdout:0/12: creat d1/f4 x:0 0 0 2024-04-06T03:40:27.035 INFO:tasks.workunit.client.1.smithi136.stdout:9/20: fsync f2 0 2024-04-06T03:40:27.036 INFO:tasks.workunit.client.1.smithi136.stdout:9/21: chown f2 94 1 2024-04-06T03:40:27.041 INFO:tasks.workunit.client.1.smithi136.stdout:6/40: dread d2/f5 [0,4194304] 0 2024-04-06T03:40:27.041 INFO:tasks.workunit.client.1.smithi136.stdout:7/22: dwrite f1 [0,4194304] 0 2024-04-06T03:40:27.041 INFO:tasks.workunit.client.1.smithi136.stdout:7/23: chown f3 6 1 2024-04-06T03:40:27.089 INFO:tasks.workunit.client.0.smithi094.stdout:6/49: mknod dc/c14 0 2024-04-06T03:40:27.091 INFO:tasks.workunit.client.1.smithi136.stdout:1/64: symlink d12/l15 0 2024-04-06T03:40:27.091 INFO:tasks.workunit.client.1.smithi136.stdout:1/65: truncate f11 101974 0 2024-04-06T03:40:27.095 INFO:tasks.workunit.client.0.smithi094.stdout:7/68: mkdir d2/d6/dc/dd/df/d14 0 2024-04-06T03:40:27.095 INFO:tasks.workunit.client.0.smithi094.stdout:7/69: readlink d2/l7 0 2024-04-06T03:40:27.095 INFO:tasks.workunit.client.1.smithi136.stdout:8/43: mkdir d3/da 0 2024-04-06T03:40:27.098 INFO:tasks.workunit.client.1.smithi136.stdout:2/119: link d3/c6 d3/d9/d10/d18/d1d/c2c 0 2024-04-06T03:40:27.098 INFO:tasks.workunit.client.1.smithi136.stdout:2/120: creat d3/d9/d10/d18/f2d x:0 0 0 2024-04-06T03:40:27.098 INFO:tasks.workunit.client.1.smithi136.stdout:2/121: fdatasync d3/d15/f2b 0 2024-04-06T03:40:27.098 INFO:tasks.workunit.client.1.smithi136.stdout:5/53: write d1/f4 [2158363,34859] 0 2024-04-06T03:40:27.102 INFO:tasks.workunit.client.0.smithi094.stdout:6/50: rename l4 to dc/l15 0 2024-04-06T03:40:27.105 INFO:tasks.workunit.client.0.smithi094.stdout:7/70: creat d2/d6/dc/f15 x:0 0 0 2024-04-06T03:40:27.107 INFO:tasks.workunit.client.1.smithi136.stdout:5/54: dread d1/f4 [0,4194304] 0 2024-04-06T03:40:27.107 INFO:tasks.workunit.client.1.smithi136.stdout:5/55: fsync d1/f4 0 2024-04-06T03:40:27.107 INFO:tasks.workunit.client.1.smithi136.stdout:4/55: symlink l13 0 2024-04-06T03:40:27.107 INFO:tasks.workunit.client.1.smithi136.stdout:4/56: creat f14 x:0 0 0 2024-04-06T03:40:27.109 INFO:tasks.workunit.client.0.smithi094.stdout:6/51: dread f5 [0,4194304] 0 2024-04-06T03:40:27.109 INFO:tasks.workunit.client.0.smithi094.stdout:6/52: chown dc/ff 103860700 1 2024-04-06T03:40:27.110 INFO:tasks.workunit.client.0.smithi094.stdout:7/71: symlink d2/d6/l16 0 2024-04-06T03:40:27.113 INFO:tasks.workunit.client.0.smithi094.stdout:6/53: creat dc/d11/f16 x:0 0 0 2024-04-06T03:40:27.114 INFO:tasks.workunit.client.0.smithi094.stdout:7/72: getdents d2/d6/dc/dd/df 0 2024-04-06T03:40:27.117 INFO:tasks.workunit.client.0.smithi094.stdout:4/89: dwrite d0/d4/fd [0,4194304] 0 2024-04-06T03:40:27.118 INFO:tasks.workunit.client.1.smithi136.stdout:8/44: dwrite d3/f8 [0,4194304] 0 2024-04-06T03:40:27.118 INFO:tasks.workunit.client.1.smithi136.stdout:8/45: stat d3/f4 0 2024-04-06T03:40:27.120 INFO:tasks.workunit.client.0.smithi094.stdout:7/73: dread d2/d6/fa [0,4194304] 0 2024-04-06T03:40:27.120 INFO:tasks.workunit.client.0.smithi094.stdout:6/54: getdents dc/d11 0 2024-04-06T03:40:27.125 INFO:tasks.workunit.client.0.smithi094.stdout:4/90: dread d0/d4/fc [0,4194304] 0 2024-04-06T03:40:27.125 INFO:tasks.workunit.client.0.smithi094.stdout:4/91: stat d0 0 2024-04-06T03:40:27.125 INFO:tasks.workunit.client.0.smithi094.stdout:7/74: unlink d2/d6/l10 0 2024-04-06T03:40:27.125 INFO:tasks.workunit.client.1.smithi136.stdout:6/41: creat d2/de/f10 x:0 0 0 2024-04-06T03:40:27.127 INFO:tasks.workunit.client.0.smithi094.stdout:6/55: rename dc/d11/f16 to dc/f17 0 2024-04-06T03:40:27.128 INFO:tasks.workunit.client.1.smithi136.stdout:7/24: mkdir d5 0 2024-04-06T03:40:27.130 INFO:tasks.workunit.client.1.smithi136.stdout:1/66: creat d12/f16 x:0 0 0 2024-04-06T03:40:27.131 INFO:tasks.workunit.client.0.smithi094.stdout:3/108: sync 2024-04-06T03:40:27.131 INFO:tasks.workunit.client.0.smithi094.stdout:1/106: sync 2024-04-06T03:40:27.131 INFO:tasks.workunit.client.0.smithi094.stdout:3/109: creat d9/d10/f22 x:0 0 0 2024-04-06T03:40:27.131 INFO:tasks.workunit.client.0.smithi094.stdout:3/110: creat d9/d20/f23 x:0 0 0 2024-04-06T03:40:27.131 INFO:tasks.workunit.client.0.smithi094.stdout:6/56: symlink dc/l18 0 2024-04-06T03:40:27.133 INFO:tasks.workunit.client.0.smithi094.stdout:4/92: link d0/d4/f14 d0/df/f19 0 2024-04-06T03:40:27.133 INFO:tasks.workunit.client.0.smithi094.stdout:4/93: write d0/f12 [1104147,96050] 0 2024-04-06T03:40:27.134 INFO:tasks.workunit.client.1.smithi136.stdout:2/122: rename d3/d9/d10/l28 to d3/d1e/l2e 0 2024-04-06T03:40:27.134 INFO:tasks.workunit.client.1.smithi136.stdout:2/123: readlink d3/d9/d10/d18/l1c 0 2024-04-06T03:40:27.134 INFO:tasks.workunit.client.1.smithi136.stdout:2/124: read - d3/d15/d19/f1b zero size 2024-04-06T03:40:27.134 INFO:tasks.workunit.client.0.smithi094.stdout:1/107: mkdir d0/d1/d27 0 2024-04-06T03:40:27.136 INFO:tasks.workunit.client.1.smithi136.stdout:5/56: mkdir d1/dc 0 2024-04-06T03:40:27.138 INFO:tasks.workunit.client.0.smithi094.stdout:3/111: symlink d9/d20/l24 0 2024-04-06T03:40:27.138 INFO:tasks.workunit.client.0.smithi094.stdout:3/112: chown d9/ce 139 1 2024-04-06T03:40:27.139 INFO:tasks.workunit.client.0.smithi094.stdout:7/75: dwrite d2/fb [0,4194304] 0 2024-04-06T03:40:27.139 INFO:tasks.workunit.client.0.smithi094.stdout:7/76: fdatasync d2/d6/dc/f15 0 2024-04-06T03:40:27.139 INFO:tasks.workunit.client.0.smithi094.stdout:7/77: truncate d2/f11 1292504 0 2024-04-06T03:40:27.139 INFO:tasks.workunit.client.1.smithi136.stdout:4/57: mknod c15 0 2024-04-06T03:40:27.140 INFO:tasks.workunit.client.0.smithi094.stdout:6/57: symlink dc/l19 0 2024-04-06T03:40:27.141 INFO:tasks.workunit.client.1.smithi136.stdout:0/13: link d1/c3 d1/c5 0 2024-04-06T03:40:27.141 INFO:tasks.workunit.client.1.smithi136.stdout:0/14: fdatasync d1/f4 0 2024-04-06T03:40:27.144 INFO:tasks.workunit.client.0.smithi094.stdout:4/94: rmdir d0/d15 39 2024-04-06T03:40:27.144 INFO:tasks.workunit.client.0.smithi094.stdout:4/95: write d0/df/d13/f18 [606757,107903] 0 2024-04-06T03:40:27.145 INFO:tasks.workunit.client.1.smithi136.stdout:2/125: dread d3/d9/fe [0,4194304] 0 2024-04-06T03:40:27.145 INFO:tasks.workunit.client.1.smithi136.stdout:2/126: chown l1 9298271 1 2024-04-06T03:40:27.145 INFO:tasks.workunit.client.1.smithi136.stdout:2/127: dread - d3/d9/d10/d18/f2d zero size 2024-04-06T03:40:27.145 INFO:tasks.workunit.client.1.smithi136.stdout:9/22: truncate f2 4252799 0 2024-04-06T03:40:27.147 INFO:tasks.workunit.client.0.smithi094.stdout:1/108: symlink d0/d1/d16/d24/l28 0 2024-04-06T03:40:27.148 INFO:tasks.workunit.client.0.smithi094.stdout:3/113: creat d9/d1d/f25 x:0 0 0 2024-04-06T03:40:27.149 INFO:tasks.workunit.client.0.smithi094.stdout:3/114: dread - d9/d20/f23 zero size 2024-04-06T03:40:27.149 INFO:tasks.workunit.client.0.smithi094.stdout:3/115: write d9/d1d/f25 [775412,72531] 0 2024-04-06T03:40:27.151 INFO:tasks.workunit.client.0.smithi094.stdout:6/58: symlink dc/l1a 0 2024-04-06T03:40:27.156 INFO:tasks.workunit.client.0.smithi094.stdout:4/96: getdents d0/df 0 2024-04-06T03:40:27.158 INFO:tasks.workunit.client.0.smithi094.stdout:7/78: dwrite d2/d6/fa [0,4194304] 0 2024-04-06T03:40:27.158 INFO:tasks.workunit.client.1.smithi136.stdout:6/42: mknod d2/d7/c11 0 2024-04-06T03:40:27.158 INFO:tasks.workunit.client.1.smithi136.stdout:4/58: dwrite fc [0,4194304] 0 2024-04-06T03:40:27.159 INFO:tasks.workunit.client.0.smithi094.stdout:6/59: dread f8 [0,4194304] 0 2024-04-06T03:40:27.159 INFO:tasks.workunit.client.0.smithi094.stdout:1/109: rename d0/d4/f25 to d0/d4/f29 0 2024-04-06T03:40:27.160 INFO:tasks.workunit.client.1.smithi136.stdout:7/25: mknod d5/c6 0 2024-04-06T03:40:27.160 INFO:tasks.workunit.client.0.smithi094.stdout:4/97: readlink d0/d4/l16 0 2024-04-06T03:40:27.161 INFO:tasks.workunit.client.0.smithi094.stdout:4/98: read d0/d4/f6 [4823477,109843] 0 2024-04-06T03:40:27.161 INFO:tasks.workunit.client.1.smithi136.stdout:1/67: mknod d12/c17 0 2024-04-06T03:40:27.163 INFO:tasks.workunit.client.0.smithi094.stdout:7/79: rename l1 to d2/d6/dc/d12/l17 0 2024-04-06T03:40:27.163 INFO:tasks.workunit.client.0.smithi094.stdout:7/80: fsync d2/d6/f9 0 2024-04-06T03:40:27.163 INFO:tasks.workunit.client.0.smithi094.stdout:7/81: fdatasync d2/d6/fa 0 2024-04-06T03:40:27.164 INFO:tasks.workunit.client.0.smithi094.stdout:7/82: chown d2 3655942 1 2024-04-06T03:40:27.164 INFO:tasks.workunit.client.1.smithi136.stdout:5/57: symlink d1/ld 0 2024-04-06T03:40:27.164 INFO:tasks.workunit.client.1.smithi136.stdout:5/58: readlink d1/ld 0 2024-04-06T03:40:27.166 INFO:tasks.workunit.client.0.smithi094.stdout:3/116: dwrite d9/f15 [0,4194304] 0 2024-04-06T03:40:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:26 smithi094 ceph-mon[29713]: pgmap v13: 65 pgs: 65 active+clean; 276 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 654 op/s 2024-04-06T03:40:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:26 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:26 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:27.169 INFO:tasks.workunit.client.0.smithi094.stdout:4/99: truncate d0/d4/fd 1619990 0 2024-04-06T03:40:27.170 INFO:tasks.workunit.client.0.smithi094.stdout:1/110: write d0/d4/f9 [3462422,99209] 0 2024-04-06T03:40:27.171 INFO:tasks.workunit.client.0.smithi094.stdout:7/83: dread d2/d6/fa [0,4194304] 0 2024-04-06T03:40:27.171 INFO:tasks.workunit.client.0.smithi094.stdout:7/84: getdents d2/d6/dc/dd/df 0 2024-04-06T03:40:27.171 INFO:tasks.workunit.client.0.smithi094.stdout:7/85: fdatasync d2/d6/f9 0 2024-04-06T03:40:27.171 INFO:tasks.workunit.client.1.smithi136.stdout:3/12: sync 2024-04-06T03:40:27.174 INFO:tasks.workunit.client.1.smithi136.stdout:2/128: rename d3/l5 to d3/d9/d10/l2f 0 2024-04-06T03:40:27.174 INFO:tasks.workunit.client.1.smithi136.stdout:2/129: dread - d3/d15/d19/f1b zero size 2024-04-06T03:40:27.174 INFO:tasks.workunit.client.1.smithi136.stdout:2/130: fsync d3/d9/d10/f17 0 2024-04-06T03:40:27.174 INFO:tasks.workunit.client.1.smithi136.stdout:2/131: chown d3/d9/c1a 8443122 1 2024-04-06T03:40:27.174 INFO:tasks.workunit.client.1.smithi136.stdout:2/132: creat d3/d15/f30 x:0 0 0 2024-04-06T03:40:27.174 INFO:tasks.workunit.client.0.smithi094.stdout:7/86: dread d2/f11 [0,4194304] 0 2024-04-06T03:40:27.175 INFO:tasks.workunit.client.0.smithi094.stdout:7/87: write d2/d6/f8 [1776894,24238] 0 2024-04-06T03:40:27.178 INFO:tasks.workunit.client.1.smithi136.stdout:8/46: link f1 d3/da/fb 0 2024-04-06T03:40:27.178 INFO:tasks.workunit.client.1.smithi136.stdout:8/47: fsync d3/f4 0 2024-04-06T03:40:27.178 INFO:tasks.workunit.client.0.smithi094.stdout:1/111: write d0/d4/f29 [2990899,90518] 0 2024-04-06T03:40:27.179 INFO:tasks.workunit.client.0.smithi094.stdout:3/117: mkdir d9/d1e/d26 0 2024-04-06T03:40:27.180 INFO:tasks.workunit.client.0.smithi094.stdout:6/60: getdents dc 0 2024-04-06T03:40:27.180 INFO:tasks.workunit.client.0.smithi094.stdout:3/118: write d9/d1d/f25 [453995,250] 0 2024-04-06T03:40:27.180 INFO:tasks.workunit.client.0.smithi094.stdout:3/119: write d9/d10/f16 [111993,125050] 0 2024-04-06T03:40:27.180 INFO:tasks.workunit.client.0.smithi094.stdout:3/120: truncate d9/d20/f21 176272 0 2024-04-06T03:40:27.181 INFO:tasks.workunit.client.1.smithi136.stdout:6/43: creat d2/d7/f12 x:0 0 0 2024-04-06T03:40:27.185 INFO:tasks.workunit.client.1.smithi136.stdout:8/48: dread d3/f4 [0,4194304] 0 2024-04-06T03:40:27.186 INFO:tasks.workunit.client.0.smithi094.stdout:4/100: dread d0/d4/fb [0,4194304] 0 2024-04-06T03:40:27.186 INFO:tasks.workunit.client.0.smithi094.stdout:7/88: symlink d2/d6/l18 0 2024-04-06T03:40:27.186 INFO:tasks.workunit.client.0.smithi094.stdout:4/101: write d0/d4/fc [1179228,125954] 0 2024-04-06T03:40:27.187 INFO:tasks.workunit.client.1.smithi136.stdout:1/68: dwrite f3 [4194304,4194304] 0 2024-04-06T03:40:27.189 INFO:tasks.workunit.client.0.smithi094.stdout:4/102: write d0/fa [815157,118224] 0 2024-04-06T03:40:27.191 INFO:tasks.workunit.client.1.smithi136.stdout:4/59: rename le to l16 0 2024-04-06T03:40:27.191 INFO:tasks.workunit.client.1.smithi136.stdout:4/60: creat f17 x:0 0 0 2024-04-06T03:40:27.191 INFO:tasks.workunit.client.1.smithi136.stdout:4/61: chown f14 0 1 2024-04-06T03:40:27.192 INFO:tasks.workunit.client.1.smithi136.stdout:4/62: fsync f7 0 2024-04-06T03:40:27.192 INFO:tasks.workunit.client.1.smithi136.stdout:4/63: rmdir - no directory 2024-04-06T03:40:27.192 INFO:tasks.workunit.client.0.smithi094.stdout:1/112: mknod d0/d1/d16/c2a 0 2024-04-06T03:40:27.196 INFO:tasks.workunit.client.1.smithi136.stdout:7/26: creat d5/f7 x:0 0 0 2024-04-06T03:40:27.198 INFO:tasks.workunit.client.1.smithi136.stdout:5/59: mknod d1/dc/ce 0 2024-04-06T03:40:27.203 INFO:tasks.workunit.client.0.smithi094.stdout:4/103: creat d0/d15/f1a x:0 0 0 2024-04-06T03:40:27.203 INFO:tasks.workunit.client.0.smithi094.stdout:3/121: dwrite d9/d20/f23 [0,4194304] 0 2024-04-06T03:40:27.203 INFO:tasks.workunit.client.0.smithi094.stdout:6/61: mkdir dc/d11/d1b 0 2024-04-06T03:40:27.204 INFO:tasks.workunit.client.1.smithi136.stdout:0/15: mkdir d1/d6 0 2024-04-06T03:40:27.204 INFO:tasks.workunit.client.1.smithi136.stdout:0/16: dread - d1/f4 zero size 2024-04-06T03:40:27.204 INFO:tasks.workunit.client.1.smithi136.stdout:4/64: write ff [2433508,34560] 0 2024-04-06T03:40:27.204 INFO:tasks.workunit.client.1.smithi136.stdout:8/49: dwrite d3/f4 [0,4194304] 0 2024-04-06T03:40:27.206 INFO:tasks.workunit.client.1.smithi136.stdout:8/50: dread f1 [0,4194304] 0 2024-04-06T03:40:27.208 INFO:tasks.workunit.client.1.smithi136.stdout:9/23: truncate f2 3010258 0 2024-04-06T03:40:27.208 INFO:tasks.workunit.client.1.smithi136.stdout:9/24: chown f2 230387 1 2024-04-06T03:40:27.211 INFO:tasks.workunit.client.0.smithi094.stdout:3/122: mkdir d9/d10/d27 0 2024-04-06T03:40:27.212 INFO:tasks.workunit.client.0.smithi094.stdout:7/89: dwrite d2/d6/f9 [0,4194304] 0 2024-04-06T03:40:27.215 INFO:tasks.workunit.client.1.smithi136.stdout:6/44: mkdir d2/d7/d13 0 2024-04-06T03:40:27.215 INFO:tasks.workunit.client.1.smithi136.stdout:6/45: read - d2/fd zero size 2024-04-06T03:40:27.222 INFO:tasks.workunit.client.0.smithi094.stdout:4/104: dwrite d0/f8 [0,4194304] 0 2024-04-06T03:40:27.222 INFO:tasks.workunit.client.0.smithi094.stdout:4/105: stat d0/df/d13/f18 0 2024-04-06T03:40:27.222 INFO:tasks.workunit.client.1.smithi136.stdout:7/27: rename c2 to d5/c8 0 2024-04-06T03:40:27.223 INFO:tasks.workunit.client.1.smithi136.stdout:7/28: chown c4 1657 1 2024-04-06T03:40:27.225 INFO:tasks.workunit.client.1.smithi136.stdout:5/60: creat d1/dc/ff x:0 0 0 2024-04-06T03:40:27.228 INFO:tasks.workunit.client.1.smithi136.stdout:6/46: dwrite d2/de/f10 [0,4194304] 0 2024-04-06T03:40:27.228 INFO:tasks.workunit.client.1.smithi136.stdout:6/47: write d2/fd [327629,126354] 0 2024-04-06T03:40:27.231 INFO:tasks.workunit.client.1.smithi136.stdout:4/65: symlink l18 0 2024-04-06T03:40:27.231 INFO:tasks.workunit.client.1.smithi136.stdout:4/66: write f14 [756665,2096] 0 2024-04-06T03:40:27.231 INFO:tasks.workunit.client.1.smithi136.stdout:4/67: chown l16 1975 1 2024-04-06T03:40:27.232 INFO:tasks.workunit.client.0.smithi094.stdout:3/123: creat d9/d1e/d26/f28 x:0 0 0 2024-04-06T03:40:27.232 INFO:tasks.workunit.client.0.smithi094.stdout:3/124: chown d9/d1d/f25 6 1 2024-04-06T03:40:27.233 INFO:tasks.workunit.client.1.smithi136.stdout:2/133: rmdir d3/d9/d10/d18 39 2024-04-06T03:40:27.235 INFO:tasks.workunit.client.1.smithi136.stdout:8/51: truncate d3/f8 1063641 0 2024-04-06T03:40:27.238 INFO:tasks.workunit.client.1.smithi136.stdout:7/29: dwrite f1 [0,4194304] 0 2024-04-06T03:40:27.238 INFO:tasks.workunit.client.1.smithi136.stdout:7/30: write f3 [462811,130043] 0 2024-04-06T03:40:27.240 INFO:tasks.workunit.client.1.smithi136.stdout:1/69: rmdir d12 39 2024-04-06T03:40:27.240 INFO:tasks.workunit.client.1.smithi136.stdout:1/70: stat d12/l15 0 2024-04-06T03:40:27.243 INFO:tasks.workunit.client.1.smithi136.stdout:5/61: rmdir d1 39 2024-04-06T03:40:27.244 INFO:tasks.workunit.client.1.smithi136.stdout:0/17: rmdir d1 39 2024-04-06T03:40:27.246 INFO:tasks.workunit.client.0.smithi094.stdout:7/90: mknod d2/d6/dc/c19 0 2024-04-06T03:40:27.248 INFO:tasks.workunit.client.1.smithi136.stdout:2/134: dread d3/d9/d10/d18/f2a [0,4194304] 0 2024-04-06T03:40:27.248 INFO:tasks.workunit.client.1.smithi136.stdout:2/135: read - d3/d9/d10/d18/f2d zero size 2024-04-06T03:40:27.248 INFO:tasks.workunit.client.1.smithi136.stdout:2/136: write d3/d9/f12 [1160591,49582] 0 2024-04-06T03:40:27.248 INFO:tasks.workunit.client.1.smithi136.stdout:2/137: write d3/d15/f2b [941872,57157] 0 2024-04-06T03:40:27.249 INFO:tasks.workunit.client.0.smithi094.stdout:6/62: symlink dc/d11/l1c 0 2024-04-06T03:40:27.250 INFO:tasks.workunit.client.1.smithi136.stdout:6/48: mkdir d2/d14 0 2024-04-06T03:40:27.250 INFO:tasks.workunit.client.1.smithi136.stdout:6/49: creat d2/de/f15 x:0 0 0 2024-04-06T03:40:27.250 INFO:tasks.workunit.client.1.smithi136.stdout:7/31: dread f3 [0,4194304] 0 2024-04-06T03:40:27.250 INFO:tasks.workunit.client.1.smithi136.stdout:7/32: creat d5/f9 x:0 0 0 2024-04-06T03:40:27.250 INFO:tasks.workunit.client.0.smithi094.stdout:4/106: creat d0/df/f1b x:0 0 0 2024-04-06T03:40:27.251 INFO:tasks.workunit.client.1.smithi136.stdout:7/33: dread f3 [0,4194304] 0 2024-04-06T03:40:27.251 INFO:tasks.workunit.client.1.smithi136.stdout:7/34: read - d5/f7 zero size 2024-04-06T03:40:27.251 INFO:tasks.workunit.client.1.smithi136.stdout:8/52: truncate f1 1504685 0 2024-04-06T03:40:27.251 INFO:tasks.workunit.client.1.smithi136.stdout:8/53: readlink d3/l6 0 2024-04-06T03:40:27.253 INFO:tasks.workunit.client.1.smithi136.stdout:9/25: rename f2 to f3 0 2024-04-06T03:40:27.254 INFO:tasks.workunit.client.1.smithi136.stdout:4/68: dwrite f7 [0,4194304] 0 2024-04-06T03:40:27.254 INFO:tasks.workunit.client.1.smithi136.stdout:1/71: rename lc to d12/l18 0 2024-04-06T03:40:27.256 INFO:tasks.workunit.client.0.smithi094.stdout:3/125: rename l3 to d9/l29 0 2024-04-06T03:40:27.257 INFO:tasks.workunit.client.1.smithi136.stdout:5/62: mkdir d1/dc/d10 0 2024-04-06T03:40:27.258 INFO:tasks.workunit.client.0.smithi094.stdout:4/107: mkdir d0/df/d1c 0 2024-04-06T03:40:27.259 INFO:tasks.workunit.client.1.smithi136.stdout:0/18: creat d1/f7 x:0 0 0 2024-04-06T03:40:27.261 INFO:tasks.workunit.client.0.smithi094.stdout:6/63: rename dc/l19 to dc/d11/d1b/l1d 0 2024-04-06T03:40:27.263 INFO:tasks.workunit.client.1.smithi136.stdout:2/138: rename d3/d9/cf to d3/d9/c31 0 2024-04-06T03:40:27.264 INFO:tasks.workunit.client.0.smithi094.stdout:3/126: symlink d9/l2a 0 2024-04-06T03:40:27.264 INFO:tasks.workunit.client.0.smithi094.stdout:3/127: fdatasync d9/fd 0 2024-04-06T03:40:27.265 INFO:tasks.workunit.client.1.smithi136.stdout:8/54: dwrite d3/f4 [0,4194304] 0 2024-04-06T03:40:27.268 INFO:tasks.workunit.client.1.smithi136.stdout:6/50: mknod d2/de/c16 0 2024-04-06T03:40:27.268 INFO:tasks.workunit.client.1.smithi136.stdout:6/51: readlink d2/d7/lb 0 2024-04-06T03:40:27.268 INFO:tasks.workunit.client.1.smithi136.stdout:6/52: fsync d2/fa 0 2024-04-06T03:40:27.271 INFO:tasks.workunit.client.1.smithi136.stdout:7/35: mknod d5/ca 0 2024-04-06T03:40:27.273 INFO:tasks.workunit.client.1.smithi136.stdout:9/26: rename f3 to f4 0 2024-04-06T03:40:27.275 INFO:tasks.workunit.client.0.smithi094.stdout:6/64: symlink dc/d11/l1e 0 2024-04-06T03:40:27.278 INFO:tasks.workunit.client.1.smithi136.stdout:4/69: unlink c11 0 2024-04-06T03:40:27.278 INFO:tasks.workunit.client.1.smithi136.stdout:1/72: stat d12/c13 0 2024-04-06T03:40:27.279 INFO:tasks.workunit.client.0.smithi094.stdout:6/65: creat dc/d11/d1b/f1f x:0 0 0 2024-04-06T03:40:27.281 INFO:tasks.workunit.client.0.smithi094.stdout:3/128: dwrite d9/f15 [0,4194304] 0 2024-04-06T03:40:27.281 INFO:tasks.workunit.client.1.smithi136.stdout:0/19: dwrite d1/f7 [0,4194304] 0 2024-04-06T03:40:27.282 INFO:tasks.workunit.client.1.smithi136.stdout:5/63: creat d1/dc/d10/f11 x:0 0 0 2024-04-06T03:40:27.282 INFO:tasks.workunit.client.1.smithi136.stdout:5/64: chown d1/ca 1102451712 1 2024-04-06T03:40:27.286 INFO:tasks.workunit.client.1.smithi136.stdout:2/139: dwrite d3/d9/d10/d18/f2a [0,4194304] 0 2024-04-06T03:40:27.286 INFO:tasks.workunit.client.1.smithi136.stdout:8/55: truncate d3/f8 236686 0 2024-04-06T03:40:27.287 INFO:tasks.workunit.client.1.smithi136.stdout:8/56: truncate d3/f9 278500 0 2024-04-06T03:40:27.288 INFO:tasks.workunit.client.1.smithi136.stdout:6/53: creat d2/d14/f17 x:0 0 0 2024-04-06T03:40:27.289 INFO:tasks.workunit.client.0.smithi094.stdout:4/108: dwrite d0/fa [0,4194304] 0 2024-04-06T03:40:27.289 INFO:tasks.workunit.client.0.smithi094.stdout:4/109: write d0/df/f1b [167270,123820] 0 2024-04-06T03:40:27.290 INFO:tasks.workunit.client.1.smithi136.stdout:7/36: rename d5/c8 to d5/cb 0 2024-04-06T03:40:27.292 INFO:tasks.workunit.client.0.smithi094.stdout:2/120: sync 2024-04-06T03:40:27.293 INFO:tasks.workunit.client.1.smithi136.stdout:9/27: mknod c5 0 2024-04-06T03:40:27.298 INFO:tasks.workunit.client.0.smithi094.stdout:4/110: symlink d0/d4/l1d 0 2024-04-06T03:40:27.298 INFO:tasks.workunit.client.0.smithi094.stdout:4/111: stat d0/fa 0 2024-04-06T03:40:27.298 INFO:tasks.workunit.client.0.smithi094.stdout:4/112: chown d0/df/d1c 787465 1 2024-04-06T03:40:27.299 INFO:tasks.workunit.client.1.smithi136.stdout:8/57: dread d3/f4 [0,4194304] 0 2024-04-06T03:40:27.301 INFO:tasks.workunit.client.0.smithi094.stdout:4/113: mkdir d0/df/d1e 0 2024-04-06T03:40:27.303 INFO:tasks.workunit.client.1.smithi136.stdout:4/70: dwrite f17 [0,4194304] 0 2024-04-06T03:40:27.303 INFO:tasks.workunit.client.1.smithi136.stdout:1/73: dwrite f11 [0,4194304] 0 2024-04-06T03:40:27.304 INFO:tasks.workunit.client.1.smithi136.stdout:5/65: rename d1/dc/ff to d1/dc/d10/f12 0 2024-04-06T03:40:27.304 INFO:tasks.workunit.client.1.smithi136.stdout:5/66: truncate d1/dc/d10/f12 632291 0 2024-04-06T03:40:27.305 INFO:tasks.workunit.client.0.smithi094.stdout:6/66: dwrite dc/f17 [0,4194304] 0 2024-04-06T03:40:27.305 INFO:tasks.workunit.client.0.smithi094.stdout:6/67: truncate dc/fd 863934 0 2024-04-06T03:40:27.306 INFO:tasks.workunit.client.1.smithi136.stdout:9/28: mkdir d6 0 2024-04-06T03:40:27.306 INFO:tasks.workunit.client.1.smithi136.stdout:9/29: write f4 [3505647,106107] 0 2024-04-06T03:40:27.307 INFO:tasks.workunit.client.1.smithi136.stdout:5/67: write d1/dc/d10/f12 [69114,41402] 0 2024-04-06T03:40:27.309 INFO:tasks.workunit.client.0.smithi094.stdout:4/114: mknod d0/df/c1f 0 2024-04-06T03:40:27.312 INFO:tasks.workunit.client.1.smithi136.stdout:0/20: rmdir d1 39 2024-04-06T03:40:27.315 INFO:tasks.workunit.client.0.smithi094.stdout:3/129: dwrite d9/f19 [0,4194304] 0 2024-04-06T03:40:27.315 INFO:tasks.workunit.client.0.smithi094.stdout:3/130: creat d9/d10/f2b x:0 0 0 2024-04-06T03:40:27.323 INFO:tasks.workunit.client.1.smithi136.stdout:4/71: rename l10 to l19 0 2024-04-06T03:40:27.324 INFO:tasks.workunit.client.0.smithi094.stdout:4/115: dread d0/fa [0,4194304] 0 2024-04-06T03:40:27.324 INFO:tasks.workunit.client.0.smithi094.stdout:4/116: write d0/d4/fd [2268315,79776] 0 2024-04-06T03:40:27.325 INFO:tasks.workunit.client.0.smithi094.stdout:6/68: creat dc/d11/f20 x:0 0 0 2024-04-06T03:40:27.327 INFO:tasks.workunit.client.1.smithi136.stdout:7/37: dwrite f3 [0,4194304] 0 2024-04-06T03:40:27.328 INFO:tasks.workunit.client.1.smithi136.stdout:8/58: dwrite f1 [0,4194304] 0 2024-04-06T03:40:27.329 INFO:tasks.workunit.client.1.smithi136.stdout:2/140: dwrite d3/d9/d10/f17 [0,4194304] 0 2024-04-06T03:40:27.329 INFO:tasks.workunit.client.1.smithi136.stdout:2/141: write d3/d9/d10/d18/f2d [830415,68930] 0 2024-04-06T03:40:27.333 INFO:tasks.workunit.client.1.smithi136.stdout:0/21: dwrite d1/f4 [0,4194304] 0 2024-04-06T03:40:27.336 INFO:tasks.workunit.client.0.smithi094.stdout:3/131: dwrite d9/fd [0,4194304] 0 2024-04-06T03:40:27.336 INFO:tasks.workunit.client.0.smithi094.stdout:3/132: truncate d9/f1b 1002124 0 2024-04-06T03:40:27.338 INFO:tasks.workunit.client.1.smithi136.stdout:7/38: dread f1 [0,4194304] 0 2024-04-06T03:40:27.342 INFO:tasks.workunit.client.0.smithi094.stdout:9/57: sync 2024-04-06T03:40:27.342 INFO:tasks.workunit.client.0.smithi094.stdout:0/103: sync 2024-04-06T03:40:27.342 INFO:tasks.workunit.client.0.smithi094.stdout:5/90: sync 2024-04-06T03:40:27.342 INFO:tasks.workunit.client.0.smithi094.stdout:9/58: chown d2/l3 15370672 1 2024-04-06T03:40:27.343 INFO:tasks.workunit.client.1.smithi136.stdout:8/59: dwrite d3/da/fb [0,4194304] 0 2024-04-06T03:40:27.343 INFO:tasks.workunit.client.1.smithi136.stdout:8/60: fdatasync d3/f8 0 2024-04-06T03:40:27.344 INFO:tasks.workunit.client.0.smithi094.stdout:4/117: link d0/d4/f6 d0/f20 0 2024-04-06T03:40:27.346 INFO:tasks.workunit.client.0.smithi094.stdout:6/69: mknod dc/c21 0 2024-04-06T03:40:27.348 INFO:tasks.workunit.client.0.smithi094.stdout:8/109: sync 2024-04-06T03:40:27.348 INFO:tasks.workunit.client.0.smithi094.stdout:8/110: dread - d18/f22 zero size 2024-04-06T03:40:27.348 INFO:tasks.workunit.client.0.smithi094.stdout:8/111: creat d18/f27 x:0 0 0 2024-04-06T03:40:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:26 smithi136 ceph-mon[36436]: pgmap v13: 65 pgs: 65 active+clean; 276 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s rd, 15 MiB/s wr, 654 op/s 2024-04-06T03:40:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:26 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:26 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:27.350 INFO:tasks.workunit.client.0.smithi094.stdout:9/59: write d2/fe [1592724,1913] 0 2024-04-06T03:40:27.352 INFO:tasks.workunit.client.0.smithi094.stdout:4/118: creat d0/d4/f21 x:0 0 0 2024-04-06T03:40:27.353 INFO:tasks.workunit.client.0.smithi094.stdout:0/104: dread d1/f5 [0,4194304] 0 2024-04-06T03:40:27.357 INFO:tasks.workunit.client.0.smithi094.stdout:3/133: rmdir d9/d20 39 2024-04-06T03:40:27.359 INFO:tasks.workunit.client.0.smithi094.stdout:9/60: symlink d2/dc/d10/l13 0 2024-04-06T03:40:27.359 INFO:tasks.workunit.client.0.smithi094.stdout:9/61: chown d2/l3 231972 1 2024-04-06T03:40:27.360 INFO:tasks.workunit.client.0.smithi094.stdout:4/119: creat d0/f22 x:0 0 0 2024-04-06T03:40:27.361 INFO:tasks.workunit.client.0.smithi094.stdout:0/105: mknod d1/c1f 0 2024-04-06T03:40:27.362 INFO:tasks.workunit.client.0.smithi094.stdout:5/91: dread db/f11 [0,4194304] 0 2024-04-06T03:40:27.363 INFO:tasks.workunit.client.0.smithi094.stdout:6/70: mkdir dc/d11/d22 0 2024-04-06T03:40:27.365 INFO:tasks.workunit.client.0.smithi094.stdout:5/92: symlink db/d19/l22 0 2024-04-06T03:40:27.369 INFO:tasks.workunit.client.0.smithi094.stdout:5/93: creat db/f23 x:0 0 0 2024-04-06T03:40:27.370 INFO:tasks.workunit.client.0.smithi094.stdout:6/71: mknod dc/d11/c23 0 2024-04-06T03:40:27.370 INFO:tasks.workunit.client.0.smithi094.stdout:6/72: fsync f8 0 2024-04-06T03:40:27.370 INFO:tasks.workunit.client.0.smithi094.stdout:6/73: stat dc 0 2024-04-06T03:40:27.370 INFO:tasks.workunit.client.0.smithi094.stdout:8/112: dwrite d18/f22 [0,4194304] 0 2024-04-06T03:40:27.371 INFO:tasks.workunit.client.0.smithi094.stdout:6/74: read dc/fd [843165,93561] 0 2024-04-06T03:40:27.371 INFO:tasks.workunit.client.0.smithi094.stdout:6/75: chown dc/l15 56 1 2024-04-06T03:40:27.373 INFO:tasks.workunit.client.0.smithi094.stdout:5/94: mkdir db/d19/d24 0 2024-04-06T03:40:27.373 INFO:tasks.workunit.client.0.smithi094.stdout:5/95: fsync f9 0 2024-04-06T03:40:27.373 INFO:tasks.workunit.client.0.smithi094.stdout:3/134: read d9/f19 [2415284,53413] 0 2024-04-06T03:40:27.379 INFO:tasks.workunit.client.0.smithi094.stdout:9/62: dread d2/dc/d10/ff [0,4194304] 0 2024-04-06T03:40:27.379 INFO:tasks.workunit.client.1.smithi136.stdout:9/30: truncate f4 957873 0 2024-04-06T03:40:27.380 INFO:tasks.workunit.client.0.smithi094.stdout:6/76: mkdir dc/d11/d1b/d24 0 2024-04-06T03:40:27.382 INFO:tasks.workunit.client.1.smithi136.stdout:7/39: mkdir d5/dc 0 2024-04-06T03:40:27.382 INFO:tasks.workunit.client.1.smithi136.stdout:7/40: creat d5/fd x:0 0 0 2024-04-06T03:40:27.384 INFO:tasks.workunit.client.0.smithi094.stdout:9/63: link d2/f9 d2/f14 0 2024-04-06T03:40:27.384 INFO:tasks.workunit.client.0.smithi094.stdout:9/64: write d2/f4 [152443,102877] 0 2024-04-06T03:40:27.387 INFO:tasks.workunit.client.0.smithi094.stdout:6/77: creat dc/d11/d1b/d24/f25 x:0 0 0 2024-04-06T03:40:27.387 INFO:tasks.workunit.client.0.smithi094.stdout:6/78: dread - dc/d11/f20 zero size 2024-04-06T03:40:27.387 INFO:tasks.workunit.client.0.smithi094.stdout:6/79: fdatasync fa 0 2024-04-06T03:40:27.387 INFO:tasks.workunit.client.0.smithi094.stdout:6/80: dread - dc/f10 zero size 2024-04-06T03:40:27.387 INFO:tasks.workunit.client.0.smithi094.stdout:6/81: dread - dc/d11/f20 zero size 2024-04-06T03:40:27.387 INFO:tasks.workunit.client.0.smithi094.stdout:6/82: write dc/fd [1344517,21735] 0 2024-04-06T03:40:27.388 INFO:tasks.workunit.client.0.smithi094.stdout:3/135: write d9/d20/f23 [3622362,3332] 0 2024-04-06T03:40:27.388 INFO:tasks.workunit.client.0.smithi094.stdout:3/136: creat d9/d1e/d26/f2c x:0 0 0 2024-04-06T03:40:27.388 INFO:tasks.workunit.client.0.smithi094.stdout:3/137: truncate d9/d1e/d26/f2c 97032 0 2024-04-06T03:40:27.388 INFO:tasks.workunit.client.0.smithi094.stdout:3/138: write d9/d10/f2b [612005,121074] 0 2024-04-06T03:40:27.388 INFO:tasks.workunit.client.1.smithi136.stdout:4/72: getdents . 0 2024-04-06T03:40:27.391 INFO:tasks.workunit.client.0.smithi094.stdout:9/65: link d2/dc/d10/l13 d2/dc/d10/l15 0 2024-04-06T03:40:27.393 INFO:tasks.workunit.client.1.smithi136.stdout:6/54: rename d2/d14 to d2/d7/d13/d18 0 2024-04-06T03:40:27.394 INFO:tasks.workunit.client.0.smithi094.stdout:4/120: dwrite d0/d4/f14 [0,4194304] 0 2024-04-06T03:40:27.395 INFO:tasks.workunit.client.0.smithi094.stdout:4/121: fdatasync d0/d4/fb 0 2024-04-06T03:40:27.395 INFO:tasks.workunit.client.0.smithi094.stdout:8/113: dwrite d18/f1c [0,4194304] 0 2024-04-06T03:40:27.395 INFO:tasks.workunit.client.0.smithi094.stdout:8/114: fdatasync f14 0 2024-04-06T03:40:27.395 INFO:tasks.workunit.client.0.smithi094.stdout:0/106: dwrite d1/f7 [0,4194304] 0 2024-04-06T03:40:27.395 INFO:tasks.workunit.client.0.smithi094.stdout:0/107: creat d1/d9/f20 x:0 0 0 2024-04-06T03:40:27.396 INFO:tasks.workunit.client.1.smithi136.stdout:0/22: unlink d1/c2 0 2024-04-06T03:40:27.397 INFO:tasks.workunit.client.0.smithi094.stdout:0/108: dread d1/d9/f14 [0,4194304] 0 2024-04-06T03:40:27.401 INFO:tasks.workunit.client.0.smithi094.stdout:5/96: dwrite db/f23 [0,4194304] 0 2024-04-06T03:40:27.401 INFO:tasks.workunit.client.1.smithi136.stdout:9/31: truncate f4 1797570 0 2024-04-06T03:40:27.402 INFO:tasks.workunit.client.0.smithi094.stdout:9/66: creat d2/dc/d10/f16 x:0 0 0 2024-04-06T03:40:27.405 INFO:tasks.workunit.client.1.smithi136.stdout:4/73: creat f1a x:0 0 0 2024-04-06T03:40:27.406 INFO:tasks.workunit.client.1.smithi136.stdout:0/23: dread d1/f4 [0,4194304] 0 2024-04-06T03:40:27.408 INFO:tasks.workunit.client.1.smithi136.stdout:2/142: rename d3/d9/d10/l2f to d3/d1e/l32 0 2024-04-06T03:40:27.410 INFO:tasks.workunit.client.0.smithi094.stdout:8/115: dwrite d18/f19 [0,4194304] 0 2024-04-06T03:40:27.411 INFO:tasks.workunit.client.0.smithi094.stdout:8/116: write d18/f25 [754349,109718] 0 2024-04-06T03:40:27.411 INFO:tasks.workunit.client.0.smithi094.stdout:8/117: creat d18/f28 x:0 0 0 2024-04-06T03:40:27.411 INFO:tasks.workunit.client.1.smithi136.stdout:6/55: symlink d2/l19 0 2024-04-06T03:40:27.412 INFO:tasks.workunit.client.1.smithi136.stdout:3/13: sync 2024-04-06T03:40:27.412 INFO:tasks.workunit.client.1.smithi136.stdout:3/14: creat d0/f3 x:0 0 0 2024-04-06T03:40:27.416 INFO:tasks.workunit.client.0.smithi094.stdout:6/83: rmdir dc/d11 39 2024-04-06T03:40:27.418 INFO:tasks.workunit.client.1.smithi136.stdout:9/32: rmdir d6 0 2024-04-06T03:40:27.418 INFO:tasks.workunit.client.1.smithi136.stdout:6/56: write d2/f5 [1462253,74161] 0 2024-04-06T03:40:27.418 INFO:tasks.workunit.client.1.smithi136.stdout:9/33: creat f7 x:0 0 0 2024-04-06T03:40:27.418 INFO:tasks.workunit.client.1.smithi136.stdout:6/57: fsync d2/f9 0 2024-04-06T03:40:27.418 INFO:tasks.workunit.client.1.smithi136.stdout:8/61: truncate f1 2898508 0 2024-04-06T03:40:27.418 INFO:tasks.workunit.client.1.smithi136.stdout:8/62: chown d3/f9 8706093 1 2024-04-06T03:40:27.418 INFO:tasks.workunit.client.1.smithi136.stdout:8/63: creat d3/fc x:0 0 0 2024-04-06T03:40:27.420 INFO:tasks.workunit.client.1.smithi136.stdout:4/74: creat f1b x:0 0 0 2024-04-06T03:40:27.420 INFO:tasks.workunit.client.1.smithi136.stdout:0/24: dwrite d1/f7 [4194304,4194304] 0 2024-04-06T03:40:27.421 INFO:tasks.workunit.client.1.smithi136.stdout:2/143: rename d3/d9/cc to d3/d1e/c33 0 2024-04-06T03:40:27.422 INFO:tasks.workunit.client.0.smithi094.stdout:8/118: write d18/f22 [2742412,97578] 0 2024-04-06T03:40:27.423 INFO:tasks.workunit.client.0.smithi094.stdout:4/122: rename d0/df/d1c to d0/d23 0 2024-04-06T03:40:27.426 INFO:tasks.workunit.client.1.smithi136.stdout:3/15: mkdir d0/d4 0 2024-04-06T03:40:27.427 INFO:tasks.workunit.client.1.smithi136.stdout:3/16: creat d0/f5 x:0 0 0 2024-04-06T03:40:27.433 INFO:tasks.workunit.client.1.smithi136.stdout:9/34: truncate f4 995725 0 2024-04-06T03:40:27.434 INFO:tasks.workunit.client.1.smithi136.stdout:5/68: sync 2024-04-06T03:40:27.434 INFO:tasks.workunit.client.1.smithi136.stdout:1/74: sync 2024-04-06T03:40:27.434 INFO:tasks.workunit.client.1.smithi136.stdout:5/69: readlink d1/l9 0 2024-04-06T03:40:27.434 INFO:tasks.workunit.client.1.smithi136.stdout:1/75: fdatasync f10 0 2024-04-06T03:40:27.434 INFO:tasks.workunit.client.1.smithi136.stdout:0/25: dwrite d1/f7 [4194304,4194304] 0 2024-04-06T03:40:27.437 INFO:tasks.workunit.client.1.smithi136.stdout:8/64: creat d3/da/fd x:0 0 0 2024-04-06T03:40:27.437 INFO:tasks.workunit.client.1.smithi136.stdout:8/65: dread - d3/fc zero size 2024-04-06T03:40:27.443 INFO:tasks.workunit.client.0.smithi094.stdout:0/109: unlink d1/d9/dd/c1b 0 2024-04-06T03:40:27.445 INFO:tasks.workunit.client.1.smithi136.stdout:1/76: read f11 [908504,94484] 0 2024-04-06T03:40:27.445 INFO:tasks.workunit.client.1.smithi136.stdout:1/77: truncate f10 5036138 0 2024-04-06T03:40:27.445 INFO:tasks.workunit.client.1.smithi136.stdout:1/78: fdatasync fb 0 2024-04-06T03:40:27.446 INFO:tasks.workunit.client.0.smithi094.stdout:4/123: dread d0/fa [0,4194304] 0 2024-04-06T03:40:27.446 INFO:tasks.workunit.client.0.smithi094.stdout:3/139: getdents d9/d1d 0 2024-04-06T03:40:27.452 INFO:tasks.workunit.client.1.smithi136.stdout:9/35: rename c5 to c8 0 2024-04-06T03:40:27.452 INFO:tasks.workunit.client.1.smithi136.stdout:9/36: chown c8 14700795 1 2024-04-06T03:40:27.456 INFO:tasks.workunit.client.0.smithi094.stdout:8/119: dwrite f1 [4194304,4194304] 0 2024-04-06T03:40:27.457 INFO:tasks.workunit.client.0.smithi094.stdout:5/97: creat db/d13/f25 x:0 0 0 2024-04-06T03:40:27.459 INFO:tasks.workunit.client.1.smithi136.stdout:3/17: dwrite d0/f1 [0,4194304] 0 2024-04-06T03:40:27.459 INFO:tasks.workunit.client.1.smithi136.stdout:3/18: creat d0/f6 x:0 0 0 2024-04-06T03:40:27.460 INFO:tasks.workunit.client.1.smithi136.stdout:3/19: write d0/f6 [715851,52771] 0 2024-04-06T03:40:27.460 INFO:tasks.workunit.client.1.smithi136.stdout:3/20: creat d0/f7 x:0 0 0 2024-04-06T03:40:27.460 INFO:tasks.workunit.client.0.smithi094.stdout:9/67: symlink d2/dc/l17 0 2024-04-06T03:40:27.462 INFO:tasks.workunit.client.1.smithi136.stdout:2/144: rmdir d3 39 2024-04-06T03:40:27.463 INFO:tasks.workunit.client.0.smithi094.stdout:6/84: symlink dc/l26 0 2024-04-06T03:40:27.464 INFO:tasks.workunit.client.1.smithi136.stdout:1/79: rmdir d12 39 2024-04-06T03:40:27.464 INFO:tasks.workunit.client.1.smithi136.stdout:1/80: write d12/f16 [104436,64443] 0 2024-04-06T03:40:27.465 INFO:tasks.workunit.client.1.smithi136.stdout:1/81: readlink l7 0 2024-04-06T03:40:27.467 INFO:tasks.workunit.client.1.smithi136.stdout:0/26: mknod d1/c8 0 2024-04-06T03:40:27.468 INFO:tasks.workunit.client.0.smithi094.stdout:0/110: unlink d1/c1f 0 2024-04-06T03:40:27.469 INFO:tasks.workunit.client.0.smithi094.stdout:0/111: truncate d1/d9/dd/f16 1943801 0 2024-04-06T03:40:27.469 INFO:tasks.workunit.client.0.smithi094.stdout:0/112: write d1/f5 [4261438,109456] 0 2024-04-06T03:40:27.469 INFO:tasks.workunit.client.1.smithi136.stdout:5/70: dwrite d1/dc/d10/f12 [0,4194304] 0 2024-04-06T03:40:27.471 INFO:tasks.workunit.client.0.smithi094.stdout:8/120: read d18/f1b [2980787,59265] 0 2024-04-06T03:40:27.471 INFO:tasks.workunit.client.0.smithi094.stdout:8/121: dread - f17 zero size 2024-04-06T03:40:27.471 INFO:tasks.workunit.client.0.smithi094.stdout:8/122: dread - d18/f27 zero size 2024-04-06T03:40:27.473 INFO:tasks.workunit.client.1.smithi136.stdout:4/75: dwrite f7 [0,4194304] 0 2024-04-06T03:40:27.474 INFO:tasks.workunit.client.1.smithi136.stdout:4/76: rmdir - no directory 2024-04-06T03:40:27.474 INFO:tasks.workunit.client.0.smithi094.stdout:4/124: rename d0/d4/fc to d0/df/d1e/f24 0 2024-04-06T03:40:27.477 INFO:tasks.workunit.client.0.smithi094.stdout:3/140: mkdir d9/d20/d2d 0 2024-04-06T03:40:27.479 INFO:tasks.workunit.client.1.smithi136.stdout:0/27: write d1/f7 [7209149,61330] 0 2024-04-06T03:40:27.479 INFO:tasks.workunit.client.1.smithi136.stdout:0/28: stat d1 0 2024-04-06T03:40:27.480 INFO:tasks.workunit.client.1.smithi136.stdout:9/37: rename c8 to c9 0 2024-04-06T03:40:27.482 INFO:tasks.workunit.client.0.smithi094.stdout:4/125: write d0/f8 [1896932,30659] 0 2024-04-06T03:40:27.483 INFO:tasks.workunit.client.1.smithi136.stdout:0/29: write d1/f4 [3610559,84474] 0 2024-04-06T03:40:27.484 INFO:tasks.workunit.client.1.smithi136.stdout:1/82: mknod d12/c19 0 2024-04-06T03:40:27.485 INFO:tasks.workunit.client.0.smithi094.stdout:5/98: creat db/d19/d21/f26 x:0 0 0 2024-04-06T03:40:27.488 INFO:tasks.workunit.client.0.smithi094.stdout:6/85: creat dc/d11/d22/f27 x:0 0 0 2024-04-06T03:40:27.488 INFO:tasks.workunit.client.1.smithi136.stdout:2/145: dwrite d3/d9/f12 [0,4194304] 0 2024-04-06T03:40:27.488 INFO:tasks.workunit.client.1.smithi136.stdout:2/146: write d3/d9/d10/f22 [321265,56986] 0 2024-04-06T03:40:27.489 INFO:tasks.workunit.client.0.smithi094.stdout:8/123: rename d18/c1d to d18/c29 0 2024-04-06T03:40:27.490 INFO:tasks.workunit.client.1.smithi136.stdout:3/21: unlink d0/f7 0 2024-04-06T03:40:27.490 INFO:tasks.workunit.client.1.smithi136.stdout:3/22: dread - d0/f3 zero size 2024-04-06T03:40:27.492 INFO:tasks.workunit.client.0.smithi094.stdout:9/68: rename d2/dc/d10/ff to d2/dc/f18 0 2024-04-06T03:40:27.492 INFO:tasks.workunit.client.0.smithi094.stdout:9/69: fdatasync d2/fb 0 2024-04-06T03:40:27.492 INFO:tasks.workunit.client.0.smithi094.stdout:0/113: dwrite d1/d9/dd/f1c [0,4194304] 0 2024-04-06T03:40:27.493 INFO:tasks.workunit.client.0.smithi094.stdout:3/141: mknod d9/d10/c2e 0 2024-04-06T03:40:27.493 INFO:tasks.workunit.client.0.smithi094.stdout:3/142: write d9/d20/f21 [777673,103201] 0 2024-04-06T03:40:27.494 INFO:tasks.workunit.client.1.smithi136.stdout:1/83: rmdir d12 39 2024-04-06T03:40:27.496 INFO:tasks.workunit.client.0.smithi094.stdout:5/99: mknod db/d19/c27 0 2024-04-06T03:40:27.498 INFO:tasks.workunit.client.0.smithi094.stdout:6/86: creat dc/f28 x:0 0 0 2024-04-06T03:40:27.498 INFO:tasks.workunit.client.0.smithi094.stdout:6/87: dread - dc/f10 zero size 2024-04-06T03:40:27.498 INFO:tasks.workunit.client.0.smithi094.stdout:6/88: write dc/f10 [680702,62823] 0 2024-04-06T03:40:27.499 INFO:tasks.workunit.client.1.smithi136.stdout:2/147: write d3/d9/d10/d18/f2a [4017875,96495] 0 2024-04-06T03:40:27.499 INFO:tasks.workunit.client.1.smithi136.stdout:2/148: chown d3/d15/d19 4383 1 2024-04-06T03:40:27.500 INFO:tasks.workunit.client.0.smithi094.stdout:8/124: mknod d18/c2a 0 2024-04-06T03:40:27.500 INFO:tasks.workunit.client.0.smithi094.stdout:8/125: write d18/f25 [959568,50925] 0 2024-04-06T03:40:27.502 INFO:tasks.workunit.client.1.smithi136.stdout:3/23: symlink d0/d4/l8 0 2024-04-06T03:40:27.502 INFO:tasks.workunit.client.1.smithi136.stdout:3/24: dread - d0/f5 zero size 2024-04-06T03:40:27.505 INFO:tasks.workunit.client.0.smithi094.stdout:1/113: sync 2024-04-06T03:40:27.506 INFO:tasks.workunit.client.1.smithi136.stdout:9/38: dwrite f7 [0,4194304] 0 2024-04-06T03:40:27.506 INFO:tasks.workunit.client.0.smithi094.stdout:4/126: dwrite d0/d4/f21 [0,4194304] 0 2024-04-06T03:40:27.507 INFO:tasks.workunit.client.1.smithi136.stdout:4/77: rename f17 to f1c 0 2024-04-06T03:40:27.508 INFO:tasks.workunit.client.0.smithi094.stdout:0/114: symlink d1/d9/dd/d18/d1d/l21 0 2024-04-06T03:40:27.510 INFO:tasks.workunit.client.0.smithi094.stdout:7/91: sync 2024-04-06T03:40:27.510 INFO:tasks.workunit.client.0.smithi094.stdout:2/121: sync 2024-04-06T03:40:27.510 INFO:tasks.workunit.client.1.smithi136.stdout:2/149: symlink d3/d15/l34 0 2024-04-06T03:40:27.510 INFO:tasks.workunit.client.1.smithi136.stdout:1/84: dread fd [0,4194304] 0 2024-04-06T03:40:27.511 INFO:tasks.workunit.client.1.smithi136.stdout:5/71: truncate d1/dc/d10/f12 2263890 0 2024-04-06T03:40:27.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/114: dread d0/d1/d16/f1c [0,4194304] 0 2024-04-06T03:40:27.513 INFO:tasks.workunit.client.0.smithi094.stdout:3/143: mkdir d9/d20/d2d/d2f 0 2024-04-06T03:40:27.514 INFO:tasks.workunit.client.0.smithi094.stdout:3/144: write d9/d20/f21 [1845846,46103] 0 2024-04-06T03:40:27.518 INFO:tasks.workunit.client.0.smithi094.stdout:5/100: mkdir db/d19/d24/d28 0 2024-04-06T03:40:27.518 INFO:tasks.workunit.client.0.smithi094.stdout:5/101: fdatasync db/f12 0 2024-04-06T03:40:27.519 INFO:tasks.workunit.client.0.smithi094.stdout:5/102: fdatasync db/f11 0 2024-04-06T03:40:27.523 INFO:tasks.workunit.client.1.smithi136.stdout:0/30: truncate d1/f4 620693 0 2024-04-06T03:40:27.524 INFO:tasks.workunit.client.0.smithi094.stdout:6/89: dwrite dc/fd [0,4194304] 0 2024-04-06T03:40:27.524 INFO:tasks.workunit.client.0.smithi094.stdout:6/90: creat dc/f29 x:0 0 0 2024-04-06T03:40:27.524 INFO:tasks.workunit.client.0.smithi094.stdout:6/91: truncate dc/f28 352379 0 2024-04-06T03:40:27.525 INFO:tasks.workunit.client.0.smithi094.stdout:8/126: mkdir d18/d2b 0 2024-04-06T03:40:27.527 INFO:tasks.workunit.client.0.smithi094.stdout:9/70: dwrite d2/f14 [0,4194304] 0 2024-04-06T03:40:27.529 INFO:tasks.workunit.client.1.smithi136.stdout:4/78: chown l6 0 1 2024-04-06T03:40:27.531 INFO:tasks.workunit.client.1.smithi136.stdout:2/150: rmdir d3/d9/d10 39 2024-04-06T03:40:27.534 INFO:tasks.workunit.client.0.smithi094.stdout:4/127: rename d0/f20 to d0/df/d1e/f25 0 2024-04-06T03:40:27.534 INFO:tasks.workunit.client.0.smithi094.stdout:4/128: chown d0/d4/l16 0 1 2024-04-06T03:40:27.536 INFO:tasks.workunit.client.1.smithi136.stdout:5/72: link d1/l9 d1/l13 0 2024-04-06T03:40:27.538 INFO:tasks.workunit.client.0.smithi094.stdout:7/92: rmdir d2/d6/dc/dd/df/d14 0 2024-04-06T03:40:27.538 INFO:tasks.workunit.client.0.smithi094.stdout:0/115: dwrite d1/d9/f14 [0,4194304] 0 2024-04-06T03:40:27.540 INFO:tasks.workunit.client.0.smithi094.stdout:9/71: read d2/dc/d10/f12 [4075166,60078] 0 2024-04-06T03:40:27.541 INFO:tasks.workunit.client.0.smithi094.stdout:2/122: dread - d0/dd/d1b/f1c zero size 2024-04-06T03:40:27.541 INFO:tasks.workunit.client.1.smithi136.stdout:3/25: dwrite d0/f1 [0,4194304] 0 2024-04-06T03:40:27.545 INFO:tasks.workunit.client.0.smithi094.stdout:0/116: read d1/d9/f14 [3388165,51900] 0 2024-04-06T03:40:27.545 INFO:tasks.workunit.client.0.smithi094.stdout:0/117: truncate d1/d9/dd/f16 4720457 0 2024-04-06T03:40:27.545 INFO:tasks.workunit.client.0.smithi094.stdout:0/118: readlink d1/l6 0 2024-04-06T03:40:27.546 INFO:tasks.workunit.client.0.smithi094.stdout:3/145: mkdir d9/d20/d30 0 2024-04-06T03:40:27.548 INFO:tasks.workunit.client.1.smithi136.stdout:0/31: dwrite d1/f7 [0,4194304] 0 2024-04-06T03:40:27.549 INFO:tasks.workunit.client.1.smithi136.stdout:9/39: truncate f7 3050680 0 2024-04-06T03:40:27.550 INFO:tasks.workunit.client.0.smithi094.stdout:5/103: creat db/f29 x:0 0 0 2024-04-06T03:40:27.550 INFO:tasks.workunit.client.1.smithi136.stdout:3/26: write d0/f1 [582753,19604] 0 2024-04-06T03:40:27.550 INFO:tasks.workunit.client.1.smithi136.stdout:3/27: write d0/f1 [1593311,73650] 0 2024-04-06T03:40:27.550 INFO:tasks.workunit.client.1.smithi136.stdout:1/85: dwrite f3 [0,4194304] 0 2024-04-06T03:40:27.558 INFO:tasks.workunit.client.1.smithi136.stdout:0/32: dread d1/f7 [0,4194304] 0 2024-04-06T03:40:27.558 INFO:tasks.workunit.client.1.smithi136.stdout:0/33: chown d1/f4 121056429 1 2024-04-06T03:40:27.558 INFO:tasks.workunit.client.1.smithi136.stdout:0/34: getdents d1/d6 0 2024-04-06T03:40:27.561 INFO:tasks.workunit.client.0.smithi094.stdout:3/146: dwrite d9/d20/f23 [0,4194304] 0 2024-04-06T03:40:27.561 INFO:tasks.workunit.client.1.smithi136.stdout:2/151: truncate d3/d9/d10/f17 459398 0 2024-04-06T03:40:27.564 INFO:tasks.workunit.client.0.smithi094.stdout:8/127: mkdir d18/d2c 0 2024-04-06T03:40:27.564 INFO:tasks.workunit.client.0.smithi094.stdout:8/128: getdents d18/d2c 0 2024-04-06T03:40:27.569 INFO:tasks.workunit.client.1.smithi136.stdout:3/28: symlink d0/l9 0 2024-04-06T03:40:27.569 INFO:tasks.workunit.client.1.smithi136.stdout:3/29: write d0/f2 [198311,55740] 0 2024-04-06T03:40:27.570 INFO:tasks.workunit.client.1.smithi136.stdout:3/30: write d0/f2 [1204227,120555] 0 2024-04-06T03:40:27.572 INFO:tasks.workunit.client.1.smithi136.stdout:9/40: write f7 [1132058,18244] 0 2024-04-06T03:40:27.573 INFO:tasks.workunit.client.0.smithi094.stdout:4/129: mknod d0/df/c26 0 2024-04-06T03:40:27.575 INFO:tasks.workunit.client.1.smithi136.stdout:1/86: symlink d12/l1a 0 2024-04-06T03:40:27.575 INFO:tasks.workunit.client.1.smithi136.stdout:7/41: sync 2024-04-06T03:40:27.577 INFO:tasks.workunit.client.1.smithi136.stdout:4/79: mkdir d1d 0 2024-04-06T03:40:27.577 INFO:tasks.workunit.client.0.smithi094.stdout:8/129: dwrite d18/f1c [0,4194304] 0 2024-04-06T03:40:27.580 INFO:tasks.workunit.client.1.smithi136.stdout:0/35: creat d1/d6/f9 x:0 0 0 2024-04-06T03:40:27.580 INFO:tasks.workunit.client.1.smithi136.stdout:0/36: rename d1 to d1/d6/da 22 2024-04-06T03:40:27.580 INFO:tasks.workunit.client.1.smithi136.stdout:0/37: creat d1/d6/fb x:0 0 0 2024-04-06T03:40:27.581 INFO:tasks.workunit.client.0.smithi094.stdout:9/72: mknod d2/c19 0 2024-04-06T03:40:27.581 INFO:tasks.workunit.client.1.smithi136.stdout:2/152: creat d3/d9/d10/d18/d1d/f35 x:0 0 0 2024-04-06T03:40:27.583 INFO:tasks.workunit.client.0.smithi094.stdout:2/123: rename d0/dd/c14 to d0/dd/d1b/c2a 0 2024-04-06T03:40:27.584 INFO:tasks.workunit.client.1.smithi136.stdout:5/73: getdents d1/dc 0 2024-04-06T03:40:27.585 INFO:tasks.workunit.client.0.smithi094.stdout:0/119: mkdir d1/d9/dd/d18/d1e/d22 0 2024-04-06T03:40:27.585 INFO:tasks.workunit.client.0.smithi094.stdout:0/120: chown d1/d9/dd/d18/d1e 7346438 1 2024-04-06T03:40:27.585 INFO:tasks.workunit.client.1.smithi136.stdout:6/58: sync 2024-04-06T03:40:27.585 INFO:tasks.workunit.client.1.smithi136.stdout:8/66: sync 2024-04-06T03:40:27.586 INFO:tasks.workunit.client.1.smithi136.stdout:3/31: creat d0/d4/fa x:0 0 0 2024-04-06T03:40:27.587 INFO:tasks.workunit.client.0.smithi094.stdout:5/104: creat db/f2a x:0 0 0 2024-04-06T03:40:27.587 INFO:tasks.workunit.client.1.smithi136.stdout:3/32: read d0/f6 [107795,49451] 0 2024-04-06T03:40:27.588 INFO:tasks.workunit.client.1.smithi136.stdout:3/33: dread - d0/f5 zero size 2024-04-06T03:40:27.588 INFO:tasks.workunit.client.0.smithi094.stdout:9/73: write d2/f5 [1246740,125864] 0 2024-04-06T03:40:27.588 INFO:tasks.workunit.client.0.smithi094.stdout:6/92: symlink dc/d11/l2a 0 2024-04-06T03:40:27.590 INFO:tasks.workunit.client.1.smithi136.stdout:1/87: mkdir d12/d1b 0 2024-04-06T03:40:27.591 INFO:tasks.workunit.client.0.smithi094.stdout:3/147: rmdir d9/d20/d2d/d2f 0 2024-04-06T03:40:27.593 INFO:tasks.workunit.client.1.smithi136.stdout:7/42: creat d5/dc/fe x:0 0 0 2024-04-06T03:40:27.593 INFO:tasks.workunit.client.1.smithi136.stdout:8/67: dread f1 [0,4194304] 0 2024-04-06T03:40:27.593 INFO:tasks.workunit.client.0.smithi094.stdout:4/130: symlink d0/df/d1e/l27 0 2024-04-06T03:40:27.594 INFO:tasks.workunit.client.1.smithi136.stdout:4/80: mknod d1d/c1e 0 2024-04-06T03:40:27.596 INFO:tasks.workunit.client.0.smithi094.stdout:7/93: getdents d2/d6/dc 0 2024-04-06T03:40:27.596 INFO:tasks.workunit.client.0.smithi094.stdout:0/121: dread d1/d9/dd/f1c [0,4194304] 0 2024-04-06T03:40:27.596 INFO:tasks.workunit.client.0.smithi094.stdout:7/94: dread - d2/d6/dc/f15 zero size 2024-04-06T03:40:27.596 INFO:tasks.workunit.client.0.smithi094.stdout:7/95: chown d2/d6 650193690 1 2024-04-06T03:40:27.599 INFO:tasks.workunit.client.0.smithi094.stdout:7/96: dread d2/d6/f8 [0,4194304] 0 2024-04-06T03:40:27.600 INFO:tasks.workunit.client.1.smithi136.stdout:2/153: link d3/d9/fe d3/d15/f36 0 2024-04-06T03:40:27.602 INFO:tasks.workunit.client.1.smithi136.stdout:5/74: creat d1/f14 x:0 0 0 2024-04-06T03:40:27.603 INFO:tasks.workunit.client.1.smithi136.stdout:5/75: dread - d1/f14 zero size 2024-04-06T03:40:27.603 INFO:tasks.workunit.client.1.smithi136.stdout:3/34: dread d0/f1 [0,4194304] 0 2024-04-06T03:40:27.603 INFO:tasks.workunit.client.1.smithi136.stdout:5/76: readlink d1/l9 0 2024-04-06T03:40:27.603 INFO:tasks.workunit.client.1.smithi136.stdout:3/35: creat d0/fb x:0 0 0 2024-04-06T03:40:27.603 INFO:tasks.workunit.client.1.smithi136.stdout:6/59: symlink d2/d7/d13/l1a 0 2024-04-06T03:40:27.603 INFO:tasks.workunit.client.1.smithi136.stdout:7/43: write f1 [3276952,82059] 0 2024-04-06T03:40:27.604 INFO:tasks.workunit.client.1.smithi136.stdout:9/41: getdents . 0 2024-04-06T03:40:27.606 INFO:tasks.workunit.client.1.smithi136.stdout:1/88: symlink d12/d1b/l1c 0 2024-04-06T03:40:27.607 INFO:tasks.workunit.client.0.smithi094.stdout:6/93: link dc/c14 dc/d11/d1b/d24/c2b 0 2024-04-06T03:40:27.607 INFO:tasks.workunit.client.0.smithi094.stdout:6/94: write dc/f10 [1120747,48063] 0 2024-04-06T03:40:27.608 INFO:tasks.workunit.client.0.smithi094.stdout:6/95: fdatasync dc/f28 0 2024-04-06T03:40:27.608 INFO:tasks.workunit.client.1.smithi136.stdout:8/68: mkdir d3/de 0 2024-04-06T03:40:27.609 INFO:tasks.workunit.client.1.smithi136.stdout:8/69: write d3/f9 [1190644,122157] 0 2024-04-06T03:40:27.610 INFO:tasks.workunit.client.1.smithi136.stdout:4/81: creat d1d/f1f x:0 0 0 2024-04-06T03:40:27.610 INFO:tasks.workunit.client.1.smithi136.stdout:4/82: write f1a [732567,13881] 0 2024-04-06T03:40:27.611 INFO:tasks.workunit.client.0.smithi094.stdout:3/148: mknod d9/d1d/c31 0 2024-04-06T03:40:27.613 INFO:tasks.workunit.client.1.smithi136.stdout:0/38: rename d1/d6/f9 to d1/fc 0 2024-04-06T03:40:27.613 INFO:tasks.workunit.client.1.smithi136.stdout:0/39: write d1/fc [692918,40189] 0 2024-04-06T03:40:27.613 INFO:tasks.workunit.client.1.smithi136.stdout:0/40: creat d1/d6/fd x:0 0 0 2024-04-06T03:40:27.613 INFO:tasks.workunit.client.0.smithi094.stdout:4/131: symlink d0/l28 0 2024-04-06T03:40:27.614 INFO:tasks.workunit.client.1.smithi136.stdout:1/89: dread fd [0,4194304] 0 2024-04-06T03:40:27.615 INFO:tasks.workunit.client.1.smithi136.stdout:2/154: mkdir d3/d9/d10/d18/d1d/d37 0 2024-04-06T03:40:27.615 INFO:tasks.workunit.client.1.smithi136.stdout:2/155: dread - d3/f29 zero size 2024-04-06T03:40:27.615 INFO:tasks.workunit.client.1.smithi136.stdout:2/156: readlink d3/d1e/l2e 0 2024-04-06T03:40:27.615 INFO:tasks.workunit.client.1.smithi136.stdout:2/157: dread - d3/d9/d10/d18/f23 zero size 2024-04-06T03:40:27.615 INFO:tasks.workunit.client.0.smithi094.stdout:8/130: truncate d18/f19 2350989 0 2024-04-06T03:40:27.615 INFO:tasks.workunit.client.0.smithi094.stdout:8/131: read - f13 zero size 2024-04-06T03:40:27.616 INFO:tasks.workunit.client.0.smithi094.stdout:7/97: dwrite d2/d6/f9 [0,4194304] 0 2024-04-06T03:40:27.616 INFO:tasks.workunit.client.1.smithi136.stdout:5/77: mkdir d1/dc/d15 0 2024-04-06T03:40:27.617 INFO:tasks.workunit.client.0.smithi094.stdout:7/98: write d2/f11 [119787,118424] 0 2024-04-06T03:40:27.619 INFO:tasks.workunit.client.0.smithi094.stdout:8/132: dread f11 [0,4194304] 0 2024-04-06T03:40:27.619 INFO:tasks.workunit.client.0.smithi094.stdout:8/133: creat d18/f2d x:0 0 0 2024-04-06T03:40:27.619 INFO:tasks.workunit.client.0.smithi094.stdout:2/124: mkdir d0/d8/d27/d2b 0 2024-04-06T03:40:27.619 INFO:tasks.workunit.client.0.smithi094.stdout:2/125: chown d0 86369421 1 2024-04-06T03:40:27.620 INFO:tasks.workunit.client.0.smithi094.stdout:6/96: mknod dc/d11/d1b/c2c 0 2024-04-06T03:40:27.622 INFO:tasks.workunit.client.1.smithi136.stdout:3/36: symlink d0/d4/lc 0 2024-04-06T03:40:27.622 INFO:tasks.workunit.client.1.smithi136.stdout:3/37: creat d0/d4/fd x:0 0 0 2024-04-06T03:40:27.622 INFO:tasks.workunit.client.1.smithi136.stdout:3/38: read - d0/f3 zero size 2024-04-06T03:40:27.623 INFO:tasks.workunit.client.1.smithi136.stdout:3/39: fsync d0/f5 0 2024-04-06T03:40:27.624 INFO:tasks.workunit.client.0.smithi094.stdout:3/149: creat d9/d10/d27/f32 x:0 0 0 2024-04-06T03:40:27.626 INFO:tasks.workunit.client.0.smithi094.stdout:3/150: dread d9/d1d/f25 [0,4194304] 0 2024-04-06T03:40:27.626 INFO:tasks.workunit.client.1.smithi136.stdout:9/42: truncate f4 757682 0 2024-04-06T03:40:27.628 INFO:tasks.workunit.client.0.smithi094.stdout:7/99: creat d2/d6/dc/dd/df/f1a x:0 0 0 2024-04-06T03:40:27.628 INFO:tasks.workunit.client.0.smithi094.stdout:7/100: truncate d2/d6/dc/dd/df/f1a 627617 0 2024-04-06T03:40:27.628 INFO:tasks.workunit.client.0.smithi094.stdout:7/101: chown d2/d6/dc 93755 1 2024-04-06T03:40:27.629 INFO:tasks.workunit.client.0.smithi094.stdout:7/102: read d2/d6/dc/dd/df/f1a [141041,19332] 0 2024-04-06T03:40:27.629 INFO:tasks.workunit.client.0.smithi094.stdout:7/103: write d2/d6/f8 [2736991,97192] 0 2024-04-06T03:40:27.629 INFO:tasks.workunit.client.0.smithi094.stdout:7/104: truncate d2/d6/f9 4346874 0 2024-04-06T03:40:27.629 INFO:tasks.workunit.client.0.smithi094.stdout:7/105: chown d2/f11 1 1 2024-04-06T03:40:27.629 INFO:tasks.workunit.client.0.smithi094.stdout:3/151: dread d9/d20/f21 [0,4194304] 0 2024-04-06T03:40:27.629 INFO:tasks.workunit.client.0.smithi094.stdout:3/152: write d9/d1d/f25 [140412,110973] 0 2024-04-06T03:40:27.633 INFO:tasks.workunit.client.0.smithi094.stdout:7/106: write d2/d6/fa [762030,114018] 0 2024-04-06T03:40:27.636 INFO:tasks.workunit.client.0.smithi094.stdout:8/134: creat d18/d2c/f2e x:0 0 0 2024-04-06T03:40:27.639 INFO:tasks.workunit.client.1.smithi136.stdout:8/70: creat d3/de/ff x:0 0 0 2024-04-06T03:40:27.639 INFO:tasks.workunit.client.1.smithi136.stdout:8/71: creat d3/f10 x:0 0 0 2024-04-06T03:40:27.640 INFO:tasks.workunit.client.1.smithi136.stdout:8/72: creat d3/de/f11 x:0 0 0 2024-04-06T03:40:27.641 INFO:tasks.workunit.client.0.smithi094.stdout:6/97: symlink dc/d11/d22/l2d 0 2024-04-06T03:40:27.641 INFO:tasks.workunit.client.0.smithi094.stdout:6/98: write dc/f10 [1315828,122558] 0 2024-04-06T03:40:27.642 INFO:tasks.workunit.client.0.smithi094.stdout:3/153: dwrite d9/d20/f21 [0,4194304] 0 2024-04-06T03:40:27.642 INFO:tasks.workunit.client.0.smithi094.stdout:3/154: fdatasync d9/f1b 0 2024-04-06T03:40:27.643 INFO:tasks.workunit.client.0.smithi094.stdout:3/155: write d9/f18 [481343,107192] 0 2024-04-06T03:40:27.643 INFO:tasks.workunit.client.0.smithi094.stdout:3/156: chown d9/d10/d27 3696236 1 2024-04-06T03:40:27.643 INFO:tasks.workunit.client.0.smithi094.stdout:3/157: fdatasync d9/d20/f21 0 2024-04-06T03:40:27.643 INFO:tasks.workunit.client.0.smithi094.stdout:4/132: rmdir d0/d4 39 2024-04-06T03:40:27.643 INFO:tasks.workunit.client.0.smithi094.stdout:4/133: creat d0/f29 x:0 0 0 2024-04-06T03:40:27.643 INFO:tasks.workunit.client.0.smithi094.stdout:4/134: getdents d0/d23 0 2024-04-06T03:40:27.644 INFO:tasks.workunit.client.1.smithi136.stdout:8/73: write d3/f4 [4141464,70147] 0 2024-04-06T03:40:27.644 INFO:tasks.workunit.client.1.smithi136.stdout:8/74: write d3/f8 [1153221,12852] 0 2024-04-06T03:40:27.644 INFO:tasks.workunit.client.0.smithi094.stdout:7/107: dread d2/d6/f9 [0,4194304] 0 2024-04-06T03:40:27.646 INFO:tasks.workunit.client.0.smithi094.stdout:8/135: symlink d18/l2f 0 2024-04-06T03:40:27.646 INFO:tasks.workunit.client.0.smithi094.stdout:8/136: dread - d18/f28 zero size 2024-04-06T03:40:27.648 INFO:tasks.workunit.client.0.smithi094.stdout:2/126: getdents d0/d8/d1d 0 2024-04-06T03:40:27.649 INFO:tasks.workunit.client.0.smithi094.stdout:6/99: dread f5 [0,4194304] 0 2024-04-06T03:40:27.650 INFO:tasks.workunit.client.1.smithi136.stdout:1/90: mknod d12/c1d 0 2024-04-06T03:40:27.653 INFO:tasks.workunit.client.1.smithi136.stdout:2/158: link d3/d9/d10/c27 d3/d15/d19/c38 0 2024-04-06T03:40:27.655 INFO:tasks.workunit.client.1.smithi136.stdout:5/78: mkdir d1/d16 0 2024-04-06T03:40:27.655 INFO:tasks.workunit.client.1.smithi136.stdout:5/79: fdatasync d1/f4 0 2024-04-06T03:40:27.658 INFO:tasks.workunit.client.0.smithi094.stdout:2/127: dread d0/dd/ff [0,4194304] 0 2024-04-06T03:40:27.660 INFO:tasks.workunit.client.0.smithi094.stdout:6/100: write dc/f17 [3500595,118244] 0 2024-04-06T03:40:27.660 INFO:tasks.workunit.client.0.smithi094.stdout:6/101: dread - dc/d11/d1b/f1f zero size 2024-04-06T03:40:27.660 INFO:tasks.workunit.client.0.smithi094.stdout:7/108: rename d2/d6/l16 to d2/d6/dc/dd/l1b 0 2024-04-06T03:40:27.663 INFO:tasks.workunit.client.0.smithi094.stdout:8/137: mkdir d18/d30 0 2024-04-06T03:40:27.664 INFO:tasks.workunit.client.0.smithi094.stdout:3/158: rmdir d9/d1e/d26 39 2024-04-06T03:40:27.665 INFO:tasks.workunit.client.0.smithi094.stdout:4/135: dwrite d0/d4/f21 [0,4194304] 0 2024-04-06T03:40:27.665 INFO:tasks.workunit.client.0.smithi094.stdout:6/102: dread f8 [0,4194304] 0 2024-04-06T03:40:27.665 INFO:tasks.workunit.client.0.smithi094.stdout:6/103: truncate dc/d11/d1b/d24/f25 67066 0 2024-04-06T03:40:27.665 INFO:tasks.workunit.client.0.smithi094.stdout:6/104: fdatasync dc/d11/d1b/d24/f25 0 2024-04-06T03:40:27.665 INFO:tasks.workunit.client.1.smithi136.stdout:4/83: dwrite ff [0,4194304] 0 2024-04-06T03:40:27.667 INFO:tasks.workunit.client.0.smithi094.stdout:2/128: dread - d0/f20 zero size 2024-04-06T03:40:27.668 INFO:tasks.workunit.client.0.smithi094.stdout:7/109: dread d2/f11 [0,4194304] 0 2024-04-06T03:40:27.669 INFO:tasks.workunit.client.0.smithi094.stdout:7/110: stat d2/l7 0 2024-04-06T03:40:27.669 INFO:tasks.workunit.client.0.smithi094.stdout:7/111: readlink d2/d6/l18 0 2024-04-06T03:40:27.669 INFO:tasks.workunit.client.0.smithi094.stdout:7/112: read - d2/d6/dc/f15 zero size 2024-04-06T03:40:27.669 INFO:tasks.workunit.client.0.smithi094.stdout:7/113: write d2/d6/fa [213490,113245] 0 2024-04-06T03:40:27.675 INFO:tasks.workunit.client.1.smithi136.stdout:7/44: getdents d5/dc 0 2024-04-06T03:40:27.675 INFO:tasks.workunit.client.1.smithi136.stdout:7/45: chown c4 537518 1 2024-04-06T03:40:27.675 INFO:tasks.workunit.client.1.smithi136.stdout:7/46: write f1 [1123296,40140] 0 2024-04-06T03:40:27.678 INFO:tasks.workunit.client.1.smithi136.stdout:9/43: creat fa x:0 0 0 2024-04-06T03:40:27.678 INFO:tasks.workunit.client.1.smithi136.stdout:9/44: creat fb x:0 0 0 2024-04-06T03:40:27.678 INFO:tasks.workunit.client.1.smithi136.stdout:7/47: fdatasync f1 0 2024-04-06T03:40:27.678 INFO:tasks.workunit.client.0.smithi094.stdout:3/159: dwrite d9/ff [0,4194304] 0 2024-04-06T03:40:27.679 INFO:tasks.workunit.client.0.smithi094.stdout:3/160: read - d9/d1e/d26/f28 zero size 2024-04-06T03:40:27.679 INFO:tasks.workunit.client.0.smithi094.stdout:4/136: rename d0/fa to d0/df/d1e/f2a 0 2024-04-06T03:40:27.694 INFO:tasks.workunit.client.0.smithi094.stdout:4/137: dread d0/df/d1e/f2a [0,4194304] 0 2024-04-06T03:40:27.702 INFO:tasks.workunit.client.1.smithi136.stdout:7/48: dwrite d5/dc/fe [0,4194304] 0 2024-04-06T03:40:27.702 INFO:tasks.workunit.client.1.smithi136.stdout:9/45: dwrite f4 [0,4194304] 0 2024-04-06T03:40:27.702 INFO:tasks.workunit.client.1.smithi136.stdout:9/46: read - fb zero size 2024-04-06T03:40:27.702 INFO:tasks.workunit.client.1.smithi136.stdout:9/47: write fa [138781,2939] 0 2024-04-06T03:40:27.745 INFO:tasks.workunit.client.0.smithi094.stdout:6/105: creat dc/d11/d1b/d24/f2e x:0 0 0 2024-04-06T03:40:27.745 INFO:tasks.workunit.client.0.smithi094.stdout:6/106: chown dc/f12 1700000455 1 2024-04-06T03:40:27.749 INFO:tasks.workunit.client.1.smithi136.stdout:8/75: mknod d3/da/c12 0 2024-04-06T03:40:27.749 INFO:tasks.workunit.client.1.smithi136.stdout:8/76: creat d3/da/f13 x:0 0 0 2024-04-06T03:40:27.750 INFO:tasks.workunit.client.0.smithi094.stdout:7/114: creat d2/d6/dc/d12/f1c x:0 0 0 2024-04-06T03:40:27.751 INFO:tasks.workunit.client.1.smithi136.stdout:0/41: symlink d1/le 0 2024-04-06T03:40:27.751 INFO:tasks.workunit.client.1.smithi136.stdout:0/42: chown d1/fc 186 1 2024-04-06T03:40:27.752 INFO:tasks.workunit.client.1.smithi136.stdout:1/91: mkdir d12/d1b/d1e 0 2024-04-06T03:40:27.752 INFO:tasks.workunit.client.1.smithi136.stdout:1/92: chown c2 61169838 1 2024-04-06T03:40:27.754 INFO:tasks.workunit.client.1.smithi136.stdout:2/159: link d3/d15/d19/f1b d3/d15/d19/f39 0 2024-04-06T03:40:27.758 INFO:tasks.workunit.client.1.smithi136.stdout:5/80: mkdir d1/d16/d17 0 2024-04-06T03:40:27.758 INFO:tasks.workunit.client.1.smithi136.stdout:5/81: read - d1/dc/d10/f11 zero size 2024-04-06T03:40:27.759 INFO:tasks.workunit.client.1.smithi136.stdout:3/40: rmdir d0/d4 39 2024-04-06T03:40:27.759 INFO:tasks.workunit.client.1.smithi136.stdout:3/41: dread - d0/f5 zero size 2024-04-06T03:40:27.759 INFO:tasks.workunit.client.1.smithi136.stdout:3/42: fsync d0/f3 0 2024-04-06T03:40:27.759 INFO:tasks.workunit.client.1.smithi136.stdout:3/43: truncate d0/d4/fa 86359 0 2024-04-06T03:40:27.760 INFO:tasks.workunit.client.0.smithi094.stdout:8/138: rmdir d18/d2c 39 2024-04-06T03:40:27.760 INFO:tasks.workunit.client.0.smithi094.stdout:8/139: truncate d18/f27 984750 0 2024-04-06T03:40:27.765 INFO:tasks.workunit.client.1.smithi136.stdout:7/49: mkdir d5/df 0 2024-04-06T03:40:27.772 INFO:tasks.workunit.client.0.smithi094.stdout:3/161: truncate d9/fc 3352922 0 2024-04-06T03:40:27.774 INFO:tasks.workunit.client.1.smithi136.stdout:1/93: dwrite f11 [0,4194304] 0 2024-04-06T03:40:27.776 INFO:tasks.workunit.client.0.smithi094.stdout:7/115: dwrite d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:27.778 INFO:tasks.workunit.client.1.smithi136.stdout:7/50: read f3 [3316069,18455] 0 2024-04-06T03:40:27.778 INFO:tasks.workunit.client.1.smithi136.stdout:7/51: getdents d5/df 0 2024-04-06T03:40:27.778 INFO:tasks.workunit.client.1.smithi136.stdout:7/52: chown d5/df 1 1 2024-04-06T03:40:27.778 INFO:tasks.workunit.client.1.smithi136.stdout:5/82: dwrite d1/dc/d10/f11 [0,4194304] 0 2024-04-06T03:40:27.779 INFO:tasks.workunit.client.0.smithi094.stdout:4/138: unlink d0/d4/l16 0 2024-04-06T03:40:27.780 INFO:tasks.workunit.client.1.smithi136.stdout:9/48: mkdir dc 0 2024-04-06T03:40:27.786 INFO:tasks.workunit.client.0.smithi094.stdout:6/107: creat dc/d11/d1b/f2f x:0 0 0 2024-04-06T03:40:27.786 INFO:tasks.workunit.client.0.smithi094.stdout:6/108: creat dc/d11/d22/f30 x:0 0 0 2024-04-06T03:40:27.786 INFO:tasks.workunit.client.0.smithi094.stdout:6/109: truncate dc/d11/d22/f30 661126 0 2024-04-06T03:40:27.786 INFO:tasks.workunit.client.0.smithi094.stdout:6/110: creat dc/d11/d1b/f31 x:0 0 0 2024-04-06T03:40:27.786 INFO:tasks.workunit.client.0.smithi094.stdout:6/111: dread - dc/d11/d1b/f2f zero size 2024-04-06T03:40:27.786 INFO:tasks.workunit.client.0.smithi094.stdout:6/112: fdatasync dc/f29 0 2024-04-06T03:40:27.787 INFO:tasks.workunit.client.1.smithi136.stdout:8/77: symlink d3/l14 0 2024-04-06T03:40:27.787 INFO:tasks.workunit.client.1.smithi136.stdout:8/78: stat d3/de/ff 0 2024-04-06T03:40:27.787 INFO:tasks.workunit.client.1.smithi136.stdout:8/79: chown d3/f10 891693620 1 2024-04-06T03:40:27.788 INFO:tasks.workunit.client.1.smithi136.stdout:5/83: write d1/dc/d10/f11 [3139880,101696] 0 2024-04-06T03:40:27.788 INFO:tasks.workunit.client.1.smithi136.stdout:5/84: write d1/f4 [1015762,93946] 0 2024-04-06T03:40:27.788 INFO:tasks.workunit.client.0.smithi094.stdout:3/162: creat d9/d20/f33 x:0 0 0 2024-04-06T03:40:27.789 INFO:tasks.workunit.client.0.smithi094.stdout:3/163: dread - d9/d10/f22 zero size 2024-04-06T03:40:27.790 INFO:tasks.workunit.client.1.smithi136.stdout:8/80: dread f1 [0,4194304] 0 2024-04-06T03:40:27.790 INFO:tasks.workunit.client.1.smithi136.stdout:8/81: chown d3/de/f11 12778 1 2024-04-06T03:40:27.790 INFO:tasks.workunit.client.1.smithi136.stdout:8/82: fdatasync d3/da/fd 0 2024-04-06T03:40:27.791 INFO:tasks.workunit.client.1.smithi136.stdout:0/43: symlink d1/d6/lf 0 2024-04-06T03:40:27.793 INFO:tasks.workunit.client.0.smithi094.stdout:8/140: dwrite f14 [4194304,4194304] 0 2024-04-06T03:40:27.793 INFO:tasks.workunit.client.1.smithi136.stdout:2/160: mknod d3/d9/d10/c3a 0 2024-04-06T03:40:27.796 INFO:tasks.workunit.client.1.smithi136.stdout:3/44: creat d0/d4/fe x:0 0 0 2024-04-06T03:40:27.796 INFO:tasks.workunit.client.1.smithi136.stdout:3/45: write d0/f5 [12763,61201] 0 2024-04-06T03:40:27.797 INFO:tasks.workunit.client.1.smithi136.stdout:3/46: write d0/f6 [1579940,39391] 0 2024-04-06T03:40:27.797 INFO:tasks.workunit.client.1.smithi136.stdout:3/47: dread - d0/d4/fd zero size 2024-04-06T03:40:27.797 INFO:tasks.workunit.client.0.smithi094.stdout:7/116: dwrite d2/d6/dc/f15 [0,4194304] 0 2024-04-06T03:40:27.797 INFO:tasks.workunit.client.0.smithi094.stdout:7/117: fsync d2/fb 0 2024-04-06T03:40:27.798 INFO:tasks.workunit.client.1.smithi136.stdout:4/84: rmdir d1d 39 2024-04-06T03:40:27.809 INFO:tasks.workunit.client.1.smithi136.stdout:1/94: mkdir d12/d1b/d1e/d1f 0 2024-04-06T03:40:27.809 INFO:tasks.workunit.client.1.smithi136.stdout:1/95: stat d12/c19 0 2024-04-06T03:40:27.809 INFO:tasks.workunit.client.1.smithi136.stdout:1/96: write f3 [8808658,105099] 0 2024-04-06T03:40:27.809 INFO:tasks.workunit.client.1.smithi136.stdout:9/49: dwrite f4 [0,4194304] 0 2024-04-06T03:40:27.809 INFO:tasks.workunit.client.1.smithi136.stdout:9/50: chown fa 24825 1 2024-04-06T03:40:27.811 INFO:tasks.workunit.client.0.smithi094.stdout:6/113: link dc/d11/d1b/f1f dc/d11/d1b/d24/f32 0 2024-04-06T03:40:27.812 INFO:tasks.workunit.client.0.smithi094.stdout:6/114: write fa [151721,46558] 0 2024-04-06T03:40:27.813 INFO:tasks.workunit.client.1.smithi136.stdout:7/53: creat d5/df/f10 x:0 0 0 2024-04-06T03:40:27.813 INFO:tasks.workunit.client.1.smithi136.stdout:7/54: stat d5/df 0 2024-04-06T03:40:27.813 INFO:tasks.workunit.client.1.smithi136.stdout:0/44: dwrite d1/d6/fb [0,4194304] 0 2024-04-06T03:40:27.813 INFO:tasks.workunit.client.1.smithi136.stdout:8/83: dwrite d3/fc [0,4194304] 0 2024-04-06T03:40:27.813 INFO:tasks.workunit.client.1.smithi136.stdout:8/84: readlink d3/l14 0 2024-04-06T03:40:27.814 INFO:tasks.workunit.client.0.smithi094.stdout:3/164: rename d9/d20/f23 to d9/d20/f34 0 2024-04-06T03:40:27.817 INFO:tasks.workunit.client.0.smithi094.stdout:8/141: symlink d18/d2b/l31 0 2024-04-06T03:40:27.818 INFO:tasks.workunit.client.1.smithi136.stdout:3/48: dwrite d0/f2 [0,4194304] 0 2024-04-06T03:40:27.818 INFO:tasks.workunit.client.0.smithi094.stdout:4/139: mknod d0/d4/c2b 0 2024-04-06T03:40:27.821 INFO:tasks.workunit.client.1.smithi136.stdout:5/85: rename d1/dc/d10/f12 to d1/f18 0 2024-04-06T03:40:27.823 INFO:tasks.workunit.client.1.smithi136.stdout:2/161: mkdir d3/d9/d3b 0 2024-04-06T03:40:27.825 INFO:tasks.workunit.client.1.smithi136.stdout:1/97: mknod d12/c20 0 2024-04-06T03:40:27.825 INFO:tasks.workunit.client.1.smithi136.stdout:2/162: dread d3/d15/f2b [0,4194304] 0 2024-04-06T03:40:27.826 INFO:tasks.workunit.client.0.smithi094.stdout:7/118: getdents d2/d6/dc/dd 0 2024-04-06T03:40:27.830 INFO:tasks.workunit.client.1.smithi136.stdout:9/51: rmdir dc 0 2024-04-06T03:40:27.830 INFO:tasks.workunit.client.1.smithi136.stdout:9/52: write f7 [1266109,5555] 0 2024-04-06T03:40:27.830 INFO:tasks.workunit.client.1.smithi136.stdout:9/53: creat fd x:0 0 0 2024-04-06T03:40:27.830 INFO:tasks.workunit.client.1.smithi136.stdout:9/54: chown c9 5048802 1 2024-04-06T03:40:27.831 INFO:tasks.workunit.client.0.smithi094.stdout:6/115: link dc/l1a dc/l33 0 2024-04-06T03:40:27.832 INFO:tasks.workunit.client.1.smithi136.stdout:4/85: dwrite f7 [0,4194304] 0 2024-04-06T03:40:27.832 INFO:tasks.workunit.client.0.smithi094.stdout:1/115: sync 2024-04-06T03:40:27.832 INFO:tasks.workunit.client.1.smithi136.stdout:7/55: mkdir d5/df/d11 0 2024-04-06T03:40:27.835 INFO:tasks.workunit.client.0.smithi094.stdout:3/165: rename d9/l2a to d9/d1e/d26/l35 0 2024-04-06T03:40:27.835 INFO:tasks.workunit.client.0.smithi094.stdout:3/166: chown d9/f1b 1041020533 1 2024-04-06T03:40:27.839 INFO:tasks.workunit.client.0.smithi094.stdout:8/142: truncate f3 112717 0 2024-04-06T03:40:27.843 INFO:tasks.workunit.client.1.smithi136.stdout:0/45: mknod d1/d6/c10 0 2024-04-06T03:40:27.846 INFO:tasks.workunit.client.1.smithi136.stdout:1/98: dwrite f10 [4194304,4194304] 0 2024-04-06T03:40:27.846 INFO:tasks.workunit.client.1.smithi136.stdout:1/99: write fb [617996,33974] 0 2024-04-06T03:40:27.846 INFO:tasks.workunit.client.1.smithi136.stdout:1/100: write d12/f16 [147441,91560] 0 2024-04-06T03:40:27.846 INFO:tasks.workunit.client.1.smithi136.stdout:1/101: write d12/f16 [824930,10684] 0 2024-04-06T03:40:27.847 INFO:tasks.workunit.client.0.smithi094.stdout:6/116: dwrite dc/d11/d22/f27 [0,4194304] 0 2024-04-06T03:40:27.847 INFO:tasks.workunit.client.0.smithi094.stdout:6/117: chown dc/d11/d1b/f2f 1440 1 2024-04-06T03:40:27.848 INFO:tasks.workunit.client.1.smithi136.stdout:8/85: symlink d3/l15 0 2024-04-06T03:40:27.849 INFO:tasks.workunit.client.1.smithi136.stdout:6/60: sync 2024-04-06T03:40:27.854 INFO:tasks.workunit.client.0.smithi094.stdout:7/119: mknod d2/d6/dc/dd/df/c1d 0 2024-04-06T03:40:27.854 INFO:tasks.workunit.client.0.smithi094.stdout:8/143: write f1 [5261346,21324] 0 2024-04-06T03:40:27.856 INFO:tasks.workunit.client.1.smithi136.stdout:3/49: symlink d0/lf 0 2024-04-06T03:40:27.859 INFO:tasks.workunit.client.1.smithi136.stdout:4/86: dwrite f1a [0,4194304] 0 2024-04-06T03:40:27.860 INFO:tasks.workunit.client.0.smithi094.stdout:1/116: mknod d0/d7/d1a/c2b 0 2024-04-06T03:40:27.861 INFO:tasks.workunit.client.1.smithi136.stdout:5/86: unlink d1/f14 0 2024-04-06T03:40:27.863 INFO:tasks.workunit.client.0.smithi094.stdout:3/167: creat d9/d1d/f36 x:0 0 0 2024-04-06T03:40:27.867 INFO:tasks.workunit.client.1.smithi136.stdout:2/163: mkdir d3/d9/d10/d18/d1d/d3c 0 2024-04-06T03:40:27.867 INFO:tasks.workunit.client.1.smithi136.stdout:2/164: dread - d3/d15/d19/f1b zero size 2024-04-06T03:40:27.872 INFO:tasks.workunit.client.0.smithi094.stdout:6/118: dwrite dc/fd [0,4194304] 0 2024-04-06T03:40:27.872 INFO:tasks.workunit.client.0.smithi094.stdout:6/119: write dc/f17 [1793438,125123] 0 2024-04-06T03:40:27.874 INFO:tasks.workunit.client.1.smithi136.stdout:9/55: rename f7 to fe 0 2024-04-06T03:40:27.877 INFO:tasks.workunit.client.1.smithi136.stdout:7/56: mkdir d5/df/d12 0 2024-04-06T03:40:27.879 INFO:tasks.workunit.client.1.smithi136.stdout:0/46: truncate d1/f7 141765 0 2024-04-06T03:40:27.879 INFO:tasks.workunit.client.0.smithi094.stdout:1/117: dwrite d0/d7/f26 [0,4194304] 0 2024-04-06T03:40:27.881 INFO:tasks.workunit.client.0.smithi094.stdout:7/120: dwrite d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:27.881 INFO:tasks.workunit.client.0.smithi094.stdout:7/121: chown d2/d6/dc/d12/f1c 226632961 1 2024-04-06T03:40:27.882 INFO:tasks.workunit.client.1.smithi136.stdout:1/102: rename d12/c17 to d12/d1b/c21 0 2024-04-06T03:40:27.883 INFO:tasks.workunit.client.1.smithi136.stdout:8/86: mknod d3/c16 0 2024-04-06T03:40:27.885 INFO:tasks.workunit.client.0.smithi094.stdout:1/118: dread d0/d4/f29 [0,4194304] 0 2024-04-06T03:40:27.885 INFO:tasks.workunit.client.0.smithi094.stdout:1/119: fsync d0/d1/f10 0 2024-04-06T03:40:27.886 INFO:tasks.workunit.client.0.smithi094.stdout:4/140: rmdir d0 39 2024-04-06T03:40:27.886 INFO:tasks.workunit.client.0.smithi094.stdout:4/141: fdatasync d0/d4/f14 0 2024-04-06T03:40:27.887 INFO:tasks.workunit.client.0.smithi094.stdout:3/168: dwrite d9/d10/f22 [0,4194304] 0 2024-04-06T03:40:27.887 INFO:tasks.workunit.client.1.smithi136.stdout:5/87: dwrite d1/f4 [0,4194304] 0 2024-04-06T03:40:27.888 INFO:tasks.workunit.client.0.smithi094.stdout:7/122: read d2/d6/f9 [1058403,28106] 0 2024-04-06T03:40:27.889 INFO:tasks.workunit.client.0.smithi094.stdout:7/123: fdatasync d2/d6/dc/f15 0 2024-04-06T03:40:27.889 INFO:tasks.workunit.client.0.smithi094.stdout:7/124: read d2/d6/fa [1197734,87004] 0 2024-04-06T03:40:27.892 INFO:tasks.workunit.client.1.smithi136.stdout:5/88: dread d1/dc/d10/f11 [0,4194304] 0 2024-04-06T03:40:27.892 INFO:tasks.workunit.client.0.smithi094.stdout:8/144: mknod d18/d30/c32 0 2024-04-06T03:40:27.892 INFO:tasks.workunit.client.0.smithi094.stdout:8/145: dread - d18/f20 zero size 2024-04-06T03:40:27.893 INFO:tasks.workunit.client.1.smithi136.stdout:7/57: write d5/dc/fe [4020415,49916] 0 2024-04-06T03:40:27.893 INFO:tasks.workunit.client.0.smithi094.stdout:1/120: write d0/d7/f26 [1666610,8989] 0 2024-04-06T03:40:27.896 INFO:tasks.workunit.client.0.smithi094.stdout:6/120: unlink dc/l18 0 2024-04-06T03:40:27.896 INFO:tasks.workunit.client.0.smithi094.stdout:6/121: creat dc/d11/d22/f34 x:0 0 0 2024-04-06T03:40:27.909 INFO:tasks.workunit.client.0.smithi094.stdout:5/105: sync 2024-04-06T03:40:27.914 INFO:tasks.workunit.client.0.smithi094.stdout:8/146: dwrite d18/f22 [0,4194304] 0 2024-04-06T03:40:27.914 INFO:tasks.workunit.client.0.smithi094.stdout:8/147: truncate d18/f19 2397873 0 2024-04-06T03:40:27.915 INFO:tasks.workunit.client.0.smithi094.stdout:4/142: getdents d0/df 0 2024-04-06T03:40:27.921 INFO:tasks.workunit.client.1.smithi136.stdout:0/47: fsync d1/f7 0 2024-04-06T03:40:27.922 INFO:tasks.workunit.client.1.smithi136.stdout:4/87: read f8 [2542318,33850] 0 2024-04-06T03:40:27.923 INFO:tasks.workunit.client.0.smithi094.stdout:3/169: truncate d9/ff 2669119 0 2024-04-06T03:40:27.924 INFO:tasks.workunit.client.0.smithi094.stdout:6/122: dwrite dc/f29 [0,4194304] 0 2024-04-06T03:40:27.925 INFO:tasks.workunit.client.1.smithi136.stdout:9/56: mknod cf 0 2024-04-06T03:40:27.925 INFO:tasks.workunit.client.1.smithi136.stdout:9/57: dread - fd zero size 2024-04-06T03:40:27.926 INFO:tasks.workunit.client.0.smithi094.stdout:1/121: dwrite d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:27.926 INFO:tasks.workunit.client.0.smithi094.stdout:1/122: chown d0/d7/d1a/c2b 92143 1 2024-04-06T03:40:27.926 INFO:tasks.workunit.client.1.smithi136.stdout:5/89: dwrite d1/f18 [0,4194304] 0 2024-04-06T03:40:27.926 INFO:tasks.workunit.client.0.smithi094.stdout:7/125: rename d2/l7 to d2/d6/dc/l1e 0 2024-04-06T03:40:27.927 INFO:tasks.workunit.client.0.smithi094.stdout:7/126: chown d2/d6/dc/d12/f1c 91 1 2024-04-06T03:40:27.929 INFO:tasks.workunit.client.1.smithi136.stdout:2/165: rename d3/f29 to d3/d15/d19/f3d 0 2024-04-06T03:40:27.931 INFO:tasks.workunit.client.0.smithi094.stdout:5/106: link db/d19/l22 db/d19/l2b 0 2024-04-06T03:40:27.932 INFO:tasks.workunit.client.0.smithi094.stdout:5/107: dread db/f11 [4194304,4194304] 0 2024-04-06T03:40:27.932 INFO:tasks.workunit.client.0.smithi094.stdout:5/108: readlink db/d19/l2b 0 2024-04-06T03:40:27.932 INFO:tasks.workunit.client.1.smithi136.stdout:2/166: dread d3/d9/d10/f22 [0,4194304] 0 2024-04-06T03:40:27.932 INFO:tasks.workunit.client.1.smithi136.stdout:2/167: stat d3/d9/d10/f22 0 2024-04-06T03:40:27.932 INFO:tasks.workunit.client.1.smithi136.stdout:2/168: fsync d3/d9/d10/d18/f2d 0 2024-04-06T03:40:27.932 INFO:tasks.workunit.client.1.smithi136.stdout:2/169: fsync d3/d9/d10/d18/f23 0 2024-04-06T03:40:27.933 INFO:tasks.workunit.client.1.smithi136.stdout:1/103: unlink d12/l15 0 2024-04-06T03:40:27.934 INFO:tasks.workunit.client.0.smithi094.stdout:4/143: symlink d0/d15/l2c 0 2024-04-06T03:40:27.936 INFO:tasks.workunit.client.1.smithi136.stdout:2/170: dread d3/d9/d10/d18/f2d [0,4194304] 0 2024-04-06T03:40:27.939 INFO:tasks.workunit.client.1.smithi136.stdout:0/48: unlink d1/c3 0 2024-04-06T03:40:27.939 INFO:tasks.workunit.client.1.smithi136.stdout:0/49: write d1/d6/fd [67435,9153] 0 2024-04-06T03:40:27.939 INFO:tasks.workunit.client.1.smithi136.stdout:0/50: creat d1/d6/f11 x:0 0 0 2024-04-06T03:40:27.941 INFO:tasks.workunit.client.1.smithi136.stdout:4/88: creat d1d/f20 x:0 0 0 2024-04-06T03:40:27.942 INFO:tasks.workunit.client.1.smithi136.stdout:9/58: symlink l10 0 2024-04-06T03:40:27.942 INFO:tasks.workunit.client.1.smithi136.stdout:9/59: write fa [853813,50191] 0 2024-04-06T03:40:27.942 INFO:tasks.workunit.client.1.smithi136.stdout:9/60: creat f11 x:0 0 0 2024-04-06T03:40:27.942 INFO:tasks.workunit.client.1.smithi136.stdout:9/61: write fd [662752,41439] 0 2024-04-06T03:40:27.943 INFO:tasks.workunit.client.1.smithi136.stdout:9/62: readlink l10 0 2024-04-06T03:40:27.943 INFO:tasks.workunit.client.1.smithi136.stdout:0/51: dread d1/f4 [0,4194304] 0 2024-04-06T03:40:27.944 INFO:tasks.workunit.client.1.smithi136.stdout:0/52: creat d1/d6/f12 x:0 0 0 2024-04-06T03:40:27.945 INFO:tasks.workunit.client.0.smithi094.stdout:1/123: creat d0/d4/d1b/f2c x:0 0 0 2024-04-06T03:40:27.945 INFO:tasks.workunit.client.0.smithi094.stdout:1/124: truncate d0/d4/f29 5073260 0 2024-04-06T03:40:27.945 INFO:tasks.workunit.client.0.smithi094.stdout:9/74: sync 2024-04-06T03:40:27.945 INFO:tasks.workunit.client.0.smithi094.stdout:9/75: chown d2/c19 6739 1 2024-04-06T03:40:27.945 INFO:tasks.workunit.client.0.smithi094.stdout:3/170: dread d9/f19 [0,4194304] 0 2024-04-06T03:40:27.946 INFO:tasks.workunit.client.0.smithi094.stdout:3/171: write d9/d1e/d26/f2c [1066685,17647] 0 2024-04-06T03:40:27.947 INFO:tasks.workunit.client.0.smithi094.stdout:5/109: mkdir db/d19/d2c 0 2024-04-06T03:40:27.947 INFO:tasks.workunit.client.1.smithi136.stdout:0/53: dread d1/fc [0,4194304] 0 2024-04-06T03:40:27.947 INFO:tasks.workunit.client.1.smithi136.stdout:0/54: chown d1/d6/f12 8764322 1 2024-04-06T03:40:27.948 INFO:tasks.workunit.client.1.smithi136.stdout:7/58: rename f3 to d5/df/d12/f13 0 2024-04-06T03:40:27.949 INFO:tasks.workunit.client.0.smithi094.stdout:0/122: sync 2024-04-06T03:40:27.949 INFO:tasks.workunit.client.0.smithi094.stdout:2/129: sync 2024-04-06T03:40:27.951 INFO:tasks.workunit.client.1.smithi136.stdout:5/90: dwrite d1/dc/d10/f11 [0,4194304] 0 2024-04-06T03:40:27.952 INFO:tasks.workunit.client.1.smithi136.stdout:4/89: write f1a [1692854,6029] 0 2024-04-06T03:40:27.952 INFO:tasks.workunit.client.1.smithi136.stdout:4/90: dread - d1d/f1f zero size 2024-04-06T03:40:27.953 INFO:tasks.workunit.client.0.smithi094.stdout:3/172: dread d9/d20/f21 [0,4194304] 0 2024-04-06T03:40:27.953 INFO:tasks.workunit.client.0.smithi094.stdout:6/123: link l1 dc/d11/l35 0 2024-04-06T03:40:27.954 INFO:tasks.workunit.client.1.smithi136.stdout:2/171: mknod d3/d9/d10/c3e 0 2024-04-06T03:40:27.954 INFO:tasks.workunit.client.1.smithi136.stdout:2/172: dread - d3/d9/d10/d18/d1d/f35 zero size 2024-04-06T03:40:27.956 INFO:tasks.workunit.client.0.smithi094.stdout:1/125: rename d0/d7/d17 to d0/d7/d2d 0 2024-04-06T03:40:27.956 INFO:tasks.workunit.client.0.smithi094.stdout:1/126: write d0/d4/d1b/f2c [638084,28690] 0 2024-04-06T03:40:27.959 INFO:tasks.workunit.client.0.smithi094.stdout:9/76: symlink d2/l1a 0 2024-04-06T03:40:27.959 INFO:tasks.workunit.client.0.smithi094.stdout:9/77: fdatasync d2/fe 0 2024-04-06T03:40:27.959 INFO:tasks.workunit.client.0.smithi094.stdout:9/78: write d2/dc/d10/f12 [4711233,107969] 0 2024-04-06T03:40:27.960 INFO:tasks.workunit.client.0.smithi094.stdout:9/79: truncate d2/f4 4778442 0 2024-04-06T03:40:27.960 INFO:tasks.workunit.client.0.smithi094.stdout:9/80: stat d2/dc/f18 0 2024-04-06T03:40:27.960 INFO:tasks.workunit.client.0.smithi094.stdout:9/81: creat d2/dc/d10/f1b x:0 0 0 2024-04-06T03:40:27.960 INFO:tasks.workunit.client.0.smithi094.stdout:7/127: dwrite d2/d6/dc/d12/f1c [0,4194304] 0 2024-04-06T03:40:27.960 INFO:tasks.workunit.client.1.smithi136.stdout:1/104: dwrite fb [0,4194304] 0 2024-04-06T03:40:27.960 INFO:tasks.workunit.client.1.smithi136.stdout:1/105: chown f10 458701 1 2024-04-06T03:40:27.963 INFO:tasks.workunit.client.0.smithi094.stdout:5/110: creat db/d19/d2c/f2d x:0 0 0 2024-04-06T03:40:27.963 INFO:tasks.workunit.client.0.smithi094.stdout:5/111: write db/d13/f25 [398293,106070] 0 2024-04-06T03:40:27.963 INFO:tasks.workunit.client.0.smithi094.stdout:5/112: chown c7 1 1 2024-04-06T03:40:27.965 INFO:tasks.workunit.client.1.smithi136.stdout:9/63: mknod c12 0 2024-04-06T03:40:27.965 INFO:tasks.workunit.client.1.smithi136.stdout:9/64: creat f13 x:0 0 0 2024-04-06T03:40:27.965 INFO:tasks.workunit.client.1.smithi136.stdout:9/65: truncate f13 447362 0 2024-04-06T03:40:27.965 INFO:tasks.workunit.client.1.smithi136.stdout:5/91: write d1/f18 [3298241,40833] 0 2024-04-06T03:40:27.965 INFO:tasks.workunit.client.1.smithi136.stdout:5/92: rename d1 to d1/d16/d19 22 2024-04-06T03:40:27.967 INFO:tasks.workunit.client.0.smithi094.stdout:0/123: rmdir d1/d9/dd/d18/d1e 39 2024-04-06T03:40:27.971 INFO:tasks.workunit.client.0.smithi094.stdout:7/128: dread d2/d6/dc/d12/f1c [0,4194304] 0 2024-04-06T03:40:27.971 INFO:tasks.workunit.client.0.smithi094.stdout:7/129: chown d2/d6/dc/dd/df/f1a 106375 1 2024-04-06T03:40:27.971 INFO:tasks.workunit.client.0.smithi094.stdout:7/130: readlink d2/d6/dc/l1e 0 2024-04-06T03:40:27.971 INFO:tasks.workunit.client.0.smithi094.stdout:7/131: fdatasync d2/d6/f9 0 2024-04-06T03:40:27.971 INFO:tasks.workunit.client.0.smithi094.stdout:7/132: creat d2/f1f x:0 0 0 2024-04-06T03:40:27.971 INFO:tasks.workunit.client.0.smithi094.stdout:7/133: write d2/d6/f9 [2786938,60731] 0 2024-04-06T03:40:27.971 INFO:tasks.workunit.client.0.smithi094.stdout:7/134: chown d2/d6/dc/d12 32114 1 2024-04-06T03:40:27.971 INFO:tasks.workunit.client.0.smithi094.stdout:2/130: rename d0/f20 to d0/d8/d27/d2b/f2c 0 2024-04-06T03:40:27.972 INFO:tasks.workunit.client.1.smithi136.stdout:0/55: mkdir d1/d6/d13 0 2024-04-06T03:40:27.972 INFO:tasks.workunit.client.1.smithi136.stdout:0/56: write d1/f7 [855559,114633] 0 2024-04-06T03:40:27.973 INFO:tasks.workunit.client.0.smithi094.stdout:3/173: creat d9/d20/d30/f37 x:0 0 0 2024-04-06T03:40:27.974 INFO:tasks.workunit.client.0.smithi094.stdout:8/148: dwrite f14 [4194304,4194304] 0 2024-04-06T03:40:27.975 INFO:tasks.workunit.client.0.smithi094.stdout:2/131: dread d0/d8/f11 [0,4194304] 0 2024-04-06T03:40:27.975 INFO:tasks.workunit.client.1.smithi136.stdout:7/59: dwrite d5/fd [0,4194304] 0 2024-04-06T03:40:27.975 INFO:tasks.workunit.client.0.smithi094.stdout:6/124: symlink dc/d11/d1b/l36 0 2024-04-06T03:40:27.976 INFO:tasks.workunit.client.0.smithi094.stdout:4/144: getdents d0/d15 0 2024-04-06T03:40:27.977 INFO:tasks.workunit.client.0.smithi094.stdout:4/145: write d0/df/d1e/f25 [8340368,69702] 0 2024-04-06T03:40:27.977 INFO:tasks.workunit.client.0.smithi094.stdout:4/146: stat d0/df/c26 0 2024-04-06T03:40:27.978 INFO:tasks.workunit.client.0.smithi094.stdout:1/127: truncate d0/d1/d16/f1c 1292498 0 2024-04-06T03:40:27.985 INFO:tasks.workunit.client.1.smithi136.stdout:2/173: symlink d3/d9/d10/l3f 0 2024-04-06T03:40:27.987 INFO:tasks.workunit.client.0.smithi094.stdout:9/82: symlink d2/dc/d10/l1c 0 2024-04-06T03:40:27.991 INFO:tasks.workunit.client.1.smithi136.stdout:1/106: mkdir d12/d1b/d22 0 2024-04-06T03:40:27.995 INFO:tasks.workunit.client.0.smithi094.stdout:0/124: truncate d1/d9/dd/f16 3056152 0 2024-04-06T03:40:27.999 INFO:tasks.workunit.client.0.smithi094.stdout:4/147: dwrite d0/d4/f21 [0,4194304] 0 2024-04-06T03:40:27.999 INFO:tasks.workunit.client.0.smithi094.stdout:4/148: dread - d0/f29 zero size 2024-04-06T03:40:27.999 INFO:tasks.workunit.client.0.smithi094.stdout:5/113: write db/ff [3553557,4227] 0 2024-04-06T03:40:27.999 INFO:tasks.workunit.client.0.smithi094.stdout:6/125: dwrite dc/f28 [0,4194304] 0 2024-04-06T03:40:28.000 INFO:tasks.workunit.client.1.smithi136.stdout:5/93: creat d1/d16/d17/f1a x:0 0 0 2024-04-06T03:40:28.001 INFO:tasks.workunit.client.0.smithi094.stdout:7/135: link d2/d6/dc/c19 d2/d6/dc/dd/c20 0 2024-04-06T03:40:28.001 INFO:tasks.workunit.client.0.smithi094.stdout:7/136: fsync d2/d6/dc/d12/f1c 0 2024-04-06T03:40:28.004 INFO:tasks.workunit.client.0.smithi094.stdout:4/149: dread d0/df/d1e/f2a [0,4194304] 0 2024-04-06T03:40:28.005 INFO:tasks.workunit.client.0.smithi094.stdout:3/174: getdents d9/d20/d2d 0 2024-04-06T03:40:28.011 INFO:tasks.workunit.client.0.smithi094.stdout:2/132: creat d0/d8/d27/d2b/f2d x:0 0 0 2024-04-06T03:40:28.017 INFO:tasks.workunit.client.0.smithi094.stdout:9/83: rename d2/dc/d10/f16 to d2/f1d 0 2024-04-06T03:40:28.018 INFO:tasks.workunit.client.0.smithi094.stdout:9/84: fdatasync d2/fb 0 2024-04-06T03:40:28.018 INFO:tasks.workunit.client.0.smithi094.stdout:9/85: chown d2/dc 11706328 1 2024-04-06T03:40:28.018 INFO:tasks.workunit.client.1.smithi136.stdout:1/107: write fb [4757468,57170] 0 2024-04-06T03:40:28.019 INFO:tasks.workunit.client.0.smithi094.stdout:7/137: dwrite d2/fb [0,4194304] 0 2024-04-06T03:40:28.019 INFO:tasks.workunit.client.0.smithi094.stdout:7/138: chown d2/d6/dc/l1e 2 1 2024-04-06T03:40:28.020 INFO:tasks.workunit.client.0.smithi094.stdout:0/125: unlink d1/f5 0 2024-04-06T03:40:28.021 INFO:tasks.workunit.client.1.smithi136.stdout:4/91: rmdir d1d 39 2024-04-06T03:40:28.022 INFO:tasks.workunit.client.0.smithi094.stdout:6/126: dwrite dc/d11/f20 [0,4194304] 0 2024-04-06T03:40:28.022 INFO:tasks.workunit.client.0.smithi094.stdout:6/127: chown cb 27834819 1 2024-04-06T03:40:28.022 INFO:tasks.workunit.client.0.smithi094.stdout:6/128: write dc/d11/d1b/d24/f32 [259192,57466] 0 2024-04-06T03:40:28.022 INFO:tasks.workunit.client.0.smithi094.stdout:6/129: readlink dc/d11/d1b/l36 0 2024-04-06T03:40:28.023 INFO:tasks.workunit.client.0.smithi094.stdout:6/130: read fa [35007,112470] 0 2024-04-06T03:40:28.030 INFO:tasks.workunit.client.1.smithi136.stdout:1/108: write fb [3852922,101906] 0 2024-04-06T03:40:28.031 INFO:tasks.workunit.client.1.smithi136.stdout:1/109: getdents d12/d1b/d22 0 2024-04-06T03:40:28.031 INFO:tasks.workunit.client.0.smithi094.stdout:7/139: dread d2/d6/f9 [0,4194304] 0 2024-04-06T03:40:28.034 INFO:tasks.workunit.client.0.smithi094.stdout:8/149: getdents d18 0 2024-04-06T03:40:28.035 INFO:tasks.workunit.client.0.smithi094.stdout:4/150: dwrite d0/f12 [0,4194304] 0 2024-04-06T03:40:28.036 INFO:tasks.workunit.client.0.smithi094.stdout:2/133: mknod d0/dd/d1b/c2e 0 2024-04-06T03:40:28.039 INFO:tasks.workunit.client.0.smithi094.stdout:3/175: dwrite d9/d10/d27/f32 [0,4194304] 0 2024-04-06T03:40:28.042 INFO:tasks.workunit.client.0.smithi094.stdout:0/126: dwrite d1/d9/dd/f16 [0,4194304] 0 2024-04-06T03:40:28.043 INFO:tasks.workunit.client.0.smithi094.stdout:9/86: symlink d2/l1e 0 2024-04-06T03:40:28.043 INFO:tasks.workunit.client.0.smithi094.stdout:9/87: stat d2/dc/l17 0 2024-04-06T03:40:28.045 INFO:tasks.workunit.client.0.smithi094.stdout:4/151: dread d0/d4/f6 [0,4194304] 0 2024-04-06T03:40:28.045 INFO:tasks.workunit.client.0.smithi094.stdout:4/152: write d0/df/d13/f18 [1028177,24039] 0 2024-04-06T03:40:28.051 INFO:tasks.workunit.client.1.smithi136.stdout:1/110: mkdir d12/d1b/d1e/d23 0 2024-04-06T03:40:28.057 INFO:tasks.workunit.client.1.smithi136.stdout:5/94: rmdir d1/d16/d17 39 2024-04-06T03:40:28.061 INFO:tasks.workunit.client.1.smithi136.stdout:1/111: truncate d12/f16 642610 0 2024-04-06T03:40:28.061 INFO:tasks.workunit.client.1.smithi136.stdout:0/57: unlink d1/c5 0 2024-04-06T03:40:28.061 INFO:tasks.workunit.client.0.smithi094.stdout:8/150: creat d18/d2c/f33 x:0 0 0 2024-04-06T03:40:28.063 INFO:tasks.workunit.client.0.smithi094.stdout:1/128: truncate d0/d4/f29 4195226 0 2024-04-06T03:40:28.064 INFO:tasks.workunit.client.1.smithi136.stdout:7/60: dwrite d5/fd [4194304,4194304] 0 2024-04-06T03:40:28.064 INFO:tasks.workunit.client.1.smithi136.stdout:0/58: truncate d1/d6/f11 110878 0 2024-04-06T03:40:28.069 INFO:tasks.workunit.client.0.smithi094.stdout:2/134: dwrite d0/d8/d27/d2b/f2c [0,4194304] 0 2024-04-06T03:40:28.069 INFO:tasks.workunit.client.0.smithi094.stdout:2/135: fdatasync d0/d8/fe 0 2024-04-06T03:40:28.072 INFO:tasks.workunit.client.1.smithi136.stdout:7/61: getdents d5 0 2024-04-06T03:40:28.074 INFO:tasks.workunit.client.1.smithi136.stdout:0/59: unlink d1/c8 0 2024-04-06T03:40:28.075 INFO:tasks.workunit.client.1.smithi136.stdout:7/62: creat d5/df/d11/f14 x:0 0 0 2024-04-06T03:40:28.076 INFO:tasks.workunit.client.0.smithi094.stdout:0/127: creat d1/f23 x:0 0 0 2024-04-06T03:40:28.076 INFO:tasks.workunit.client.0.smithi094.stdout:0/128: chown d1/d9/dd/d18 2653501 1 2024-04-06T03:40:28.076 INFO:tasks.workunit.client.1.smithi136.stdout:0/60: mknod d1/d6/d13/c14 0 2024-04-06T03:40:28.077 INFO:tasks.workunit.client.1.smithi136.stdout:7/63: symlink d5/df/d11/l15 0 2024-04-06T03:40:28.077 INFO:tasks.workunit.client.1.smithi136.stdout:7/64: dread - d5/df/d11/f14 zero size 2024-04-06T03:40:28.078 INFO:tasks.workunit.client.0.smithi094.stdout:6/131: symlink dc/d11/l37 0 2024-04-06T03:40:28.078 INFO:tasks.workunit.client.0.smithi094.stdout:6/132: chown f7 1660032 1 2024-04-06T03:40:28.079 INFO:tasks.workunit.client.0.smithi094.stdout:8/151: rename d18/d2b to d18/d34 0 2024-04-06T03:40:28.082 INFO:tasks.workunit.client.0.smithi094.stdout:3/176: dwrite d9/fd [0,4194304] 0 2024-04-06T03:40:28.082 INFO:tasks.workunit.client.0.smithi094.stdout:5/114: rename db/d13 to db/d19/d24/d2e 0 2024-04-06T03:40:28.082 INFO:tasks.workunit.client.0.smithi094.stdout:4/153: dwrite d0/d15/f1a [0,4194304] 0 2024-04-06T03:40:28.083 INFO:tasks.workunit.client.1.smithi136.stdout:5/95: dwrite d1/f18 [0,4194304] 0 2024-04-06T03:40:28.083 INFO:tasks.workunit.client.0.smithi094.stdout:9/88: dwrite d2/f7 [0,4194304] 0 2024-04-06T03:40:28.083 INFO:tasks.workunit.client.0.smithi094.stdout:0/129: mknod d1/d9/dd/d18/d1e/c24 0 2024-04-06T03:40:28.083 INFO:tasks.workunit.client.0.smithi094.stdout:0/130: chown d1/d9/dd/d18/d1e/d22 506544 1 2024-04-06T03:40:28.084 INFO:tasks.workunit.client.1.smithi136.stdout:3/50: sync 2024-04-06T03:40:28.084 INFO:tasks.workunit.client.1.smithi136.stdout:8/87: sync 2024-04-06T03:40:28.084 INFO:tasks.workunit.client.1.smithi136.stdout:6/61: sync 2024-04-06T03:40:28.084 INFO:tasks.workunit.client.1.smithi136.stdout:3/51: truncate d0/f3 586521 0 2024-04-06T03:40:28.084 INFO:tasks.workunit.client.1.smithi136.stdout:6/62: write d2/f5 [1312125,42792] 0 2024-04-06T03:40:28.084 INFO:tasks.workunit.client.1.smithi136.stdout:6/63: creat d2/d7/f1b x:0 0 0 2024-04-06T03:40:28.085 INFO:tasks.workunit.client.1.smithi136.stdout:6/64: readlink d2/d7/lc 0 2024-04-06T03:40:28.085 INFO:tasks.workunit.client.0.smithi094.stdout:6/133: truncate fa 174955 0 2024-04-06T03:40:28.086 INFO:tasks.workunit.client.0.smithi094.stdout:8/152: mkdir d18/d2c/d35 0 2024-04-06T03:40:28.089 INFO:tasks.workunit.client.0.smithi094.stdout:2/136: truncate d0/d18/f22 537956 0 2024-04-06T03:40:28.089 INFO:tasks.workunit.client.0.smithi094.stdout:2/137: fsync d0/dd/ff 0 2024-04-06T03:40:28.090 INFO:tasks.workunit.client.0.smithi094.stdout:2/138: dread d0/d8/f11 [0,4194304] 0 2024-04-06T03:40:28.090 INFO:tasks.workunit.client.0.smithi094.stdout:2/139: write d0/d8/d27/d2b/f2d [870339,101052] 0 2024-04-06T03:40:28.091 INFO:tasks.workunit.client.0.smithi094.stdout:0/131: write d1/d9/f14 [721091,7897] 0 2024-04-06T03:40:28.092 INFO:tasks.workunit.client.1.smithi136.stdout:5/96: creat d1/d16/f1b x:0 0 0 2024-04-06T03:40:28.092 INFO:tasks.workunit.client.1.smithi136.stdout:5/97: chown d1/d16 677642 1 2024-04-06T03:40:28.094 INFO:tasks.workunit.client.1.smithi136.stdout:8/88: mknod d3/c17 0 2024-04-06T03:40:28.097 INFO:tasks.workunit.client.0.smithi094.stdout:3/177: rename d9/d20/f21 to d9/d10/d27/f38 0 2024-04-06T03:40:28.097 INFO:tasks.workunit.client.0.smithi094.stdout:3/178: chown d9/d20/f33 1339 1 2024-04-06T03:40:28.098 INFO:tasks.workunit.client.0.smithi094.stdout:3/179: dread d9/f1b [0,4194304] 0 2024-04-06T03:40:28.101 INFO:tasks.workunit.client.0.smithi094.stdout:5/115: creat db/f2f x:0 0 0 2024-04-06T03:40:28.105 INFO:tasks.workunit.client.0.smithi094.stdout:4/154: mknod d0/df/d13/c2d 0 2024-04-06T03:40:28.106 INFO:tasks.workunit.client.1.smithi136.stdout:6/65: mknod d2/d7/c1c 0 2024-04-06T03:40:28.111 INFO:tasks.workunit.client.1.smithi136.stdout:3/52: rmdir d0 39 2024-04-06T03:40:28.112 INFO:tasks.workunit.client.1.smithi136.stdout:6/66: mknod d2/de/c1d 0 2024-04-06T03:40:28.113 INFO:tasks.workunit.client.0.smithi094.stdout:0/132: dwrite d1/d9/fc [4194304,4194304] 0 2024-04-06T03:40:28.114 INFO:tasks.workunit.client.0.smithi094.stdout:9/89: rename d2/l1e to d2/dc/l1f 0 2024-04-06T03:40:28.114 INFO:tasks.workunit.client.0.smithi094.stdout:9/90: chown d2 4 1 2024-04-06T03:40:28.114 INFO:tasks.workunit.client.0.smithi094.stdout:9/91: write d2/dc/d10/f12 [4927039,74981] 0 2024-04-06T03:40:28.114 INFO:tasks.workunit.client.0.smithi094.stdout:9/92: fdatasync d2/fb 0 2024-04-06T03:40:28.114 INFO:tasks.workunit.client.1.smithi136.stdout:3/53: rmdir d0 39 2024-04-06T03:40:28.115 INFO:tasks.workunit.client.0.smithi094.stdout:5/116: mkdir db/d19/d30 0 2024-04-06T03:40:28.115 INFO:tasks.workunit.client.0.smithi094.stdout:4/155: truncate d0/d4/f21 1590814 0 2024-04-06T03:40:28.116 INFO:tasks.workunit.client.0.smithi094.stdout:2/140: fdatasync d0/d8/d27/d2b/f2d 0 2024-04-06T03:40:28.116 INFO:tasks.workunit.client.0.smithi094.stdout:2/141: stat d0/d18 0 2024-04-06T03:40:28.117 INFO:tasks.workunit.client.1.smithi136.stdout:3/54: symlink d0/d4/l10 0 2024-04-06T03:40:28.117 INFO:tasks.workunit.client.0.smithi094.stdout:6/134: rmdir dc/d11/d1b 39 2024-04-06T03:40:28.119 INFO:tasks.workunit.client.0.smithi094.stdout:2/142: read d0/d18/f22 [379612,89455] 0 2024-04-06T03:40:28.120 INFO:tasks.workunit.client.1.smithi136.stdout:5/98: dwrite d1/d16/f1b [0,4194304] 0 2024-04-06T03:40:28.122 INFO:tasks.workunit.client.0.smithi094.stdout:0/133: link d1/f7 d1/d9/dd/d18/d1a/f25 0 2024-04-06T03:40:28.122 INFO:tasks.workunit.client.0.smithi094.stdout:3/180: dwrite d9/fd [0,4194304] 0 2024-04-06T03:40:28.126 INFO:tasks.workunit.client.0.smithi094.stdout:9/93: link d2/c19 d2/c20 0 2024-04-06T03:40:28.126 INFO:tasks.workunit.client.0.smithi094.stdout:5/117: truncate f9 1691534 0 2024-04-06T03:40:28.126 INFO:tasks.workunit.client.0.smithi094.stdout:5/118: creat db/d19/d2c/f31 x:0 0 0 2024-04-06T03:40:28.126 INFO:tasks.workunit.client.0.smithi094.stdout:5/119: fdatasync db/fc 0 2024-04-06T03:40:28.129 INFO:tasks.workunit.client.0.smithi094.stdout:4/156: symlink d0/df/d1e/l2e 0 2024-04-06T03:40:28.131 INFO:tasks.workunit.client.1.smithi136.stdout:5/99: mkdir d1/dc/d1c 0 2024-04-06T03:40:28.131 INFO:tasks.workunit.client.1.smithi136.stdout:8/89: dwrite d3/de/ff [0,4194304] 0 2024-04-06T03:40:28.132 INFO:tasks.workunit.client.1.smithi136.stdout:8/90: read d3/f4 [3870048,126296] 0 2024-04-06T03:40:28.132 INFO:tasks.workunit.client.1.smithi136.stdout:8/91: chown d3/c17 11290630 1 2024-04-06T03:40:28.132 INFO:tasks.workunit.client.1.smithi136.stdout:8/92: chown d3/f4 8542 1 2024-04-06T03:40:28.132 INFO:tasks.workunit.client.1.smithi136.stdout:8/93: fdatasync d3/f8 0 2024-04-06T03:40:28.133 INFO:tasks.workunit.client.0.smithi094.stdout:6/135: rename dc/d11/d22/f27 to dc/d11/d1b/f38 0 2024-04-06T03:40:28.133 INFO:tasks.workunit.client.0.smithi094.stdout:6/136: fdatasync dc/f12 0 2024-04-06T03:40:28.133 INFO:tasks.workunit.client.0.smithi094.stdout:6/137: chown dc/d11/d1b/l36 906 1 2024-04-06T03:40:28.133 INFO:tasks.workunit.client.0.smithi094.stdout:6/138: fdatasync dc/f29 0 2024-04-06T03:40:28.135 INFO:tasks.workunit.client.0.smithi094.stdout:2/143: dwrite d0/d8/f11 [0,4194304] 0 2024-04-06T03:40:28.138 INFO:tasks.workunit.client.1.smithi136.stdout:5/100: mknod d1/dc/c1d 0 2024-04-06T03:40:28.140 INFO:tasks.workunit.client.0.smithi094.stdout:9/94: dread d2/dc/f18 [0,4194304] 0 2024-04-06T03:40:28.143 INFO:tasks.workunit.client.0.smithi094.stdout:5/120: symlink db/d19/d24/d28/l32 0 2024-04-06T03:40:28.143 INFO:tasks.workunit.client.0.smithi094.stdout:5/121: write db/f23 [4939149,114533] 0 2024-04-06T03:40:28.143 INFO:tasks.workunit.client.0.smithi094.stdout:3/181: dwrite d9/d20/f33 [0,4194304] 0 2024-04-06T03:40:28.143 INFO:tasks.workunit.client.0.smithi094.stdout:5/122: fdatasync db/fd 0 2024-04-06T03:40:28.144 INFO:tasks.workunit.client.0.smithi094.stdout:4/157: dread d0/df/f19 [0,4194304] 0 2024-04-06T03:40:28.144 INFO:tasks.workunit.client.0.smithi094.stdout:4/158: chown d0/l28 1128 1 2024-04-06T03:40:28.144 INFO:tasks.workunit.client.1.smithi136.stdout:8/94: mkdir d3/de/d18 0 2024-04-06T03:40:28.146 INFO:tasks.workunit.client.1.smithi136.stdout:5/101: mkdir d1/dc/d10/d1e 0 2024-04-06T03:40:28.146 INFO:tasks.workunit.client.1.smithi136.stdout:5/102: fsync d1/f18 0 2024-04-06T03:40:28.146 INFO:tasks.workunit.client.1.smithi136.stdout:5/103: chown d1/dc 6865 1 2024-04-06T03:40:28.146 INFO:tasks.workunit.client.1.smithi136.stdout:5/104: write d1/d16/d17/f1a [556688,85331] 0 2024-04-06T03:40:28.149 INFO:tasks.workunit.client.1.smithi136.stdout:8/95: read d3/da/fb [744,90411] 0 2024-04-06T03:40:28.149 INFO:tasks.workunit.client.1.smithi136.stdout:8/96: truncate d3/de/f11 671246 0 2024-04-06T03:40:28.150 INFO:tasks.workunit.client.1.smithi136.stdout:8/97: creat d3/da/f19 x:0 0 0 2024-04-06T03:40:28.150 INFO:tasks.workunit.client.1.smithi136.stdout:8/98: chown d3/da 223 1 2024-04-06T03:40:28.150 INFO:tasks.workunit.client.0.smithi094.stdout:3/182: write d9/d20/f34 [1155144,22220] 0 2024-04-06T03:40:28.150 INFO:tasks.workunit.client.0.smithi094.stdout:3/183: creat d9/d1d/f39 x:0 0 0 2024-04-06T03:40:28.150 INFO:tasks.workunit.client.0.smithi094.stdout:3/184: fsync d9/fd 0 2024-04-06T03:40:28.151 INFO:tasks.workunit.client.0.smithi094.stdout:3/185: stat d9/fd 0 2024-04-06T03:40:28.154 INFO:tasks.workunit.client.1.smithi136.stdout:8/99: symlink d3/de/l1a 0 2024-04-06T03:40:28.154 INFO:tasks.workunit.client.1.smithi136.stdout:8/100: chown d3/f4 7550 1 2024-04-06T03:40:28.155 INFO:tasks.workunit.client.1.smithi136.stdout:5/105: dread d1/f4 [0,4194304] 0 2024-04-06T03:40:28.157 INFO:tasks.workunit.client.1.smithi136.stdout:8/101: mkdir d3/de/d1b 0 2024-04-06T03:40:28.161 INFO:tasks.workunit.client.0.smithi094.stdout:6/139: truncate dc/fd 1417978 0 2024-04-06T03:40:28.162 INFO:tasks.workunit.client.1.smithi136.stdout:8/102: mkdir d3/da/d1c 0 2024-04-06T03:40:28.166 INFO:tasks.workunit.client.0.smithi094.stdout:1/129: truncate d0/d4/f29 788585 0 2024-04-06T03:40:28.167 INFO:tasks.workunit.client.1.smithi136.stdout:9/66: sync 2024-04-06T03:40:28.167 INFO:tasks.workunit.client.0.smithi094.stdout:6/140: dread dc/f29 [0,4194304] 0 2024-04-06T03:40:28.168 INFO:tasks.workunit.client.0.smithi094.stdout:6/141: write dc/f10 [109298,120393] 0 2024-04-06T03:40:28.170 INFO:tasks.workunit.client.0.smithi094.stdout:9/95: mkdir d2/dc/d21 0 2024-04-06T03:40:28.172 INFO:tasks.workunit.client.1.smithi136.stdout:9/67: symlink l14 0 2024-04-06T03:40:28.173 INFO:tasks.workunit.client.1.smithi136.stdout:9/68: dread f13 [0,4194304] 0 2024-04-06T03:40:28.175 INFO:tasks.workunit.client.0.smithi094.stdout:2/144: dread d0/f1 [0,4194304] 0 2024-04-06T03:40:28.176 INFO:tasks.workunit.client.0.smithi094.stdout:5/123: rename db/ff to db/d19/d30/f33 0 2024-04-06T03:40:28.177 INFO:tasks.workunit.client.1.smithi136.stdout:8/103: dwrite d3/de/ff [0,4194304] 0 2024-04-06T03:40:28.178 INFO:tasks.workunit.client.0.smithi094.stdout:4/159: mknod d0/df/d1e/c2f 0 2024-04-06T03:40:28.178 INFO:tasks.workunit.client.0.smithi094.stdout:4/160: chown d0/df/d1e/l27 54337528 1 2024-04-06T03:40:28.179 INFO:tasks.workunit.client.1.smithi136.stdout:5/106: rmdir d1 39 2024-04-06T03:40:28.181 INFO:tasks.workunit.client.0.smithi094.stdout:0/134: truncate d1/d9/fc 5751324 0 2024-04-06T03:40:28.184 INFO:tasks.workunit.client.0.smithi094.stdout:3/186: mkdir d9/d10/d27/d3a 0 2024-04-06T03:40:28.185 INFO:tasks.workunit.client.0.smithi094.stdout:3/187: write d9/d10/f16 [545611,118151] 0 2024-04-06T03:40:28.185 INFO:tasks.workunit.client.0.smithi094.stdout:3/188: chown d9/f19 2 1 2024-04-06T03:40:28.185 INFO:tasks.workunit.client.0.smithi094.stdout:3/189: readlink d9/d20/l24 0 2024-04-06T03:40:28.186 INFO:tasks.workunit.client.0.smithi094.stdout:1/130: creat d0/d1/d27/f2e x:0 0 0 2024-04-06T03:40:28.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/96: unlink d2/f9 0 2024-04-06T03:40:28.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/97: fsync d2/f14 0 2024-04-06T03:40:28.189 INFO:tasks.workunit.client.0.smithi094.stdout:9/98: read d2/dc/f18 [2188082,91675] 0 2024-04-06T03:40:28.195 INFO:tasks.workunit.client.0.smithi094.stdout:2/145: symlink d0/d8/d1d/l2f 0 2024-04-06T03:40:28.197 INFO:tasks.workunit.client.0.smithi094.stdout:4/161: symlink d0/df/l30 0 2024-04-06T03:40:28.198 INFO:tasks.workunit.client.0.smithi094.stdout:0/135: symlink d1/d9/dd/d18/d1e/l26 0 2024-04-06T03:40:28.198 INFO:tasks.workunit.client.0.smithi094.stdout:0/136: creat d1/d9/f27 x:0 0 0 2024-04-06T03:40:28.199 INFO:tasks.workunit.client.1.smithi136.stdout:9/69: dwrite fd [0,4194304] 0 2024-04-06T03:40:28.199 INFO:tasks.workunit.client.1.smithi136.stdout:9/70: truncate f11 666322 0 2024-04-06T03:40:28.199 INFO:tasks.workunit.client.1.smithi136.stdout:9/71: creat f15 x:0 0 0 2024-04-06T03:40:28.200 INFO:tasks.workunit.client.1.smithi136.stdout:9/72: creat f16 x:0 0 0 2024-04-06T03:40:28.200 INFO:tasks.workunit.client.0.smithi094.stdout:3/190: rename d9/f1b to d9/d10/f3b 0 2024-04-06T03:40:28.203 INFO:tasks.workunit.client.0.smithi094.stdout:2/146: dread - d0/d18/f24 zero size 2024-04-06T03:40:28.205 INFO:tasks.workunit.client.0.smithi094.stdout:4/162: mknod d0/c31 0 2024-04-06T03:40:28.209 INFO:tasks.workunit.client.1.smithi136.stdout:9/73: dread fe [0,4194304] 0 2024-04-06T03:40:28.210 INFO:tasks.workunit.client.1.smithi136.stdout:5/107: dwrite d1/d16/f1b [0,4194304] 0 2024-04-06T03:40:28.210 INFO:tasks.workunit.client.0.smithi094.stdout:3/191: rename d9/ff to d9/d1d/f3c 0 2024-04-06T03:40:28.210 INFO:tasks.workunit.client.0.smithi094.stdout:3/192: write d9/d1d/f25 [261318,130328] 0 2024-04-06T03:40:28.211 INFO:tasks.workunit.client.1.smithi136.stdout:9/74: dread fa [0,4194304] 0 2024-04-06T03:40:28.212 INFO:tasks.workunit.client.0.smithi094.stdout:4/163: creat d0/d23/f32 x:0 0 0 2024-04-06T03:40:28.215 INFO:tasks.workunit.client.1.smithi136.stdout:8/104: dwrite d3/de/f11 [0,4194304] 0 2024-04-06T03:40:28.216 INFO:tasks.workunit.client.0.smithi094.stdout:4/164: mknod d0/df/c33 0 2024-04-06T03:40:28.218 INFO:tasks.workunit.client.0.smithi094.stdout:3/193: dread d9/d1e/d26/f2c [0,4194304] 0 2024-04-06T03:40:28.218 INFO:tasks.workunit.client.0.smithi094.stdout:3/194: creat d9/d20/f3d x:0 0 0 2024-04-06T03:40:28.218 INFO:tasks.workunit.client.0.smithi094.stdout:3/195: fsync d9/f18 0 2024-04-06T03:40:28.219 INFO:tasks.workunit.client.1.smithi136.stdout:8/105: dread f1 [0,4194304] 0 2024-04-06T03:40:28.220 INFO:tasks.workunit.client.0.smithi094.stdout:2/147: rename d0/l4 to d0/dd/d19/l30 0 2024-04-06T03:40:28.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/124: dwrite db/f11 [4194304,4194304] 0 2024-04-06T03:40:28.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/125: truncate db/f1d 104588 0 2024-04-06T03:40:28.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/126: write db/f2f [386311,15321] 0 2024-04-06T03:40:28.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/127: stat db/f23 0 2024-04-06T03:40:28.225 INFO:tasks.workunit.client.1.smithi136.stdout:5/108: mkdir d1/d1f 0 2024-04-06T03:40:28.227 INFO:tasks.workunit.client.1.smithi136.stdout:9/75: truncate f4 1928136 0 2024-04-06T03:40:28.227 INFO:tasks.workunit.client.1.smithi136.stdout:9/76: stat fd 0 2024-04-06T03:40:28.229 INFO:tasks.workunit.client.0.smithi094.stdout:9/99: dwrite d2/dc/f18 [0,4194304] 0 2024-04-06T03:40:28.229 INFO:tasks.workunit.client.0.smithi094.stdout:4/165: dread d0/df/d1e/f25 [4194304,4194304] 0 2024-04-06T03:40:28.230 INFO:tasks.workunit.client.0.smithi094.stdout:4/166: fdatasync d0/d23/f32 0 2024-04-06T03:40:28.232 INFO:tasks.workunit.client.0.smithi094.stdout:5/128: rename f9 to db/d19/d24/f34 0 2024-04-06T03:40:28.232 INFO:tasks.workunit.client.0.smithi094.stdout:0/137: dwrite d1/d9/fc [0,4194304] 0 2024-04-06T03:40:28.232 INFO:tasks.workunit.client.0.smithi094.stdout:5/129: truncate db/d19/f1f 728901 0 2024-04-06T03:40:28.232 INFO:tasks.workunit.client.0.smithi094.stdout:5/130: dread - f6 zero size 2024-04-06T03:40:28.235 INFO:tasks.workunit.client.1.smithi136.stdout:5/109: mkdir d1/dc/d15/d20 0 2024-04-06T03:40:28.238 INFO:tasks.workunit.client.1.smithi136.stdout:9/77: mkdir d17 0 2024-04-06T03:40:28.240 INFO:tasks.workunit.client.1.smithi136.stdout:9/78: mknod d17/c18 0 2024-04-06T03:40:28.241 INFO:tasks.workunit.client.0.smithi094.stdout:4/167: mkdir d0/df/d1e/d34 0 2024-04-06T03:40:28.242 INFO:tasks.workunit.client.0.smithi094.stdout:0/138: symlink d1/l28 0 2024-04-06T03:40:28.243 INFO:tasks.workunit.client.1.smithi136.stdout:9/79: mkdir d17/d19 0 2024-04-06T03:40:28.243 INFO:tasks.workunit.client.0.smithi094.stdout:5/131: truncate db/fc 4053507 0 2024-04-06T03:40:28.245 INFO:tasks.workunit.client.0.smithi094.stdout:4/168: creat d0/df/d13/f35 x:0 0 0 2024-04-06T03:40:28.245 INFO:tasks.workunit.client.0.smithi094.stdout:4/169: write d0/df/f1b [181846,89659] 0 2024-04-06T03:40:28.245 INFO:tasks.workunit.client.1.smithi136.stdout:5/110: write d1/dc/d10/f11 [4107836,4860] 0 2024-04-06T03:40:28.246 INFO:tasks.workunit.client.1.smithi136.stdout:9/80: mknod d17/d19/c1a 0 2024-04-06T03:40:28.247 INFO:tasks.workunit.client.1.smithi136.stdout:8/106: dwrite d3/da/fd [0,4194304] 0 2024-04-06T03:40:28.247 INFO:tasks.workunit.client.0.smithi094.stdout:0/139: symlink d1/d9/dd/d18/d1e/l29 0 2024-04-06T03:40:28.250 INFO:tasks.workunit.client.0.smithi094.stdout:2/148: getdents d0/dd/d19 0 2024-04-06T03:40:28.252 INFO:tasks.workunit.client.0.smithi094.stdout:9/100: dread d2/dc/f18 [0,4194304] 0 2024-04-06T03:40:28.254 INFO:tasks.workunit.client.1.smithi136.stdout:5/111: creat d1/d1f/f21 x:0 0 0 2024-04-06T03:40:28.255 INFO:tasks.workunit.client.0.smithi094.stdout:0/140: mknod d1/c2a 0 2024-04-06T03:40:28.258 INFO:tasks.workunit.client.0.smithi094.stdout:0/141: mknod d1/d9/dd/d18/d1a/c2b 0 2024-04-06T03:40:28.259 INFO:tasks.workunit.client.0.smithi094.stdout:0/142: creat d1/f2c x:0 0 0 2024-04-06T03:40:28.260 INFO:tasks.workunit.client.0.smithi094.stdout:4/170: dwrite d0/df/d1e/f25 [0,4194304] 0 2024-04-06T03:40:28.262 INFO:tasks.workunit.client.0.smithi094.stdout:2/149: dread d0/d18/f22 [0,4194304] 0 2024-04-06T03:40:28.264 INFO:tasks.workunit.client.0.smithi094.stdout:0/143: mkdir d1/d9/dd/d18/d2d 0 2024-04-06T03:40:28.264 INFO:tasks.workunit.client.0.smithi094.stdout:0/144: chown d1/d9/c12 1496 1 2024-04-06T03:40:28.269 INFO:tasks.workunit.client.0.smithi094.stdout:5/132: dwrite db/d19/d2c/f31 [0,4194304] 0 2024-04-06T03:40:28.273 INFO:tasks.workunit.client.1.smithi136.stdout:8/107: rename d3/de/f11 to d3/de/d1b/f1d 0 2024-04-06T03:40:28.274 INFO:tasks.workunit.client.1.smithi136.stdout:8/108: stat f1 0 2024-04-06T03:40:28.274 INFO:tasks.workunit.client.1.smithi136.stdout:9/81: dwrite fe [0,4194304] 0 2024-04-06T03:40:28.276 INFO:tasks.workunit.client.0.smithi094.stdout:4/171: link d0/df/d13/f18 d0/df/d13/f36 0 2024-04-06T03:40:28.276 INFO:tasks.workunit.client.0.smithi094.stdout:4/172: creat d0/df/d13/f37 x:0 0 0 2024-04-06T03:40:28.277 INFO:tasks.workunit.client.1.smithi136.stdout:8/109: creat d3/de/d18/f1e x:0 0 0 2024-04-06T03:40:28.277 INFO:tasks.workunit.client.1.smithi136.stdout:8/110: chown d3/c16 58 1 2024-04-06T03:40:28.278 INFO:tasks.workunit.client.0.smithi094.stdout:2/150: mkdir d0/dd/d19/d31 0 2024-04-06T03:40:28.279 INFO:tasks.workunit.client.0.smithi094.stdout:0/145: dwrite d1/d9/fc [0,4194304] 0 2024-04-06T03:40:28.280 INFO:tasks.workunit.client.1.smithi136.stdout:5/112: dwrite d1/d1f/f21 [0,4194304] 0 2024-04-06T03:40:28.285 INFO:tasks.workunit.client.0.smithi094.stdout:4/173: symlink d0/l38 0 2024-04-06T03:40:28.285 INFO:tasks.workunit.client.0.smithi094.stdout:4/174: chown d0/df/d13/c2d 61072 1 2024-04-06T03:40:28.285 INFO:tasks.workunit.client.0.smithi094.stdout:4/175: chown d0/df/d13/f35 317898 1 2024-04-06T03:40:28.287 INFO:tasks.workunit.client.0.smithi094.stdout:2/151: mknod d0/d8/d27/d2b/c32 0 2024-04-06T03:40:28.289 INFO:tasks.workunit.client.1.smithi136.stdout:5/113: mknod d1/dc/d15/c22 0 2024-04-06T03:40:28.289 INFO:tasks.workunit.client.1.smithi136.stdout:9/82: write f4 [393941,15127] 0 2024-04-06T03:40:28.290 INFO:tasks.workunit.client.0.smithi094.stdout:4/176: creat d0/df/d1e/d34/f39 x:0 0 0 2024-04-06T03:40:28.290 INFO:tasks.workunit.client.0.smithi094.stdout:4/177: chown d0/df/d13/f36 14964370 1 2024-04-06T03:40:28.291 INFO:tasks.workunit.client.0.smithi094.stdout:4/178: write d0/f22 [711426,26058] 0 2024-04-06T03:40:28.291 INFO:tasks.workunit.client.0.smithi094.stdout:5/133: read db/fc [2212059,84878] 0 2024-04-06T03:40:28.292 INFO:tasks.workunit.client.1.smithi136.stdout:9/83: symlink d17/d19/l1b 0 2024-04-06T03:40:28.293 INFO:tasks.workunit.client.1.smithi136.stdout:9/84: chown cf 188044 1 2024-04-06T03:40:28.293 INFO:tasks.workunit.client.0.smithi094.stdout:2/152: readlink d0/l3 0 2024-04-06T03:40:28.293 INFO:tasks.workunit.client.0.smithi094.stdout:2/153: creat d0/d8/f33 x:0 0 0 2024-04-06T03:40:28.293 INFO:tasks.workunit.client.0.smithi094.stdout:2/154: dread - d0/d18/f24 zero size 2024-04-06T03:40:28.293 INFO:tasks.workunit.client.1.smithi136.stdout:8/111: dwrite d3/de/ff [0,4194304] 0 2024-04-06T03:40:28.294 INFO:tasks.workunit.client.1.smithi136.stdout:8/112: stat d3/c16 0 2024-04-06T03:40:28.294 INFO:tasks.workunit.client.1.smithi136.stdout:5/114: symlink d1/dc/d1c/l23 0 2024-04-06T03:40:28.295 INFO:tasks.workunit.client.0.smithi094.stdout:5/134: mknod db/d19/d24/c35 0 2024-04-06T03:40:28.296 INFO:tasks.workunit.client.1.smithi136.stdout:9/85: dread fa [0,4194304] 0 2024-04-06T03:40:28.296 INFO:tasks.workunit.client.1.smithi136.stdout:9/86: write f11 [919358,11189] 0 2024-04-06T03:40:28.296 INFO:tasks.workunit.client.1.smithi136.stdout:9/87: fsync fb 0 2024-04-06T03:40:28.296 INFO:tasks.workunit.client.1.smithi136.stdout:9/88: write f16 [668875,97720] 0 2024-04-06T03:40:28.298 INFO:tasks.workunit.client.0.smithi094.stdout:4/179: dread d0/d4/f6 [0,4194304] 0 2024-04-06T03:40:28.299 INFO:tasks.workunit.client.1.smithi136.stdout:8/113: rename d3/da/c12 to d3/de/c1f 0 2024-04-06T03:40:28.300 INFO:tasks.workunit.client.0.smithi094.stdout:2/155: symlink d0/dd/d1b/l34 0 2024-04-06T03:40:28.300 INFO:tasks.workunit.client.0.smithi094.stdout:0/146: dwrite d1/f23 [0,4194304] 0 2024-04-06T03:40:28.302 INFO:tasks.workunit.client.0.smithi094.stdout:5/135: creat db/d19/d24/d2e/f36 x:0 0 0 2024-04-06T03:40:28.303 INFO:tasks.workunit.client.1.smithi136.stdout:9/89: rmdir d17 39 2024-04-06T03:40:28.303 INFO:tasks.workunit.client.1.smithi136.stdout:9/90: chown d17/d19 6486 1 2024-04-06T03:40:28.303 INFO:tasks.workunit.client.1.smithi136.stdout:9/91: write f4 [224600,68067] 0 2024-04-06T03:40:28.303 INFO:tasks.workunit.client.1.smithi136.stdout:9/92: fsync f15 0 2024-04-06T03:40:28.304 INFO:tasks.workunit.client.0.smithi094.stdout:4/180: mkdir d0/d15/d3a 0 2024-04-06T03:40:28.305 INFO:tasks.workunit.client.1.smithi136.stdout:8/114: mknod d3/de/d1b/c20 0 2024-04-06T03:40:28.305 INFO:tasks.workunit.client.1.smithi136.stdout:8/115: truncate d3/da/f19 576010 0 2024-04-06T03:40:28.306 INFO:tasks.workunit.client.0.smithi094.stdout:2/156: creat d0/d18/f35 x:0 0 0 2024-04-06T03:40:28.306 INFO:tasks.workunit.client.1.smithi136.stdout:5/115: getdents d1/d1f 0 2024-04-06T03:40:28.308 INFO:tasks.workunit.client.1.smithi136.stdout:9/93: getdents d17 0 2024-04-06T03:40:28.309 INFO:tasks.workunit.client.1.smithi136.stdout:5/116: rmdir d1/dc/d10 39 2024-04-06T03:40:28.310 INFO:tasks.workunit.client.1.smithi136.stdout:5/117: read d1/dc/d10/f11 [1978803,31545] 0 2024-04-06T03:40:28.310 INFO:tasks.workunit.client.0.smithi094.stdout:0/147: creat d1/d10/f2e x:0 0 0 2024-04-06T03:40:28.310 INFO:tasks.workunit.client.0.smithi094.stdout:0/148: fdatasync d1/d9/dd/f1c 0 2024-04-06T03:40:28.312 INFO:tasks.workunit.client.1.smithi136.stdout:9/94: symlink d17/l1c 0 2024-04-06T03:40:28.313 INFO:tasks.workunit.client.0.smithi094.stdout:2/157: unlink d0/d8/d1d/f25 0 2024-04-06T03:40:28.315 INFO:tasks.workunit.client.1.smithi136.stdout:5/118: symlink d1/dc/d15/l24 0 2024-04-06T03:40:28.315 INFO:tasks.workunit.client.1.smithi136.stdout:4/92: sync 2024-04-06T03:40:28.316 INFO:tasks.workunit.client.0.smithi094.stdout:0/149: truncate d1/d9/dd/d18/d1a/f25 1058051 0 2024-04-06T03:40:28.317 INFO:tasks.workunit.client.1.smithi136.stdout:2/174: sync 2024-04-06T03:40:28.320 INFO:tasks.workunit.client.1.smithi136.stdout:4/93: symlink d1d/l21 0 2024-04-06T03:40:28.320 INFO:tasks.workunit.client.1.smithi136.stdout:4/94: truncate f1a 4686862 0 2024-04-06T03:40:28.322 INFO:tasks.workunit.client.0.smithi094.stdout:0/150: unlink d1/d9/dd/f16 0 2024-04-06T03:40:28.322 INFO:tasks.workunit.client.0.smithi094.stdout:0/151: chown d1/d9/dd/f1c 4 1 2024-04-06T03:40:28.323 INFO:tasks.workunit.client.1.smithi136.stdout:9/95: link c12 d17/c1d 0 2024-04-06T03:40:28.325 INFO:tasks.workunit.client.1.smithi136.stdout:2/175: creat d3/d9/d10/d18/d1d/d3c/f40 x:0 0 0 2024-04-06T03:40:28.326 INFO:tasks.workunit.client.1.smithi136.stdout:4/95: rmdir d1d 39 2024-04-06T03:40:28.328 INFO:tasks.workunit.client.0.smithi094.stdout:5/136: dwrite db/d19/d24/d2e/f25 [0,4194304] 0 2024-04-06T03:40:28.328 INFO:tasks.workunit.client.1.smithi136.stdout:2/176: mknod d3/d9/c41 0 2024-04-06T03:40:28.328 INFO:tasks.workunit.client.1.smithi136.stdout:2/177: dread - d3/d15/f30 zero size 2024-04-06T03:40:28.330 INFO:tasks.workunit.client.1.smithi136.stdout:5/119: dread d1/d1f/f21 [0,4194304] 0 2024-04-06T03:40:28.332 INFO:tasks.workunit.client.1.smithi136.stdout:2/178: rename d3/c6 to d3/d9/d3b/c42 0 2024-04-06T03:40:28.332 INFO:tasks.workunit.client.1.smithi136.stdout:2/179: truncate d3/d15/d19/f1b 675853 0 2024-04-06T03:40:28.332 INFO:tasks.workunit.client.1.smithi136.stdout:8/116: dwrite d3/de/ff [0,4194304] 0 2024-04-06T03:40:28.332 INFO:tasks.workunit.client.1.smithi136.stdout:8/117: creat d3/f21 x:0 0 0 2024-04-06T03:40:28.333 INFO:tasks.workunit.client.0.smithi094.stdout:2/158: dread d0/f5 [0,4194304] 0 2024-04-06T03:40:28.333 INFO:tasks.workunit.client.0.smithi094.stdout:4/181: dwrite d0/df/d1e/f2a [0,4194304] 0 2024-04-06T03:40:28.340 INFO:tasks.workunit.client.1.smithi136.stdout:4/96: getdents d1d 0 2024-04-06T03:40:28.340 INFO:tasks.workunit.client.1.smithi136.stdout:5/120: write d1/f18 [3455337,73834] 0 2024-04-06T03:40:28.341 INFO:tasks.workunit.client.0.smithi094.stdout:0/152: rmdir d1 39 2024-04-06T03:40:28.348 INFO:tasks.workunit.client.1.smithi136.stdout:8/118: read d3/fc [1851095,49281] 0 2024-04-06T03:40:28.349 INFO:tasks.workunit.client.0.smithi094.stdout:4/182: write d0/d4/f14 [95127,105592] 0 2024-04-06T03:40:28.349 INFO:tasks.workunit.client.0.smithi094.stdout:4/183: dread - d0/f29 zero size 2024-04-06T03:40:28.352 INFO:tasks.workunit.client.0.smithi094.stdout:4/184: dread d0/d4/fb [0,4194304] 0 2024-04-06T03:40:28.355 INFO:tasks.workunit.client.0.smithi094.stdout:2/159: rename d0/d8/la to d0/d8/d27/l36 0 2024-04-06T03:40:28.356 INFO:tasks.workunit.client.0.smithi094.stdout:2/160: stat d0/d8/d27/d2b/c32 0 2024-04-06T03:40:28.356 INFO:tasks.workunit.client.0.smithi094.stdout:2/161: fsync d0/dd/ff 0 2024-04-06T03:40:28.356 INFO:tasks.workunit.client.1.smithi136.stdout:8/119: write d3/de/ff [3887758,102632] 0 2024-04-06T03:40:28.357 INFO:tasks.workunit.client.0.smithi094.stdout:0/153: symlink d1/d9/dd/d18/d1e/d22/l2f 0 2024-04-06T03:40:28.357 INFO:tasks.workunit.client.0.smithi094.stdout:0/154: dread - d1/d9/f20 zero size 2024-04-06T03:40:28.357 INFO:tasks.workunit.client.0.smithi094.stdout:0/155: write d1/f2c [712735,104867] 0 2024-04-06T03:40:28.358 INFO:tasks.workunit.client.1.smithi136.stdout:9/96: dwrite f4 [0,4194304] 0 2024-04-06T03:40:28.358 INFO:tasks.workunit.client.1.smithi136.stdout:2/180: dwrite d3/d9/d10/d18/f2a [0,4194304] 0 2024-04-06T03:40:28.359 INFO:tasks.workunit.client.1.smithi136.stdout:4/97: creat d1d/f22 x:0 0 0 2024-04-06T03:40:28.360 INFO:tasks.workunit.client.0.smithi094.stdout:4/185: mknod d0/df/d1e/d34/c3b 0 2024-04-06T03:40:28.360 INFO:tasks.workunit.client.0.smithi094.stdout:4/186: readlink d0/d4/l1d 0 2024-04-06T03:40:28.361 INFO:tasks.workunit.client.1.smithi136.stdout:5/121: mknod d1/d16/c25 0 2024-04-06T03:40:28.363 INFO:tasks.workunit.client.1.smithi136.stdout:8/120: mknod d3/de/d18/c22 0 2024-04-06T03:40:28.364 INFO:tasks.workunit.client.0.smithi094.stdout:0/156: mknod d1/c30 0 2024-04-06T03:40:28.366 INFO:tasks.workunit.client.0.smithi094.stdout:4/187: symlink d0/d15/l3c 0 2024-04-06T03:40:28.368 INFO:tasks.workunit.client.1.smithi136.stdout:2/181: mkdir d3/d15/d19/d43 0 2024-04-06T03:40:28.368 INFO:tasks.workunit.client.1.smithi136.stdout:0/61: sync 2024-04-06T03:40:28.369 INFO:tasks.workunit.client.0.smithi094.stdout:2/162: mkdir d0/d37 0 2024-04-06T03:40:28.372 INFO:tasks.workunit.client.1.smithi136.stdout:1/112: sync 2024-04-06T03:40:28.373 INFO:tasks.workunit.client.1.smithi136.stdout:8/121: write d3/da/fd [3499445,84329] 0 2024-04-06T03:40:28.373 INFO:tasks.workunit.client.0.smithi094.stdout:0/157: creat d1/d9/dd/d18/f31 x:0 0 0 2024-04-06T03:40:28.373 INFO:tasks.workunit.client.0.smithi094.stdout:0/158: write d1/f2c [511329,98088] 0 2024-04-06T03:40:28.373 INFO:tasks.workunit.client.0.smithi094.stdout:0/159: write d1/d10/f2e [588811,122687] 0 2024-04-06T03:40:28.373 INFO:tasks.workunit.client.0.smithi094.stdout:0/160: truncate d1/d9/f27 880086 0 2024-04-06T03:40:28.374 INFO:tasks.workunit.client.1.smithi136.stdout:5/122: link l0 d1/dc/d15/d20/l26 0 2024-04-06T03:40:28.374 INFO:tasks.workunit.client.1.smithi136.stdout:5/123: fdatasync d1/d16/d17/f1a 0 2024-04-06T03:40:28.374 INFO:tasks.workunit.client.1.smithi136.stdout:9/97: creat d17/d19/f1e x:0 0 0 2024-04-06T03:40:28.376 INFO:tasks.workunit.client.1.smithi136.stdout:2/182: symlink d3/d15/d19/l44 0 2024-04-06T03:40:28.376 INFO:tasks.workunit.client.1.smithi136.stdout:2/183: chown d3/d9 26023026 1 2024-04-06T03:40:28.376 INFO:tasks.workunit.client.1.smithi136.stdout:2/184: write d3/d9/d10/d18/d1d/f35 [89821,85694] 0 2024-04-06T03:40:28.376 INFO:tasks.workunit.client.1.smithi136.stdout:2/185: fdatasync d3/d9/d10/d18/d1d/f20 0 2024-04-06T03:40:28.376 INFO:tasks.workunit.client.1.smithi136.stdout:0/62: symlink d1/d6/l15 0 2024-04-06T03:40:28.377 INFO:tasks.workunit.client.0.smithi094.stdout:4/188: write d0/df/d1e/f2a [2723191,80376] 0 2024-04-06T03:40:28.377 INFO:tasks.workunit.client.0.smithi094.stdout:4/189: creat d0/df/d1e/d34/f3d x:0 0 0 2024-04-06T03:40:28.378 INFO:tasks.workunit.client.0.smithi094.stdout:2/163: rmdir d0/d37 0 2024-04-06T03:40:28.379 INFO:tasks.workunit.client.0.smithi094.stdout:0/161: unlink d1/d9/dd/d18/d1e/c24 0 2024-04-06T03:40:28.380 INFO:tasks.workunit.client.1.smithi136.stdout:1/113: write f10 [2927694,44800] 0 2024-04-06T03:40:28.381 INFO:tasks.workunit.client.1.smithi136.stdout:8/122: link d3/c16 d3/de/d1b/c23 0 2024-04-06T03:40:28.381 INFO:tasks.workunit.client.0.smithi094.stdout:4/190: mkdir d0/d15/d3a/d3e 0 2024-04-06T03:40:28.382 INFO:tasks.workunit.client.1.smithi136.stdout:5/124: mknod d1/c27 0 2024-04-06T03:40:28.382 INFO:tasks.workunit.client.1.smithi136.stdout:5/125: readlink d1/dc/d15/d20/l26 0 2024-04-06T03:40:28.383 INFO:tasks.workunit.client.0.smithi094.stdout:2/164: creat d0/d8/d1d/f38 x:0 0 0 2024-04-06T03:40:28.383 INFO:tasks.workunit.client.0.smithi094.stdout:2/165: truncate d0/dd/d1b/f1c 454678 0 2024-04-06T03:40:28.383 INFO:tasks.workunit.client.0.smithi094.stdout:2/166: chown d0/d8/d27 4348 1 2024-04-06T03:40:28.385 INFO:tasks.workunit.client.1.smithi136.stdout:2/186: write d3/d9/f12 [3728993,10533] 0 2024-04-06T03:40:28.386 INFO:tasks.workunit.client.1.smithi136.stdout:9/98: write fe [754087,37307] 0 2024-04-06T03:40:28.386 INFO:tasks.workunit.client.1.smithi136.stdout:9/99: creat d17/d19/f1f x:0 0 0 2024-04-06T03:40:28.386 INFO:tasks.workunit.client.0.smithi094.stdout:2/167: read d0/dd/ff [2401777,61842] 0 2024-04-06T03:40:28.386 INFO:tasks.workunit.client.0.smithi094.stdout:2/168: chown d0/dd/d1b/l34 7190121 1 2024-04-06T03:40:28.387 INFO:tasks.workunit.client.0.smithi094.stdout:0/162: mkdir d1/d9/dd/d18/d32 0 2024-04-06T03:40:28.390 INFO:tasks.workunit.client.0.smithi094.stdout:4/191: mknod d0/d15/d3a/d3e/c3f 0 2024-04-06T03:40:28.390 INFO:tasks.workunit.client.0.smithi094.stdout:4/192: write d0/d15/f1a [5195123,57401] 0 2024-04-06T03:40:28.395 INFO:tasks.workunit.client.0.smithi094.stdout:0/163: mkdir d1/d9/dd/d18/d1e/d33 0 2024-04-06T03:40:28.397 INFO:tasks.workunit.client.1.smithi136.stdout:8/123: dwrite d3/de/d1b/f1d [4194304,4194304] 0 2024-04-06T03:40:28.397 INFO:tasks.workunit.client.1.smithi136.stdout:8/124: dread - d3/da/f13 zero size 2024-04-06T03:40:28.399 INFO:tasks.workunit.client.0.smithi094.stdout:4/193: rename d0/f12 to d0/d15/f40 0 2024-04-06T03:40:28.400 INFO:tasks.workunit.client.1.smithi136.stdout:1/114: write f3 [7898662,73843] 0 2024-04-06T03:40:28.400 INFO:tasks.workunit.client.1.smithi136.stdout:1/115: fdatasync fb 0 2024-04-06T03:40:28.405 INFO:tasks.workunit.client.1.smithi136.stdout:5/126: dread d1/f18 [0,4194304] 0 2024-04-06T03:40:28.405 INFO:tasks.workunit.client.1.smithi136.stdout:5/127: fsync d1/d16/d17/f1a 0 2024-04-06T03:40:28.406 INFO:tasks.workunit.client.1.smithi136.stdout:4/98: link cb d1d/c23 0 2024-04-06T03:40:28.408 INFO:tasks.workunit.client.0.smithi094.stdout:4/194: mknod d0/d15/d3a/c41 0 2024-04-06T03:40:28.408 INFO:tasks.workunit.client.0.smithi094.stdout:4/195: dread - d0/f29 zero size 2024-04-06T03:40:28.408 INFO:tasks.workunit.client.0.smithi094.stdout:0/164: dread d1/f23 [0,4194304] 0 2024-04-06T03:40:28.409 INFO:tasks.workunit.client.1.smithi136.stdout:0/63: rmdir d1 39 2024-04-06T03:40:28.409 INFO:tasks.workunit.client.1.smithi136.stdout:0/64: dread - d1/d6/f12 zero size 2024-04-06T03:40:28.409 INFO:tasks.workunit.client.0.smithi094.stdout:2/169: dwrite d0/d8/f33 [0,4194304] 0 2024-04-06T03:40:28.410 INFO:tasks.workunit.client.0.smithi094.stdout:2/170: chown d0/d18 351439006 1 2024-04-06T03:40:28.410 INFO:tasks.workunit.client.1.smithi136.stdout:2/187: mkdir d3/d45 0 2024-04-06T03:40:28.411 INFO:tasks.workunit.client.0.smithi094.stdout:4/196: link d0/df/d1e/l2e d0/df/d1e/l42 0 2024-04-06T03:40:28.416 INFO:tasks.workunit.client.0.smithi094.stdout:0/165: symlink d1/d9/dd/d18/d1d/l34 0 2024-04-06T03:40:28.416 INFO:tasks.workunit.client.0.smithi094.stdout:0/166: readlink d1/l6 0 2024-04-06T03:40:28.423 INFO:tasks.workunit.client.1.smithi136.stdout:8/125: rmdir d3/de/d1b 39 2024-04-06T03:40:28.423 INFO:tasks.workunit.client.1.smithi136.stdout:8/126: readlink d3/l15 0 2024-04-06T03:40:28.424 INFO:tasks.workunit.client.0.smithi094.stdout:2/171: stat d0/dd/l12 0 2024-04-06T03:40:28.424 INFO:tasks.workunit.client.0.smithi094.stdout:2/172: stat d0/dd/d1b/f1c 0 2024-04-06T03:40:28.425 INFO:tasks.workunit.client.1.smithi136.stdout:5/128: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:28.426 INFO:tasks.workunit.client.1.smithi136.stdout:0/65: symlink d1/d6/l16 0 2024-04-06T03:40:28.426 INFO:tasks.workunit.client.1.smithi136.stdout:0/66: write d1/d6/fd [264348,12840] 0 2024-04-06T03:40:28.426 INFO:tasks.workunit.client.0.smithi094.stdout:0/167: dread d1/d9/fc [0,4194304] 0 2024-04-06T03:40:28.426 INFO:tasks.workunit.client.0.smithi094.stdout:0/168: stat d1/d9/dd/d18/d1e 0 2024-04-06T03:40:28.427 INFO:tasks.workunit.client.1.smithi136.stdout:1/116: dwrite fd [0,4194304] 0 2024-04-06T03:40:28.427 INFO:tasks.workunit.client.0.smithi094.stdout:4/197: creat d0/d15/d3a/f43 x:0 0 0 2024-04-06T03:40:28.430 INFO:tasks.workunit.client.0.smithi094.stdout:2/173: mknod d0/dd/d19/d31/c39 0 2024-04-06T03:40:28.430 INFO:tasks.workunit.client.1.smithi136.stdout:2/188: getdents d3/d9 0 2024-04-06T03:40:28.430 INFO:tasks.workunit.client.1.smithi136.stdout:2/189: write d3/d9/d10/f22 [324804,30414] 0 2024-04-06T03:40:28.431 INFO:tasks.workunit.client.1.smithi136.stdout:9/100: mkdir d17/d20 0 2024-04-06T03:40:28.432 INFO:tasks.workunit.client.0.smithi094.stdout:0/169: rename d1/d9/dd/d18/d1e/d33 to d1/d9/dd/d18/d1e/d22/d35 0 2024-04-06T03:40:28.432 INFO:tasks.workunit.client.0.smithi094.stdout:0/170: truncate d1/d9/dd/d18/f31 222873 0 2024-04-06T03:40:28.432 INFO:tasks.workunit.client.0.smithi094.stdout:4/198: dread d0/d4/f21 [0,4194304] 0 2024-04-06T03:40:28.434 INFO:tasks.workunit.client.1.smithi136.stdout:8/127: rmdir d3/da/d1c 0 2024-04-06T03:40:28.435 INFO:tasks.workunit.client.0.smithi094.stdout:4/199: dread d0/f22 [0,4194304] 0 2024-04-06T03:40:28.435 INFO:tasks.workunit.client.0.smithi094.stdout:2/174: rmdir d0 39 2024-04-06T03:40:28.439 INFO:tasks.workunit.client.1.smithi136.stdout:6/67: sync 2024-04-06T03:40:28.439 INFO:tasks.workunit.client.1.smithi136.stdout:3/55: sync 2024-04-06T03:40:28.439 INFO:tasks.workunit.client.1.smithi136.stdout:7/65: sync 2024-04-06T03:40:28.439 INFO:tasks.workunit.client.1.smithi136.stdout:3/56: stat d0/d4 0 2024-04-06T03:40:28.439 INFO:tasks.workunit.client.1.smithi136.stdout:3/57: stat d0/l9 0 2024-04-06T03:40:28.439 INFO:tasks.workunit.client.1.smithi136.stdout:6/68: chown d2/de/c16 9451 1 2024-04-06T03:40:28.440 INFO:tasks.workunit.client.1.smithi136.stdout:6/69: fsync d2/de/f15 0 2024-04-06T03:40:28.440 INFO:tasks.workunit.client.0.smithi094.stdout:0/171: mknod d1/d9/dd/d18/d1d/c36 0 2024-04-06T03:40:28.440 INFO:tasks.workunit.client.1.smithi136.stdout:4/99: dwrite f1a [0,4194304] 0 2024-04-06T03:40:28.440 INFO:tasks.workunit.client.1.smithi136.stdout:0/67: mknod d1/d6/d13/c17 0 2024-04-06T03:40:28.441 INFO:tasks.workunit.client.0.smithi094.stdout:2/175: creat d0/dd/d19/d31/f3a x:0 0 0 2024-04-06T03:40:28.444 INFO:tasks.workunit.client.1.smithi136.stdout:1/117: creat d12/f24 x:0 0 0 2024-04-06T03:40:28.445 INFO:tasks.workunit.client.0.smithi094.stdout:2/176: symlink d0/dd/l3b 0 2024-04-06T03:40:28.445 INFO:tasks.workunit.client.1.smithi136.stdout:2/190: link d3/d9/c31 d3/c46 0 2024-04-06T03:40:28.445 INFO:tasks.workunit.client.1.smithi136.stdout:2/191: write d3/d9/f12 [3475466,105490] 0 2024-04-06T03:40:28.445 INFO:tasks.workunit.client.1.smithi136.stdout:2/192: truncate d3/d9/d10/d18/d1d/f20 1261947 0 2024-04-06T03:40:28.445 INFO:tasks.workunit.client.1.smithi136.stdout:2/193: truncate d3/d9/d10/d18/d1d/f20 1697839 0 2024-04-06T03:40:28.445 INFO:tasks.workunit.client.1.smithi136.stdout:2/194: truncate d3/d9/d10/d18/d1d/f20 2178596 0 2024-04-06T03:40:28.446 INFO:tasks.workunit.client.1.smithi136.stdout:2/195: fsync d3/d9/d10/d18/d1d/d3c/f40 0 2024-04-06T03:40:28.446 INFO:tasks.workunit.client.0.smithi094.stdout:8/153: sync 2024-04-06T03:40:28.447 INFO:tasks.workunit.client.0.smithi094.stdout:7/140: sync 2024-04-06T03:40:28.447 INFO:tasks.workunit.client.1.smithi136.stdout:3/58: creat d0/f11 x:0 0 0 2024-04-06T03:40:28.447 INFO:tasks.workunit.client.0.smithi094.stdout:0/172: dread d1/f7 [0,4194304] 0 2024-04-06T03:40:28.449 INFO:tasks.workunit.client.1.smithi136.stdout:7/66: creat d5/df/d12/f16 x:0 0 0 2024-04-06T03:40:28.449 INFO:tasks.workunit.client.1.smithi136.stdout:9/101: dread fe [0,4194304] 0 2024-04-06T03:40:28.450 INFO:tasks.workunit.client.1.smithi136.stdout:2/196: dread d3/d15/f2b [0,4194304] 0 2024-04-06T03:40:28.451 INFO:tasks.workunit.client.1.smithi136.stdout:6/70: mkdir d2/d7/d1e 0 2024-04-06T03:40:28.451 INFO:tasks.workunit.client.1.smithi136.stdout:6/71: chown d2/de/c16 9135055 1 2024-04-06T03:40:28.451 INFO:tasks.workunit.client.1.smithi136.stdout:8/128: dread d3/da/fd [0,4194304] 0 2024-04-06T03:40:28.451 INFO:tasks.workunit.client.1.smithi136.stdout:8/129: truncate d3/da/f13 563742 0 2024-04-06T03:40:28.451 INFO:tasks.workunit.client.1.smithi136.stdout:5/129: dwrite d1/f18 [4194304,4194304] 0 2024-04-06T03:40:28.452 INFO:tasks.workunit.client.0.smithi094.stdout:8/154: creat d18/f36 x:0 0 0 2024-04-06T03:40:28.452 INFO:tasks.workunit.client.0.smithi094.stdout:8/155: fsync d18/f22 0 2024-04-06T03:40:28.453 INFO:tasks.workunit.client.1.smithi136.stdout:4/100: creat d1d/f24 x:0 0 0 2024-04-06T03:40:28.453 INFO:tasks.workunit.client.1.smithi136.stdout:4/101: chown cb 1086938850 1 2024-04-06T03:40:28.454 INFO:tasks.workunit.client.0.smithi094.stdout:7/141: link d2/d6/dc/c19 d2/d6/dc/c21 0 2024-04-06T03:40:28.455 INFO:tasks.workunit.client.0.smithi094.stdout:7/142: write d2/f1f [478132,6438] 0 2024-04-06T03:40:28.455 INFO:tasks.workunit.client.0.smithi094.stdout:7/143: chown d2/f1f 10438630 1 2024-04-06T03:40:28.455 INFO:tasks.workunit.client.0.smithi094.stdout:7/144: readlink d2/d6/dc/l1e 0 2024-04-06T03:40:28.457 INFO:tasks.workunit.client.0.smithi094.stdout:4/200: dwrite d0/d4/f6 [4194304,4194304] 0 2024-04-06T03:40:28.457 INFO:tasks.workunit.client.0.smithi094.stdout:0/173: truncate d1/d9/dd/d18/d1a/f25 102521 0 2024-04-06T03:40:28.460 INFO:tasks.workunit.client.0.smithi094.stdout:8/156: dread f11 [0,4194304] 0 2024-04-06T03:40:28.461 INFO:tasks.workunit.client.1.smithi136.stdout:1/118: dread fd [0,4194304] 0 2024-04-06T03:40:28.461 INFO:tasks.workunit.client.0.smithi094.stdout:4/201: creat d0/f44 x:0 0 0 2024-04-06T03:40:28.462 INFO:tasks.workunit.client.1.smithi136.stdout:3/59: mkdir d0/d4/d12 0 2024-04-06T03:40:28.462 INFO:tasks.workunit.client.1.smithi136.stdout:3/60: creat d0/f13 x:0 0 0 2024-04-06T03:40:28.464 INFO:tasks.workunit.client.1.smithi136.stdout:2/197: symlink d3/d9/d10/l47 0 2024-04-06T03:40:28.466 INFO:tasks.workunit.client.1.smithi136.stdout:5/130: dread d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:28.467 INFO:tasks.workunit.client.1.smithi136.stdout:7/67: dread d5/dc/fe [0,4194304] 0 2024-04-06T03:40:28.467 INFO:tasks.workunit.client.1.smithi136.stdout:7/68: creat d5/df/d12/f17 x:0 0 0 2024-04-06T03:40:28.467 INFO:tasks.workunit.client.1.smithi136.stdout:7/69: dread - d5/df/d12/f16 zero size 2024-04-06T03:40:28.468 INFO:tasks.workunit.client.1.smithi136.stdout:6/72: symlink d2/d7/l1f 0 2024-04-06T03:40:28.470 INFO:tasks.workunit.client.0.smithi094.stdout:8/157: mkdir d18/d2c/d37 0 2024-04-06T03:40:28.470 INFO:tasks.workunit.client.0.smithi094.stdout:8/158: dread - d18/f2d zero size 2024-04-06T03:40:28.471 INFO:tasks.workunit.client.1.smithi136.stdout:8/130: truncate d3/da/fd 3399542 0 2024-04-06T03:40:28.471 INFO:tasks.workunit.client.1.smithi136.stdout:8/131: fsync d3/f4 0 2024-04-06T03:40:28.471 INFO:tasks.workunit.client.0.smithi094.stdout:7/145: write d2/d6/dc/dd/df/f1a [2817779,37265] 0 2024-04-06T03:40:28.471 INFO:tasks.workunit.client.0.smithi094.stdout:2/177: dwrite d0/d8/f11 [4194304,4194304] 0 2024-04-06T03:40:28.472 INFO:tasks.workunit.client.0.smithi094.stdout:2/178: fsync d0/f1 0 2024-04-06T03:40:28.472 INFO:tasks.workunit.client.0.smithi094.stdout:7/146: creat d2/d6/f22 x:0 0 0 2024-04-06T03:40:28.472 INFO:tasks.workunit.client.0.smithi094.stdout:7/147: creat d2/f23 x:0 0 0 2024-04-06T03:40:28.472 INFO:tasks.workunit.client.0.smithi094.stdout:2/179: write d0/d18/f24 [996369,104840] 0 2024-04-06T03:40:28.472 INFO:tasks.workunit.client.0.smithi094.stdout:7/148: creat d2/d6/f24 x:0 0 0 2024-04-06T03:40:28.472 INFO:tasks.workunit.client.0.smithi094.stdout:0/174: rmdir d1/d9/dd/d18/d1e 39 2024-04-06T03:40:28.479 INFO:tasks.workunit.client.1.smithi136.stdout:4/102: creat d1d/f25 x:0 0 0 2024-04-06T03:40:28.482 INFO:tasks.workunit.client.1.smithi136.stdout:1/119: symlink d12/d1b/d1e/d23/l25 0 2024-04-06T03:40:28.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/202: dwrite d0/f8 [0,4194304] 0 2024-04-06T03:40:28.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/203: write d0/d23/f32 [930530,6294] 0 2024-04-06T03:40:28.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/204: stat d0/c31 0 2024-04-06T03:40:28.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/205: write d0/f29 [251382,39626] 0 2024-04-06T03:40:28.485 INFO:tasks.workunit.client.1.smithi136.stdout:9/102: mkdir d17/d21 0 2024-04-06T03:40:28.487 INFO:tasks.workunit.client.1.smithi136.stdout:0/68: dwrite d1/fc [0,4194304] 0 2024-04-06T03:40:28.488 INFO:tasks.workunit.client.1.smithi136.stdout:0/69: write d1/d6/fd [154658,116971] 0 2024-04-06T03:40:28.491 INFO:tasks.workunit.client.0.smithi094.stdout:2/180: rename d0/dd/d19/l30 to d0/dd/d1b/l3c 0 2024-04-06T03:40:28.492 INFO:tasks.workunit.client.1.smithi136.stdout:2/198: unlink d3/d9/d10/d18/f2d 0 2024-04-06T03:40:28.492 INFO:tasks.workunit.client.1.smithi136.stdout:2/199: readlink d3/d9/d10/l3f 0 2024-04-06T03:40:28.493 INFO:tasks.workunit.client.1.smithi136.stdout:2/200: write d3/d9/d10/f22 [65637,56714] 0 2024-04-06T03:40:28.494 INFO:tasks.workunit.client.0.smithi094.stdout:7/149: mknod d2/d6/dc/d12/c25 0 2024-04-06T03:40:28.494 INFO:tasks.workunit.client.0.smithi094.stdout:8/159: dwrite f11 [0,4194304] 0 2024-04-06T03:40:28.494 INFO:tasks.workunit.client.0.smithi094.stdout:8/160: write d18/f25 [982225,89032] 0 2024-04-06T03:40:28.494 INFO:tasks.workunit.client.1.smithi136.stdout:3/61: dwrite d0/d4/fe [0,4194304] 0 2024-04-06T03:40:28.495 INFO:tasks.workunit.client.1.smithi136.stdout:5/131: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:28.496 INFO:tasks.workunit.client.0.smithi094.stdout:0/175: creat d1/d9/dd/d18/d1e/d22/f37 x:0 0 0 2024-04-06T03:40:28.498 INFO:tasks.workunit.client.1.smithi136.stdout:7/70: mknod d5/df/d12/c18 0 2024-04-06T03:40:28.498 INFO:tasks.workunit.client.1.smithi136.stdout:7/71: chown d5/f7 2394197 1 2024-04-06T03:40:28.498 INFO:tasks.workunit.client.1.smithi136.stdout:7/72: creat d5/df/d12/f19 x:0 0 0 2024-04-06T03:40:28.498 INFO:tasks.workunit.client.1.smithi136.stdout:7/73: stat d5/f9 0 2024-04-06T03:40:28.498 INFO:tasks.workunit.client.1.smithi136.stdout:7/74: write d5/df/d12/f19 [888450,93295] 0 2024-04-06T03:40:28.499 INFO:tasks.workunit.client.0.smithi094.stdout:4/206: symlink d0/l45 0 2024-04-06T03:40:28.499 INFO:tasks.workunit.client.0.smithi094.stdout:4/207: stat d0/df/c1f 0 2024-04-06T03:40:28.499 INFO:tasks.workunit.client.0.smithi094.stdout:4/208: dread - d0/df/d13/f37 zero size 2024-04-06T03:40:28.499 INFO:tasks.workunit.client.0.smithi094.stdout:4/209: write d0/df/d1e/f25 [9255726,52183] 0 2024-04-06T03:40:28.500 INFO:tasks.workunit.client.0.smithi094.stdout:2/181: truncate d0/d8/d27/d2b/f2c 2625135 0 2024-04-06T03:40:28.503 INFO:tasks.workunit.client.1.smithi136.stdout:8/132: mknod d3/da/c24 0 2024-04-06T03:40:28.504 INFO:tasks.workunit.client.0.smithi094.stdout:7/150: symlink d2/d6/dc/d12/l26 0 2024-04-06T03:40:28.506 INFO:tasks.workunit.client.1.smithi136.stdout:4/103: creat d1d/f26 x:0 0 0 2024-04-06T03:40:28.507 INFO:tasks.workunit.client.0.smithi094.stdout:0/176: mknod d1/d9/dd/d18/d1e/d22/c38 0 2024-04-06T03:40:28.508 INFO:tasks.workunit.client.0.smithi094.stdout:8/161: write d18/f1a [3617390,50389] 0 2024-04-06T03:40:28.510 INFO:tasks.workunit.client.1.smithi136.stdout:1/120: rename fd to d12/d1b/d1e/d1f/f26 0 2024-04-06T03:40:28.511 INFO:tasks.workunit.client.0.smithi094.stdout:4/210: mkdir d0/d15/d3a/d3e/d46 0 2024-04-06T03:40:28.512 INFO:tasks.workunit.client.1.smithi136.stdout:9/103: symlink d17/d20/l22 0 2024-04-06T03:40:28.514 INFO:tasks.workunit.client.0.smithi094.stdout:2/182: mkdir d0/dd/d3d 0 2024-04-06T03:40:28.514 INFO:tasks.workunit.client.0.smithi094.stdout:2/183: creat d0/dd/d1b/f3e x:0 0 0 2024-04-06T03:40:28.514 INFO:tasks.workunit.client.1.smithi136.stdout:5/132: dwrite d1/f18 [4194304,4194304] 0 2024-04-06T03:40:28.516 INFO:tasks.workunit.client.1.smithi136.stdout:2/201: dwrite d3/d9/d10/d18/d1d/f35 [0,4194304] 0 2024-04-06T03:40:28.521 INFO:tasks.workunit.client.1.smithi136.stdout:0/70: truncate d1/f4 651623 0 2024-04-06T03:40:28.526 INFO:tasks.workunit.client.1.smithi136.stdout:3/62: truncate d0/f2 49032 0 2024-04-06T03:40:28.527 INFO:tasks.workunit.client.0.smithi094.stdout:7/151: dwrite d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:28.527 INFO:tasks.workunit.client.0.smithi094.stdout:7/152: write d2/d6/fa [3192787,19287] 0 2024-04-06T03:40:28.532 INFO:tasks.workunit.client.0.smithi094.stdout:2/184: dwrite d0/d8/d27/d2b/f2d [0,4194304] 0 2024-04-06T03:40:28.574 INFO:tasks.workunit.client.1.smithi136.stdout:7/75: symlink d5/df/d11/l1a 0 2024-04-06T03:40:28.574 INFO:tasks.workunit.client.1.smithi136.stdout:6/73: getdents d2/d7 0 2024-04-06T03:40:28.576 INFO:tasks.workunit.client.1.smithi136.stdout:8/133: symlink d3/da/l25 0 2024-04-06T03:40:28.576 INFO:tasks.workunit.client.1.smithi136.stdout:8/134: stat d3/de/d18 0 2024-04-06T03:40:28.580 INFO:tasks.workunit.client.1.smithi136.stdout:1/121: unlink d12/d1b/d1e/d1f/f26 0 2024-04-06T03:40:28.580 INFO:tasks.workunit.client.1.smithi136.stdout:1/122: read f10 [4015794,28154] 0 2024-04-06T03:40:28.584 INFO:tasks.workunit.client.0.smithi094.stdout:0/177: symlink d1/d9/dd/d18/d1d/l39 0 2024-04-06T03:40:28.584 INFO:tasks.workunit.client.0.smithi094.stdout:0/178: truncate d1/d9/dd/d18/d1e/d22/f37 802854 0 2024-04-06T03:40:28.584 INFO:tasks.workunit.client.1.smithi136.stdout:5/133: rmdir d1/dc/d15/d20 39 2024-04-06T03:40:28.584 INFO:tasks.workunit.client.1.smithi136.stdout:5/134: chown d1/dc/d10 1 1 2024-04-06T03:40:28.584 INFO:tasks.workunit.client.1.smithi136.stdout:5/135: chown d1/d16/d17/f1a 2081 1 2024-04-06T03:40:28.584 INFO:tasks.workunit.client.1.smithi136.stdout:5/136: chown d1/dc/d15/d20 924 1 2024-04-06T03:40:28.587 INFO:tasks.workunit.client.0.smithi094.stdout:8/162: creat d18/d34/f38 x:0 0 0 2024-04-06T03:40:28.593 INFO:tasks.workunit.client.1.smithi136.stdout:2/202: creat d3/d9/d10/d18/f48 x:0 0 0 2024-04-06T03:40:28.594 INFO:tasks.workunit.client.0.smithi094.stdout:4/211: symlink d0/d15/d3a/d3e/d46/l47 0 2024-04-06T03:40:28.596 INFO:tasks.workunit.client.1.smithi136.stdout:1/123: write f10 [7148474,67741] 0 2024-04-06T03:40:28.596 INFO:tasks.workunit.client.1.smithi136.stdout:7/76: dwrite d5/df/d12/f16 [0,4194304] 0 2024-04-06T03:40:28.596 INFO:tasks.workunit.client.1.smithi136.stdout:7/77: truncate f1 4809045 0 2024-04-06T03:40:28.596 INFO:tasks.workunit.client.1.smithi136.stdout:0/71: creat d1/f18 x:0 0 0 2024-04-06T03:40:28.596 INFO:tasks.workunit.client.1.smithi136.stdout:0/72: dread - d1/f18 zero size 2024-04-06T03:40:28.597 INFO:tasks.workunit.client.1.smithi136.stdout:3/63: unlink d0/f3 0 2024-04-06T03:40:28.601 INFO:tasks.workunit.client.0.smithi094.stdout:7/153: creat d2/d6/dc/dd/f27 x:0 0 0 2024-04-06T03:40:28.601 INFO:tasks.workunit.client.0.smithi094.stdout:7/154: creat d2/d6/dc/dd/f28 x:0 0 0 2024-04-06T03:40:28.601 INFO:tasks.workunit.client.0.smithi094.stdout:7/155: read - d2/d6/dc/dd/f27 zero size 2024-04-06T03:40:28.602 INFO:tasks.workunit.client.1.smithi136.stdout:6/74: creat d2/d7/f20 x:0 0 0 2024-04-06T03:40:28.602 INFO:tasks.workunit.client.1.smithi136.stdout:6/75: creat d2/f21 x:0 0 0 2024-04-06T03:40:28.603 INFO:tasks.workunit.client.1.smithi136.stdout:6/76: chown d2/de/f10 10 1 2024-04-06T03:40:28.606 INFO:tasks.workunit.client.1.smithi136.stdout:8/135: mkdir d3/da/d26 0 2024-04-06T03:40:28.607 INFO:tasks.workunit.client.1.smithi136.stdout:4/104: rmdir d1d 39 2024-04-06T03:40:28.612 INFO:tasks.workunit.client.1.smithi136.stdout:9/104: mknod d17/c23 0 2024-04-06T03:40:28.612 INFO:tasks.workunit.client.1.smithi136.stdout:9/105: chown f13 712101772 1 2024-04-06T03:40:28.612 INFO:tasks.workunit.client.1.smithi136.stdout:9/106: fsync f13 0 2024-04-06T03:40:28.612 INFO:tasks.workunit.client.1.smithi136.stdout:9/107: write d17/d19/f1f [103787,9144] 0 2024-04-06T03:40:28.614 INFO:tasks.workunit.client.1.smithi136.stdout:5/137: rmdir d1/dc 39 2024-04-06T03:40:28.614 INFO:tasks.workunit.client.1.smithi136.stdout:3/64: dwrite d0/d4/fd [0,4194304] 0 2024-04-06T03:40:28.614 INFO:tasks.workunit.client.1.smithi136.stdout:3/65: write d0/f6 [2134014,54992] 0 2024-04-06T03:40:28.616 INFO:tasks.workunit.client.1.smithi136.stdout:2/203: rename d3/d9/c41 to d3/d9/d3b/c49 0 2024-04-06T03:40:28.617 INFO:tasks.workunit.client.0.smithi094.stdout:0/179: unlink d1/d9/dd/d18/d1e/d22/l2f 0 2024-04-06T03:40:28.618 INFO:tasks.workunit.client.0.smithi094.stdout:0/180: creat d1/d9/dd/d18/f3a x:0 0 0 2024-04-06T03:40:28.618 INFO:tasks.workunit.client.1.smithi136.stdout:8/136: dread f1 [0,4194304] 0 2024-04-06T03:40:28.619 INFO:tasks.workunit.client.0.smithi094.stdout:4/212: mknod d0/df/d1e/c48 0 2024-04-06T03:40:28.620 INFO:tasks.workunit.client.1.smithi136.stdout:6/77: dwrite d2/fd [0,4194304] 0 2024-04-06T03:40:28.621 INFO:tasks.workunit.client.1.smithi136.stdout:6/78: fdatasync d2/d7/f1b 0 2024-04-06T03:40:28.624 INFO:tasks.workunit.client.1.smithi136.stdout:1/124: mkdir d12/d1b/d1e/d27 0 2024-04-06T03:40:28.624 INFO:tasks.workunit.client.1.smithi136.stdout:8/137: dread d3/da/fd [0,4194304] 0 2024-04-06T03:40:28.625 INFO:tasks.workunit.client.0.smithi094.stdout:7/156: mkdir d2/d6/dc/d12/d29 0 2024-04-06T03:40:28.625 INFO:tasks.workunit.client.0.smithi094.stdout:7/157: fdatasync d2/d6/dc/f15 0 2024-04-06T03:40:28.625 INFO:tasks.workunit.client.0.smithi094.stdout:7/158: creat d2/d6/dc/d12/f2a x:0 0 0 2024-04-06T03:40:28.626 INFO:tasks.workunit.client.0.smithi094.stdout:4/213: write d0/f8 [1898339,119588] 0 2024-04-06T03:40:28.628 INFO:tasks.workunit.client.0.smithi094.stdout:4/214: dread d0/f22 [0,4194304] 0 2024-04-06T03:40:28.629 INFO:tasks.workunit.client.1.smithi136.stdout:7/78: mkdir d5/df/d12/d1b 0 2024-04-06T03:40:28.629 INFO:tasks.workunit.client.0.smithi094.stdout:2/185: mkdir d0/d18/d3f 0 2024-04-06T03:40:28.631 INFO:tasks.workunit.client.1.smithi136.stdout:0/73: creat d1/d6/f19 x:0 0 0 2024-04-06T03:40:28.632 INFO:tasks.workunit.client.1.smithi136.stdout:0/74: write d1/d6/fd [61202,119026] 0 2024-04-06T03:40:28.632 INFO:tasks.workunit.client.1.smithi136.stdout:0/75: truncate d1/f18 104632 0 2024-04-06T03:40:28.633 INFO:tasks.workunit.client.1.smithi136.stdout:0/76: dread d1/d6/f11 [0,4194304] 0 2024-04-06T03:40:28.635 INFO:tasks.workunit.client.0.smithi094.stdout:0/181: rename d1/d9/f27 to d1/d9/dd/f3b 0 2024-04-06T03:40:28.636 INFO:tasks.workunit.client.1.smithi136.stdout:6/79: dwrite d2/de/f15 [0,4194304] 0 2024-04-06T03:40:28.636 INFO:tasks.workunit.client.1.smithi136.stdout:6/80: chown d2/d7/d1e 1 1 2024-04-06T03:40:28.636 INFO:tasks.workunit.client.1.smithi136.stdout:6/81: chown d2/d7 7 1 2024-04-06T03:40:28.637 INFO:tasks.workunit.client.1.smithi136.stdout:6/82: write d2/f5 [5163342,23757] 0 2024-04-06T03:40:28.637 INFO:tasks.workunit.client.1.smithi136.stdout:4/105: creat d1d/f27 x:0 0 0 2024-04-06T03:40:28.639 INFO:tasks.workunit.client.0.smithi094.stdout:0/182: dread d1/d9/fc [4194304,4194304] 0 2024-04-06T03:40:28.639 INFO:tasks.workunit.client.0.smithi094.stdout:0/183: write d1/d9/dd/d18/d1a/f25 [472171,48100] 0 2024-04-06T03:40:28.639 INFO:tasks.workunit.client.0.smithi094.stdout:0/184: chown d1/d9/dd/d18/d1e/d22/f37 1738422 1 2024-04-06T03:40:28.639 INFO:tasks.workunit.client.0.smithi094.stdout:0/185: chown d1/f2c 6138235 1 2024-04-06T03:40:28.640 INFO:tasks.workunit.client.1.smithi136.stdout:9/108: mknod d17/d20/c24 0 2024-04-06T03:40:28.641 INFO:tasks.workunit.client.0.smithi094.stdout:7/159: creat d2/d6/dc/f2b x:0 0 0 2024-04-06T03:40:28.642 INFO:tasks.workunit.client.1.smithi136.stdout:5/138: truncate d1/f4 2770000 0 2024-04-06T03:40:28.644 INFO:tasks.workunit.client.0.smithi094.stdout:4/215: mkdir d0/d15/d3a/d3e/d49 0 2024-04-06T03:40:28.649 INFO:tasks.workunit.client.0.smithi094.stdout:0/186: creat d1/d9/dd/d18/d1e/d22/d35/f3c x:0 0 0 2024-04-06T03:40:28.650 INFO:tasks.workunit.client.1.smithi136.stdout:6/83: dread d2/de/f15 [0,4194304] 0 2024-04-06T03:40:28.650 INFO:tasks.workunit.client.1.smithi136.stdout:6/84: write d2/d7/f12 [36960,30041] 0 2024-04-06T03:40:28.650 INFO:tasks.workunit.client.1.smithi136.stdout:4/106: write f7 [4182177,19098] 0 2024-04-06T03:40:28.650 INFO:tasks.workunit.client.1.smithi136.stdout:3/66: symlink d0/d4/l14 0 2024-04-06T03:40:28.650 INFO:tasks.workunit.client.1.smithi136.stdout:3/67: chown d0/f2 638421 1 2024-04-06T03:40:28.650 INFO:tasks.workunit.client.1.smithi136.stdout:3/68: fsync d0/f5 0 2024-04-06T03:40:28.653 INFO:tasks.workunit.client.1.smithi136.stdout:2/204: rename d3/d1e to d3/d4a 0 2024-04-06T03:40:28.656 INFO:tasks.workunit.client.0.smithi094.stdout:2/186: dwrite d0/d18/f22 [0,4194304] 0 2024-04-06T03:40:28.656 INFO:tasks.workunit.client.0.smithi094.stdout:4/216: dread d0/d4/f14 [0,4194304] 0 2024-04-06T03:40:28.656 INFO:tasks.workunit.client.0.smithi094.stdout:4/217: readlink d0/l38 0 2024-04-06T03:40:28.657 INFO:tasks.workunit.client.1.smithi136.stdout:9/109: dread f16 [0,4194304] 0 2024-04-06T03:40:28.657 INFO:tasks.workunit.client.1.smithi136.stdout:2/205: dread d3/d9/d10/d18/d1d/f20 [0,4194304] 0 2024-04-06T03:40:28.657 INFO:tasks.workunit.client.1.smithi136.stdout:2/206: write d3/d15/d19/f39 [852819,82802] 0 2024-04-06T03:40:28.657 INFO:tasks.workunit.client.1.smithi136.stdout:2/207: chown l1 7 1 2024-04-06T03:40:28.658 INFO:tasks.workunit.client.1.smithi136.stdout:1/125: mkdir d12/d1b/d28 0 2024-04-06T03:40:28.659 INFO:tasks.workunit.client.0.smithi094.stdout:7/160: dwrite d2/f1f [0,4194304] 0 2024-04-06T03:40:28.663 INFO:tasks.workunit.client.1.smithi136.stdout:7/79: rename d5/dc/fe to d5/df/d11/f1c 0 2024-04-06T03:40:28.665 INFO:tasks.workunit.client.0.smithi094.stdout:0/187: dwrite d1/d9/dd/d18/d1e/d22/d35/f3c [0,4194304] 0 2024-04-06T03:40:28.667 INFO:tasks.workunit.client.1.smithi136.stdout:0/77: mkdir d1/d6/d13/d1a 0 2024-04-06T03:40:28.667 INFO:tasks.workunit.client.1.smithi136.stdout:0/78: truncate d1/d6/fd 974022 0 2024-04-06T03:40:28.667 INFO:tasks.workunit.client.0.smithi094.stdout:2/187: truncate d0/d18/f24 1046027 0 2024-04-06T03:40:28.673 INFO:tasks.workunit.client.1.smithi136.stdout:5/139: creat d1/dc/d10/f28 x:0 0 0 2024-04-06T03:40:28.673 INFO:tasks.workunit.client.1.smithi136.stdout:5/140: dread - d1/dc/d10/f28 zero size 2024-04-06T03:40:28.675 INFO:tasks.workunit.client.0.smithi094.stdout:0/188: dread d1/d9/dd/d18/d1e/d22/d35/f3c [0,4194304] 0 2024-04-06T03:40:28.676 INFO:tasks.workunit.client.1.smithi136.stdout:6/85: symlink d2/d7/d1e/l22 0 2024-04-06T03:40:28.676 INFO:tasks.workunit.client.0.smithi094.stdout:0/189: write d1/d9/dd/f3b [737776,96092] 0 2024-04-06T03:40:28.677 INFO:tasks.workunit.client.0.smithi094.stdout:2/188: readlink d0/d18/l26 0 2024-04-06T03:40:28.677 INFO:tasks.workunit.client.0.smithi094.stdout:2/189: chown d0/l3 539647 1 2024-04-06T03:40:28.677 INFO:tasks.workunit.client.0.smithi094.stdout:4/218: dwrite d0/df/f1b [0,4194304] 0 2024-04-06T03:40:28.678 INFO:tasks.workunit.client.1.smithi136.stdout:4/107: readlink l13 0 2024-04-06T03:40:28.679 INFO:tasks.workunit.client.1.smithi136.stdout:4/108: write f8 [2743125,80610] 0 2024-04-06T03:40:28.680 INFO:tasks.workunit.client.0.smithi094.stdout:7/161: dwrite d2/d6/f8 [0,4194304] 0 2024-04-06T03:40:28.680 INFO:tasks.workunit.client.0.smithi094.stdout:7/162: chown d2/d6/dc/l1e 57310 1 2024-04-06T03:40:28.680 INFO:tasks.workunit.client.1.smithi136.stdout:3/69: truncate d0/d4/fe 2586643 0 2024-04-06T03:40:28.684 INFO:tasks.workunit.client.1.smithi136.stdout:3/70: write d0/f6 [1206057,35047] 0 2024-04-06T03:40:28.692 INFO:tasks.workunit.client.0.smithi094.stdout:2/190: getdents d0/dd/d19 0 2024-04-06T03:40:28.693 INFO:tasks.workunit.client.1.smithi136.stdout:2/208: mkdir d3/d9/d10/d18/d4b 0 2024-04-06T03:40:28.694 INFO:tasks.workunit.client.1.smithi136.stdout:8/138: rmdir d3/de/d18 39 2024-04-06T03:40:28.695 INFO:tasks.workunit.client.0.smithi094.stdout:4/219: mkdir d0/df/d13/d4a 0 2024-04-06T03:40:28.697 INFO:tasks.workunit.client.1.smithi136.stdout:1/126: truncate f11 2084539 0 2024-04-06T03:40:28.698 INFO:tasks.workunit.client.1.smithi136.stdout:5/141: dwrite d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:28.701 INFO:tasks.workunit.client.0.smithi094.stdout:0/190: dwrite d1/f2c [0,4194304] 0 2024-04-06T03:40:28.702 INFO:tasks.workunit.client.0.smithi094.stdout:0/191: chown d1/d10/f2e 66636 1 2024-04-06T03:40:28.702 INFO:tasks.workunit.client.1.smithi136.stdout:2/209: dread d3/d15/d19/f39 [0,4194304] 0 2024-04-06T03:40:28.702 INFO:tasks.workunit.client.1.smithi136.stdout:2/210: dread - d3/d15/d19/f3d zero size 2024-04-06T03:40:28.702 INFO:tasks.workunit.client.1.smithi136.stdout:2/211: chown d3/d9/d3b/c49 11642090 1 2024-04-06T03:40:28.703 INFO:tasks.workunit.client.1.smithi136.stdout:0/79: write d1/f7 [1926818,124029] 0 2024-04-06T03:40:28.706 INFO:tasks.workunit.client.1.smithi136.stdout:5/142: write d1/d1f/f21 [797002,93898] 0 2024-04-06T03:40:28.706 INFO:tasks.workunit.client.1.smithi136.stdout:6/86: symlink d2/d7/l23 0 2024-04-06T03:40:28.707 INFO:tasks.workunit.client.0.smithi094.stdout:7/163: dwrite d2/d6/f8 [4194304,4194304] 0 2024-04-06T03:40:28.710 INFO:tasks.workunit.client.1.smithi136.stdout:8/139: truncate d3/da/fd 582307 0 2024-04-06T03:40:28.710 INFO:tasks.workunit.client.1.smithi136.stdout:8/140: creat d3/de/f27 x:0 0 0 2024-04-06T03:40:28.710 INFO:tasks.workunit.client.1.smithi136.stdout:8/141: chown d3/f10 5646277 1 2024-04-06T03:40:28.711 INFO:tasks.workunit.client.1.smithi136.stdout:6/87: write d2/de/f15 [2441930,97444] 0 2024-04-06T03:40:28.713 INFO:tasks.workunit.client.0.smithi094.stdout:0/192: mknod d1/d9/dd/c3d 0 2024-04-06T03:40:28.715 INFO:tasks.workunit.client.0.smithi094.stdout:2/191: dwrite d0/d8/fe [0,4194304] 0 2024-04-06T03:40:28.716 INFO:tasks.workunit.client.0.smithi094.stdout:4/220: mkdir d0/d4/d4b 0 2024-04-06T03:40:28.718 INFO:tasks.workunit.client.1.smithi136.stdout:2/212: read d3/d9/fe [4137047,105638] 0 2024-04-06T03:40:28.720 INFO:tasks.workunit.client.0.smithi094.stdout:7/164: mknod d2/d6/dc/c2c 0 2024-04-06T03:40:28.723 INFO:tasks.workunit.client.0.smithi094.stdout:2/192: readlink d0/l21 0 2024-04-06T03:40:28.723 INFO:tasks.workunit.client.0.smithi094.stdout:2/193: dread - d0/dd/d1b/f3e zero size 2024-04-06T03:40:28.723 INFO:tasks.workunit.client.0.smithi094.stdout:2/194: creat d0/dd/f40 x:0 0 0 2024-04-06T03:40:28.723 INFO:tasks.workunit.client.1.smithi136.stdout:2/213: dread d3/d15/f2b [0,4194304] 0 2024-04-06T03:40:28.728 INFO:tasks.workunit.client.1.smithi136.stdout:0/80: dwrite d1/d6/f12 [0,4194304] 0 2024-04-06T03:40:28.728 INFO:tasks.workunit.client.1.smithi136.stdout:0/81: dread - d1/d6/f19 zero size 2024-04-06T03:40:28.728 INFO:tasks.workunit.client.0.smithi094.stdout:7/165: link d2/d6/dc/dd/f27 d2/d6/dc/dd/df/f2d 0 2024-04-06T03:40:28.735 INFO:tasks.workunit.client.0.smithi094.stdout:0/193: dwrite d1/d9/dd/d18/f3a [0,4194304] 0 2024-04-06T03:40:28.735 INFO:tasks.workunit.client.0.smithi094.stdout:0/194: write d1/d10/f2e [1558592,22271] 0 2024-04-06T03:40:28.736 INFO:tasks.workunit.client.0.smithi094.stdout:1/131: sync 2024-04-06T03:40:28.736 INFO:tasks.workunit.client.0.smithi094.stdout:6/142: sync 2024-04-06T03:40:28.736 INFO:tasks.workunit.client.0.smithi094.stdout:6/143: chown dc/d11/d1b/c2c 1734870 1 2024-04-06T03:40:28.736 INFO:tasks.workunit.client.0.smithi094.stdout:6/144: chown cb 996120782 1 2024-04-06T03:40:28.736 INFO:tasks.workunit.client.0.smithi094.stdout:3/196: sync 2024-04-06T03:40:28.737 INFO:tasks.workunit.client.0.smithi094.stdout:6/145: creat dc/d11/d22/f39 x:0 0 0 2024-04-06T03:40:28.739 INFO:tasks.workunit.client.1.smithi136.stdout:6/88: dwrite d2/f5 [0,4194304] 0 2024-04-06T03:40:28.739 INFO:tasks.workunit.client.1.smithi136.stdout:2/214: dwrite d3/d9/d10/f17 [0,4194304] 0 2024-04-06T03:40:28.743 INFO:tasks.workunit.client.1.smithi136.stdout:5/143: mknod d1/dc/c29 0 2024-04-06T03:40:28.743 INFO:tasks.workunit.client.1.smithi136.stdout:5/144: write d1/d1f/f21 [2733387,30319] 0 2024-04-06T03:40:28.743 INFO:tasks.workunit.client.1.smithi136.stdout:5/145: read d1/d1f/f21 [1092753,93859] 0 2024-04-06T03:40:28.744 INFO:tasks.workunit.client.0.smithi094.stdout:1/132: mkdir d0/d4/d1b/d2f 0 2024-04-06T03:40:28.744 INFO:tasks.workunit.client.0.smithi094.stdout:9/101: sync 2024-04-06T03:40:28.749 INFO:tasks.workunit.client.0.smithi094.stdout:3/197: mkdir d9/d20/d3e 0 2024-04-06T03:40:28.749 INFO:tasks.workunit.client.0.smithi094.stdout:3/198: creat d9/f3f x:0 0 0 2024-04-06T03:40:28.749 INFO:tasks.workunit.client.0.smithi094.stdout:3/199: readlink d9/l1c 0 2024-04-06T03:40:28.751 INFO:tasks.workunit.client.0.smithi094.stdout:6/146: mkdir dc/d3a 0 2024-04-06T03:40:28.751 INFO:tasks.workunit.client.0.smithi094.stdout:6/147: fsync dc/fd 0 2024-04-06T03:40:28.751 INFO:tasks.workunit.client.0.smithi094.stdout:0/195: write d1/f2c [1092102,50847] 0 2024-04-06T03:40:28.751 INFO:tasks.workunit.client.0.smithi094.stdout:0/196: chown d1/d9/dd/d18/d1d 43 1 2024-04-06T03:40:28.752 INFO:tasks.workunit.client.0.smithi094.stdout:7/166: dwrite d2/f11 [0,4194304] 0 2024-04-06T03:40:28.755 INFO:tasks.workunit.client.1.smithi136.stdout:8/142: link d3/de/d1b/f1d d3/da/d26/f28 0 2024-04-06T03:40:28.756 INFO:tasks.workunit.client.1.smithi136.stdout:8/143: chown d3/de/f27 3215403 1 2024-04-06T03:40:28.756 INFO:tasks.workunit.client.1.smithi136.stdout:8/144: creat d3/de/f29 x:0 0 0 2024-04-06T03:40:28.756 INFO:tasks.workunit.client.0.smithi094.stdout:3/200: dread d9/d10/f22 [0,4194304] 0 2024-04-06T03:40:28.756 INFO:tasks.workunit.client.0.smithi094.stdout:3/201: write d9/d1d/f39 [251362,117717] 0 2024-04-06T03:40:28.757 INFO:tasks.workunit.client.1.smithi136.stdout:2/215: dwrite d3/d9/fe [0,4194304] 0 2024-04-06T03:40:28.757 INFO:tasks.workunit.client.1.smithi136.stdout:2/216: read - d3/d9/d10/d18/f23 zero size 2024-04-06T03:40:28.758 INFO:tasks.workunit.client.0.smithi094.stdout:6/148: rmdir dc/d11/d1b/d24 39 2024-04-06T03:40:28.761 INFO:tasks.workunit.client.0.smithi094.stdout:5/137: sync 2024-04-06T03:40:28.761 INFO:tasks.workunit.client.0.smithi094.stdout:5/138: readlink db/l20 0 2024-04-06T03:40:28.762 INFO:tasks.workunit.client.1.smithi136.stdout:8/145: dread f1 [0,4194304] 0 2024-04-06T03:40:28.764 INFO:tasks.workunit.client.0.smithi094.stdout:1/133: dwrite d0/d7/d1a/f1e [0,4194304] 0 2024-04-06T03:40:28.765 INFO:tasks.workunit.client.0.smithi094.stdout:1/134: chown d0/d1/cd 530267 1 2024-04-06T03:40:28.766 INFO:tasks.workunit.client.1.smithi136.stdout:9/110: getdents d17 0 2024-04-06T03:40:28.766 INFO:tasks.workunit.client.1.smithi136.stdout:9/111: write fb [728745,18741] 0 2024-04-06T03:40:28.766 INFO:tasks.workunit.client.1.smithi136.stdout:9/112: fsync f11 0 2024-04-06T03:40:28.767 INFO:tasks.workunit.client.1.smithi136.stdout:9/113: fsync f15 0 2024-04-06T03:40:28.767 INFO:tasks.workunit.client.0.smithi094.stdout:0/197: mkdir d1/d9/dd/d18/d1e/d22/d3e 0 2024-04-06T03:40:28.773 INFO:tasks.workunit.client.0.smithi094.stdout:3/202: unlink d9/d1e/d26/l35 0 2024-04-06T03:40:28.776 INFO:tasks.workunit.client.0.smithi094.stdout:6/149: rename dc/d11/c23 to dc/d11/d1b/d24/c3b 0 2024-04-06T03:40:28.777 INFO:tasks.workunit.client.0.smithi094.stdout:5/139: dwrite db/d19/d21/f26 [0,4194304] 0 2024-04-06T03:40:28.777 INFO:tasks.workunit.client.0.smithi094.stdout:5/140: fdatasync db/f2a 0 2024-04-06T03:40:28.778 INFO:tasks.workunit.client.0.smithi094.stdout:7/167: fsync d2/d6/dc/dd/f28 0 2024-04-06T03:40:28.778 INFO:tasks.workunit.client.0.smithi094.stdout:1/135: mkdir d0/d1/d16/d24/d30 0 2024-04-06T03:40:28.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/168: dwrite d2/fb [4194304,4194304] 0 2024-04-06T03:40:28.842 INFO:tasks.workunit.client.0.smithi094.stdout:3/203: creat d9/d10/d27/d3a/f40 x:0 0 0 2024-04-06T03:40:28.844 INFO:tasks.workunit.client.0.smithi094.stdout:6/150: mknod dc/d3a/c3c 0 2024-04-06T03:40:28.848 INFO:tasks.workunit.client.1.smithi136.stdout:0/82: symlink d1/d6/d13/d1a/l1b 0 2024-04-06T03:40:28.848 INFO:tasks.workunit.client.1.smithi136.stdout:0/83: readlink d1/d6/d13/d1a/l1b 0 2024-04-06T03:40:28.851 INFO:tasks.workunit.client.1.smithi136.stdout:6/89: rename d2/d7/d1e/l22 to d2/de/l24 0 2024-04-06T03:40:28.851 INFO:tasks.workunit.client.1.smithi136.stdout:6/90: fdatasync d2/f5 0 2024-04-06T03:40:28.851 INFO:tasks.workunit.client.1.smithi136.stdout:4/109: rmdir d1d 39 2024-04-06T03:40:28.851 INFO:tasks.workunit.client.1.smithi136.stdout:4/110: dread - d1d/f24 zero size 2024-04-06T03:40:28.854 INFO:tasks.workunit.client.0.smithi094.stdout:5/141: link db/f1d db/d19/d24/f37 0 2024-04-06T03:40:28.854 INFO:tasks.workunit.client.0.smithi094.stdout:5/142: chown db/d19/d21/f26 2955 1 2024-04-06T03:40:28.855 INFO:tasks.workunit.client.0.smithi094.stdout:3/204: dwrite d9/f3f [0,4194304] 0 2024-04-06T03:40:28.855 INFO:tasks.workunit.client.1.smithi136.stdout:5/146: getdents d1/d16 0 2024-04-06T03:40:28.856 INFO:tasks.workunit.client.0.smithi094.stdout:1/136: symlink d0/d1/d16/l31 0 2024-04-06T03:40:28.857 INFO:tasks.workunit.client.0.smithi094.stdout:1/137: dread - d0/d1/d27/f2e zero size 2024-04-06T03:40:28.857 INFO:tasks.workunit.client.0.smithi094.stdout:1/138: dread - d0/d1/d27/f2e zero size 2024-04-06T03:40:28.860 INFO:tasks.workunit.client.0.smithi094.stdout:3/205: write d9/d1d/f3c [2278594,64524] 0 2024-04-06T03:40:28.864 INFO:tasks.workunit.client.0.smithi094.stdout:6/151: symlink dc/l3d 0 2024-04-06T03:40:28.866 INFO:tasks.workunit.client.0.smithi094.stdout:5/143: creat db/d19/d24/f38 x:0 0 0 2024-04-06T03:40:28.866 INFO:tasks.workunit.client.1.smithi136.stdout:9/114: truncate fa 267900 0 2024-04-06T03:40:28.874 INFO:tasks.workunit.client.0.smithi094.stdout:3/206: symlink d9/d1e/d26/l41 0 2024-04-06T03:40:28.876 INFO:tasks.workunit.client.0.smithi094.stdout:3/207: write d9/d10/f2b [258992,107317] 0 2024-04-06T03:40:28.876 INFO:tasks.workunit.client.0.smithi094.stdout:3/208: fsync d9/d20/f34 0 2024-04-06T03:40:28.876 INFO:tasks.workunit.client.0.smithi094.stdout:3/209: creat d9/d1d/f42 x:0 0 0 2024-04-06T03:40:28.876 INFO:tasks.workunit.client.0.smithi094.stdout:7/169: rmdir d2/d6/dc/d12 39 2024-04-06T03:40:28.876 INFO:tasks.workunit.client.0.smithi094.stdout:7/170: fsync d2/f1f 0 2024-04-06T03:40:28.879 INFO:tasks.workunit.client.1.smithi136.stdout:7/80: getdents d5/df/d11 0 2024-04-06T03:40:28.879 INFO:tasks.workunit.client.1.smithi136.stdout:7/81: write d5/df/f10 [266601,82360] 0 2024-04-06T03:40:28.880 INFO:tasks.workunit.client.1.smithi136.stdout:0/84: mkdir d1/d6/d13/d1a/d1c 0 2024-04-06T03:40:28.882 INFO:tasks.workunit.client.0.smithi094.stdout:6/152: symlink dc/l3e 0 2024-04-06T03:40:28.884 INFO:tasks.workunit.client.1.smithi136.stdout:6/91: rename d2/d7/d13/d18 to d2/d25 0 2024-04-06T03:40:28.890 INFO:tasks.workunit.client.0.smithi094.stdout:3/210: mknod d9/d10/d27/c43 0 2024-04-06T03:40:28.890 INFO:tasks.workunit.client.0.smithi094.stdout:3/211: truncate d9/d10/f3b 1126685 0 2024-04-06T03:40:28.891 INFO:tasks.workunit.client.0.smithi094.stdout:7/171: write d2/f1f [4668986,43139] 0 2024-04-06T03:40:28.895 INFO:tasks.workunit.client.0.smithi094.stdout:3/212: mknod d9/d1e/c44 0 2024-04-06T03:40:28.895 INFO:tasks.workunit.client.0.smithi094.stdout:3/213: read d9/d10/f2b [163507,93877] 0 2024-04-06T03:40:28.896 INFO:tasks.workunit.client.0.smithi094.stdout:3/214: write d9/d10/f3b [1800303,123861] 0 2024-04-06T03:40:28.896 INFO:tasks.workunit.client.0.smithi094.stdout:3/215: creat d9/d1d/f45 x:0 0 0 2024-04-06T03:40:28.897 INFO:tasks.workunit.client.1.smithi136.stdout:9/115: mkdir d17/d20/d25 0 2024-04-06T03:40:28.897 INFO:tasks.workunit.client.1.smithi136.stdout:9/116: stat cf 0 2024-04-06T03:40:28.897 INFO:tasks.workunit.client.1.smithi136.stdout:5/147: fsync d1/dc/d10/f28 0 2024-04-06T03:40:28.898 INFO:tasks.workunit.client.0.smithi094.stdout:7/172: mknod d2/d6/dc/dd/c2e 0 2024-04-06T03:40:28.901 INFO:tasks.workunit.client.1.smithi136.stdout:7/82: unlink d5/df/d12/f13 0 2024-04-06T03:40:28.901 INFO:tasks.workunit.client.1.smithi136.stdout:7/83: getdents d5/df/d12/d1b 0 2024-04-06T03:40:28.903 INFO:tasks.workunit.client.1.smithi136.stdout:6/92: dwrite d2/fa [0,4194304] 0 2024-04-06T03:40:28.905 INFO:tasks.workunit.client.1.smithi136.stdout:0/85: creat d1/d6/d13/d1a/f1d x:0 0 0 2024-04-06T03:40:28.906 INFO:tasks.workunit.client.0.smithi094.stdout:6/153: dwrite dc/f29 [0,4194304] 0 2024-04-06T03:40:28.906 INFO:tasks.workunit.client.0.smithi094.stdout:6/154: truncate dc/d11/d22/f34 381267 0 2024-04-06T03:40:28.907 INFO:tasks.workunit.client.1.smithi136.stdout:4/111: mknod d1d/c28 0 2024-04-06T03:40:28.910 INFO:tasks.workunit.client.1.smithi136.stdout:5/148: mknod d1/dc/d15/c2a 0 2024-04-06T03:40:28.916 INFO:tasks.workunit.client.0.smithi094.stdout:4/221: rmdir d0/df/d13 39 2024-04-06T03:40:28.916 INFO:tasks.workunit.client.0.smithi094.stdout:4/222: write d0/df/d13/f37 [360261,31256] 0 2024-04-06T03:40:28.918 INFO:tasks.workunit.client.0.smithi094.stdout:3/216: mknod d9/d20/d30/c46 0 2024-04-06T03:40:28.918 INFO:tasks.workunit.client.0.smithi094.stdout:3/217: creat d9/d10/f47 x:0 0 0 2024-04-06T03:40:28.922 INFO:tasks.workunit.client.1.smithi136.stdout:0/86: write d1/d6/f12 [972688,94343] 0 2024-04-06T03:40:28.924 INFO:tasks.workunit.client.0.smithi094.stdout:6/155: rename dc/d11/d1b/d24/c2b to dc/d11/c3f 0 2024-04-06T03:40:28.931 INFO:tasks.workunit.client.1.smithi136.stdout:5/149: write d1/d16/f1b [3687667,28153] 0 2024-04-06T03:40:28.931 INFO:tasks.workunit.client.1.smithi136.stdout:5/150: rename d1 to d1/dc/d10/d1e/d2b 22 2024-04-06T03:40:28.941 INFO:tasks.workunit.client.1.smithi136.stdout:5/151: read d1/d16/d17/f1a [2575509,42610] 0 2024-04-06T03:40:28.943 INFO:tasks.workunit.client.0.smithi094.stdout:4/223: dwrite d0/df/d13/f37 [0,4194304] 0 2024-04-06T03:40:28.943 INFO:tasks.workunit.client.0.smithi094.stdout:4/224: fdatasync d0/d15/f1a 0 2024-04-06T03:40:28.945 INFO:tasks.workunit.client.0.smithi094.stdout:4/225: symlink d0/df/d13/d4a/l4c 0 2024-04-06T03:40:28.945 INFO:tasks.workunit.client.0.smithi094.stdout:4/226: fsync d0/f22 0 2024-04-06T03:40:28.945 INFO:tasks.workunit.client.0.smithi094.stdout:4/227: truncate d0/df/d1e/d34/f39 335310 0 2024-04-06T03:40:28.946 INFO:tasks.workunit.client.0.smithi094.stdout:4/228: creat d0/d15/d3a/d3e/d49/f4d x:0 0 0 2024-04-06T03:40:28.946 INFO:tasks.workunit.client.0.smithi094.stdout:4/229: stat d0/d4/f21 0 2024-04-06T03:40:28.946 INFO:tasks.workunit.client.0.smithi094.stdout:4/230: truncate d0/d4/fb 997351 0 2024-04-06T03:40:28.947 INFO:tasks.workunit.client.0.smithi094.stdout:4/231: truncate d0/df/f19 749351 0 2024-04-06T03:40:28.947 INFO:tasks.workunit.client.0.smithi094.stdout:4/232: stat d0/d15/f1a 0 2024-04-06T03:40:28.947 INFO:tasks.workunit.client.0.smithi094.stdout:4/233: creat d0/f4e x:0 0 0 2024-04-06T03:40:28.961 INFO:tasks.workunit.client.1.smithi136.stdout:2/217: truncate d3/d9/d10/f17 782044 0 2024-04-06T03:40:28.963 INFO:tasks.workunit.client.1.smithi136.stdout:2/218: rename l1 to d3/d15/l4c 0 2024-04-06T03:40:28.994 INFO:tasks.workunit.client.0.smithi094.stdout:7/173: dwrite d2/d6/dc/dd/f28 [0,4194304] 0 2024-04-06T03:40:28.994 INFO:tasks.workunit.client.0.smithi094.stdout:7/174: read - d2/d6/dc/f2b zero size 2024-04-06T03:40:28.994 INFO:tasks.workunit.client.0.smithi094.stdout:7/175: creat d2/d6/dc/dd/f2f x:0 0 0 2024-04-06T03:40:28.994 INFO:tasks.workunit.client.0.smithi094.stdout:7/176: chown d2/d6/dc/dd/df/f1a 763 1 2024-04-06T03:40:28.995 INFO:tasks.workunit.client.1.smithi136.stdout:5/152: dwrite d1/dc/d10/f11 [0,4194304] 0 2024-04-06T03:40:28.996 INFO:tasks.workunit.client.0.smithi094.stdout:7/177: link d2/d6/dc/dd/df/f1a d2/d6/f30 0 2024-04-06T03:40:28.996 INFO:tasks.workunit.client.0.smithi094.stdout:7/178: truncate d2/f23 689811 0 2024-04-06T03:40:28.996 INFO:tasks.workunit.client.0.smithi094.stdout:7/179: creat d2/d6/dc/dd/f31 x:0 0 0 2024-04-06T03:40:28.998 INFO:tasks.workunit.client.1.smithi136.stdout:7/84: dwrite d5/fd [8388608,4194304] 0 2024-04-06T03:40:28.998 INFO:tasks.workunit.client.1.smithi136.stdout:7/85: truncate d5/fd 12681581 0 2024-04-06T03:40:28.999 INFO:tasks.workunit.client.1.smithi136.stdout:4/112: dwrite f14 [0,4194304] 0 2024-04-06T03:40:28.999 INFO:tasks.workunit.client.1.smithi136.stdout:6/93: dwrite d2/d7/f20 [0,4194304] 0 2024-04-06T03:40:29.000 INFO:tasks.workunit.client.1.smithi136.stdout:0/87: dread d1/d6/fb [0,4194304] 0 2024-04-06T03:40:29.000 INFO:tasks.workunit.client.1.smithi136.stdout:0/88: stat d1/d6/f19 0 2024-04-06T03:40:29.000 INFO:tasks.workunit.client.1.smithi136.stdout:7/86: mkdir d5/df/d1d 0 2024-04-06T03:40:29.000 INFO:tasks.workunit.client.1.smithi136.stdout:7/87: chown d5/df/d11/f14 0 1 2024-04-06T03:40:29.000 INFO:tasks.workunit.client.1.smithi136.stdout:7/88: chown d5/df/d1d 200322 1 2024-04-06T03:40:29.005 INFO:tasks.workunit.client.1.smithi136.stdout:5/153: dread d1/f18 [4194304,4194304] 0 2024-04-06T03:40:29.005 INFO:tasks.workunit.client.1.smithi136.stdout:4/113: creat d1d/f29 x:0 0 0 2024-04-06T03:40:29.005 INFO:tasks.workunit.client.1.smithi136.stdout:4/114: stat d1d/f26 0 2024-04-06T03:40:29.006 INFO:tasks.workunit.client.0.smithi094.stdout:7/180: read d2/d6/dc/dd/f28 [3410301,117747] 0 2024-04-06T03:40:29.006 INFO:tasks.workunit.client.0.smithi094.stdout:7/181: dread - d2/d6/f24 zero size 2024-04-06T03:40:29.006 INFO:tasks.workunit.client.1.smithi136.stdout:1/127: read f11 [962658,93743] 0 2024-04-06T03:40:29.006 INFO:tasks.workunit.client.1.smithi136.stdout:1/128: write d12/f16 [1359980,72847] 0 2024-04-06T03:40:29.006 INFO:tasks.workunit.client.1.smithi136.stdout:1/129: write fb [1847022,13525] 0 2024-04-06T03:40:29.007 INFO:tasks.workunit.client.1.smithi136.stdout:8/146: dread d3/da/fd [0,4194304] 0 2024-04-06T03:40:29.007 INFO:tasks.workunit.client.1.smithi136.stdout:1/130: write fb [4336523,75731] 0 2024-04-06T03:40:29.009 INFO:tasks.workunit.client.1.smithi136.stdout:6/94: write d2/d7/f20 [2234880,3416] 0 2024-04-06T03:40:29.010 INFO:tasks.workunit.client.0.smithi094.stdout:7/182: getdents d2/d6/dc/dd/df 0 2024-04-06T03:40:29.013 INFO:tasks.workunit.client.1.smithi136.stdout:7/89: unlink d5/df/d12/f16 0 2024-04-06T03:40:29.015 INFO:tasks.workunit.client.1.smithi136.stdout:5/154: truncate d1/d16/d17/f1a 933696 0 2024-04-06T03:40:29.018 INFO:tasks.workunit.client.1.smithi136.stdout:8/147: link d3/f21 d3/f2a 0 2024-04-06T03:40:29.019 INFO:tasks.workunit.client.1.smithi136.stdout:1/131: rmdir d12/d1b 39 2024-04-06T03:40:29.024 INFO:tasks.workunit.client.1.smithi136.stdout:5/155: creat d1/dc/d1c/f2c x:0 0 0 2024-04-06T03:40:29.024 INFO:tasks.workunit.client.1.smithi136.stdout:5/156: write d1/dc/d1c/f2c [812301,46520] 0 2024-04-06T03:40:29.025 INFO:tasks.workunit.client.1.smithi136.stdout:4/115: mknod d1d/c2a 0 2024-04-06T03:40:29.027 INFO:tasks.workunit.client.1.smithi136.stdout:8/148: symlink d3/de/d1b/l2b 0 2024-04-06T03:40:29.027 INFO:tasks.workunit.client.1.smithi136.stdout:8/149: read d3/da/f13 [250122,34440] 0 2024-04-06T03:40:29.027 INFO:tasks.workunit.client.1.smithi136.stdout:8/150: dread - d3/de/f27 zero size 2024-04-06T03:40:29.029 INFO:tasks.workunit.client.1.smithi136.stdout:0/89: rename d1/d6/d13 to d1/d1e 0 2024-04-06T03:40:29.029 INFO:tasks.workunit.client.1.smithi136.stdout:0/90: write d1/d6/f12 [4098477,37357] 0 2024-04-06T03:40:29.029 INFO:tasks.workunit.client.1.smithi136.stdout:0/91: creat d1/d6/f1f x:0 0 0 2024-04-06T03:40:29.029 INFO:tasks.workunit.client.1.smithi136.stdout:6/95: dwrite d2/f21 [0,4194304] 0 2024-04-06T03:40:29.030 INFO:tasks.workunit.client.1.smithi136.stdout:5/157: symlink d1/dc/d15/l2d 0 2024-04-06T03:40:29.030 INFO:tasks.workunit.client.1.smithi136.stdout:4/116: creat d1d/f2b x:0 0 0 2024-04-06T03:40:29.032 INFO:tasks.workunit.client.1.smithi136.stdout:8/151: link d3/de/d1b/c20 d3/de/c2c 0 2024-04-06T03:40:29.032 INFO:tasks.workunit.client.1.smithi136.stdout:8/152: chown d3/da 5 1 2024-04-06T03:40:29.032 INFO:tasks.workunit.client.1.smithi136.stdout:1/132: dread d12/f16 [0,4194304] 0 2024-04-06T03:40:29.033 INFO:tasks.workunit.client.0.smithi094.stdout:7/183: dwrite d2/d6/dc/dd/df/f2d [0,4194304] 0 2024-04-06T03:40:29.033 INFO:tasks.workunit.client.0.smithi094.stdout:7/184: dread - d2/d6/dc/dd/f2f zero size 2024-04-06T03:40:29.033 INFO:tasks.workunit.client.0.smithi094.stdout:7/185: creat d2/d6/f32 x:0 0 0 2024-04-06T03:40:29.033 INFO:tasks.workunit.client.1.smithi136.stdout:7/90: dwrite d5/df/d11/f1c [0,4194304] 0 2024-04-06T03:40:29.034 INFO:tasks.workunit.client.1.smithi136.stdout:7/91: readlink d5/df/d11/l15 0 2024-04-06T03:40:29.034 INFO:tasks.workunit.client.1.smithi136.stdout:7/92: dread - d5/f7 zero size 2024-04-06T03:40:29.034 INFO:tasks.workunit.client.1.smithi136.stdout:7/93: write d5/f7 [570817,130897] 0 2024-04-06T03:40:29.034 INFO:tasks.workunit.client.1.smithi136.stdout:7/94: truncate d5/f7 1386087 0 2024-04-06T03:40:29.037 INFO:tasks.workunit.client.1.smithi136.stdout:0/92: creat d1/d1e/f20 x:0 0 0 2024-04-06T03:40:29.041 INFO:tasks.workunit.client.1.smithi136.stdout:1/133: dread fb [0,4194304] 0 2024-04-06T03:40:29.042 INFO:tasks.workunit.client.1.smithi136.stdout:1/134: write f11 [542096,44944] 0 2024-04-06T03:40:29.042 INFO:tasks.workunit.client.1.smithi136.stdout:1/135: chown d12/d1b/d1e/d27 531670322 1 2024-04-06T03:40:29.042 INFO:tasks.workunit.client.1.smithi136.stdout:1/136: write f10 [1892116,82621] 0 2024-04-06T03:40:29.042 INFO:tasks.workunit.client.0.smithi094.stdout:7/186: mknod d2/d6/c33 0 2024-04-06T03:40:29.046 INFO:tasks.workunit.client.1.smithi136.stdout:6/96: mkdir d2/d7/d26 0 2024-04-06T03:40:29.046 INFO:tasks.workunit.client.1.smithi136.stdout:6/97: creat d2/de/f27 x:0 0 0 2024-04-06T03:40:29.046 INFO:tasks.workunit.client.1.smithi136.stdout:5/158: truncate d1/d1f/f21 364505 0 2024-04-06T03:40:29.046 INFO:tasks.workunit.client.1.smithi136.stdout:5/159: write d1/dc/d1c/f2c [1086319,106341] 0 2024-04-06T03:40:29.047 INFO:tasks.workunit.client.0.smithi094.stdout:7/187: link d2/f1f d2/d6/dc/d12/d29/f34 0 2024-04-06T03:40:29.048 INFO:tasks.workunit.client.1.smithi136.stdout:7/95: dwrite d5/df/f10 [0,4194304] 0 2024-04-06T03:40:29.048 INFO:tasks.workunit.client.1.smithi136.stdout:3/71: sync 2024-04-06T03:40:29.048 INFO:tasks.workunit.client.1.smithi136.stdout:3/72: fdatasync d0/d4/fd 0 2024-04-06T03:40:29.048 INFO:tasks.workunit.client.1.smithi136.stdout:3/73: stat d0/fb 0 2024-04-06T03:40:29.052 INFO:tasks.workunit.client.1.smithi136.stdout:8/153: mkdir d3/d2d 0 2024-04-06T03:40:29.053 INFO:tasks.workunit.client.1.smithi136.stdout:0/93: creat d1/d1e/d1a/d1c/f21 x:0 0 0 2024-04-06T03:40:29.053 INFO:tasks.workunit.client.1.smithi136.stdout:1/137: dread f10 [4194304,4194304] 0 2024-04-06T03:40:29.056 INFO:tasks.workunit.client.1.smithi136.stdout:6/98: rename d2/d7/d1e to d2/d7/d28 0 2024-04-06T03:40:29.057 INFO:tasks.workunit.client.1.smithi136.stdout:6/99: write d2/d7/f20 [786444,16300] 0 2024-04-06T03:40:29.057 INFO:tasks.workunit.client.0.smithi094.stdout:8/163: sync 2024-04-06T03:40:29.057 INFO:tasks.workunit.client.0.smithi094.stdout:2/195: sync 2024-04-06T03:40:29.057 INFO:tasks.workunit.client.0.smithi094.stdout:8/164: readlink l10 0 2024-04-06T03:40:29.057 INFO:tasks.workunit.client.0.smithi094.stdout:8/165: write f3 [444624,72767] 0 2024-04-06T03:40:29.060 INFO:tasks.workunit.client.1.smithi136.stdout:3/74: rename d0/f6 to d0/d4/d12/f15 0 2024-04-06T03:40:29.061 INFO:tasks.workunit.client.1.smithi136.stdout:3/75: chown d0/d4 11356 1 2024-04-06T03:40:29.061 INFO:tasks.workunit.client.1.smithi136.stdout:3/76: write d0/d4/fd [5137102,84705] 0 2024-04-06T03:40:29.063 INFO:tasks.workunit.client.0.smithi094.stdout:7/188: dread d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:29.063 INFO:tasks.workunit.client.0.smithi094.stdout:2/196: symlink d0/d8/d27/l41 0 2024-04-06T03:40:29.063 INFO:tasks.workunit.client.0.smithi094.stdout:7/189: fsync d2/f11 0 2024-04-06T03:40:29.065 INFO:tasks.workunit.client.1.smithi136.stdout:4/117: symlink d1d/l2c 0 2024-04-06T03:40:29.069 INFO:tasks.workunit.client.0.smithi094.stdout:7/190: mkdir d2/d6/dc/d12/d29/d35 0 2024-04-06T03:40:29.069 INFO:tasks.workunit.client.0.smithi094.stdout:7/191: write d2/d6/dc/dd/f31 [1032547,56839] 0 2024-04-06T03:40:29.070 INFO:tasks.workunit.client.0.smithi094.stdout:8/166: dread fd [0,4194304] 0 2024-04-06T03:40:29.070 INFO:tasks.workunit.client.1.smithi136.stdout:8/154: mknod d3/de/d18/c2e 0 2024-04-06T03:40:29.075 INFO:tasks.workunit.client.0.smithi094.stdout:7/192: mkdir d2/d6/dc/dd/df/d36 0 2024-04-06T03:40:29.076 INFO:tasks.workunit.client.1.smithi136.stdout:3/77: symlink d0/d4/l16 0 2024-04-06T03:40:29.076 INFO:tasks.workunit.client.1.smithi136.stdout:3/78: chown d0/l9 925 1 2024-04-06T03:40:29.077 INFO:tasks.workunit.client.1.smithi136.stdout:4/118: creat d1d/f2d x:0 0 0 2024-04-06T03:40:29.079 INFO:tasks.workunit.client.0.smithi094.stdout:2/197: dread d0/d8/fe [0,4194304] 0 2024-04-06T03:40:29.079 INFO:tasks.workunit.client.0.smithi094.stdout:2/198: write d0/dd/d19/d31/f3a [827725,21342] 0 2024-04-06T03:40:29.079 INFO:tasks.workunit.client.0.smithi094.stdout:2/199: readlink d0/d8/d27/l41 0 2024-04-06T03:40:29.079 INFO:tasks.workunit.client.0.smithi094.stdout:2/200: creat d0/dd/d19/d31/f42 x:0 0 0 2024-04-06T03:40:29.082 INFO:tasks.workunit.client.0.smithi094.stdout:7/193: mknod d2/d6/c37 0 2024-04-06T03:40:29.083 INFO:tasks.workunit.client.1.smithi136.stdout:7/96: dwrite d5/df/d12/f17 [0,4194304] 0 2024-04-06T03:40:29.084 INFO:tasks.workunit.client.1.smithi136.stdout:2/219: dwrite d3/d9/d10/f17 [0,4194304] 0 2024-04-06T03:40:29.085 INFO:tasks.workunit.client.0.smithi094.stdout:8/167: read f11 [8022345,127702] 0 2024-04-06T03:40:29.085 INFO:tasks.workunit.client.0.smithi094.stdout:7/194: dread d2/f23 [0,4194304] 0 2024-04-06T03:40:29.085 INFO:tasks.workunit.client.1.smithi136.stdout:6/100: dwrite d2/f9 [0,4194304] 0 2024-04-06T03:40:29.085 INFO:tasks.workunit.client.1.smithi136.stdout:0/94: dwrite d1/d1e/f20 [0,4194304] 0 2024-04-06T03:40:29.086 INFO:tasks.workunit.client.1.smithi136.stdout:3/79: symlink d0/d4/d12/l17 0 2024-04-06T03:40:29.087 INFO:tasks.workunit.client.0.smithi094.stdout:2/201: stat d0/d18/c23 0 2024-04-06T03:40:29.096 INFO:tasks.workunit.client.1.smithi136.stdout:7/97: symlink d5/l1e 0 2024-04-06T03:40:29.178 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:29.178 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:29.178 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:29.178 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:29.178 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:29.179 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: Adjusting osd_memory_target on smithi094 to 1557M 2024-04-06T03:40:29.179 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:29.179 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:40:29.179 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:40:29.179 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-06T03:40:29.179 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-06T03:40:29.179 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:28 smithi136 ceph-mon[36436]: pgmap v14: 65 pgs: 65 active+clean; 514 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 30 MiB/s rd, 50 MiB/s wr, 710 op/s 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:8/168: rmdir d18/d2c/d35 0 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:7/195: symlink d2/d6/dc/dd/df/l38 0 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:7/196: chown d2/d6/dc/d12/d29/f34 106776855 1 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:2/202: truncate d0/f5 1765976 0 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:2/203: dread - d0/d8/d1d/f38 zero size 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:2/204: write d0/d8/f33 [4510535,15546] 0 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:2/205: mknod d0/d8/d27/d2b/c43 0 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:2/206: fsync d0/d18/f22 0 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:2/207: fdatasync d0/f5 0 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:2/208: rename d0/d8/d27/l41 to d0/d18/d3f/l44 0 2024-04-06T03:40:29.181 INFO:tasks.workunit.client.0.smithi094.stdout:8/169: dwrite d18/d34/f38 [0,4194304] 0 2024-04-06T03:40:29.182 INFO:tasks.workunit.client.0.smithi094.stdout:8/170: dread f3 [0,4194304] 0 2024-04-06T03:40:29.182 INFO:tasks.workunit.client.0.smithi094.stdout:2/209: dwrite d0/dd/f40 [0,4194304] 0 2024-04-06T03:40:29.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:29.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:29.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:29.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:29.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-06T03:40:29.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: Adjusting osd_memory_target on smithi094 to 1557M 2024-04-06T03:40:29.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:29.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:40:29.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:40:29.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-06T03:40:29.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-06T03:40:29.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:28 smithi094 ceph-mon[29713]: pgmap v14: 65 pgs: 65 active+clean; 514 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 30 MiB/s rd, 50 MiB/s wr, 710 op/s 2024-04-06T03:40:29.185 INFO:tasks.workunit.client.1.smithi136.stdout:2/220: rename d3/d15/d19/l44 to d3/l4d 0 2024-04-06T03:40:29.185 INFO:tasks.workunit.client.1.smithi136.stdout:8/155: rmdir d3/da 39 2024-04-06T03:40:29.185 INFO:tasks.workunit.client.1.smithi136.stdout:0/95: dwrite d1/d6/f19 [0,4194304] 0 2024-04-06T03:40:29.185 INFO:tasks.workunit.client.1.smithi136.stdout:0/96: chown d1/d6/lf 0 1 2024-04-06T03:40:29.185 INFO:tasks.workunit.client.1.smithi136.stdout:1/138: dwrite f3 [0,4194304] 0 2024-04-06T03:40:29.185 INFO:tasks.workunit.client.1.smithi136.stdout:1/139: truncate d12/f16 2147267 0 2024-04-06T03:40:29.185 INFO:tasks.workunit.client.1.smithi136.stdout:6/101: mkdir d2/de/d29 0 2024-04-06T03:40:29.185 INFO:tasks.workunit.client.1.smithi136.stdout:3/80: unlink d0/d4/l16 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:2/221: dwrite d3/d9/fe [0,4194304] 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:2/222: write d3/d9/d10/d18/d1d/d3c/f40 [159594,53052] 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:2/223: truncate d3/d15/d19/f1b 1131841 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:5/160: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:4/119: dwrite fc [0,4194304] 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:2/224: read d3/d9/fe [1497262,48751] 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:8/156: dwrite d3/da/f19 [0,4194304] 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:8/157: creat d3/f2f x:0 0 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:0/97: dwrite d1/d1e/f20 [0,4194304] 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:0/98: readlink d1/d1e/d1a/l1b 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:0/99: fsync d1/d1e/d1a/f1d 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:0/100: fsync d1/d6/f11 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:2/225: dwrite d3/d15/f36 [0,4194304] 0 2024-04-06T03:40:29.186 INFO:tasks.workunit.client.1.smithi136.stdout:2/226: chown d3/d15/f30 7978707 1 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:2/227: stat d3/d9/d10/d18/d1d 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:7/98: creat d5/df/d1d/f1f x:0 0 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:7/99: write d5/df/f10 [563924,98532] 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:7/100: write d5/df/d11/f14 [763593,67696] 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:6/102: rmdir d2/de/d29 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:3/81: link d0/fb d0/d4/f18 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:3/82: truncate d0/f2 208634 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:3/83: write d0/f5 [876545,19713] 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:3/84: dread - d0/f11 zero size 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:5/161: mknod d1/dc/d15/c2e 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:0/101: getdents d1/d1e/d1a/d1c 0 2024-04-06T03:40:29.187 INFO:tasks.workunit.client.1.smithi136.stdout:0/102: write d1/d6/f11 [384488,94450] 0 2024-04-06T03:40:29.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/102: sync 2024-04-06T03:40:29.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/103: readlink d2/dc/d10/l15 0 2024-04-06T03:40:29.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/104: stat d2/dc 0 2024-04-06T03:40:29.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/105: dread d2/f4 [4194304,4194304] 0 2024-04-06T03:40:29.190 INFO:tasks.workunit.client.1.smithi136.stdout:2/228: mkdir d3/d15/d19/d4e 0 2024-04-06T03:40:29.191 INFO:tasks.workunit.client.0.smithi094.stdout:9/106: dread - d2/f1d zero size 2024-04-06T03:40:29.191 INFO:tasks.workunit.client.1.smithi136.stdout:5/162: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:29.193 INFO:tasks.workunit.client.0.smithi094.stdout:9/107: symlink d2/dc/d21/l22 0 2024-04-06T03:40:29.194 INFO:tasks.workunit.client.0.smithi094.stdout:9/108: write d2/f1d [459894,25321] 0 2024-04-06T03:40:29.195 INFO:tasks.workunit.client.1.smithi136.stdout:6/103: symlink d2/d7/l2a 0 2024-04-06T03:40:29.197 INFO:tasks.workunit.client.0.smithi094.stdout:9/109: mkdir d2/dc/d21/d23 0 2024-04-06T03:40:29.199 INFO:tasks.workunit.client.1.smithi136.stdout:3/85: mkdir d0/d4/d12/d19 0 2024-04-06T03:40:29.199 INFO:tasks.workunit.client.1.smithi136.stdout:3/86: write d0/f1 [3894890,11098] 0 2024-04-06T03:40:29.200 INFO:tasks.workunit.client.1.smithi136.stdout:4/120: symlink d1d/l2e 0 2024-04-06T03:40:29.200 INFO:tasks.workunit.client.1.smithi136.stdout:4/121: dread - d1d/f29 zero size 2024-04-06T03:40:29.205 INFO:tasks.workunit.client.1.smithi136.stdout:0/103: unlink d1/d6/c10 0 2024-04-06T03:40:29.205 INFO:tasks.workunit.client.1.smithi136.stdout:0/104: write d1/d1e/d1a/d1c/f21 [78568,53146] 0 2024-04-06T03:40:29.206 INFO:tasks.workunit.client.1.smithi136.stdout:2/229: unlink d3/d15/d19/f3d 0 2024-04-06T03:40:29.206 INFO:tasks.workunit.client.1.smithi136.stdout:2/230: readlink d3/d9/d10/l3f 0 2024-04-06T03:40:29.210 INFO:tasks.workunit.client.1.smithi136.stdout:7/101: rmdir d5/dc 0 2024-04-06T03:40:29.210 INFO:tasks.workunit.client.1.smithi136.stdout:7/102: truncate d5/df/d12/f19 1434183 0 2024-04-06T03:40:29.213 INFO:tasks.workunit.client.1.smithi136.stdout:4/122: mkdir d1d/d2f 0 2024-04-06T03:40:29.214 INFO:tasks.workunit.client.1.smithi136.stdout:3/87: write d0/d4/fd [1471359,90468] 0 2024-04-06T03:40:29.214 INFO:tasks.workunit.client.1.smithi136.stdout:3/88: write d0/f2 [278740,67716] 0 2024-04-06T03:40:29.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/210: dwrite d0/f5 [0,4194304] 0 2024-04-06T03:40:29.215 INFO:tasks.workunit.client.1.smithi136.stdout:5/163: dwrite d1/d16/f1b [0,4194304] 0 2024-04-06T03:40:29.215 INFO:tasks.workunit.client.1.smithi136.stdout:5/164: chown d1/dc/d1c 402436305 1 2024-04-06T03:40:29.215 INFO:tasks.workunit.client.0.smithi094.stdout:9/110: write d2/f7 [1908990,16448] 0 2024-04-06T03:40:29.216 INFO:tasks.workunit.client.1.smithi136.stdout:7/103: creat d5/df/d12/d1b/f20 x:0 0 0 2024-04-06T03:40:29.216 INFO:tasks.workunit.client.1.smithi136.stdout:4/123: creat d1d/d2f/f30 x:0 0 0 2024-04-06T03:40:29.221 INFO:tasks.workunit.client.0.smithi094.stdout:9/111: mknod d2/dc/c24 0 2024-04-06T03:40:29.223 INFO:tasks.workunit.client.1.smithi136.stdout:3/89: creat d0/d4/d12/f1a x:0 0 0 2024-04-06T03:40:29.223 INFO:tasks.workunit.client.1.smithi136.stdout:3/90: readlink d0/lf 0 2024-04-06T03:40:29.226 INFO:tasks.workunit.client.1.smithi136.stdout:6/104: dwrite d2/d7/f20 [0,4194304] 0 2024-04-06T03:40:29.226 INFO:tasks.workunit.client.1.smithi136.stdout:6/105: fsync d2/d25/f17 0 2024-04-06T03:40:29.231 INFO:tasks.workunit.client.0.smithi094.stdout:2/211: write d0/dd/f40 [3289864,127774] 0 2024-04-06T03:40:29.232 INFO:tasks.workunit.client.1.smithi136.stdout:7/104: mkdir d5/d21 0 2024-04-06T03:40:29.232 INFO:tasks.workunit.client.1.smithi136.stdout:2/231: dwrite d3/d9/f12 [0,4194304] 0 2024-04-06T03:40:29.235 INFO:tasks.workunit.client.1.smithi136.stdout:3/91: mkdir d0/d4/d12/d19/d1b 0 2024-04-06T03:40:29.235 INFO:tasks.workunit.client.1.smithi136.stdout:9/117: sync 2024-04-06T03:40:29.235 INFO:tasks.workunit.client.1.smithi136.stdout:9/118: truncate fe 4479574 0 2024-04-06T03:40:29.237 INFO:tasks.workunit.client.1.smithi136.stdout:9/119: write f11 [628512,7104] 0 2024-04-06T03:40:29.238 INFO:tasks.workunit.client.0.smithi094.stdout:9/112: dread d2/dc/f18 [0,4194304] 0 2024-04-06T03:40:29.245 INFO:tasks.workunit.client.1.smithi136.stdout:0/105: dwrite d1/d1e/d1a/d1c/f21 [0,4194304] 0 2024-04-06T03:40:29.247 INFO:tasks.workunit.client.1.smithi136.stdout:5/165: dwrite d1/f18 [4194304,4194304] 0 2024-04-06T03:40:29.247 INFO:tasks.workunit.client.0.smithi094.stdout:9/113: rename d2/ca to d2/c25 0 2024-04-06T03:40:29.249 INFO:tasks.workunit.client.1.smithi136.stdout:7/105: mkdir d5/df/d11/d22 0 2024-04-06T03:40:29.249 INFO:tasks.workunit.client.1.smithi136.stdout:7/106: write d5/f7 [1948890,56554] 0 2024-04-06T03:40:29.249 INFO:tasks.workunit.client.1.smithi136.stdout:7/107: chown d5/ca 107032 1 2024-04-06T03:40:29.249 INFO:tasks.workunit.client.1.smithi136.stdout:7/108: write d5/df/d12/d1b/f20 [388294,74123] 0 2024-04-06T03:40:29.251 INFO:tasks.workunit.client.0.smithi094.stdout:9/114: mkdir d2/dc/d21/d26 0 2024-04-06T03:40:29.251 INFO:tasks.workunit.client.0.smithi094.stdout:9/115: fdatasync d2/dc/d10/f12 0 2024-04-06T03:40:29.252 INFO:tasks.workunit.client.0.smithi094.stdout:2/212: dwrite d0/f1 [0,4194304] 0 2024-04-06T03:40:29.252 INFO:tasks.workunit.client.1.smithi136.stdout:9/120: creat d17/d20/d25/f26 x:0 0 0 2024-04-06T03:40:29.252 INFO:tasks.workunit.client.1.smithi136.stdout:9/121: truncate f13 1425560 0 2024-04-06T03:40:29.254 INFO:tasks.workunit.client.0.smithi094.stdout:9/116: mknod d2/dc/c27 0 2024-04-06T03:40:29.255 INFO:tasks.workunit.client.1.smithi136.stdout:0/106: symlink d1/d6/l22 0 2024-04-06T03:40:29.255 INFO:tasks.workunit.client.1.smithi136.stdout:0/107: truncate d1/d6/fd 1619840 0 2024-04-06T03:40:29.256 INFO:tasks.workunit.client.0.smithi094.stdout:2/213: creat d0/d8/d27/f45 x:0 0 0 2024-04-06T03:40:29.258 INFO:tasks.workunit.client.1.smithi136.stdout:5/166: rename d1/f18 to d1/dc/d10/d1e/f2f 0 2024-04-06T03:40:29.258 INFO:tasks.workunit.client.1.smithi136.stdout:5/167: fdatasync d1/dc/d1c/f2c 0 2024-04-06T03:40:29.258 INFO:tasks.workunit.client.1.smithi136.stdout:5/168: chown d1/dc/d10 3278384 1 2024-04-06T03:40:29.259 INFO:tasks.workunit.client.0.smithi094.stdout:9/117: symlink d2/dc/l28 0 2024-04-06T03:40:29.262 INFO:tasks.workunit.client.1.smithi136.stdout:7/109: symlink d5/d21/l23 0 2024-04-06T03:40:29.263 INFO:tasks.workunit.client.0.smithi094.stdout:9/118: fdatasync d2/f5 0 2024-04-06T03:40:29.266 INFO:tasks.workunit.client.1.smithi136.stdout:0/108: creat d1/d1e/d1a/d1c/f23 x:0 0 0 2024-04-06T03:40:29.268 INFO:tasks.workunit.client.1.smithi136.stdout:9/122: rename f15 to d17/d19/f27 0 2024-04-06T03:40:29.268 INFO:tasks.workunit.client.1.smithi136.stdout:9/123: write f11 [526373,96912] 0 2024-04-06T03:40:29.269 INFO:tasks.workunit.client.1.smithi136.stdout:5/169: symlink d1/dc/d10/l30 0 2024-04-06T03:40:29.271 INFO:tasks.workunit.client.1.smithi136.stdout:0/109: symlink d1/d1e/d1a/d1c/l24 0 2024-04-06T03:40:29.271 INFO:tasks.workunit.client.1.smithi136.stdout:0/110: write d1/d1e/d1a/d1c/f23 [417877,96550] 0 2024-04-06T03:40:29.272 INFO:tasks.workunit.client.0.smithi094.stdout:9/119: fdatasync d2/fe 0 2024-04-06T03:40:29.274 INFO:tasks.workunit.client.1.smithi136.stdout:9/124: mkdir d17/d20/d28 0 2024-04-06T03:40:29.275 INFO:tasks.workunit.client.1.smithi136.stdout:5/170: symlink d1/dc/d15/d20/l31 0 2024-04-06T03:40:29.275 INFO:tasks.workunit.client.1.smithi136.stdout:0/111: mknod d1/d1e/c25 0 2024-04-06T03:40:29.275 INFO:tasks.workunit.client.1.smithi136.stdout:0/112: chown d1/d1e/d1a/d1c 9 1 2024-04-06T03:40:29.276 INFO:tasks.workunit.client.1.smithi136.stdout:9/125: mkdir d17/d21/d29 0 2024-04-06T03:40:29.277 INFO:tasks.workunit.client.1.smithi136.stdout:0/113: mknod d1/d6/c26 0 2024-04-06T03:40:29.278 INFO:tasks.workunit.client.1.smithi136.stdout:0/114: creat d1/d1e/f27 x:0 0 0 2024-04-06T03:40:29.278 INFO:tasks.workunit.client.1.smithi136.stdout:0/115: chown d1/d1e/d1a/l1b 1380 1 2024-04-06T03:40:29.278 INFO:tasks.workunit.client.1.smithi136.stdout:0/116: truncate d1/f18 958589 0 2024-04-06T03:40:29.278 INFO:tasks.workunit.client.1.smithi136.stdout:0/117: write d1/d6/fb [4264581,41989] 0 2024-04-06T03:40:29.287 INFO:tasks.workunit.client.0.smithi094.stdout:5/144: sync 2024-04-06T03:40:29.287 INFO:tasks.workunit.client.0.smithi094.stdout:0/198: sync 2024-04-06T03:40:29.287 INFO:tasks.workunit.client.0.smithi094.stdout:1/139: sync 2024-04-06T03:40:29.290 INFO:tasks.workunit.client.0.smithi094.stdout:5/145: dread db/d19/d24/f34 [0,4194304] 0 2024-04-06T03:40:29.296 INFO:tasks.workunit.client.0.smithi094.stdout:1/140: creat d0/d1/d16/d24/d30/f32 x:0 0 0 2024-04-06T03:40:29.296 INFO:tasks.workunit.client.0.smithi094.stdout:1/141: creat d0/d1/d16/d24/d30/f33 x:0 0 0 2024-04-06T03:40:29.298 INFO:tasks.workunit.client.1.smithi136.stdout:6/106: dwrite d2/de/f10 [0,4194304] 0 2024-04-06T03:40:29.298 INFO:tasks.workunit.client.1.smithi136.stdout:6/107: truncate d2/d7/f12 221699 0 2024-04-06T03:40:29.298 INFO:tasks.workunit.client.1.smithi136.stdout:6/108: write d2/d7/f12 [286801,85912] 0 2024-04-06T03:40:29.299 INFO:tasks.workunit.client.0.smithi094.stdout:5/146: symlink db/d19/d30/l39 0 2024-04-06T03:40:29.300 INFO:tasks.workunit.client.0.smithi094.stdout:0/199: getdents d1/d9/dd/d18/d1e/d22 0 2024-04-06T03:40:29.300 INFO:tasks.workunit.client.1.smithi136.stdout:5/171: read d1/d16/d17/f1a [1371182,104016] 0 2024-04-06T03:40:29.301 INFO:tasks.workunit.client.0.smithi094.stdout:9/120: dwrite d2/dc/d10/f1b [0,4194304] 0 2024-04-06T03:40:29.302 INFO:tasks.workunit.client.0.smithi094.stdout:0/200: dread d1/d9/dd/d18/f31 [0,4194304] 0 2024-04-06T03:40:29.303 INFO:tasks.workunit.client.0.smithi094.stdout:1/142: symlink d0/d4/d1b/l34 0 2024-04-06T03:40:29.304 INFO:tasks.workunit.client.0.smithi094.stdout:9/121: fdatasync d2/f14 0 2024-04-06T03:40:29.304 INFO:tasks.workunit.client.0.smithi094.stdout:9/122: fsync d2/f5 0 2024-04-06T03:40:29.305 INFO:tasks.workunit.client.1.smithi136.stdout:0/118: read d1/fc [4187164,115117] 0 2024-04-06T03:40:29.305 INFO:tasks.workunit.client.1.smithi136.stdout:0/119: write d1/d6/f12 [3042108,5560] 0 2024-04-06T03:40:29.306 INFO:tasks.workunit.client.0.smithi094.stdout:0/201: symlink d1/d9/l3f 0 2024-04-06T03:40:29.308 INFO:tasks.workunit.client.1.smithi136.stdout:0/120: creat d1/d1e/d1a/f28 x:0 0 0 2024-04-06T03:40:29.310 INFO:tasks.workunit.client.1.smithi136.stdout:0/121: creat d1/f29 x:0 0 0 2024-04-06T03:40:29.314 INFO:tasks.workunit.client.0.smithi094.stdout:1/143: dread d0/d1/f10 [0,4194304] 0 2024-04-06T03:40:29.314 INFO:tasks.workunit.client.0.smithi094.stdout:1/144: chown d0/d7/c1f 505 1 2024-04-06T03:40:29.314 INFO:tasks.workunit.client.0.smithi094.stdout:1/145: write d0/d7/f26 [4054824,88564] 0 2024-04-06T03:40:29.314 INFO:tasks.workunit.client.0.smithi094.stdout:1/146: readlink d0/d1/l21 0 2024-04-06T03:40:29.314 INFO:tasks.workunit.client.0.smithi094.stdout:1/147: truncate d0/d1/d16/d24/d30/f33 961351 0 2024-04-06T03:40:29.314 INFO:tasks.workunit.client.0.smithi094.stdout:1/148: dread - d0/d1/d16/d24/d30/f32 zero size 2024-04-06T03:40:29.314 INFO:tasks.workunit.client.0.smithi094.stdout:1/149: write d0/f19 [4927587,65414] 0 2024-04-06T03:40:29.315 INFO:tasks.workunit.client.0.smithi094.stdout:1/150: stat d0/d4/d1b/d2f 0 2024-04-06T03:40:29.315 INFO:tasks.workunit.client.0.smithi094.stdout:1/151: mknod d0/d1/c35 0 2024-04-06T03:40:29.315 INFO:tasks.workunit.client.0.smithi094.stdout:1/152: creat d0/d7/f36 x:0 0 0 2024-04-06T03:40:29.316 INFO:tasks.workunit.client.1.smithi136.stdout:9/126: dwrite d17/d19/f1f [0,4194304] 0 2024-04-06T03:40:29.316 INFO:tasks.workunit.client.1.smithi136.stdout:5/172: dread d1/dc/d10/d1e/f2f [4194304,4194304] 0 2024-04-06T03:40:29.318 INFO:tasks.workunit.client.1.smithi136.stdout:5/173: mkdir d1/dc/d1c/d32 0 2024-04-06T03:40:29.318 INFO:tasks.workunit.client.1.smithi136.stdout:5/174: readlink d1/dc/d15/l24 0 2024-04-06T03:40:29.318 INFO:tasks.workunit.client.1.smithi136.stdout:5/175: chown d1/dc/d15/c2e 598 1 2024-04-06T03:40:29.319 INFO:tasks.workunit.client.1.smithi136.stdout:9/127: creat d17/d21/d29/f2a x:0 0 0 2024-04-06T03:40:29.319 INFO:tasks.workunit.client.1.smithi136.stdout:5/176: stat d1/l13 0 2024-04-06T03:40:29.320 INFO:tasks.workunit.client.1.smithi136.stdout:9/128: symlink d17/d20/d25/l2b 0 2024-04-06T03:40:29.321 INFO:tasks.workunit.client.1.smithi136.stdout:9/129: fsync f4 0 2024-04-06T03:40:29.321 INFO:tasks.workunit.client.1.smithi136.stdout:9/130: chown d17/d20/c24 97288261 1 2024-04-06T03:40:29.323 INFO:tasks.workunit.client.1.smithi136.stdout:5/177: link d1/ld d1/l33 0 2024-04-06T03:40:29.323 INFO:tasks.workunit.client.1.smithi136.stdout:5/178: creat d1/dc/d1c/f34 x:0 0 0 2024-04-06T03:40:29.324 INFO:tasks.workunit.client.1.smithi136.stdout:5/179: dread d1/d1f/f21 [0,4194304] 0 2024-04-06T03:40:29.324 INFO:tasks.workunit.client.1.smithi136.stdout:5/180: write d1/dc/d1c/f34 [567381,102404] 0 2024-04-06T03:40:29.394 INFO:tasks.workunit.client.1.smithi136.stdout:7/110: dwrite d5/df/d12/f19 [0,4194304] 0 2024-04-06T03:40:29.394 INFO:tasks.workunit.client.0.smithi094.stdout:0/202: dwrite d1/d9/dd/d18/d1e/d22/d35/f3c [0,4194304] 0 2024-04-06T03:40:29.397 INFO:tasks.workunit.client.0.smithi094.stdout:2/214: rmdir d0/d8 39 2024-04-06T03:40:29.398 INFO:tasks.workunit.client.1.smithi136.stdout:5/181: getdents d1/dc/d10/d1e 0 2024-04-06T03:40:29.400 INFO:tasks.workunit.client.0.smithi094.stdout:0/203: unlink d1/d9/c12 0 2024-04-06T03:40:29.401 INFO:tasks.workunit.client.0.smithi094.stdout:9/123: dwrite d2/fb [4194304,4194304] 0 2024-04-06T03:40:29.403 INFO:tasks.workunit.client.0.smithi094.stdout:2/215: symlink d0/d18/d3f/l46 0 2024-04-06T03:40:29.404 INFO:tasks.workunit.client.1.smithi136.stdout:2/232: truncate d3/d9/f12 3030193 0 2024-04-06T03:40:29.405 INFO:tasks.workunit.client.1.smithi136.stdout:6/109: dwrite d2/fd [0,4194304] 0 2024-04-06T03:40:29.405 INFO:tasks.workunit.client.0.smithi094.stdout:1/153: dwrite d0/d7/d1a/f1e [0,4194304] 0 2024-04-06T03:40:29.407 INFO:tasks.workunit.client.0.smithi094.stdout:5/147: dwrite db/d19/f1f [0,4194304] 0 2024-04-06T03:40:29.407 INFO:tasks.workunit.client.0.smithi094.stdout:0/204: creat d1/d9/dd/d18/d1d/f40 x:0 0 0 2024-04-06T03:40:29.407 INFO:tasks.workunit.client.0.smithi094.stdout:5/148: write f6 [110936,107895] 0 2024-04-06T03:40:29.407 INFO:tasks.workunit.client.0.smithi094.stdout:5/149: creat db/d19/d24/d2e/f3a x:0 0 0 2024-04-06T03:40:29.408 INFO:tasks.workunit.client.1.smithi136.stdout:9/131: dwrite f13 [0,4194304] 0 2024-04-06T03:40:29.408 INFO:tasks.workunit.client.1.smithi136.stdout:9/132: stat d17/l1c 0 2024-04-06T03:40:29.410 INFO:tasks.workunit.client.1.smithi136.stdout:7/111: dread d5/df/d11/f1c [0,4194304] 0 2024-04-06T03:40:29.411 INFO:tasks.workunit.client.1.smithi136.stdout:7/112: stat d5/df/d12/f17 0 2024-04-06T03:40:29.411 INFO:tasks.workunit.client.0.smithi094.stdout:9/124: read d2/dc/d10/f1b [322482,62861] 0 2024-04-06T03:40:29.412 INFO:tasks.workunit.client.1.smithi136.stdout:5/182: symlink d1/dc/d15/d20/l35 0 2024-04-06T03:40:29.413 INFO:tasks.workunit.client.0.smithi094.stdout:2/216: mknod d0/dd/d3d/c47 0 2024-04-06T03:40:29.413 INFO:tasks.workunit.client.0.smithi094.stdout:2/217: read - d0/dd/d19/d31/f42 zero size 2024-04-06T03:40:29.415 INFO:tasks.workunit.client.1.smithi136.stdout:2/233: mknod d3/c4f 0 2024-04-06T03:40:29.415 INFO:tasks.workunit.client.1.smithi136.stdout:2/234: chown d3/d9/d10/l16 2 1 2024-04-06T03:40:29.416 INFO:tasks.workunit.client.0.smithi094.stdout:9/125: dread d2/dc/d10/f1b [0,4194304] 0 2024-04-06T03:40:29.421 INFO:tasks.workunit.client.1.smithi136.stdout:5/183: write d1/f4 [640375,25820] 0 2024-04-06T03:40:29.421 INFO:tasks.workunit.client.0.smithi094.stdout:1/154: mkdir d0/d4/d37 0 2024-04-06T03:40:29.421 INFO:tasks.workunit.client.0.smithi094.stdout:1/155: write d0/d1/d16/d24/d30/f32 [396403,62590] 0 2024-04-06T03:40:29.423 INFO:tasks.workunit.client.1.smithi136.stdout:9/133: dwrite fe [0,4194304] 0 2024-04-06T03:40:29.423 INFO:tasks.workunit.client.1.smithi136.stdout:9/134: fsync d17/d19/f1e 0 2024-04-06T03:40:29.423 INFO:tasks.workunit.client.1.smithi136.stdout:9/135: chown c12 11494451 1 2024-04-06T03:40:29.423 INFO:tasks.workunit.client.1.smithi136.stdout:9/136: chown d17/d20/d28 5663 1 2024-04-06T03:40:29.427 INFO:tasks.workunit.client.0.smithi094.stdout:0/205: truncate d1/d9/dd/f3b 307921 0 2024-04-06T03:40:29.428 INFO:tasks.workunit.client.1.smithi136.stdout:5/184: dread d1/dc/d10/d1e/f2f [0,4194304] 0 2024-04-06T03:40:29.428 INFO:tasks.workunit.client.1.smithi136.stdout:5/185: truncate d1/dc/d1c/f2c 1652897 0 2024-04-06T03:40:29.432 INFO:tasks.workunit.client.0.smithi094.stdout:5/150: rename db/d19/c1b to db/d19/c3b 0 2024-04-06T03:40:29.433 INFO:tasks.workunit.client.1.smithi136.stdout:7/113: creat d5/d21/f24 x:0 0 0 2024-04-06T03:40:29.436 INFO:tasks.workunit.client.0.smithi094.stdout:9/126: creat d2/dc/d21/d26/f29 x:0 0 0 2024-04-06T03:40:29.436 INFO:tasks.workunit.client.1.smithi136.stdout:6/110: dwrite d2/de/f27 [0,4194304] 0 2024-04-06T03:40:29.436 INFO:tasks.workunit.client.0.smithi094.stdout:2/218: dwrite d0/dd/d1b/f3e [0,4194304] 0 2024-04-06T03:40:29.437 INFO:tasks.workunit.client.1.smithi136.stdout:2/235: creat d3/d15/f50 x:0 0 0 2024-04-06T03:40:29.439 INFO:tasks.workunit.client.0.smithi094.stdout:1/156: mkdir d0/d1/d16/d24/d30/d38 0 2024-04-06T03:40:29.439 INFO:tasks.workunit.client.0.smithi094.stdout:1/157: write d0/d4/fc [4645712,93996] 0 2024-04-06T03:40:29.439 INFO:tasks.workunit.client.0.smithi094.stdout:1/158: creat d0/d4/f39 x:0 0 0 2024-04-06T03:40:29.440 INFO:tasks.workunit.client.1.smithi136.stdout:9/137: mknod d17/d20/d25/c2c 0 2024-04-06T03:40:29.440 INFO:tasks.workunit.client.1.smithi136.stdout:0/122: truncate d1/f18 535390 0 2024-04-06T03:40:29.440 INFO:tasks.workunit.client.1.smithi136.stdout:0/123: chown d1/d1e/f27 0 1 2024-04-06T03:40:29.441 INFO:tasks.workunit.client.0.smithi094.stdout:0/206: creat d1/d9/f41 x:0 0 0 2024-04-06T03:40:29.442 INFO:tasks.workunit.client.1.smithi136.stdout:7/114: truncate d5/fd 5926560 0 2024-04-06T03:40:29.447 INFO:tasks.workunit.client.0.smithi094.stdout:9/127: mkdir d2/dc/d10/d2a 0 2024-04-06T03:40:29.447 INFO:tasks.workunit.client.0.smithi094.stdout:9/128: rename d2 to d2/dc/d21/d23/d2b 22 2024-04-06T03:40:29.456 INFO:tasks.workunit.client.1.smithi136.stdout:5/186: dwrite d1/dc/d10/d1e/f2f [4194304,4194304] 0 2024-04-06T03:40:29.456 INFO:tasks.workunit.client.1.smithi136.stdout:5/187: creat d1/dc/d1c/f36 x:0 0 0 2024-04-06T03:40:29.457 INFO:tasks.workunit.client.0.smithi094.stdout:1/159: dread d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:29.462 INFO:tasks.workunit.client.1.smithi136.stdout:6/111: symlink d2/d7/d28/l2b 0 2024-04-06T03:40:29.462 INFO:tasks.workunit.client.1.smithi136.stdout:6/112: creat d2/d7/f2c x:0 0 0 2024-04-06T03:40:29.462 INFO:tasks.workunit.client.1.smithi136.stdout:6/113: chown d2/d7/c11 1 1 2024-04-06T03:40:29.463 INFO:tasks.workunit.client.0.smithi094.stdout:2/219: truncate d0/d8/f33 1999520 0 2024-04-06T03:40:29.465 INFO:tasks.workunit.client.1.smithi136.stdout:2/236: rmdir d3/d9/d10 39 2024-04-06T03:40:29.473 INFO:tasks.workunit.client.1.smithi136.stdout:5/188: rename d1/ld to d1/d1f/l37 0 2024-04-06T03:40:29.473 INFO:tasks.workunit.client.0.smithi094.stdout:1/160: dwrite d0/d4/d1b/f2c [0,4194304] 0 2024-04-06T03:40:29.474 INFO:tasks.workunit.client.1.smithi136.stdout:9/138: dwrite f4 [0,4194304] 0 2024-04-06T03:40:29.475 INFO:tasks.workunit.client.0.smithi094.stdout:9/129: chown d2/l1a 458228696 1 2024-04-06T03:40:29.476 INFO:tasks.workunit.client.1.smithi136.stdout:5/189: dread d1/dc/d1c/f2c [0,4194304] 0 2024-04-06T03:40:29.479 INFO:tasks.workunit.client.1.smithi136.stdout:0/124: dwrite d1/d6/fd [0,4194304] 0 2024-04-06T03:40:29.479 INFO:tasks.workunit.client.1.smithi136.stdout:0/125: chown d1/f7 59 1 2024-04-06T03:40:29.479 INFO:tasks.workunit.client.1.smithi136.stdout:0/126: creat d1/d1e/d1a/f2a x:0 0 0 2024-04-06T03:40:29.479 INFO:tasks.workunit.client.1.smithi136.stdout:0/127: chown d1/d1e/d1a/d1c/f21 13256165 1 2024-04-06T03:40:29.480 INFO:tasks.workunit.client.1.smithi136.stdout:0/128: stat d1/d6/fd 0 2024-04-06T03:40:29.480 INFO:tasks.workunit.client.0.smithi094.stdout:2/220: dwrite d0/d8/f11 [0,4194304] 0 2024-04-06T03:40:29.481 INFO:tasks.workunit.client.1.smithi136.stdout:6/114: symlink d2/d7/d26/l2d 0 2024-04-06T03:40:29.481 INFO:tasks.workunit.client.1.smithi136.stdout:6/115: read - d2/d7/f2c zero size 2024-04-06T03:40:29.481 INFO:tasks.workunit.client.1.smithi136.stdout:6/116: creat d2/de/f2e x:0 0 0 2024-04-06T03:40:29.481 INFO:tasks.workunit.client.1.smithi136.stdout:7/115: rmdir d5/df/d1d 39 2024-04-06T03:40:29.483 INFO:tasks.workunit.client.1.smithi136.stdout:9/139: rename d17/l1c to d17/d20/d28/l2d 0 2024-04-06T03:40:29.484 INFO:tasks.workunit.client.0.smithi094.stdout:0/207: rmdir d1/d9/dd/d18 39 2024-04-06T03:40:29.485 INFO:tasks.workunit.client.1.smithi136.stdout:5/190: creat d1/dc/d1c/d32/f38 x:0 0 0 2024-04-06T03:40:29.486 INFO:tasks.workunit.client.1.smithi136.stdout:2/237: dwrite d3/d9/d10/d18/f48 [0,4194304] 0 2024-04-06T03:40:29.486 INFO:tasks.workunit.client.1.smithi136.stdout:2/238: write d3/d9/d10/d18/d1d/f20 [2399253,22653] 0 2024-04-06T03:40:29.493 INFO:tasks.workunit.client.0.smithi094.stdout:1/161: unlink d0/f14 0 2024-04-06T03:40:29.493 INFO:tasks.workunit.client.0.smithi094.stdout:1/162: dread - d0/d4/f39 zero size 2024-04-06T03:40:29.496 INFO:tasks.workunit.client.0.smithi094.stdout:5/151: rmdir db/d19/d24/d2e 39 2024-04-06T03:40:29.497 INFO:tasks.workunit.client.1.smithi136.stdout:0/129: dwrite d1/d1e/d1a/d1c/f23 [0,4194304] 0 2024-04-06T03:40:29.500 INFO:tasks.workunit.client.1.smithi136.stdout:6/117: unlink d2/de/l24 0 2024-04-06T03:40:29.502 INFO:tasks.workunit.client.1.smithi136.stdout:7/116: creat d5/df/d1d/f25 x:0 0 0 2024-04-06T03:40:29.504 INFO:tasks.workunit.client.1.smithi136.stdout:8/158: sync 2024-04-06T03:40:29.504 INFO:tasks.workunit.client.1.smithi136.stdout:1/140: sync 2024-04-06T03:40:29.505 INFO:tasks.workunit.client.1.smithi136.stdout:4/124: sync 2024-04-06T03:40:29.505 INFO:tasks.workunit.client.1.smithi136.stdout:1/141: chown d12/d1b 0 1 2024-04-06T03:40:29.505 INFO:tasks.workunit.client.1.smithi136.stdout:1/142: chown d12/d1b/d28 126348083 1 2024-04-06T03:40:29.506 INFO:tasks.workunit.client.1.smithi136.stdout:9/140: link d17/d19/f1f d17/d20/d25/f2e 0 2024-04-06T03:40:29.509 INFO:tasks.workunit.client.0.smithi094.stdout:2/221: symlink d0/d8/d27/d2b/l48 0 2024-04-06T03:40:29.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/152: dwrite db/d19/d2c/f31 [4194304,4194304] 0 2024-04-06T03:40:29.513 INFO:tasks.workunit.client.0.smithi094.stdout:0/208: mkdir d1/d9/dd/d18/d32/d42 0 2024-04-06T03:40:29.513 INFO:tasks.workunit.client.0.smithi094.stdout:0/209: readlink d1/d9/l3f 0 2024-04-06T03:40:29.514 INFO:tasks.workunit.client.1.smithi136.stdout:2/239: truncate d3/d15/f2b 442461 0 2024-04-06T03:40:29.514 INFO:tasks.workunit.client.1.smithi136.stdout:2/240: creat d3/d9/d10/d18/d1d/d3c/f51 x:0 0 0 2024-04-06T03:40:29.515 INFO:tasks.workunit.client.1.smithi136.stdout:3/92: sync 2024-04-06T03:40:29.516 INFO:tasks.workunit.client.1.smithi136.stdout:1/143: dread f3 [4194304,4194304] 0 2024-04-06T03:40:29.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/144: write d12/f16 [65759,127495] 0 2024-04-06T03:40:29.517 INFO:tasks.workunit.client.1.smithi136.stdout:2/241: dread d3/d15/d19/f1b [0,4194304] 0 2024-04-06T03:40:29.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/163: creat d0/d1/d16/f3a x:0 0 0 2024-04-06T03:40:29.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/164: creat d0/d1/d16/f3b x:0 0 0 2024-04-06T03:40:29.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/165: fdatasync d0/d4/f39 0 2024-04-06T03:40:29.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/145: write d12/f16 [860555,11531] 0 2024-04-06T03:40:29.518 INFO:tasks.workunit.client.0.smithi094.stdout:9/130: mkdir d2/d2c 0 2024-04-06T03:40:29.518 INFO:tasks.workunit.client.0.smithi094.stdout:9/131: stat d2/dc/d21/d23 0 2024-04-06T03:40:29.520 INFO:tasks.workunit.client.1.smithi136.stdout:8/159: dwrite d3/da/d26/f28 [4194304,4194304] 0 2024-04-06T03:40:29.521 INFO:tasks.workunit.client.1.smithi136.stdout:8/160: fdatasync d3/da/fd 0 2024-04-06T03:40:29.521 INFO:tasks.workunit.client.1.smithi136.stdout:8/161: write d3/f10 [458201,63237] 0 2024-04-06T03:40:29.521 INFO:tasks.workunit.client.1.smithi136.stdout:8/162: write d3/de/ff [347494,16757] 0 2024-04-06T03:40:29.521 INFO:tasks.workunit.client.0.smithi094.stdout:2/222: mkdir d0/dd/d3d/d49 0 2024-04-06T03:40:29.521 INFO:tasks.workunit.client.0.smithi094.stdout:2/223: write d0/f5 [4346623,110097] 0 2024-04-06T03:40:29.522 INFO:tasks.workunit.client.1.smithi136.stdout:0/130: rename d1/d1e/c14 to d1/d6/c2b 0 2024-04-06T03:40:29.523 INFO:tasks.workunit.client.1.smithi136.stdout:3/93: write d0/d4/fe [2165679,121114] 0 2024-04-06T03:40:29.527 INFO:tasks.workunit.client.0.smithi094.stdout:5/153: mknod db/d19/d24/c3c 0 2024-04-06T03:40:29.528 INFO:tasks.workunit.client.1.smithi136.stdout:3/94: dread d0/d4/d12/f15 [0,4194304] 0 2024-04-06T03:40:29.529 INFO:tasks.workunit.client.0.smithi094.stdout:0/210: rmdir d1/d9/dd/d18/d1a 39 2024-04-06T03:40:29.530 INFO:tasks.workunit.client.0.smithi094.stdout:0/211: read d1/d10/f2e [1435668,13087] 0 2024-04-06T03:40:29.531 INFO:tasks.workunit.client.0.smithi094.stdout:0/212: write d1/d9/f41 [910374,36721] 0 2024-04-06T03:40:29.531 INFO:tasks.workunit.client.0.smithi094.stdout:5/154: dread db/f2f [0,4194304] 0 2024-04-06T03:40:29.532 INFO:tasks.workunit.client.1.smithi136.stdout:8/163: dread d3/da/f19 [0,4194304] 0 2024-04-06T03:40:29.532 INFO:tasks.workunit.client.1.smithi136.stdout:8/164: write d3/da/fd [104358,38645] 0 2024-04-06T03:40:29.533 INFO:tasks.workunit.client.1.smithi136.stdout:6/118: getdents d2/d25 0 2024-04-06T03:40:29.533 INFO:tasks.workunit.client.1.smithi136.stdout:6/119: creat d2/de/f2f x:0 0 0 2024-04-06T03:40:29.535 INFO:tasks.workunit.client.0.smithi094.stdout:1/166: mkdir d0/d4/d1b/d2f/d3c 0 2024-04-06T03:40:29.535 INFO:tasks.workunit.client.0.smithi094.stdout:1/167: dread - d0/d4/f39 zero size 2024-04-06T03:40:29.535 INFO:tasks.workunit.client.0.smithi094.stdout:1/168: creat d0/d1/d27/f3d x:0 0 0 2024-04-06T03:40:29.536 INFO:tasks.workunit.client.0.smithi094.stdout:3/218: sync 2024-04-06T03:40:29.536 INFO:tasks.workunit.client.0.smithi094.stdout:3/219: write d9/d20/d30/f37 [1001022,118879] 0 2024-04-06T03:40:29.536 INFO:tasks.workunit.client.0.smithi094.stdout:3/220: fsync d9/f19 0 2024-04-06T03:40:29.537 INFO:tasks.workunit.client.0.smithi094.stdout:3/221: fdatasync d9/d20/f3d 0 2024-04-06T03:40:29.537 INFO:tasks.workunit.client.1.smithi136.stdout:7/117: creat d5/df/d11/f26 x:0 0 0 2024-04-06T03:40:29.538 INFO:tasks.workunit.client.0.smithi094.stdout:9/132: mkdir d2/dc/d10/d2a/d2d 0 2024-04-06T03:40:29.538 INFO:tasks.workunit.client.0.smithi094.stdout:9/133: write d2/fe [2205423,59770] 0 2024-04-06T03:40:29.541 INFO:tasks.workunit.client.0.smithi094.stdout:0/213: rename d1/c30 to d1/d9/dd/d18/d1e/c43 0 2024-04-06T03:40:29.543 INFO:tasks.workunit.client.0.smithi094.stdout:1/169: write d0/d1/f10 [1365338,35849] 0 2024-04-06T03:40:29.543 INFO:tasks.workunit.client.0.smithi094.stdout:2/224: dwrite d0/d8/d1d/f38 [0,4194304] 0 2024-04-06T03:40:29.543 INFO:tasks.workunit.client.0.smithi094.stdout:2/225: chown d0/d18/c23 0 1 2024-04-06T03:40:29.543 INFO:tasks.workunit.client.0.smithi094.stdout:5/155: creat db/d19/f3d x:0 0 0 2024-04-06T03:40:29.546 INFO:tasks.workunit.client.0.smithi094.stdout:7/197: sync 2024-04-06T03:40:29.546 INFO:tasks.workunit.client.0.smithi094.stdout:6/156: sync 2024-04-06T03:40:29.546 INFO:tasks.workunit.client.0.smithi094.stdout:4/234: sync 2024-04-06T03:40:29.546 INFO:tasks.workunit.client.0.smithi094.stdout:8/171: sync 2024-04-06T03:40:29.548 INFO:tasks.workunit.client.0.smithi094.stdout:9/134: mknod d2/dc/d10/c2e 0 2024-04-06T03:40:29.550 INFO:tasks.workunit.client.1.smithi136.stdout:1/146: truncate f3 5585914 0 2024-04-06T03:40:29.551 INFO:tasks.workunit.client.1.smithi136.stdout:1/147: fsync d12/f24 0 2024-04-06T03:40:29.551 INFO:tasks.workunit.client.0.smithi094.stdout:9/135: dread d2/f1d [0,4194304] 0 2024-04-06T03:40:29.552 INFO:tasks.workunit.client.1.smithi136.stdout:6/120: dwrite d2/f9 [0,4194304] 0 2024-04-06T03:40:29.553 INFO:tasks.workunit.client.1.smithi136.stdout:6/121: truncate d2/d25/f17 55461 0 2024-04-06T03:40:29.553 INFO:tasks.workunit.client.0.smithi094.stdout:0/214: mkdir d1/d9/dd/d18/d1a/d44 0 2024-04-06T03:40:29.554 INFO:tasks.workunit.client.1.smithi136.stdout:0/131: chown d1/le 82354 1 2024-04-06T03:40:29.557 INFO:tasks.workunit.client.1.smithi136.stdout:3/95: creat d0/d4/f1c x:0 0 0 2024-04-06T03:40:29.557 INFO:tasks.workunit.client.1.smithi136.stdout:3/96: fsync d0/f13 0 2024-04-06T03:40:29.558 INFO:tasks.workunit.client.0.smithi094.stdout:1/170: symlink d0/l3e 0 2024-04-06T03:40:29.559 INFO:tasks.workunit.client.0.smithi094.stdout:3/222: dwrite d9/d10/f16 [0,4194304] 0 2024-04-06T03:40:29.560 INFO:tasks.workunit.client.0.smithi094.stdout:7/198: write d2/d6/dc/d12/f1c [2716667,94244] 0 2024-04-06T03:40:29.563 INFO:tasks.workunit.client.1.smithi136.stdout:8/165: rename d3/de/d18/c2e to d3/de/d18/c30 0 2024-04-06T03:40:29.565 INFO:tasks.workunit.client.1.smithi136.stdout:6/122: dread d2/fd [0,4194304] 0 2024-04-06T03:40:29.566 INFO:tasks.workunit.client.1.smithi136.stdout:7/118: creat d5/f27 x:0 0 0 2024-04-06T03:40:29.567 INFO:tasks.workunit.client.0.smithi094.stdout:1/171: dread d0/d1/d16/f1c [0,4194304] 0 2024-04-06T03:40:29.567 INFO:tasks.workunit.client.0.smithi094.stdout:1/172: write d0/d7/f36 [328540,22126] 0 2024-04-06T03:40:29.567 INFO:tasks.workunit.client.0.smithi094.stdout:2/226: truncate d0/d18/f22 3453251 0 2024-04-06T03:40:29.569 INFO:tasks.workunit.client.0.smithi094.stdout:0/215: dread d1/f23 [0,4194304] 0 2024-04-06T03:40:29.569 INFO:tasks.workunit.client.0.smithi094.stdout:0/216: fdatasync d1/d9/dd/f1c 0 2024-04-06T03:40:29.569 INFO:tasks.workunit.client.1.smithi136.stdout:4/125: mkdir d1d/d31 0 2024-04-06T03:40:29.569 INFO:tasks.workunit.client.1.smithi136.stdout:4/126: creat d1d/d2f/f32 x:0 0 0 2024-04-06T03:40:29.572 INFO:tasks.workunit.client.1.smithi136.stdout:9/141: truncate fe 2027421 0 2024-04-06T03:40:29.572 INFO:tasks.workunit.client.1.smithi136.stdout:9/142: dread - d17/d19/f1e zero size 2024-04-06T03:40:29.572 INFO:tasks.workunit.client.1.smithi136.stdout:9/143: write f16 [1630475,16397] 0 2024-04-06T03:40:29.572 INFO:tasks.workunit.client.1.smithi136.stdout:9/144: chown d17/d20/d25 36 1 2024-04-06T03:40:29.572 INFO:tasks.workunit.client.1.smithi136.stdout:9/145: creat d17/d19/f2f x:0 0 0 2024-04-06T03:40:29.573 INFO:tasks.workunit.client.0.smithi094.stdout:8/172: truncate d18/f22 1371197 0 2024-04-06T03:40:29.574 INFO:tasks.workunit.client.1.smithi136.stdout:2/242: getdents d3/d15/d19 0 2024-04-06T03:40:29.575 INFO:tasks.workunit.client.1.smithi136.stdout:8/166: dread d3/de/d1b/f1d [4194304,4194304] 0 2024-04-06T03:40:29.576 INFO:tasks.workunit.client.0.smithi094.stdout:7/199: dread d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:29.576 INFO:tasks.workunit.client.0.smithi094.stdout:4/235: dwrite d0/df/d1e/f24 [0,4194304] 0 2024-04-06T03:40:29.576 INFO:tasks.workunit.client.0.smithi094.stdout:9/136: mkdir d2/dc/d21/d23/d2f 0 2024-04-06T03:40:29.576 INFO:tasks.workunit.client.0.smithi094.stdout:9/137: write d2/dc/d10/f1b [616411,5350] 0 2024-04-06T03:40:29.577 INFO:tasks.workunit.client.0.smithi094.stdout:6/157: dwrite dc/d11/d22/f39 [0,4194304] 0 2024-04-06T03:40:29.579 INFO:tasks.workunit.client.1.smithi136.stdout:1/148: creat d12/d1b/f29 x:0 0 0 2024-04-06T03:40:29.586 INFO:tasks.workunit.client.0.smithi094.stdout:3/223: mkdir d9/d48 0 2024-04-06T03:40:29.587 INFO:tasks.workunit.client.1.smithi136.stdout:7/119: dwrite d5/df/d11/f14 [0,4194304] 0 2024-04-06T03:40:29.588 INFO:tasks.workunit.client.1.smithi136.stdout:3/97: link d0/d4/d12/l17 d0/d4/d12/d19/d1b/l1d 0 2024-04-06T03:40:29.588 INFO:tasks.workunit.client.1.smithi136.stdout:3/98: creat d0/f1e x:0 0 0 2024-04-06T03:40:29.588 INFO:tasks.workunit.client.1.smithi136.stdout:3/99: chown d0/d4/d12/f1a 17218015 1 2024-04-06T03:40:29.589 INFO:tasks.workunit.client.0.smithi094.stdout:8/173: dwrite d18/d34/f38 [0,4194304] 0 2024-04-06T03:40:29.589 INFO:tasks.workunit.client.1.smithi136.stdout:3/100: dread d0/d4/fa [0,4194304] 0 2024-04-06T03:40:29.590 INFO:tasks.workunit.client.1.smithi136.stdout:3/101: chown d0/lf 6 1 2024-04-06T03:40:29.590 INFO:tasks.workunit.client.0.smithi094.stdout:1/173: mkdir d0/d1/d16/d3f 0 2024-04-06T03:40:29.590 INFO:tasks.workunit.client.1.smithi136.stdout:3/102: write d0/d4/d12/f15 [1854656,50135] 0 2024-04-06T03:40:29.591 INFO:tasks.workunit.client.1.smithi136.stdout:1/149: dread d12/f16 [0,4194304] 0 2024-04-06T03:40:29.591 INFO:tasks.workunit.client.1.smithi136.stdout:1/150: fdatasync f10 0 2024-04-06T03:40:29.591 INFO:tasks.workunit.client.0.smithi094.stdout:2/227: rmdir d0/d8/d27/d2b 39 2024-04-06T03:40:29.592 INFO:tasks.workunit.client.0.smithi094.stdout:2/228: fdatasync d0/dd/f40 0 2024-04-06T03:40:29.592 INFO:tasks.workunit.client.0.smithi094.stdout:2/229: fsync d0/dd/f40 0 2024-04-06T03:40:29.592 INFO:tasks.workunit.client.0.smithi094.stdout:1/174: dread d0/d4/fc [4194304,4194304] 0 2024-04-06T03:40:29.592 INFO:tasks.workunit.client.1.smithi136.stdout:4/127: creat d1d/d31/f33 x:0 0 0 2024-04-06T03:40:29.593 INFO:tasks.workunit.client.0.smithi094.stdout:0/217: creat d1/d9/dd/d18/d32/d42/f45 x:0 0 0 2024-04-06T03:40:29.597 INFO:tasks.workunit.client.0.smithi094.stdout:5/156: getdents db/d19/d2c 0 2024-04-06T03:40:29.597 INFO:tasks.workunit.client.0.smithi094.stdout:5/157: write db/f2f [1215946,88954] 0 2024-04-06T03:40:29.598 INFO:tasks.workunit.client.1.smithi136.stdout:9/146: getdents d17/d20/d28 0 2024-04-06T03:40:29.600 INFO:tasks.workunit.client.0.smithi094.stdout:7/200: creat d2/d6/dc/d12/d29/d35/f39 x:0 0 0 2024-04-06T03:40:29.602 INFO:tasks.workunit.client.1.smithi136.stdout:2/243: symlink d3/d9/l52 0 2024-04-06T03:40:29.604 INFO:tasks.workunit.client.0.smithi094.stdout:4/236: symlink d0/d15/d3a/l4f 0 2024-04-06T03:40:29.605 INFO:tasks.workunit.client.0.smithi094.stdout:1/175: dwrite d0/d4/d1b/f2c [0,4194304] 0 2024-04-06T03:40:29.609 INFO:tasks.workunit.client.0.smithi094.stdout:9/138: creat d2/f30 x:0 0 0 2024-04-06T03:40:29.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/224: mknod d9/d20/d2d/c49 0 2024-04-06T03:40:29.618 INFO:tasks.workunit.client.0.smithi094.stdout:3/225: creat d9/d1d/f4a x:0 0 0 2024-04-06T03:40:29.621 INFO:tasks.workunit.client.0.smithi094.stdout:7/201: dwrite d2/d6/dc/dd/f27 [0,4194304] 0 2024-04-06T03:40:29.621 INFO:tasks.workunit.client.0.smithi094.stdout:7/202: write d2/d6/dc/dd/f2f [303586,112931] 0 2024-04-06T03:40:29.624 INFO:tasks.workunit.client.0.smithi094.stdout:1/176: dwrite d0/d1/d16/d24/d30/f33 [0,4194304] 0 2024-04-06T03:40:29.624 INFO:tasks.workunit.client.0.smithi094.stdout:1/177: truncate d0/d1/d27/f3d 128679 0 2024-04-06T03:40:29.625 INFO:tasks.workunit.client.0.smithi094.stdout:1/178: readlink d0/d1/d16/l31 0 2024-04-06T03:40:29.626 INFO:tasks.workunit.client.0.smithi094.stdout:4/237: dwrite d0/df/d1e/f25 [4194304,4194304] 0 2024-04-06T03:40:29.626 INFO:tasks.workunit.client.1.smithi136.stdout:3/103: symlink d0/l1f 0 2024-04-06T03:40:29.634 INFO:tasks.workunit.client.0.smithi094.stdout:7/203: write d2/d6/dc/dd/f27 [3688616,36522] 0 2024-04-06T03:40:29.635 INFO:tasks.workunit.client.0.smithi094.stdout:8/174: symlink d18/d30/l39 0 2024-04-06T03:40:29.635 INFO:tasks.workunit.client.0.smithi094.stdout:8/175: creat d18/d2c/f3a x:0 0 0 2024-04-06T03:40:29.635 INFO:tasks.workunit.client.0.smithi094.stdout:8/176: creat d18/d34/f3b x:0 0 0 2024-04-06T03:40:29.636 INFO:tasks.workunit.client.0.smithi094.stdout:2/230: mknod d0/d8/c4a 0 2024-04-06T03:40:29.640 INFO:tasks.workunit.client.0.smithi094.stdout:0/218: creat d1/d9/dd/d18/d1a/f46 x:0 0 0 2024-04-06T03:40:29.640 INFO:tasks.workunit.client.0.smithi094.stdout:0/219: chown d1/d9/dd/d18/d32/d42 32649 1 2024-04-06T03:40:29.642 INFO:tasks.workunit.client.0.smithi094.stdout:5/158: mknod db/d19/c3e 0 2024-04-06T03:40:29.642 INFO:tasks.workunit.client.0.smithi094.stdout:4/238: dwrite d0/df/f19 [0,4194304] 0 2024-04-06T03:40:29.644 INFO:tasks.workunit.client.0.smithi094.stdout:9/139: mknod d2/dc/c31 0 2024-04-06T03:40:29.651 INFO:tasks.workunit.client.0.smithi094.stdout:3/226: mknod d9/d1d/c4b 0 2024-04-06T03:40:29.654 INFO:tasks.workunit.client.0.smithi094.stdout:0/220: dwrite d1/d10/f2e [0,4194304] 0 2024-04-06T03:40:29.654 INFO:tasks.workunit.client.0.smithi094.stdout:2/231: dwrite d0/d8/f11 [4194304,4194304] 0 2024-04-06T03:40:29.657 INFO:tasks.workunit.client.1.smithi136.stdout:2/244: symlink d3/d9/d10/d18/d4b/l53 0 2024-04-06T03:40:29.658 INFO:tasks.workunit.client.1.smithi136.stdout:2/245: dread d3/d15/f2b [0,4194304] 0 2024-04-06T03:40:29.662 INFO:tasks.workunit.client.1.smithi136.stdout:0/132: rename d1/d1e to d1/d6/d2c 0 2024-04-06T03:40:29.662 INFO:tasks.workunit.client.1.smithi136.stdout:7/120: rmdir d5/df/d12/d1b 39 2024-04-06T03:40:29.662 INFO:tasks.workunit.client.1.smithi136.stdout:7/121: stat d5/c6 0 2024-04-06T03:40:29.662 INFO:tasks.workunit.client.1.smithi136.stdout:7/122: write d5/f9 [474390,22695] 0 2024-04-06T03:40:29.663 INFO:tasks.workunit.client.1.smithi136.stdout:7/123: chown d5/f27 16129894 1 2024-04-06T03:40:29.663 INFO:tasks.workunit.client.0.smithi094.stdout:1/179: mkdir d0/d1/d16/d40 0 2024-04-06T03:40:29.663 INFO:tasks.workunit.client.0.smithi094.stdout:2/232: dread d0/d8/d27/d2b/f2d [0,4194304] 0 2024-04-06T03:40:29.663 INFO:tasks.workunit.client.0.smithi094.stdout:1/180: write d0/d1/d16/f1c [987331,119115] 0 2024-04-06T03:40:29.665 INFO:tasks.workunit.client.1.smithi136.stdout:0/133: dread d1/d6/fb [0,4194304] 0 2024-04-06T03:40:29.667 INFO:tasks.workunit.client.1.smithi136.stdout:4/128: rmdir d1d 39 2024-04-06T03:40:29.669 INFO:tasks.workunit.client.0.smithi094.stdout:8/177: mkdir d18/d3c 0 2024-04-06T03:40:29.669 INFO:tasks.workunit.client.0.smithi094.stdout:8/178: fdatasync d18/f19 0 2024-04-06T03:40:29.669 INFO:tasks.workunit.client.0.smithi094.stdout:8/179: write d18/f28 [149408,61313] 0 2024-04-06T03:40:29.669 INFO:tasks.workunit.client.0.smithi094.stdout:8/180: fsync d18/d2c/f2e 0 2024-04-06T03:40:29.669 INFO:tasks.workunit.client.0.smithi094.stdout:8/181: readlink l12 0 2024-04-06T03:40:29.670 INFO:tasks.workunit.client.1.smithi136.stdout:9/147: getdents d17/d21/d29 0 2024-04-06T03:40:29.670 INFO:tasks.workunit.client.1.smithi136.stdout:9/148: chown d17/d19/c1a 8504685 1 2024-04-06T03:40:29.671 INFO:tasks.workunit.client.0.smithi094.stdout:5/159: creat db/d19/d24/d28/f3f x:0 0 0 2024-04-06T03:40:29.673 INFO:tasks.workunit.client.1.smithi136.stdout:2/246: mkdir d3/d15/d19/d43/d54 0 2024-04-06T03:40:29.674 INFO:tasks.workunit.client.0.smithi094.stdout:4/239: mkdir d0/d15/d3a/d3e/d50 0 2024-04-06T03:40:29.676 INFO:tasks.workunit.client.1.smithi136.stdout:6/123: rename d2/d7/d28 to d2/d7/d26/d30 0 2024-04-06T03:40:29.676 INFO:tasks.workunit.client.1.smithi136.stdout:6/124: stat d2/fa 0 2024-04-06T03:40:29.676 INFO:tasks.workunit.client.1.smithi136.stdout:6/125: creat d2/d7/f31 x:0 0 0 2024-04-06T03:40:29.677 INFO:tasks.workunit.client.0.smithi094.stdout:9/140: mkdir d2/dc/d32 0 2024-04-06T03:40:29.679 INFO:tasks.workunit.client.1.smithi136.stdout:7/124: creat d5/df/d11/d22/f28 x:0 0 0 2024-04-06T03:40:29.680 INFO:tasks.workunit.client.0.smithi094.stdout:3/227: symlink d9/d48/l4c 0 2024-04-06T03:40:29.682 INFO:tasks.workunit.client.1.smithi136.stdout:3/104: getdents d0 0 2024-04-06T03:40:29.682 INFO:tasks.workunit.client.1.smithi136.stdout:3/105: creat d0/d4/d12/f20 x:0 0 0 2024-04-06T03:40:29.682 INFO:tasks.workunit.client.1.smithi136.stdout:0/134: dwrite d1/d6/f12 [0,4194304] 0 2024-04-06T03:40:29.684 INFO:tasks.workunit.client.1.smithi136.stdout:4/129: mknod d1d/d31/c34 0 2024-04-06T03:40:29.688 INFO:tasks.workunit.client.1.smithi136.stdout:8/167: rename d3/l6 to d3/de/d18/l31 0 2024-04-06T03:40:29.694 INFO:tasks.workunit.client.1.smithi136.stdout:7/125: creat d5/df/d12/d1b/f29 x:0 0 0 2024-04-06T03:40:29.699 INFO:tasks.workunit.client.1.smithi136.stdout:3/106: unlink d0/d4/f1c 0 2024-04-06T03:40:29.700 INFO:tasks.workunit.client.1.smithi136.stdout:5/191: sync 2024-04-06T03:40:29.700 INFO:tasks.workunit.client.1.smithi136.stdout:5/192: chown d1/dc/d15/l2d 99 1 2024-04-06T03:40:29.701 INFO:tasks.workunit.client.1.smithi136.stdout:0/135: symlink d1/d6/l2d 0 2024-04-06T03:40:29.703 INFO:tasks.workunit.client.1.smithi136.stdout:6/126: dwrite d2/f5 [0,4194304] 0 2024-04-06T03:40:29.703 INFO:tasks.workunit.client.1.smithi136.stdout:6/127: write d2/d7/f1b [80694,54496] 0 2024-04-06T03:40:29.703 INFO:tasks.workunit.client.1.smithi136.stdout:7/126: dread d5/df/f10 [0,4194304] 0 2024-04-06T03:40:29.704 INFO:tasks.workunit.client.1.smithi136.stdout:7/127: chown d5/l1e 364 1 2024-04-06T03:40:29.705 INFO:tasks.workunit.client.1.smithi136.stdout:1/151: rename d12/l14 to d12/d1b/d1e/d27/l2a 0 2024-04-06T03:40:29.705 INFO:tasks.workunit.client.1.smithi136.stdout:9/149: rename d17/d19 to d17/d19/d30 22 2024-04-06T03:40:29.705 INFO:tasks.workunit.client.1.smithi136.stdout:9/150: creat d17/d20/d25/f31 x:0 0 0 2024-04-06T03:40:29.707 INFO:tasks.workunit.client.1.smithi136.stdout:3/107: link d0/d4/l14 d0/d4/d12/d19/d1b/l21 0 2024-04-06T03:40:29.708 INFO:tasks.workunit.client.0.smithi094.stdout:3/228: dwrite d9/f19 [0,4194304] 0 2024-04-06T03:40:29.708 INFO:tasks.workunit.client.0.smithi094.stdout:3/229: dread - d9/d1d/f42 zero size 2024-04-06T03:40:29.708 INFO:tasks.workunit.client.0.smithi094.stdout:3/230: creat d9/d20/d30/f4d x:0 0 0 2024-04-06T03:40:29.709 INFO:tasks.workunit.client.1.smithi136.stdout:8/168: dwrite f1 [0,4194304] 0 2024-04-06T03:40:29.709 INFO:tasks.workunit.client.1.smithi136.stdout:8/169: truncate d3/da/fb 4906389 0 2024-04-06T03:40:29.712 INFO:tasks.workunit.client.0.smithi094.stdout:0/221: rmdir d1/d9/dd/d18/d1e 39 2024-04-06T03:40:29.712 INFO:tasks.workunit.client.0.smithi094.stdout:0/222: readlink d1/l6 0 2024-04-06T03:40:29.714 INFO:tasks.workunit.client.1.smithi136.stdout:9/151: write d17/d20/d25/f2e [1269212,19305] 0 2024-04-06T03:40:29.716 INFO:tasks.workunit.client.1.smithi136.stdout:6/128: symlink d2/de/l32 0 2024-04-06T03:40:29.719 INFO:tasks.workunit.client.0.smithi094.stdout:1/181: rename d0/d1/d16/c2a to d0/d4/d1b/d2f/c41 0 2024-04-06T03:40:29.721 INFO:tasks.workunit.client.1.smithi136.stdout:7/128: mkdir d5/df/d11/d22/d2a 0 2024-04-06T03:40:29.722 INFO:tasks.workunit.client.0.smithi094.stdout:7/204: truncate d2/d6/dc/dd/df/f2d 638159 0 2024-04-06T03:40:29.724 INFO:tasks.workunit.client.1.smithi136.stdout:2/247: rename d3/d4a/l2e to d3/d9/d3b/l55 0 2024-04-06T03:40:29.726 INFO:tasks.workunit.client.1.smithi136.stdout:8/170: dwrite d3/fc [0,4194304] 0 2024-04-06T03:40:29.726 INFO:tasks.workunit.client.1.smithi136.stdout:8/171: dread - d3/de/d18/f1e zero size 2024-04-06T03:40:29.727 INFO:tasks.workunit.client.0.smithi094.stdout:4/240: creat d0/d4/f51 x:0 0 0 2024-04-06T03:40:29.727 INFO:tasks.workunit.client.0.smithi094.stdout:4/241: write d0/df/d1e/d34/f39 [546093,77937] 0 2024-04-06T03:40:29.727 INFO:tasks.workunit.client.0.smithi094.stdout:4/242: chown d0/df/d1e/c48 246959 1 2024-04-06T03:40:29.727 INFO:tasks.workunit.client.0.smithi094.stdout:4/243: dread - d0/d15/d3a/d3e/d49/f4d zero size 2024-04-06T03:40:29.728 INFO:tasks.workunit.client.0.smithi094.stdout:9/141: mkdir d2/d2c/d33 0 2024-04-06T03:40:29.731 INFO:tasks.workunit.client.1.smithi136.stdout:3/108: link d0/f1e d0/d4/f22 0 2024-04-06T03:40:29.731 INFO:tasks.workunit.client.1.smithi136.stdout:3/109: dread - d0/d4/f22 zero size 2024-04-06T03:40:29.734 INFO:tasks.workunit.client.1.smithi136.stdout:7/129: dwrite d5/df/d12/d1b/f29 [0,4194304] 0 2024-04-06T03:40:29.734 INFO:tasks.workunit.client.1.smithi136.stdout:7/130: creat d5/df/d11/f2b x:0 0 0 2024-04-06T03:40:29.735 INFO:tasks.workunit.client.1.smithi136.stdout:0/136: mkdir d1/d2e 0 2024-04-06T03:40:29.737 INFO:tasks.workunit.client.0.smithi094.stdout:3/231: link d9/d10/f22 d9/d20/f4e 0 2024-04-06T03:40:29.737 INFO:tasks.workunit.client.0.smithi094.stdout:3/232: write d9/d10/f2b [300788,112508] 0 2024-04-06T03:40:29.739 INFO:tasks.workunit.client.1.smithi136.stdout:2/248: dread d3/d9/d10/f17 [0,4194304] 0 2024-04-06T03:40:29.739 INFO:tasks.workunit.client.1.smithi136.stdout:9/152: mkdir d17/d21/d32 0 2024-04-06T03:40:29.741 INFO:tasks.workunit.client.0.smithi094.stdout:0/223: symlink d1/d9/dd/d18/l47 0 2024-04-06T03:40:29.741 INFO:tasks.workunit.client.1.smithi136.stdout:3/110: dread d0/d4/fd [0,4194304] 0 2024-04-06T03:40:29.741 INFO:tasks.workunit.client.1.smithi136.stdout:3/111: dread - d0/d4/f22 zero size 2024-04-06T03:40:29.742 INFO:tasks.workunit.client.1.smithi136.stdout:3/112: dread - d0/d4/d12/f1a zero size 2024-04-06T03:40:29.743 INFO:tasks.workunit.client.1.smithi136.stdout:6/129: symlink d2/d7/d26/l33 0 2024-04-06T03:40:29.743 INFO:tasks.workunit.client.1.smithi136.stdout:6/130: dread - d2/de/f2f zero size 2024-04-06T03:40:29.744 INFO:tasks.workunit.client.0.smithi094.stdout:2/233: symlink d0/d18/l4b 0 2024-04-06T03:40:29.746 INFO:tasks.workunit.client.1.smithi136.stdout:4/130: rename d1d/d2f/f30 to d1d/d31/f35 0 2024-04-06T03:40:29.747 INFO:tasks.workunit.client.1.smithi136.stdout:7/131: dread d5/fd [0,4194304] 0 2024-04-06T03:40:29.748 INFO:tasks.workunit.client.0.smithi094.stdout:3/233: write d9/d10/d27/f32 [3599970,46259] 0 2024-04-06T03:40:29.748 INFO:tasks.workunit.client.0.smithi094.stdout:8/182: rename d18/f28 to d18/d34/f3d 0 2024-04-06T03:40:29.748 INFO:tasks.workunit.client.0.smithi094.stdout:9/142: dwrite d2/fb [0,4194304] 0 2024-04-06T03:40:29.749 INFO:tasks.workunit.client.1.smithi136.stdout:1/152: getdents d12/d1b/d1e/d1f 0 2024-04-06T03:40:29.749 INFO:tasks.workunit.client.1.smithi136.stdout:0/137: dwrite d1/d6/f1f [0,4194304] 0 2024-04-06T03:40:29.751 INFO:tasks.workunit.client.0.smithi094.stdout:1/182: unlink d0/d1/cd 0 2024-04-06T03:40:29.751 INFO:tasks.workunit.client.0.smithi094.stdout:1/183: chown d0/d7/f26 6459 1 2024-04-06T03:40:29.753 INFO:tasks.workunit.client.0.smithi094.stdout:7/205: unlink d2/d6/f32 0 2024-04-06T03:40:29.756 INFO:tasks.workunit.client.1.smithi136.stdout:6/131: read d2/f21 [1158302,83499] 0 2024-04-06T03:40:29.756 INFO:tasks.workunit.client.1.smithi136.stdout:8/172: mknod d3/d2d/c32 0 2024-04-06T03:40:29.758 INFO:tasks.workunit.client.0.smithi094.stdout:5/160: mkdir db/d40 0 2024-04-06T03:40:29.759 INFO:tasks.workunit.client.0.smithi094.stdout:5/161: chown db/l20 1155453357 1 2024-04-06T03:40:29.759 INFO:tasks.workunit.client.0.smithi094.stdout:5/162: dread - db/d19/d24/d28/f3f zero size 2024-04-06T03:40:29.761 INFO:tasks.workunit.client.0.smithi094.stdout:4/244: rmdir d0/d15/d3a/d3e 39 2024-04-06T03:40:29.764 INFO:tasks.workunit.client.0.smithi094.stdout:0/224: link d1/d9/dd/d18/d1a/c2b d1/d9/dd/d18/d1e/d22/c48 0 2024-04-06T03:40:29.764 INFO:tasks.workunit.client.1.smithi136.stdout:8/173: dread d3/da/fb [0,4194304] 0 2024-04-06T03:40:29.764 INFO:tasks.workunit.client.0.smithi094.stdout:2/234: creat d0/dd/d19/f4c x:0 0 0 2024-04-06T03:40:29.766 INFO:tasks.workunit.client.1.smithi136.stdout:3/113: creat d0/f23 x:0 0 0 2024-04-06T03:40:29.766 INFO:tasks.workunit.client.1.smithi136.stdout:3/114: write d0/f1e [227000,113179] 0 2024-04-06T03:40:29.770 INFO:tasks.workunit.client.1.smithi136.stdout:5/193: rename d1/dc/ce to d1/c39 0 2024-04-06T03:40:29.770 INFO:tasks.workunit.client.1.smithi136.stdout:5/194: write d1/f4 [2527336,123078] 0 2024-04-06T03:40:29.771 INFO:tasks.workunit.client.0.smithi094.stdout:3/234: unlink d9/f18 0 2024-04-06T03:40:29.772 INFO:tasks.workunit.client.1.smithi136.stdout:4/131: mkdir d1d/d31/d36 0 2024-04-06T03:40:29.773 INFO:tasks.workunit.client.0.smithi094.stdout:8/183: creat d18/d30/f3e x:0 0 0 2024-04-06T03:40:29.773 INFO:tasks.workunit.client.0.smithi094.stdout:8/184: readlink d18/l24 0 2024-04-06T03:40:29.773 INFO:tasks.workunit.client.0.smithi094.stdout:8/185: readlink l12 0 2024-04-06T03:40:29.774 INFO:tasks.workunit.client.1.smithi136.stdout:0/138: dread d1/fc [0,4194304] 0 2024-04-06T03:40:29.775 INFO:tasks.workunit.client.1.smithi136.stdout:0/139: write d1/d6/f11 [26181,47767] 0 2024-04-06T03:40:29.775 INFO:tasks.workunit.client.1.smithi136.stdout:7/132: unlink d5/cb 0 2024-04-06T03:40:29.776 INFO:tasks.workunit.client.1.smithi136.stdout:7/133: chown d5/df/d11/f1c 56131 1 2024-04-06T03:40:29.776 INFO:tasks.workunit.client.0.smithi094.stdout:7/206: dwrite d2/d6/dc/f2b [0,4194304] 0 2024-04-06T03:40:29.777 INFO:tasks.workunit.client.0.smithi094.stdout:9/143: creat d2/dc/d21/d23/d2f/f34 x:0 0 0 2024-04-06T03:40:29.780 INFO:tasks.workunit.client.0.smithi094.stdout:7/207: dread d2/d6/dc/dd/f2f [0,4194304] 0 2024-04-06T03:40:29.783 INFO:tasks.workunit.client.1.smithi136.stdout:9/153: dread d17/d20/d25/f2e [0,4194304] 0 2024-04-06T03:40:29.783 INFO:tasks.workunit.client.0.smithi094.stdout:1/184: symlink d0/d4/d1b/l42 0 2024-04-06T03:40:29.785 INFO:tasks.workunit.client.1.smithi136.stdout:1/153: dwrite f3 [4194304,4194304] 0 2024-04-06T03:40:29.785 INFO:tasks.workunit.client.1.smithi136.stdout:1/154: write f11 [2376915,17403] 0 2024-04-06T03:40:29.785 INFO:tasks.workunit.client.1.smithi136.stdout:1/155: chown d12/l18 454770026 1 2024-04-06T03:40:29.785 INFO:tasks.workunit.client.1.smithi136.stdout:1/156: readlink d12/d1b/d1e/d23/l25 0 2024-04-06T03:40:29.785 INFO:tasks.workunit.client.1.smithi136.stdout:1/157: chown d12/c13 3 1 2024-04-06T03:40:29.788 INFO:tasks.workunit.client.0.smithi094.stdout:4/245: unlink d0/d15/c17 0 2024-04-06T03:40:29.791 INFO:tasks.workunit.client.0.smithi094.stdout:0/225: creat d1/d9/dd/d18/d1e/d22/f49 x:0 0 0 2024-04-06T03:40:29.791 INFO:tasks.workunit.client.0.smithi094.stdout:0/226: write d1/d9/fc [6057977,71428] 0 2024-04-06T03:40:29.791 INFO:tasks.workunit.client.0.smithi094.stdout:0/227: creat d1/d9/dd/d18/d32/d42/f4a x:0 0 0 2024-04-06T03:40:29.792 INFO:tasks.workunit.client.1.smithi136.stdout:8/174: creat d3/de/d18/f33 x:0 0 0 2024-04-06T03:40:29.793 INFO:tasks.workunit.client.0.smithi094.stdout:3/235: symlink d9/l4f 0 2024-04-06T03:40:29.794 INFO:tasks.workunit.client.0.smithi094.stdout:5/163: dwrite db/f23 [4194304,4194304] 0 2024-04-06T03:40:29.794 INFO:tasks.workunit.client.0.smithi094.stdout:5/164: fsync db/d19/f3d 0 2024-04-06T03:40:29.795 INFO:tasks.workunit.client.1.smithi136.stdout:3/115: creat d0/d4/d12/d19/d1b/f24 x:0 0 0 2024-04-06T03:40:29.796 INFO:tasks.workunit.client.0.smithi094.stdout:2/235: dwrite d0/dd/d1b/f3e [0,4194304] 0 2024-04-06T03:40:29.796 INFO:tasks.workunit.client.0.smithi094.stdout:2/236: chown d0/dd/d1b 79351 1 2024-04-06T03:40:29.799 INFO:tasks.workunit.client.1.smithi136.stdout:5/195: dwrite d1/f4 [0,4194304] 0 2024-04-06T03:40:29.799 INFO:tasks.workunit.client.1.smithi136.stdout:5/196: write d1/d16/f1b [5167954,85815] 0 2024-04-06T03:40:29.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/186: rename lf to d18/d2c/l3f 0 2024-04-06T03:40:29.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/187: creat d18/d30/f40 x:0 0 0 2024-04-06T03:40:29.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/188: getdents d18/d3c 0 2024-04-06T03:40:29.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/189: creat d18/d2c/f41 x:0 0 0 2024-04-06T03:40:29.800 INFO:tasks.workunit.client.0.smithi094.stdout:8/190: dread - d18/d2c/f2e zero size 2024-04-06T03:40:29.805 INFO:tasks.workunit.client.1.smithi136.stdout:2/249: rename d3/d15/l4c to d3/d15/d19/d4e/l56 0 2024-04-06T03:40:29.807 INFO:tasks.workunit.client.1.smithi136.stdout:4/132: mknod d1d/d31/c37 0 2024-04-06T03:40:29.809 INFO:tasks.workunit.client.0.smithi094.stdout:9/144: creat d2/d2c/f35 x:0 0 0 2024-04-06T03:40:29.809 INFO:tasks.workunit.client.0.smithi094.stdout:9/145: chown d2/dc/d10/d2a/d2d 255915 1 2024-04-06T03:40:29.809 INFO:tasks.workunit.client.0.smithi094.stdout:9/146: chown d2/dc/d10/f1b 263 1 2024-04-06T03:40:29.810 INFO:tasks.workunit.client.0.smithi094.stdout:3/236: dwrite d9/d1d/f3c [0,4194304] 0 2024-04-06T03:40:29.810 INFO:tasks.workunit.client.0.smithi094.stdout:3/237: creat d9/d20/f50 x:0 0 0 2024-04-06T03:40:29.810 INFO:tasks.workunit.client.0.smithi094.stdout:3/238: dread - d9/d10/d27/d3a/f40 zero size 2024-04-06T03:40:29.811 INFO:tasks.workunit.client.1.smithi136.stdout:7/134: unlink d5/f7 0 2024-04-06T03:40:29.813 INFO:tasks.workunit.client.0.smithi094.stdout:7/208: mkdir d2/d6/dc/d12/d3a 0 2024-04-06T03:40:29.820 INFO:tasks.workunit.client.0.smithi094.stdout:4/246: fdatasync d0/d4/f9 0 2024-04-06T03:40:29.824 INFO:tasks.workunit.client.1.smithi136.stdout:3/116: dwrite d0/d4/fd [0,4194304] 0 2024-04-06T03:40:29.824 INFO:tasks.workunit.client.1.smithi136.stdout:3/117: creat d0/f25 x:0 0 0 2024-04-06T03:40:29.826 INFO:tasks.workunit.client.1.smithi136.stdout:5/197: dread d1/d16/f1b [0,4194304] 0 2024-04-06T03:40:29.826 INFO:tasks.workunit.client.0.smithi094.stdout:0/228: mknod d1/d9/dd/d18/d32/d42/c4b 0 2024-04-06T03:40:29.827 INFO:tasks.workunit.client.0.smithi094.stdout:4/247: dread d0/d4/f21 [0,4194304] 0 2024-04-06T03:40:29.828 INFO:tasks.workunit.client.0.smithi094.stdout:9/147: dwrite d2/dc/d10/f12 [4194304,4194304] 0 2024-04-06T03:40:29.828 INFO:tasks.workunit.client.0.smithi094.stdout:9/148: write d2/d2c/f35 [395210,26957] 0 2024-04-06T03:40:29.828 INFO:tasks.workunit.client.0.smithi094.stdout:9/149: chown d2 132971726 1 2024-04-06T03:40:29.828 INFO:tasks.workunit.client.0.smithi094.stdout:9/150: creat d2/d2c/f36 x:0 0 0 2024-04-06T03:40:29.828 INFO:tasks.workunit.client.0.smithi094.stdout:9/151: chown d2/dc/l1f 711635 1 2024-04-06T03:40:29.829 INFO:tasks.workunit.client.0.smithi094.stdout:2/237: chown d0/c16 2320 1 2024-04-06T03:40:29.830 INFO:tasks.workunit.client.0.smithi094.stdout:0/229: dread d1/d9/dd/d18/d1a/f25 [0,4194304] 0 2024-04-06T03:40:29.832 INFO:tasks.workunit.client.1.smithi136.stdout:1/158: mknod d12/d1b/d1e/d23/c2b 0 2024-04-06T03:40:29.832 INFO:tasks.workunit.client.1.smithi136.stdout:1/159: creat d12/f2c x:0 0 0 2024-04-06T03:40:29.832 INFO:tasks.workunit.client.0.smithi094.stdout:3/239: dwrite d9/f15 [0,4194304] 0 2024-04-06T03:40:29.832 INFO:tasks.workunit.client.0.smithi094.stdout:3/240: dread - d9/d1d/f4a zero size 2024-04-06T03:40:29.836 INFO:tasks.workunit.client.0.smithi094.stdout:7/209: dwrite d2/d6/dc/f15 [0,4194304] 0 2024-04-06T03:40:29.836 INFO:tasks.workunit.client.0.smithi094.stdout:7/210: fsync d2/d6/dc/dd/f28 0 2024-04-06T03:40:29.836 INFO:tasks.workunit.client.0.smithi094.stdout:7/211: chown d2/d6/dc/dd/df/f2d 135870 1 2024-04-06T03:40:29.836 INFO:tasks.workunit.client.0.smithi094.stdout:5/165: rename db/d19/d24/d2e/f25 to db/d19/f41 0 2024-04-06T03:40:29.837 INFO:tasks.workunit.client.1.smithi136.stdout:5/198: write d1/dc/d10/f11 [3894968,23235] 0 2024-04-06T03:40:29.839 INFO:tasks.workunit.client.1.smithi136.stdout:5/199: dread d1/d1f/f21 [0,4194304] 0 2024-04-06T03:40:29.840 INFO:tasks.workunit.client.0.smithi094.stdout:8/191: mknod d18/d2c/c42 0 2024-04-06T03:40:29.846 INFO:tasks.workunit.client.1.smithi136.stdout:6/132: getdents d2/de 0 2024-04-06T03:40:29.851 INFO:tasks.workunit.client.1.smithi136.stdout:8/175: mknod d3/de/c34 0 2024-04-06T03:40:29.857 INFO:tasks.workunit.client.0.smithi094.stdout:2/238: dwrite d0/d8/f11 [8388608,4194304] 0 2024-04-06T03:40:29.858 INFO:tasks.workunit.client.1.smithi136.stdout:2/250: truncate d3/d9/d10/d18/f2a 2546552 0 2024-04-06T03:40:29.860 INFO:tasks.workunit.client.1.smithi136.stdout:4/133: creat d1d/f38 x:0 0 0 2024-04-06T03:40:29.865 INFO:tasks.workunit.client.1.smithi136.stdout:7/135: mkdir d5/df/d11/d22/d2c 0 2024-04-06T03:40:29.865 INFO:tasks.workunit.client.0.smithi094.stdout:3/241: dwrite d9/d1e/d26/f2c [0,4194304] 0 2024-04-06T03:40:29.866 INFO:tasks.workunit.client.0.smithi094.stdout:3/242: fdatasync d9/d1d/f3c 0 2024-04-06T03:40:29.867 INFO:tasks.workunit.client.1.smithi136.stdout:2/251: read d3/d9/f12 [2913941,101119] 0 2024-04-06T03:40:29.869 INFO:tasks.workunit.client.1.smithi136.stdout:3/118: symlink d0/d4/d12/d19/d1b/l26 0 2024-04-06T03:40:29.869 INFO:tasks.workunit.client.1.smithi136.stdout:3/119: chown d0/l1f 697888 1 2024-04-06T03:40:29.873 INFO:tasks.workunit.client.0.smithi094.stdout:1/185: rmdir d0/d7/d1a 39 2024-04-06T03:40:29.875 INFO:tasks.workunit.client.1.smithi136.stdout:5/200: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:29.876 INFO:tasks.workunit.client.1.smithi136.stdout:1/160: symlink d12/d1b/d28/l2d 0 2024-04-06T03:40:29.876 INFO:tasks.workunit.client.1.smithi136.stdout:1/161: chown d12/c1d 114490712 1 2024-04-06T03:40:29.877 INFO:tasks.workunit.client.1.smithi136.stdout:6/133: dwrite d2/de/f2e [0,4194304] 0 2024-04-06T03:40:29.877 INFO:tasks.workunit.client.0.smithi094.stdout:7/212: dwrite d2/d6/dc/d12/f1c [0,4194304] 0 2024-04-06T03:40:29.877 INFO:tasks.workunit.client.0.smithi094.stdout:7/213: creat d2/d6/dc/dd/f3b x:0 0 0 2024-04-06T03:40:29.878 INFO:tasks.workunit.client.1.smithi136.stdout:7/136: dread d5/df/d11/f14 [0,4194304] 0 2024-04-06T03:40:29.878 INFO:tasks.workunit.client.1.smithi136.stdout:4/134: stat cb 0 2024-04-06T03:40:29.878 INFO:tasks.workunit.client.1.smithi136.stdout:4/135: dread - d1d/f2b zero size 2024-04-06T03:40:29.878 INFO:tasks.workunit.client.1.smithi136.stdout:4/136: truncate d1d/f29 92797 0 2024-04-06T03:40:29.879 INFO:tasks.workunit.client.0.smithi094.stdout:9/152: rmdir d2/dc/d21 39 2024-04-06T03:40:29.881 INFO:tasks.workunit.client.1.smithi136.stdout:2/252: write d3/d9/d10/d18/f48 [1572912,19315] 0 2024-04-06T03:40:29.881 INFO:tasks.workunit.client.1.smithi136.stdout:8/176: dwrite d3/f4 [0,4194304] 0 2024-04-06T03:40:29.881 INFO:tasks.workunit.client.1.smithi136.stdout:5/201: rename d1/dc/d15/c2a to d1/dc/d15/c3a 0 2024-04-06T03:40:29.882 INFO:tasks.workunit.client.0.smithi094.stdout:0/230: symlink d1/d9/dd/d18/d1e/l4c 0 2024-04-06T03:40:29.884 INFO:tasks.workunit.client.1.smithi136.stdout:1/162: symlink d12/d1b/d1e/d1f/l2e 0 2024-04-06T03:40:29.884 INFO:tasks.workunit.client.1.smithi136.stdout:1/163: dread - d12/f2c zero size 2024-04-06T03:40:29.889 INFO:tasks.workunit.client.0.smithi094.stdout:4/248: rename d0/d4/c2b to d0/d23/c52 0 2024-04-06T03:40:29.889 INFO:tasks.workunit.client.1.smithi136.stdout:3/120: dwrite d0/f23 [0,4194304] 0 2024-04-06T03:40:29.889 INFO:tasks.workunit.client.1.smithi136.stdout:3/121: fsync d0/d4/d12/f15 0 2024-04-06T03:40:29.890 INFO:tasks.workunit.client.0.smithi094.stdout:7/214: dread d2/d6/dc/d12/f1c [0,4194304] 0 2024-04-06T03:40:29.894 INFO:tasks.workunit.client.0.smithi094.stdout:8/192: mknod d18/d2c/c43 0 2024-04-06T03:40:29.901 INFO:tasks.workunit.client.1.smithi136.stdout:6/134: mknod d2/de/c34 0 2024-04-06T03:40:29.903 INFO:tasks.workunit.client.1.smithi136.stdout:7/137: rename d5/f9 to d5/df/d12/d1b/f2d 0 2024-04-06T03:40:29.903 INFO:tasks.workunit.client.1.smithi136.stdout:7/138: stat d5/df/d11/f2b 0 2024-04-06T03:40:29.903 INFO:tasks.workunit.client.0.smithi094.stdout:6/158: sync 2024-04-06T03:40:29.904 INFO:tasks.workunit.client.0.smithi094.stdout:6/159: write dc/d11/d1b/d24/f25 [943327,87746] 0 2024-04-06T03:40:29.904 INFO:tasks.workunit.client.0.smithi094.stdout:6/160: fsync dc/d11/f20 0 2024-04-06T03:40:29.904 INFO:tasks.workunit.client.0.smithi094.stdout:6/161: creat dc/f40 x:0 0 0 2024-04-06T03:40:29.904 INFO:tasks.workunit.client.0.smithi094.stdout:4/249: read d0/df/d13/f37 [1547712,35248] 0 2024-04-06T03:40:29.905 INFO:tasks.workunit.client.0.smithi094.stdout:9/153: dwrite d2/d2c/f36 [0,4194304] 0 2024-04-06T03:40:29.906 INFO:tasks.workunit.client.1.smithi136.stdout:4/137: dwrite d1d/d31/f33 [0,4194304] 0 2024-04-06T03:40:29.906 INFO:tasks.workunit.client.1.smithi136.stdout:4/138: read - d1d/f38 zero size 2024-04-06T03:40:29.907 INFO:tasks.workunit.client.0.smithi094.stdout:1/186: link d0/d1/c6 d0/d1/d16/d3f/c43 0 2024-04-06T03:40:29.909 INFO:tasks.workunit.client.1.smithi136.stdout:2/253: symlink d3/d9/d10/d18/l57 0 2024-04-06T03:40:29.910 INFO:tasks.workunit.client.1.smithi136.stdout:6/135: write d2/de/f10 [3003329,103385] 0 2024-04-06T03:40:29.911 INFO:tasks.workunit.client.0.smithi094.stdout:7/215: dwrite d2/d6/dc/d12/d29/f34 [0,4194304] 0 2024-04-06T03:40:29.914 INFO:tasks.workunit.client.0.smithi094.stdout:8/193: dwrite f13 [0,4194304] 0 2024-04-06T03:40:29.914 INFO:tasks.workunit.client.0.smithi094.stdout:8/194: truncate d18/d2c/f2e 921741 0 2024-04-06T03:40:29.914 INFO:tasks.workunit.client.0.smithi094.stdout:3/243: rename d9/lb to d9/d20/l51 0 2024-04-06T03:40:29.917 INFO:tasks.workunit.client.0.smithi094.stdout:2/239: mkdir d0/d18/d4d 0 2024-04-06T03:40:29.922 INFO:tasks.workunit.client.1.smithi136.stdout:5/202: link d1/ca d1/d16/c3b 0 2024-04-06T03:40:29.922 INFO:tasks.workunit.client.1.smithi136.stdout:5/203: fsync d1/dc/d1c/f2c 0 2024-04-06T03:40:29.924 INFO:tasks.workunit.client.0.smithi094.stdout:6/162: truncate dc/f29 3907182 0 2024-04-06T03:40:29.927 INFO:tasks.workunit.client.0.smithi094.stdout:2/240: write d0/dd/d1b/f3e [120163,84775] 0 2024-04-06T03:40:29.929 INFO:tasks.workunit.client.1.smithi136.stdout:0/140: sync 2024-04-06T03:40:29.929 INFO:tasks.workunit.client.1.smithi136.stdout:0/141: chown d1/d6 0 1 2024-04-06T03:40:29.929 INFO:tasks.workunit.client.1.smithi136.stdout:0/142: dread - d1/f29 zero size 2024-04-06T03:40:29.930 INFO:tasks.workunit.client.1.smithi136.stdout:3/122: symlink d0/d4/l27 0 2024-04-06T03:40:29.931 INFO:tasks.workunit.client.1.smithi136.stdout:4/139: dwrite d1d/f27 [0,4194304] 0 2024-04-06T03:40:29.932 INFO:tasks.workunit.client.1.smithi136.stdout:6/136: dwrite d2/d25/f17 [0,4194304] 0 2024-04-06T03:40:29.932 INFO:tasks.workunit.client.1.smithi136.stdout:6/137: read d2/f21 [3635174,32536] 0 2024-04-06T03:40:29.932 INFO:tasks.workunit.client.0.smithi094.stdout:9/154: unlink d2/d2c/f35 0 2024-04-06T03:40:29.932 INFO:tasks.workunit.client.0.smithi094.stdout:9/155: truncate d2/f4 5775322 0 2024-04-06T03:40:29.934 INFO:tasks.workunit.client.1.smithi136.stdout:1/164: rename d12/d1b/d1e/d27/l2a to d12/d1b/d28/l2f 0 2024-04-06T03:40:29.938 INFO:tasks.workunit.client.1.smithi136.stdout:7/139: truncate d5/fd 5488236 0 2024-04-06T03:40:29.938 INFO:tasks.workunit.client.1.smithi136.stdout:7/140: dread - d5/f27 zero size 2024-04-06T03:40:29.939 INFO:tasks.workunit.client.1.smithi136.stdout:7/141: creat d5/f2e x:0 0 0 2024-04-06T03:40:29.939 INFO:tasks.workunit.client.1.smithi136.stdout:7/142: fsync d5/d21/f24 0 2024-04-06T03:40:29.939 INFO:tasks.workunit.client.1.smithi136.stdout:7/143: chown d5/df/d12/c18 3791910 1 2024-04-06T03:40:29.939 INFO:tasks.workunit.client.1.smithi136.stdout:7/144: chown d5/l1e 70 1 2024-04-06T03:40:29.939 INFO:tasks.workunit.client.1.smithi136.stdout:7/145: dread - d5/f2e zero size 2024-04-06T03:40:29.939 INFO:tasks.workunit.client.1.smithi136.stdout:7/146: creat d5/df/d11/d22/f2f x:0 0 0 2024-04-06T03:40:29.939 INFO:tasks.workunit.client.1.smithi136.stdout:7/147: write d5/f2e [395447,9723] 0 2024-04-06T03:40:29.940 INFO:tasks.workunit.client.0.smithi094.stdout:1/187: unlink d0/d4/d1b/f2c 0 2024-04-06T03:40:29.940 INFO:tasks.workunit.client.0.smithi094.stdout:9/156: dread d2/dc/d10/f1b [0,4194304] 0 2024-04-06T03:40:29.942 INFO:tasks.workunit.client.0.smithi094.stdout:7/216: mknod d2/d6/dc/c3c 0 2024-04-06T03:40:29.944 INFO:tasks.workunit.client.1.smithi136.stdout:9/154: sync 2024-04-06T03:40:29.945 INFO:tasks.workunit.client.1.smithi136.stdout:2/254: mkdir d3/d9/d10/d18/d1d/d3c/d58 0 2024-04-06T03:40:29.947 INFO:tasks.workunit.client.1.smithi136.stdout:0/143: dwrite d1/d6/d2c/f20 [0,4194304] 0 2024-04-06T03:40:29.950 INFO:tasks.workunit.client.0.smithi094.stdout:8/195: rename d18/d30/l39 to d18/d30/l44 0 2024-04-06T03:40:29.950 INFO:tasks.workunit.client.0.smithi094.stdout:8/196: write f14 [6508089,118629] 0 2024-04-06T03:40:29.951 INFO:tasks.workunit.client.1.smithi136.stdout:7/148: dread f1 [0,4194304] 0 2024-04-06T03:40:29.952 INFO:tasks.workunit.client.1.smithi136.stdout:7/149: stat d5/f2e 0 2024-04-06T03:40:29.952 INFO:tasks.workunit.client.1.smithi136.stdout:5/204: link d1/dc/d15/c22 d1/dc/c3c 0 2024-04-06T03:40:29.953 INFO:tasks.workunit.client.0.smithi094.stdout:0/231: rename d1/d9/f14 to d1/d9/dd/d18/d1a/f4d 0 2024-04-06T03:40:29.954 INFO:tasks.workunit.client.1.smithi136.stdout:0/144: read d1/d6/d2c/d1a/d1c/f23 [581010,88550] 0 2024-04-06T03:40:29.955 INFO:tasks.workunit.client.0.smithi094.stdout:3/244: mkdir d9/d20/d3e/d52 0 2024-04-06T03:40:29.956 INFO:tasks.workunit.client.1.smithi136.stdout:0/145: dread d1/d6/f11 [0,4194304] 0 2024-04-06T03:40:29.956 INFO:tasks.workunit.client.1.smithi136.stdout:0/146: write d1/f18 [188504,127663] 0 2024-04-06T03:40:29.957 INFO:tasks.workunit.client.0.smithi094.stdout:6/163: creat dc/d3a/f41 x:0 0 0 2024-04-06T03:40:29.958 INFO:tasks.workunit.client.1.smithi136.stdout:4/140: creat d1d/d2f/f39 x:0 0 0 2024-04-06T03:40:29.961 INFO:tasks.workunit.client.0.smithi094.stdout:2/241: truncate d0/dd/ff 789140 0 2024-04-06T03:40:29.965 INFO:tasks.workunit.client.0.smithi094.stdout:1/188: unlink d0/d1/l11 0 2024-04-06T03:40:29.965 INFO:tasks.workunit.client.0.smithi094.stdout:1/189: readlink d0/d1/l21 0 2024-04-06T03:40:29.967 INFO:tasks.workunit.client.1.smithi136.stdout:6/138: creat d2/de/f35 x:0 0 0 2024-04-06T03:40:29.970 INFO:tasks.workunit.client.1.smithi136.stdout:3/123: rename d0/d4/d12/l17 to d0/l28 0 2024-04-06T03:40:29.974 INFO:tasks.workunit.client.1.smithi136.stdout:1/165: creat d12/d1b/d1e/d1f/f30 x:0 0 0 2024-04-06T03:40:29.974 INFO:tasks.workunit.client.1.smithi136.stdout:1/166: chown c5 4741 1 2024-04-06T03:40:29.976 INFO:tasks.workunit.client.0.smithi094.stdout:2/242: dwrite d0/dd/d19/d31/f42 [0,4194304] 0 2024-04-06T03:40:29.979 INFO:tasks.workunit.client.1.smithi136.stdout:8/177: truncate d3/f4 3547312 0 2024-04-06T03:40:29.983 INFO:tasks.workunit.client.1.smithi136.stdout:2/255: symlink d3/d45/l59 0 2024-04-06T03:40:29.984 INFO:tasks.workunit.client.1.smithi136.stdout:3/124: dwrite d0/d4/d12/f20 [0,4194304] 0 2024-04-06T03:40:29.987 INFO:tasks.workunit.client.1.smithi136.stdout:5/205: rename d1/dc/d15/c22 to d1/dc/d15/c3d 0 2024-04-06T03:40:29.987 INFO:tasks.workunit.client.1.smithi136.stdout:5/206: fsync d1/dc/d1c/f2c 0 2024-04-06T03:40:29.988 INFO:tasks.workunit.client.1.smithi136.stdout:5/207: fsync d1/dc/d1c/f2c 0 2024-04-06T03:40:29.988 INFO:tasks.workunit.client.0.smithi094.stdout:2/243: dread d0/d8/fe [0,4194304] 0 2024-04-06T03:40:29.991 INFO:tasks.workunit.client.1.smithi136.stdout:5/208: read d1/dc/d1c/f34 [288406,130620] 0 2024-04-06T03:40:29.993 INFO:tasks.workunit.client.1.smithi136.stdout:4/141: creat d1d/d31/f3a x:0 0 0 2024-04-06T03:40:29.993 INFO:tasks.workunit.client.1.smithi136.stdout:6/139: rmdir d2/d7/d13 39 2024-04-06T03:40:29.994 INFO:tasks.workunit.client.0.smithi094.stdout:0/232: getdents d1 0 2024-04-06T03:40:29.996 INFO:tasks.workunit.client.0.smithi094.stdout:0/233: dread d1/f7 [0,4194304] 0 2024-04-06T03:40:29.997 INFO:tasks.workunit.client.1.smithi136.stdout:9/155: rename l10 to d17/d19/l33 0 2024-04-06T03:40:29.998 INFO:tasks.workunit.client.0.smithi094.stdout:3/245: symlink d9/d1d/l53 0 2024-04-06T03:40:29.998 INFO:tasks.workunit.client.0.smithi094.stdout:3/246: truncate d9/d1d/f42 572073 0 2024-04-06T03:40:30.001 INFO:tasks.workunit.client.0.smithi094.stdout:6/164: unlink dc/f10 0 2024-04-06T03:40:30.001 INFO:tasks.workunit.client.0.smithi094.stdout:6/165: fdatasync dc/d11/d1b/d24/f32 0 2024-04-06T03:40:30.004 INFO:tasks.workunit.client.0.smithi094.stdout:1/190: symlink d0/d7/d2d/l44 0 2024-04-06T03:40:30.006 INFO:tasks.workunit.client.1.smithi136.stdout:8/178: dwrite d3/da/f19 [0,4194304] 0 2024-04-06T03:40:30.006 INFO:tasks.workunit.client.1.smithi136.stdout:8/179: fsync d3/de/ff 0 2024-04-06T03:40:30.008 INFO:tasks.workunit.client.0.smithi094.stdout:7/217: getdents d2/d6 0 2024-04-06T03:40:30.008 INFO:tasks.workunit.client.0.smithi094.stdout:7/218: creat d2/d6/dc/d12/d29/d35/f3d x:0 0 0 2024-04-06T03:40:30.008 INFO:tasks.workunit.client.0.smithi094.stdout:7/219: readlink d2/d6/dc/l13 0 2024-04-06T03:40:30.011 INFO:tasks.workunit.client.1.smithi136.stdout:2/256: mknod d3/d9/c5a 0 2024-04-06T03:40:30.012 INFO:tasks.workunit.client.0.smithi094.stdout:2/244: dwrite d0/dd/d19/d31/f42 [0,4194304] 0 2024-04-06T03:40:30.016 INFO:tasks.workunit.client.0.smithi094.stdout:4/250: rename d0/df/c33 to d0/d4/c53 0 2024-04-06T03:40:30.016 INFO:tasks.workunit.client.0.smithi094.stdout:4/251: readlink d0/l28 0 2024-04-06T03:40:30.017 INFO:tasks.workunit.client.1.smithi136.stdout:9/156: dwrite d17/d19/f2f [0,4194304] 0 2024-04-06T03:40:30.017 INFO:tasks.workunit.client.1.smithi136.stdout:9/157: dread - d17/d19/f27 zero size 2024-04-06T03:40:30.017 INFO:tasks.workunit.client.1.smithi136.stdout:8/180: dread d3/de/d1b/f1d [0,4194304] 0 2024-04-06T03:40:30.018 INFO:tasks.workunit.client.0.smithi094.stdout:4/252: write d0/d23/f32 [310024,82813] 0 2024-04-06T03:40:30.018 INFO:tasks.workunit.client.0.smithi094.stdout:4/253: creat d0/d15/d3a/f54 x:0 0 0 2024-04-06T03:40:30.019 INFO:tasks.workunit.client.0.smithi094.stdout:0/234: mkdir d1/d10/d4e 0 2024-04-06T03:40:30.020 INFO:tasks.workunit.client.0.smithi094.stdout:0/235: dread d1/d9/dd/d18/f31 [0,4194304] 0 2024-04-06T03:40:30.021 INFO:tasks.workunit.client.1.smithi136.stdout:3/125: creat d0/d4/d12/d19/f29 x:0 0 0 2024-04-06T03:40:30.021 INFO:tasks.workunit.client.1.smithi136.stdout:3/126: readlink d0/lf 0 2024-04-06T03:40:30.022 INFO:tasks.workunit.client.1.smithi136.stdout:3/127: write d0/f5 [1409264,18591] 0 2024-04-06T03:40:30.025 INFO:tasks.workunit.client.0.smithi094.stdout:1/191: link d0/d1/l15 d0/d1/d16/d24/d30/l45 0 2024-04-06T03:40:30.027 INFO:tasks.workunit.client.1.smithi136.stdout:0/147: rmdir d1/d6 39 2024-04-06T03:40:30.031 INFO:tasks.workunit.client.1.smithi136.stdout:5/209: mknod d1/dc/c3e 0 2024-04-06T03:40:30.033 INFO:tasks.workunit.client.0.smithi094.stdout:7/220: truncate d2/d6/dc/dd/f31 644680 0 2024-04-06T03:40:30.035 INFO:tasks.workunit.client.1.smithi136.stdout:6/140: mknod d2/de/c36 0 2024-04-06T03:40:30.035 INFO:tasks.workunit.client.1.smithi136.stdout:6/141: chown d2/l19 131510394 1 2024-04-06T03:40:30.037 INFO:tasks.workunit.client.0.smithi094.stdout:2/245: truncate d0/d8/d1d/f38 2704580 0 2024-04-06T03:40:30.038 INFO:tasks.workunit.client.1.smithi136.stdout:7/150: rename d5/df/d12/f19 to d5/df/d12/d1b/f30 0 2024-04-06T03:40:30.041 INFO:tasks.workunit.client.0.smithi094.stdout:9/157: rename d2/dc/d32 to d2/d2c/d33/d37 0 2024-04-06T03:40:30.044 INFO:tasks.workunit.client.1.smithi136.stdout:9/158: dwrite f4 [0,4194304] 0 2024-04-06T03:40:30.044 INFO:tasks.workunit.client.1.smithi136.stdout:9/159: readlink d17/d20/d25/l2b 0 2024-04-06T03:40:30.044 INFO:tasks.workunit.client.1.smithi136.stdout:8/181: dwrite d3/f2f [0,4194304] 0 2024-04-06T03:40:30.045 INFO:tasks.workunit.client.1.smithi136.stdout:8/182: dread d3/da/f13 [0,4194304] 0 2024-04-06T03:40:30.045 INFO:tasks.workunit.client.1.smithi136.stdout:8/183: chown d3/de/d1b/c20 3 1 2024-04-06T03:40:30.046 INFO:tasks.workunit.client.1.smithi136.stdout:2/257: link d3/d15/d19/d4e/l56 d3/d9/d10/l5b 0 2024-04-06T03:40:30.047 INFO:tasks.workunit.client.0.smithi094.stdout:3/247: getdents d9/d10/d27/d3a 0 2024-04-06T03:40:30.050 INFO:tasks.workunit.client.1.smithi136.stdout:2/258: dread d3/d15/d19/f1b [0,4194304] 0 2024-04-06T03:40:30.051 INFO:tasks.workunit.client.0.smithi094.stdout:4/254: dwrite d0/df/d1e/f24 [0,4194304] 0 2024-04-06T03:40:30.051 INFO:tasks.workunit.client.0.smithi094.stdout:4/255: chown d0/d15/d3a/c41 117027956 1 2024-04-06T03:40:30.051 INFO:tasks.workunit.client.0.smithi094.stdout:4/256: fdatasync d0/f29 0 2024-04-06T03:40:30.052 INFO:tasks.workunit.client.1.smithi136.stdout:3/128: symlink d0/l2a 0 2024-04-06T03:40:30.054 INFO:tasks.workunit.client.0.smithi094.stdout:8/197: rename d18/d30/l44 to d18/d2c/l45 0 2024-04-06T03:40:30.056 INFO:tasks.workunit.client.0.smithi094.stdout:7/221: dwrite d2/d6/dc/d12/d29/d35/f39 [0,4194304] 0 2024-04-06T03:40:30.056 INFO:tasks.workunit.client.0.smithi094.stdout:7/222: chown d2/d6/dc/d12/d29/f34 446 1 2024-04-06T03:40:30.056 INFO:tasks.workunit.client.1.smithi136.stdout:0/148: link d1/d6/lf d1/d2e/l2f 0 2024-04-06T03:40:30.057 INFO:tasks.workunit.client.0.smithi094.stdout:2/246: dwrite d0/d8/d27/d2b/f2d [0,4194304] 0 2024-04-06T03:40:30.057 INFO:tasks.workunit.client.0.smithi094.stdout:2/247: truncate d0/d18/f35 989912 0 2024-04-06T03:40:30.059 INFO:tasks.workunit.client.1.smithi136.stdout:5/210: truncate d1/dc/d10/f28 1599928 0 2024-04-06T03:40:30.059 INFO:tasks.workunit.client.1.smithi136.stdout:5/211: readlink d1/dc/d1c/l23 0 2024-04-06T03:40:30.060 INFO:tasks.workunit.client.0.smithi094.stdout:4/257: stat d0/df/d1e/l2e 0 2024-04-06T03:40:30.062 INFO:tasks.workunit.client.1.smithi136.stdout:6/142: mkdir d2/d7/d37 0 2024-04-06T03:40:30.062 INFO:tasks.workunit.client.1.smithi136.stdout:6/143: chown d2/d7/d26/d30/l2b 157102 1 2024-04-06T03:40:30.063 INFO:tasks.workunit.client.0.smithi094.stdout:4/258: dread d0/df/d13/f18 [0,4194304] 0 2024-04-06T03:40:30.064 INFO:tasks.workunit.client.0.smithi094.stdout:6/166: rename l1 to dc/l42 0 2024-04-06T03:40:30.066 INFO:tasks.workunit.client.1.smithi136.stdout:7/151: link d5/c6 d5/df/d12/d1b/c31 0 2024-04-06T03:40:30.070 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:40:30.070 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:40:30.070 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:40:30.070 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:40:30.070 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:40:30.070 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:40:30.070 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.071 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.071 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.071 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.071 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.071 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.071 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.071 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.071 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:29 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.074 INFO:tasks.workunit.client.1.smithi136.stdout:9/160: unlink d17/d19/c1a 0 2024-04-06T03:40:30.074 INFO:tasks.workunit.client.1.smithi136.stdout:9/161: write f16 [1507625,106212] 0 2024-04-06T03:40:30.075 INFO:tasks.workunit.client.1.smithi136.stdout:6/144: write d2/de/f27 [4101457,98075] 0 2024-04-06T03:40:30.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/167: dread dc/d11/d22/f39 [0,4194304] 0 2024-04-06T03:40:30.075 INFO:tasks.workunit.client.0.smithi094.stdout:6/168: write dc/d11/d1b/d24/f32 [661947,52601] 0 2024-04-06T03:40:30.078 INFO:tasks.workunit.client.0.smithi094.stdout:6/169: write dc/fd [269855,36331] 0 2024-04-06T03:40:30.078 INFO:tasks.workunit.client.0.smithi094.stdout:6/170: chown dc/d11/d1b/f2f 17706 1 2024-04-06T03:40:30.080 INFO:tasks.workunit.client.0.smithi094.stdout:4/259: dwrite d0/d15/d3a/d3e/d49/f4d [0,4194304] 0 2024-04-06T03:40:30.080 INFO:tasks.workunit.client.0.smithi094.stdout:4/260: chown d0/d4/f51 0 1 2024-04-06T03:40:30.081 INFO:tasks.workunit.client.1.smithi136.stdout:8/184: rename d3/de/d1b to d3/da/d26/d35 0 2024-04-06T03:40:30.083 INFO:tasks.workunit.client.0.smithi094.stdout:2/248: mknod d0/dd/d19/c4e 0 2024-04-06T03:40:30.084 INFO:tasks.workunit.client.1.smithi136.stdout:6/145: read d2/f9 [3351809,107120] 0 2024-04-06T03:40:30.084 INFO:tasks.workunit.client.1.smithi136.stdout:6/146: stat d2/de/l32 0 2024-04-06T03:40:30.085 INFO:tasks.workunit.client.1.smithi136.stdout:2/259: rmdir d3/d15/d19/d4e 39 2024-04-06T03:40:30.089 INFO:tasks.workunit.client.0.smithi094.stdout:9/158: link d2/l3 d2/dc/d21/d23/d2f/l38 0 2024-04-06T03:40:30.089 INFO:tasks.workunit.client.0.smithi094.stdout:9/159: readlink d2/dc/l17 0 2024-04-06T03:40:30.090 INFO:tasks.workunit.client.1.smithi136.stdout:8/185: read d3/da/f19 [3115268,129958] 0 2024-04-06T03:40:30.090 INFO:tasks.workunit.client.1.smithi136.stdout:8/186: fdatasync d3/de/f27 0 2024-04-06T03:40:30.090 INFO:tasks.workunit.client.1.smithi136.stdout:8/187: write d3/de/f27 [538072,91529] 0 2024-04-06T03:40:30.090 INFO:tasks.workunit.client.1.smithi136.stdout:8/188: fsync d3/f9 0 2024-04-06T03:40:30.092 INFO:tasks.workunit.client.0.smithi094.stdout:0/236: rename d1/d9/dd/d18/d1d/f40 to d1/d10/f4f 0 2024-04-06T03:40:30.092 INFO:tasks.workunit.client.0.smithi094.stdout:9/160: dread d2/dc/d10/f1b [0,4194304] 0 2024-04-06T03:40:30.092 INFO:tasks.workunit.client.0.smithi094.stdout:4/261: dwrite d0/d15/d3a/d3e/d49/f4d [0,4194304] 0 2024-04-06T03:40:30.096 INFO:tasks.workunit.client.0.smithi094.stdout:6/171: creat dc/d11/d1b/d24/f43 x:0 0 0 2024-04-06T03:40:30.097 INFO:tasks.workunit.client.0.smithi094.stdout:6/172: write fa [902580,106335] 0 2024-04-06T03:40:30.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.101 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:29 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:30.104 INFO:tasks.workunit.client.1.smithi136.stdout:5/212: mknod d1/c3f 0 2024-04-06T03:40:30.105 INFO:tasks.workunit.client.0.smithi094.stdout:2/249: mkdir d0/d8/d4f 0 2024-04-06T03:40:30.105 INFO:tasks.workunit.client.0.smithi094.stdout:2/250: readlink d0/d8/d27/d2b/l48 0 2024-04-06T03:40:30.107 INFO:tasks.workunit.client.1.smithi136.stdout:9/162: mkdir d17/d21/d29/d34 0 2024-04-06T03:40:30.108 INFO:tasks.workunit.client.0.smithi094.stdout:0/237: mknod d1/d9/dd/c50 0 2024-04-06T03:40:30.108 INFO:tasks.workunit.client.0.smithi094.stdout:6/173: dwrite dc/d11/d1b/f2f [0,4194304] 0 2024-04-06T03:40:30.112 INFO:tasks.workunit.client.0.smithi094.stdout:1/192: rename d0/d4/f29 to d0/d7/f46 0 2024-04-06T03:40:30.112 INFO:tasks.workunit.client.0.smithi094.stdout:4/262: dwrite d0/df/d1e/f2a [4194304,4194304] 0 2024-04-06T03:40:30.115 INFO:tasks.workunit.client.1.smithi136.stdout:7/152: rename d5/df/d1d/f25 to d5/f32 0 2024-04-06T03:40:30.115 INFO:tasks.workunit.client.1.smithi136.stdout:7/153: rename d5/df/d12 to d5/df/d12/d33 22 2024-04-06T03:40:30.121 INFO:tasks.workunit.client.0.smithi094.stdout:0/238: symlink d1/d10/d4e/l51 0 2024-04-06T03:40:30.121 INFO:tasks.workunit.client.0.smithi094.stdout:0/239: dread - d1/d9/dd/d18/d1a/f46 zero size 2024-04-06T03:40:30.123 INFO:tasks.workunit.client.1.smithi136.stdout:6/147: creat d2/d25/f38 x:0 0 0 2024-04-06T03:40:30.123 INFO:tasks.workunit.client.1.smithi136.stdout:2/260: symlink d3/d15/d19/d4e/l5c 0 2024-04-06T03:40:30.123 INFO:tasks.workunit.client.1.smithi136.stdout:6/148: write d2/d7/f1b [502695,13686] 0 2024-04-06T03:40:30.123 INFO:tasks.workunit.client.1.smithi136.stdout:6/149: fsync d2/f21 0 2024-04-06T03:40:30.123 INFO:tasks.workunit.client.1.smithi136.stdout:6/150: write d2/de/f10 [2714897,42657] 0 2024-04-06T03:40:30.129 INFO:tasks.workunit.client.1.smithi136.stdout:1/167: sync 2024-04-06T03:40:30.129 INFO:tasks.workunit.client.1.smithi136.stdout:1/168: write d12/d1b/d1e/d1f/f30 [53310,119281] 0 2024-04-06T03:40:30.129 INFO:tasks.workunit.client.1.smithi136.stdout:1/169: chown c8 2648568 1 2024-04-06T03:40:30.129 INFO:tasks.workunit.client.1.smithi136.stdout:1/170: truncate d12/f24 844389 0 2024-04-06T03:40:30.129 INFO:tasks.workunit.client.1.smithi136.stdout:1/171: readlink d12/d1b/d1e/d23/l25 0 2024-04-06T03:40:30.130 INFO:tasks.workunit.client.0.smithi094.stdout:6/174: dwrite dc/d11/d1b/d24/f2e [0,4194304] 0 2024-04-06T03:40:30.131 INFO:tasks.workunit.client.0.smithi094.stdout:6/175: chown dc/d11/d1b/d24/f25 635 1 2024-04-06T03:40:30.132 INFO:tasks.workunit.client.0.smithi094.stdout:8/198: rename c5 to d18/d30/c46 0 2024-04-06T03:40:30.132 INFO:tasks.workunit.client.0.smithi094.stdout:8/199: stat d18/f1a 0 2024-04-06T03:40:30.132 INFO:tasks.workunit.client.0.smithi094.stdout:8/200: creat d18/d2c/f47 x:0 0 0 2024-04-06T03:40:30.132 INFO:tasks.workunit.client.1.smithi136.stdout:9/163: dwrite f4 [0,4194304] 0 2024-04-06T03:40:30.133 INFO:tasks.workunit.client.1.smithi136.stdout:9/164: write d17/d21/d29/f2a [206455,61691] 0 2024-04-06T03:40:30.133 INFO:tasks.workunit.client.1.smithi136.stdout:0/149: mknod d1/c30 0 2024-04-06T03:40:30.133 INFO:tasks.workunit.client.0.smithi094.stdout:1/193: dwrite d0/d1/d16/f3a [0,4194304] 0 2024-04-06T03:40:30.133 INFO:tasks.workunit.client.0.smithi094.stdout:1/194: fsync d0/d4/fc 0 2024-04-06T03:40:30.134 INFO:tasks.workunit.client.0.smithi094.stdout:2/251: dwrite d0/d18/f22 [0,4194304] 0 2024-04-06T03:40:30.135 INFO:tasks.workunit.client.0.smithi094.stdout:4/263: link d0/d15/d3a/l4f d0/df/d13/l55 0 2024-04-06T03:40:30.138 INFO:tasks.workunit.client.1.smithi136.stdout:5/213: link d1/dc/d10/f11 d1/f40 0 2024-04-06T03:40:30.138 INFO:tasks.workunit.client.1.smithi136.stdout:5/214: chown d1/dc/d1c/f34 103 1 2024-04-06T03:40:30.138 INFO:tasks.workunit.client.1.smithi136.stdout:5/215: write d1/d1f/f21 [78175,29752] 0 2024-04-06T03:40:30.139 INFO:tasks.workunit.client.0.smithi094.stdout:0/240: creat d1/d9/dd/d18/d1e/d22/d35/f52 x:0 0 0 2024-04-06T03:40:30.140 INFO:tasks.workunit.client.1.smithi136.stdout:5/216: dread d1/dc/d1c/f2c [0,4194304] 0 2024-04-06T03:40:30.143 INFO:tasks.workunit.client.1.smithi136.stdout:2/261: dwrite d3/d9/d10/d18/f48 [0,4194304] 0 2024-04-06T03:40:30.143 INFO:tasks.workunit.client.1.smithi136.stdout:8/189: rename d3/c17 to d3/de/d18/c36 0 2024-04-06T03:40:30.143 INFO:tasks.workunit.client.1.smithi136.stdout:8/190: write d3/f21 [972418,61445] 0 2024-04-06T03:40:30.144 INFO:tasks.workunit.client.1.smithi136.stdout:6/151: truncate d2/d25/f17 4082158 0 2024-04-06T03:40:30.147 INFO:tasks.workunit.client.1.smithi136.stdout:1/172: creat d12/d1b/d1e/d23/f31 x:0 0 0 2024-04-06T03:40:30.150 INFO:tasks.workunit.client.1.smithi136.stdout:2/262: read d3/d9/fe [1368850,31984] 0 2024-04-06T03:40:30.150 INFO:tasks.workunit.client.1.smithi136.stdout:2/263: dread - d3/d15/f50 zero size 2024-04-06T03:40:30.150 INFO:tasks.workunit.client.1.smithi136.stdout:2/264: write d3/d9/d10/f22 [701429,34172] 0 2024-04-06T03:40:30.151 INFO:tasks.workunit.client.0.smithi094.stdout:8/201: mkdir d18/d2c/d37/d48 0 2024-04-06T03:40:30.152 INFO:tasks.workunit.client.1.smithi136.stdout:1/173: dread d12/f16 [0,4194304] 0 2024-04-06T03:40:30.152 INFO:tasks.workunit.client.1.smithi136.stdout:1/174: creat d12/d1b/f32 x:0 0 0 2024-04-06T03:40:30.153 INFO:tasks.workunit.client.1.smithi136.stdout:4/142: sync 2024-04-06T03:40:30.157 INFO:tasks.workunit.client.1.smithi136.stdout:1/175: dread d12/f16 [0,4194304] 0 2024-04-06T03:40:30.158 INFO:tasks.workunit.client.0.smithi094.stdout:1/195: truncate d0/d7/f26 1830056 0 2024-04-06T03:40:30.159 INFO:tasks.workunit.client.1.smithi136.stdout:3/129: sync 2024-04-06T03:40:30.160 INFO:tasks.workunit.client.1.smithi136.stdout:9/165: link d17/d19/f2f d17/d19/f35 0 2024-04-06T03:40:30.163 INFO:tasks.workunit.client.1.smithi136.stdout:0/150: creat d1/d6/d2c/d1a/d1c/f31 x:0 0 0 2024-04-06T03:40:30.166 INFO:tasks.workunit.client.1.smithi136.stdout:6/152: dwrite d2/de/f27 [0,4194304] 0 2024-04-06T03:40:30.166 INFO:tasks.workunit.client.1.smithi136.stdout:6/153: dread - d2/d7/f31 zero size 2024-04-06T03:40:30.167 INFO:tasks.workunit.client.0.smithi094.stdout:4/264: mknod d0/d23/c56 0 2024-04-06T03:40:30.167 INFO:tasks.workunit.client.0.smithi094.stdout:4/265: read d0/f8 [2860161,56769] 0 2024-04-06T03:40:30.169 INFO:tasks.workunit.client.0.smithi094.stdout:0/241: mknod d1/c53 0 2024-04-06T03:40:30.169 INFO:tasks.workunit.client.0.smithi094.stdout:0/242: chown d1/d9/dd/d18/d1e/d22 260604432 1 2024-04-06T03:40:30.169 INFO:tasks.workunit.client.0.smithi094.stdout:0/243: readlink d1/d9/dd/d18/l47 0 2024-04-06T03:40:30.169 INFO:tasks.workunit.client.0.smithi094.stdout:0/244: fsync d1/f23 0 2024-04-06T03:40:30.171 INFO:tasks.workunit.client.1.smithi136.stdout:2/265: dwrite d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:30.171 INFO:tasks.workunit.client.1.smithi136.stdout:2/266: chown d3/c46 0 1 2024-04-06T03:40:30.172 INFO:tasks.workunit.client.1.smithi136.stdout:5/217: creat d1/dc/d15/d20/f41 x:0 0 0 2024-04-06T03:40:30.173 INFO:tasks.workunit.client.0.smithi094.stdout:9/161: rename d2/c20 to d2/d2c/d33/d37/c39 0 2024-04-06T03:40:30.175 INFO:tasks.workunit.client.1.smithi136.stdout:8/191: symlink d3/de/d18/l37 0 2024-04-06T03:40:30.176 INFO:tasks.workunit.client.0.smithi094.stdout:8/202: truncate f11 5855481 0 2024-04-06T03:40:30.181 INFO:tasks.workunit.client.0.smithi094.stdout:4/266: mkdir d0/df/d57 0 2024-04-06T03:40:30.182 INFO:tasks.workunit.client.1.smithi136.stdout:9/166: dwrite fb [0,4194304] 0 2024-04-06T03:40:30.183 INFO:tasks.workunit.client.0.smithi094.stdout:6/176: dwrite dc/d11/d1b/d24/f2e [4194304,4194304] 0 2024-04-06T03:40:30.185 INFO:tasks.workunit.client.1.smithi136.stdout:1/176: mknod d12/d1b/d1e/c33 0 2024-04-06T03:40:30.185 INFO:tasks.workunit.client.1.smithi136.stdout:1/177: stat d12/d1b/d1e/c33 0 2024-04-06T03:40:30.186 INFO:tasks.workunit.client.1.smithi136.stdout:1/178: fdatasync f11 0 2024-04-06T03:40:30.187 INFO:tasks.workunit.client.1.smithi136.stdout:5/218: dread d1/dc/d10/d1e/f2f [0,4194304] 0 2024-04-06T03:40:30.187 INFO:tasks.workunit.client.1.smithi136.stdout:5/219: creat d1/dc/d15/d20/f42 x:0 0 0 2024-04-06T03:40:30.187 INFO:tasks.workunit.client.1.smithi136.stdout:5/220: readlink d1/dc/d1c/l23 0 2024-04-06T03:40:30.188 INFO:tasks.workunit.client.0.smithi094.stdout:1/196: dwrite d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:30.188 INFO:tasks.workunit.client.1.smithi136.stdout:3/130: symlink d0/l2b 0 2024-04-06T03:40:30.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/245: unlink d1/f7 0 2024-04-06T03:40:30.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/246: fdatasync d1/d9/dd/d18/d1a/f4d 0 2024-04-06T03:40:30.191 INFO:tasks.workunit.client.1.smithi136.stdout:2/267: dwrite d3/d9/f12 [0,4194304] 0 2024-04-06T03:40:30.192 INFO:tasks.workunit.client.1.smithi136.stdout:0/151: symlink d1/d6/d2c/d1a/l32 0 2024-04-06T03:40:30.192 INFO:tasks.workunit.client.1.smithi136.stdout:0/152: write d1/d6/d2c/d1a/f2a [89660,116254] 0 2024-04-06T03:40:30.193 INFO:tasks.workunit.client.0.smithi094.stdout:4/267: dread d0/df/f1b [0,4194304] 0 2024-04-06T03:40:30.194 INFO:tasks.workunit.client.1.smithi136.stdout:2/268: dread d3/d15/d19/f39 [0,4194304] 0 2024-04-06T03:40:30.194 INFO:tasks.workunit.client.0.smithi094.stdout:9/162: creat d2/d2c/d33/d37/f3a x:0 0 0 2024-04-06T03:40:30.200 INFO:tasks.workunit.client.0.smithi094.stdout:2/252: rename d0/dd/d19/c4e to d0/dd/d3d/d49/c50 0 2024-04-06T03:40:30.202 INFO:tasks.workunit.client.1.smithi136.stdout:2/269: dread d3/d9/d10/d18/d1d/f35 [0,4194304] 0 2024-04-06T03:40:30.202 INFO:tasks.workunit.client.0.smithi094.stdout:2/253: dread d0/d18/f24 [0,4194304] 0 2024-04-06T03:40:30.203 INFO:tasks.workunit.client.1.smithi136.stdout:6/154: rename d2/de/l32 to d2/l39 0 2024-04-06T03:40:30.203 INFO:tasks.workunit.client.1.smithi136.stdout:6/155: truncate d2/de/f2f 195801 0 2024-04-06T03:40:30.204 INFO:tasks.workunit.client.0.smithi094.stdout:8/203: mkdir d18/d3c/d49 0 2024-04-06T03:40:30.204 INFO:tasks.workunit.client.0.smithi094.stdout:8/204: readlink l15 0 2024-04-06T03:40:30.204 INFO:tasks.workunit.client.1.smithi136.stdout:3/131: read d0/f23 [2327526,127173] 0 2024-04-06T03:40:30.205 INFO:tasks.workunit.client.1.smithi136.stdout:1/179: dwrite d12/d1b/f32 [0,4194304] 0 2024-04-06T03:40:30.211 INFO:tasks.workunit.client.0.smithi094.stdout:4/268: dwrite d0/df/f1b [0,4194304] 0 2024-04-06T03:40:30.212 INFO:tasks.workunit.client.0.smithi094.stdout:4/269: readlink d0/d15/l2c 0 2024-04-06T03:40:30.212 INFO:tasks.workunit.client.1.smithi136.stdout:8/192: link d3/de/ff d3/da/d26/d35/f38 0 2024-04-06T03:40:30.212 INFO:tasks.workunit.client.1.smithi136.stdout:8/193: truncate d3/de/d18/f1e 452136 0 2024-04-06T03:40:30.213 INFO:tasks.workunit.client.1.smithi136.stdout:8/194: read d3/de/f27 [384676,20777] 0 2024-04-06T03:40:30.217 INFO:tasks.workunit.client.1.smithi136.stdout:9/167: link fd d17/d21/d29/f36 0 2024-04-06T03:40:30.217 INFO:tasks.workunit.client.1.smithi136.stdout:9/168: read d17/d20/d25/f2e [3913244,103852] 0 2024-04-06T03:40:30.219 INFO:tasks.workunit.client.1.smithi136.stdout:8/195: write d3/da/d26/f28 [2914753,12611] 0 2024-04-06T03:40:30.219 INFO:tasks.workunit.client.1.smithi136.stdout:4/143: getdents d1d/d31 0 2024-04-06T03:40:30.219 INFO:tasks.workunit.client.1.smithi136.stdout:4/144: truncate d1d/d31/f3a 311753 0 2024-04-06T03:40:30.223 INFO:tasks.workunit.client.0.smithi094.stdout:6/177: unlink dc/f29 0 2024-04-06T03:40:30.225 INFO:tasks.workunit.client.0.smithi094.stdout:2/254: dwrite d0/d8/d27/d2b/f2d [0,4194304] 0 2024-04-06T03:40:30.225 INFO:tasks.workunit.client.1.smithi136.stdout:5/221: symlink d1/l43 0 2024-04-06T03:40:30.227 INFO:tasks.workunit.client.0.smithi094.stdout:1/197: mknod d0/d1/d27/c47 0 2024-04-06T03:40:30.229 INFO:tasks.workunit.client.1.smithi136.stdout:0/153: mknod d1/d2e/c33 0 2024-04-06T03:40:30.229 INFO:tasks.workunit.client.1.smithi136.stdout:8/196: write d3/da/d26/d35/f1d [5023081,111274] 0 2024-04-06T03:40:30.231 INFO:tasks.workunit.client.1.smithi136.stdout:2/270: creat d3/d9/d10/d18/d4b/f5d x:0 0 0 2024-04-06T03:40:30.232 INFO:tasks.workunit.client.0.smithi094.stdout:8/205: symlink d18/d3c/l4a 0 2024-04-06T03:40:30.232 INFO:tasks.workunit.client.0.smithi094.stdout:8/206: creat d18/d30/f4b x:0 0 0 2024-04-06T03:40:30.233 INFO:tasks.workunit.client.1.smithi136.stdout:5/222: dread d1/d16/f1b [0,4194304] 0 2024-04-06T03:40:30.234 INFO:tasks.workunit.client.1.smithi136.stdout:6/156: creat d2/d7/d26/d30/f3a x:0 0 0 2024-04-06T03:40:30.238 INFO:tasks.workunit.client.0.smithi094.stdout:4/270: unlink d0/df/d1e/f2a 0 2024-04-06T03:40:30.242 INFO:tasks.workunit.client.0.smithi094.stdout:2/255: write d0/d8/fe [410691,67698] 0 2024-04-06T03:40:30.247 INFO:tasks.workunit.client.0.smithi094.stdout:1/198: mkdir d0/d4/d1b/d2f/d3c/d48 0 2024-04-06T03:40:30.248 INFO:tasks.workunit.client.1.smithi136.stdout:1/180: mknod d12/d1b/d1e/c34 0 2024-04-06T03:40:30.248 INFO:tasks.workunit.client.1.smithi136.stdout:1/181: write d12/d1b/d1e/d1f/f30 [1178941,97275] 0 2024-04-06T03:40:30.250 INFO:tasks.workunit.client.0.smithi094.stdout:8/207: link cb d18/d30/c4c 0 2024-04-06T03:40:30.251 INFO:tasks.workunit.client.0.smithi094.stdout:6/178: dwrite dc/ff [0,4194304] 0 2024-04-06T03:40:30.251 INFO:tasks.workunit.client.0.smithi094.stdout:6/179: creat dc/d11/d22/f44 x:0 0 0 2024-04-06T03:40:30.252 INFO:tasks.workunit.client.0.smithi094.stdout:8/208: write d18/f27 [902858,74931] 0 2024-04-06T03:40:30.252 INFO:tasks.workunit.client.0.smithi094.stdout:8/209: dread - d18/f20 zero size 2024-04-06T03:40:30.252 INFO:tasks.workunit.client.0.smithi094.stdout:8/210: truncate d18/d2c/f41 67181 0 2024-04-06T03:40:30.252 INFO:tasks.workunit.client.0.smithi094.stdout:8/211: chown d18/d2c/l45 2 1 2024-04-06T03:40:30.252 INFO:tasks.workunit.client.0.smithi094.stdout:8/212: write f14 [6323950,69433] 0 2024-04-06T03:40:30.254 INFO:tasks.workunit.client.0.smithi094.stdout:4/271: unlink d0/df/d1e/d34/f3d 0 2024-04-06T03:40:30.254 INFO:tasks.workunit.client.0.smithi094.stdout:4/272: chown d0/df/f19 486539114 1 2024-04-06T03:40:30.260 INFO:tasks.workunit.client.0.smithi094.stdout:2/256: mknod d0/dd/d1b/c51 0 2024-04-06T03:40:30.260 INFO:tasks.workunit.client.0.smithi094.stdout:2/257: read d0/dd/d1b/f3e [1082750,54955] 0 2024-04-06T03:40:30.260 INFO:tasks.workunit.client.0.smithi094.stdout:2/258: chown d0/dd/l12 804 1 2024-04-06T03:40:30.260 INFO:tasks.workunit.client.0.smithi094.stdout:2/259: chown d0/dd/c28 15131826 1 2024-04-06T03:40:30.262 INFO:tasks.workunit.client.0.smithi094.stdout:1/199: symlink d0/d4/d37/l49 0 2024-04-06T03:40:30.262 INFO:tasks.workunit.client.0.smithi094.stdout:1/200: readlink d0/l3e 0 2024-04-06T03:40:30.264 INFO:tasks.workunit.client.1.smithi136.stdout:1/182: dwrite d12/f24 [0,4194304] 0 2024-04-06T03:40:30.264 INFO:tasks.workunit.client.1.smithi136.stdout:1/183: creat d12/d1b/f35 x:0 0 0 2024-04-06T03:40:30.264 INFO:tasks.workunit.client.1.smithi136.stdout:1/184: readlink d12/d1b/d1e/d23/l25 0 2024-04-06T03:40:30.265 INFO:tasks.workunit.client.0.smithi094.stdout:8/213: dread f14 [0,4194304] 0 2024-04-06T03:40:30.265 INFO:tasks.workunit.client.0.smithi094.stdout:8/214: dread - d18/d30/f4b zero size 2024-04-06T03:40:30.265 INFO:tasks.workunit.client.0.smithi094.stdout:8/215: creat d18/f4d x:0 0 0 2024-04-06T03:40:30.269 INFO:tasks.workunit.client.0.smithi094.stdout:1/201: write d0/f19 [1752642,112302] 0 2024-04-06T03:40:30.272 INFO:tasks.workunit.client.0.smithi094.stdout:6/180: dwrite fa [0,4194304] 0 2024-04-06T03:40:30.272 INFO:tasks.workunit.client.0.smithi094.stdout:4/273: creat d0/df/d13/f58 x:0 0 0 2024-04-06T03:40:30.272 INFO:tasks.workunit.client.0.smithi094.stdout:4/274: chown d0/df/l30 3059 1 2024-04-06T03:40:30.277 INFO:tasks.workunit.client.0.smithi094.stdout:2/260: creat d0/d8/d27/d2b/f52 x:0 0 0 2024-04-06T03:40:30.277 INFO:tasks.workunit.client.0.smithi094.stdout:2/261: chown d0/d8/d1d/l2f 934205239 1 2024-04-06T03:40:30.277 INFO:tasks.workunit.client.0.smithi094.stdout:2/262: chown d0/d8/f11 30 1 2024-04-06T03:40:30.278 INFO:tasks.workunit.client.1.smithi136.stdout:9/169: unlink d17/d20/d25/f2e 0 2024-04-06T03:40:30.279 INFO:tasks.workunit.client.1.smithi136.stdout:4/145: write d1d/f1f [264993,15728] 0 2024-04-06T03:40:30.282 INFO:tasks.workunit.client.0.smithi094.stdout:8/216: unlink d18/d3c/l4a 0 2024-04-06T03:40:30.282 INFO:tasks.workunit.client.0.smithi094.stdout:8/217: chown d18/d2c/d37/d48 109843677 1 2024-04-06T03:40:30.282 INFO:tasks.workunit.client.0.smithi094.stdout:8/218: chown cb 13 1 2024-04-06T03:40:30.284 INFO:tasks.workunit.client.1.smithi136.stdout:0/154: rename d1/d6/l15 to d1/d2e/l34 0 2024-04-06T03:40:30.292 INFO:tasks.workunit.client.1.smithi136.stdout:8/197: truncate d3/f8 514497 0 2024-04-06T03:40:30.293 INFO:tasks.workunit.client.0.smithi094.stdout:1/202: dwrite d0/d4/fc [0,4194304] 0 2024-04-06T03:40:30.296 INFO:tasks.workunit.client.1.smithi136.stdout:2/271: mknod d3/d4a/c5e 0 2024-04-06T03:40:30.298 INFO:tasks.workunit.client.1.smithi136.stdout:5/223: creat d1/d16/d17/f44 x:0 0 0 2024-04-06T03:40:30.301 INFO:tasks.workunit.client.0.smithi094.stdout:6/181: dwrite f7 [0,4194304] 0 2024-04-06T03:40:30.302 INFO:tasks.workunit.client.1.smithi136.stdout:3/132: rmdir d0/d4/d12/d19/d1b 39 2024-04-06T03:40:30.302 INFO:tasks.workunit.client.1.smithi136.stdout:3/133: write d0/d4/d12/f1a [1015223,119308] 0 2024-04-06T03:40:30.303 INFO:tasks.workunit.client.1.smithi136.stdout:7/154: sync 2024-04-06T03:40:30.303 INFO:tasks.workunit.client.0.smithi094.stdout:2/263: getdents d0/dd/d3d/d49 0 2024-04-06T03:40:30.303 INFO:tasks.workunit.client.0.smithi094.stdout:2/264: stat d0/dd/d19/d31/f42 0 2024-04-06T03:40:30.304 INFO:tasks.workunit.client.1.smithi136.stdout:9/170: dwrite d17/d19/f27 [0,4194304] 0 2024-04-06T03:40:30.306 INFO:tasks.workunit.client.0.smithi094.stdout:8/219: getdents d18/d2c/d37 0 2024-04-06T03:40:30.306 INFO:tasks.workunit.client.0.smithi094.stdout:8/220: chown d18/d2c/d37/d48 308729263 1 2024-04-06T03:40:30.306 INFO:tasks.workunit.client.0.smithi094.stdout:8/221: creat d18/f4e x:0 0 0 2024-04-06T03:40:30.306 INFO:tasks.workunit.client.0.smithi094.stdout:8/222: truncate d18/f4d 743555 0 2024-04-06T03:40:30.308 INFO:tasks.workunit.client.0.smithi094.stdout:1/203: mknod d0/c4a 0 2024-04-06T03:40:30.309 INFO:tasks.workunit.client.0.smithi094.stdout:1/204: write d0/d1/d27/f3d [1130247,73122] 0 2024-04-06T03:40:30.310 INFO:tasks.workunit.client.0.smithi094.stdout:4/275: creat d0/df/d1e/f59 x:0 0 0 2024-04-06T03:40:30.310 INFO:tasks.workunit.client.0.smithi094.stdout:4/276: creat d0/df/d13/f5a x:0 0 0 2024-04-06T03:40:30.311 INFO:tasks.workunit.client.1.smithi136.stdout:3/134: dread d0/d4/fd [0,4194304] 0 2024-04-06T03:40:30.312 INFO:tasks.workunit.client.1.smithi136.stdout:4/146: dwrite d1d/f27 [0,4194304] 0 2024-04-06T03:40:30.312 INFO:tasks.workunit.client.1.smithi136.stdout:4/147: write d1d/d2f/f39 [827426,29978] 0 2024-04-06T03:40:30.314 INFO:tasks.workunit.client.1.smithi136.stdout:4/148: fdatasync d1d/d2f/f39 0 2024-04-06T03:40:30.314 INFO:tasks.workunit.client.0.smithi094.stdout:2/265: dread d0/dd/d19/d31/f42 [0,4194304] 0 2024-04-06T03:40:30.314 INFO:tasks.workunit.client.0.smithi094.stdout:2/266: truncate d0/f5 5106140 0 2024-04-06T03:40:30.315 INFO:tasks.workunit.client.0.smithi094.stdout:2/267: fdatasync d0/dd/d1b/f1c 0 2024-04-06T03:40:30.315 INFO:tasks.workunit.client.1.smithi136.stdout:3/135: dread d0/d4/fe [0,4194304] 0 2024-04-06T03:40:30.315 INFO:tasks.workunit.client.1.smithi136.stdout:3/136: truncate d0/d4/d12/d19/d1b/f24 23045 0 2024-04-06T03:40:30.316 INFO:tasks.workunit.client.1.smithi136.stdout:1/185: symlink d12/d1b/d1e/l36 0 2024-04-06T03:40:30.318 INFO:tasks.workunit.client.1.smithi136.stdout:0/155: symlink d1/d6/d2c/d1a/l35 0 2024-04-06T03:40:30.320 INFO:tasks.workunit.client.1.smithi136.stdout:8/198: mkdir d3/da/d26/d35/d39 0 2024-04-06T03:40:30.322 INFO:tasks.workunit.client.1.smithi136.stdout:2/272: mkdir d3/d45/d5f 0 2024-04-06T03:40:30.324 INFO:tasks.workunit.client.0.smithi094.stdout:8/223: mknod d18/d34/c4f 0 2024-04-06T03:40:30.324 INFO:tasks.workunit.client.0.smithi094.stdout:8/224: write d18/d30/f40 [50433,15255] 0 2024-04-06T03:40:30.325 INFO:tasks.workunit.client.1.smithi136.stdout:5/224: symlink d1/dc/d1c/d32/l45 0 2024-04-06T03:40:30.327 INFO:tasks.workunit.client.0.smithi094.stdout:1/205: mkdir d0/d4/d1b/d2f/d4b 0 2024-04-06T03:40:30.331 INFO:tasks.workunit.client.0.smithi094.stdout:2/268: dwrite d0/f1 [4194304,4194304] 0 2024-04-06T03:40:30.331 INFO:tasks.workunit.client.0.smithi094.stdout:2/269: fsync d0/dd/d19/d31/f42 0 2024-04-06T03:40:30.333 INFO:tasks.workunit.client.1.smithi136.stdout:6/157: rmdir d2/d7 39 2024-04-06T03:40:30.333 INFO:tasks.workunit.client.1.smithi136.stdout:6/158: creat d2/d25/f3b x:0 0 0 2024-04-06T03:40:30.335 INFO:tasks.workunit.client.1.smithi136.stdout:2/273: dread d3/d9/d10/d18/f48 [0,4194304] 0 2024-04-06T03:40:30.335 INFO:tasks.workunit.client.1.smithi136.stdout:2/274: write d3/d9/d10/d18/d1d/d3c/f40 [564433,27458] 0 2024-04-06T03:40:30.337 INFO:tasks.workunit.client.1.smithi136.stdout:7/155: symlink d5/l34 0 2024-04-06T03:40:30.340 INFO:tasks.workunit.client.0.smithi094.stdout:6/182: mknod dc/d11/c45 0 2024-04-06T03:40:30.340 INFO:tasks.workunit.client.0.smithi094.stdout:8/225: dwrite d18/f20 [0,4194304] 0 2024-04-06T03:40:30.341 INFO:tasks.workunit.client.0.smithi094.stdout:2/270: rmdir d0/d8/d27 39 2024-04-06T03:40:30.342 INFO:tasks.workunit.client.0.smithi094.stdout:2/271: dread d0/d18/f35 [0,4194304] 0 2024-04-06T03:40:30.342 INFO:tasks.workunit.client.1.smithi136.stdout:9/171: creat d17/d21/d32/f37 x:0 0 0 2024-04-06T03:40:30.347 INFO:tasks.workunit.client.0.smithi094.stdout:4/277: rmdir d0/d15/d3a/d3e/d49 39 2024-04-06T03:40:30.347 INFO:tasks.workunit.client.1.smithi136.stdout:3/137: mkdir d0/d4/d12/d19/d2c 0 2024-04-06T03:40:30.347 INFO:tasks.workunit.client.1.smithi136.stdout:3/138: truncate d0/f1e 468603 0 2024-04-06T03:40:30.349 INFO:tasks.workunit.client.1.smithi136.stdout:1/186: mkdir d12/d1b/d1e/d27/d37 0 2024-04-06T03:40:30.349 INFO:tasks.workunit.client.1.smithi136.stdout:1/187: fdatasync d12/d1b/f29 0 2024-04-06T03:40:30.349 INFO:tasks.workunit.client.1.smithi136.stdout:1/188: creat d12/f38 x:0 0 0 2024-04-06T03:40:30.349 INFO:tasks.workunit.client.1.smithi136.stdout:1/189: readlink d12/l1a 0 2024-04-06T03:40:30.349 INFO:tasks.workunit.client.1.smithi136.stdout:1/190: write f10 [4516836,29909] 0 2024-04-06T03:40:30.349 INFO:tasks.workunit.client.1.smithi136.stdout:0/156: mkdir d1/d6/d2c/d1a/d36 0 2024-04-06T03:40:30.350 INFO:tasks.workunit.client.1.smithi136.stdout:0/157: chown d1/d6/d2c/c17 51581 1 2024-04-06T03:40:30.353 INFO:tasks.workunit.client.1.smithi136.stdout:8/199: rename d3/f2f to d3/de/f3a 0 2024-04-06T03:40:30.353 INFO:tasks.workunit.client.1.smithi136.stdout:8/200: truncate d3/de/ff 4346322 0 2024-04-06T03:40:30.355 INFO:tasks.workunit.client.0.smithi094.stdout:1/206: dwrite d0/d1/d27/f3d [0,4194304] 0 2024-04-06T03:40:30.355 INFO:tasks.workunit.client.0.smithi094.stdout:1/207: stat d0/d4 0 2024-04-06T03:40:30.357 INFO:tasks.workunit.client.0.smithi094.stdout:2/272: creat d0/dd/d3d/d49/f53 x:0 0 0 2024-04-06T03:40:30.360 INFO:tasks.workunit.client.0.smithi094.stdout:4/278: symlink d0/d15/d3a/d3e/d50/l5b 0 2024-04-06T03:40:30.362 INFO:tasks.workunit.client.1.smithi136.stdout:7/156: mknod d5/c35 0 2024-04-06T03:40:30.362 INFO:tasks.workunit.client.1.smithi136.stdout:7/157: stat d5/d21 0 2024-04-06T03:40:30.362 INFO:tasks.workunit.client.1.smithi136.stdout:7/158: truncate d5/df/d12/f17 4803798 0 2024-04-06T03:40:30.362 INFO:tasks.workunit.client.1.smithi136.stdout:7/159: creat d5/f36 x:0 0 0 2024-04-06T03:40:30.364 INFO:tasks.workunit.client.0.smithi094.stdout:2/273: dread d0/d8/d27/d2b/f2c [0,4194304] 0 2024-04-06T03:40:30.364 INFO:tasks.workunit.client.1.smithi136.stdout:9/172: getdents d17/d21/d29 0 2024-04-06T03:40:30.367 INFO:tasks.workunit.client.1.smithi136.stdout:4/149: rmdir d1d/d2f 39 2024-04-06T03:40:30.370 INFO:tasks.workunit.client.0.smithi094.stdout:6/183: dwrite dc/d3a/f41 [0,4194304] 0 2024-04-06T03:40:30.371 INFO:tasks.workunit.client.1.smithi136.stdout:2/275: dwrite d3/d15/f2b [0,4194304] 0 2024-04-06T03:40:30.371 INFO:tasks.workunit.client.1.smithi136.stdout:2/276: chown d3/d15/d19/d43 57 1 2024-04-06T03:40:30.371 INFO:tasks.workunit.client.1.smithi136.stdout:6/159: dwrite d2/de/f10 [4194304,4194304] 0 2024-04-06T03:40:30.371 INFO:tasks.workunit.client.0.smithi094.stdout:4/279: symlink d0/d15/d3a/l5c 0 2024-04-06T03:40:30.372 INFO:tasks.workunit.client.1.smithi136.stdout:3/139: mkdir d0/d4/d12/d2d 0 2024-04-06T03:40:30.376 INFO:tasks.workunit.client.1.smithi136.stdout:7/160: write d5/fd [3658440,36243] 0 2024-04-06T03:40:30.376 INFO:tasks.workunit.client.1.smithi136.stdout:7/161: dread - d5/df/d11/d22/f2f zero size 2024-04-06T03:40:30.376 INFO:tasks.workunit.client.1.smithi136.stdout:7/162: stat d5/df/d1d 0 2024-04-06T03:40:30.376 INFO:tasks.workunit.client.1.smithi136.stdout:7/163: stat d5/l1e 0 2024-04-06T03:40:30.376 INFO:tasks.workunit.client.1.smithi136.stdout:7/164: chown d5/df/d12/c18 336423811 1 2024-04-06T03:40:30.376 INFO:tasks.workunit.client.1.smithi136.stdout:7/165: write d5/df/d11/d22/f28 [957218,71071] 0 2024-04-06T03:40:30.376 INFO:tasks.workunit.client.0.smithi094.stdout:2/274: dwrite d0/d18/f35 [0,4194304] 0 2024-04-06T03:40:30.378 INFO:tasks.workunit.client.1.smithi136.stdout:0/158: truncate d1/d6/f12 3694229 0 2024-04-06T03:40:30.379 INFO:tasks.workunit.client.1.smithi136.stdout:1/191: rename f11 to d12/f39 0 2024-04-06T03:40:30.379 INFO:tasks.workunit.client.1.smithi136.stdout:1/192: truncate d12/f39 3111358 0 2024-04-06T03:40:30.380 INFO:tasks.workunit.client.1.smithi136.stdout:5/225: getdents d1/d16/d17 0 2024-04-06T03:40:30.382 INFO:tasks.workunit.client.1.smithi136.stdout:5/226: write d1/dc/d1c/f2c [146097,99423] 0 2024-04-06T03:40:30.383 INFO:tasks.workunit.client.1.smithi136.stdout:9/173: symlink d17/d19/l38 0 2024-04-06T03:40:30.384 INFO:tasks.workunit.client.1.smithi136.stdout:4/150: creat d1d/d31/f3b x:0 0 0 2024-04-06T03:40:30.387 INFO:tasks.workunit.client.1.smithi136.stdout:0/159: read d1/d6/f19 [3098215,20247] 0 2024-04-06T03:40:30.387 INFO:tasks.workunit.client.1.smithi136.stdout:2/277: creat d3/d15/d19/f60 x:0 0 0 2024-04-06T03:40:30.388 INFO:tasks.workunit.client.0.smithi094.stdout:6/184: mkdir dc/d11/d1b/d24/d46 0 2024-04-06T03:40:30.388 INFO:tasks.workunit.client.0.smithi094.stdout:6/185: truncate dc/f40 830127 0 2024-04-06T03:40:30.390 INFO:tasks.workunit.client.1.smithi136.stdout:6/160: link d2/de/f15 d2/f3c 0 2024-04-06T03:40:30.391 INFO:tasks.workunit.client.0.smithi094.stdout:4/280: link d0/df/l30 d0/df/l5d 0 2024-04-06T03:40:30.391 INFO:tasks.workunit.client.1.smithi136.stdout:3/140: mkdir d0/d4/d12/d19/d2e 0 2024-04-06T03:40:30.392 INFO:tasks.workunit.client.1.smithi136.stdout:1/193: dwrite d12/d1b/d1e/d23/f31 [0,4194304] 0 2024-04-06T03:40:30.393 INFO:tasks.workunit.client.1.smithi136.stdout:7/166: unlink d5/f27 0 2024-04-06T03:40:30.393 INFO:tasks.workunit.client.1.smithi136.stdout:3/141: dread d0/d4/f22 [0,4194304] 0 2024-04-06T03:40:30.397 INFO:tasks.workunit.client.0.smithi094.stdout:4/281: creat d0/d15/d3a/d3e/d50/f5e x:0 0 0 2024-04-06T03:40:30.397 INFO:tasks.workunit.client.0.smithi094.stdout:4/282: fsync d0/d15/f40 0 2024-04-06T03:40:30.397 INFO:tasks.workunit.client.0.smithi094.stdout:4/283: chown d0/df/d13/c2d 656828855 1 2024-04-06T03:40:30.398 INFO:tasks.workunit.client.1.smithi136.stdout:5/227: rename d1/c7 to d1/dc/d10/c46 0 2024-04-06T03:40:30.398 INFO:tasks.workunit.client.1.smithi136.stdout:5/228: truncate d1/d1f/f21 1326948 0 2024-04-06T03:40:30.398 INFO:tasks.workunit.client.1.smithi136.stdout:5/229: stat d1/d16/c25 0 2024-04-06T03:40:30.398 INFO:tasks.workunit.client.1.smithi136.stdout:0/160: write d1/d6/fb [3944017,126758] 0 2024-04-06T03:40:30.398 INFO:tasks.workunit.client.1.smithi136.stdout:9/174: symlink d17/d21/d29/d34/l39 0 2024-04-06T03:40:30.399 INFO:tasks.workunit.client.1.smithi136.stdout:4/151: unlink d1d/f24 0 2024-04-06T03:40:30.402 INFO:tasks.workunit.client.1.smithi136.stdout:2/278: creat d3/f61 x:0 0 0 2024-04-06T03:40:30.402 INFO:tasks.workunit.client.1.smithi136.stdout:2/279: fdatasync d3/d15/d19/f1b 0 2024-04-06T03:40:30.405 INFO:tasks.workunit.client.1.smithi136.stdout:5/230: dread d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:30.407 INFO:tasks.workunit.client.0.smithi094.stdout:6/186: dwrite dc/d11/d22/f30 [0,4194304] 0 2024-04-06T03:40:30.408 INFO:tasks.workunit.client.1.smithi136.stdout:1/194: link c2 d12/d1b/d1e/d27/d37/c3a 0 2024-04-06T03:40:30.410 INFO:tasks.workunit.client.1.smithi136.stdout:7/167: rename d5/df/d12/d1b/f20 to d5/df/d12/f37 0 2024-04-06T03:40:30.411 INFO:tasks.workunit.client.1.smithi136.stdout:7/168: dread - d5/df/d11/d22/f2f zero size 2024-04-06T03:40:30.411 INFO:tasks.workunit.client.1.smithi136.stdout:7/169: chown d5/df/d11/l1a 4 1 2024-04-06T03:40:30.411 INFO:tasks.workunit.client.1.smithi136.stdout:0/161: mknod d1/d2e/c37 0 2024-04-06T03:40:30.413 INFO:tasks.workunit.client.1.smithi136.stdout:4/152: symlink d1d/d2f/l3c 0 2024-04-06T03:40:30.413 INFO:tasks.workunit.client.1.smithi136.stdout:4/153: truncate d1d/f2b 22341 0 2024-04-06T03:40:30.413 INFO:tasks.workunit.client.1.smithi136.stdout:4/154: chown d1d/d2f 107428413 1 2024-04-06T03:40:30.413 INFO:tasks.workunit.client.1.smithi136.stdout:4/155: fdatasync d1d/f38 0 2024-04-06T03:40:30.414 INFO:tasks.workunit.client.1.smithi136.stdout:2/280: mknod d3/d9/c62 0 2024-04-06T03:40:30.414 INFO:tasks.workunit.client.1.smithi136.stdout:2/281: chown d3/d9/d10/d18/d4b/l53 3 1 2024-04-06T03:40:30.414 INFO:tasks.workunit.client.1.smithi136.stdout:5/231: truncate d1/d16/d17/f1a 2505430 0 2024-04-06T03:40:30.415 INFO:tasks.workunit.client.1.smithi136.stdout:7/170: rename d5/f36 to d5/df/d11/f38 0 2024-04-06T03:40:30.417 INFO:tasks.workunit.client.1.smithi136.stdout:4/156: mknod d1d/d31/d36/c3d 0 2024-04-06T03:40:30.418 INFO:tasks.workunit.client.1.smithi136.stdout:5/232: mknod d1/dc/d10/d1e/c47 0 2024-04-06T03:40:30.418 INFO:tasks.workunit.client.1.smithi136.stdout:5/233: write d1/dc/d1c/f34 [201736,129358] 0 2024-04-06T03:40:30.420 INFO:tasks.workunit.client.1.smithi136.stdout:7/171: rename d5/df/d12/f17 to d5/d21/f39 0 2024-04-06T03:40:30.438 INFO:tasks.workunit.client.1.smithi136.stdout:2/282: dwrite d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:30.445 INFO:tasks.workunit.client.1.smithi136.stdout:2/283: dread d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:30.446 INFO:tasks.workunit.client.1.smithi136.stdout:4/157: dwrite d1d/f27 [0,4194304] 0 2024-04-06T03:40:30.447 INFO:tasks.workunit.client.1.smithi136.stdout:2/284: rename d3/d9/d10/f17 to d3/d45/f63 0 2024-04-06T03:40:30.488 INFO:tasks.workunit.client.1.smithi136.stdout:8/201: dwrite d3/f9 [0,4194304] 0 2024-04-06T03:40:30.488 INFO:tasks.workunit.client.1.smithi136.stdout:8/202: read d3/da/d26/d35/f1d [4155116,120448] 0 2024-04-06T03:40:30.494 INFO:tasks.workunit.client.1.smithi136.stdout:6/161: getdents d2/d25 0 2024-04-06T03:40:30.495 INFO:tasks.workunit.client.1.smithi136.stdout:9/175: read fd [2661728,45823] 0 2024-04-06T03:40:30.497 INFO:tasks.workunit.client.1.smithi136.stdout:3/142: dwrite d0/f5 [0,4194304] 0 2024-04-06T03:40:30.497 INFO:tasks.workunit.client.1.smithi136.stdout:3/143: fsync d0/f1 0 2024-04-06T03:40:30.498 INFO:tasks.workunit.client.1.smithi136.stdout:0/162: dwrite d1/d6/d2c/f20 [0,4194304] 0 2024-04-06T03:40:30.498 INFO:tasks.workunit.client.1.smithi136.stdout:0/163: creat d1/d6/d2c/d1a/d1c/f38 x:0 0 0 2024-04-06T03:40:30.499 INFO:tasks.workunit.client.1.smithi136.stdout:6/162: mkdir d2/de/d3d 0 2024-04-06T03:40:30.499 INFO:tasks.workunit.client.1.smithi136.stdout:6/163: getdents d2/de/d3d 0 2024-04-06T03:40:30.499 INFO:tasks.workunit.client.1.smithi136.stdout:6/164: truncate d2/d7/f2c 1031971 0 2024-04-06T03:40:30.499 INFO:tasks.workunit.client.1.smithi136.stdout:9/176: rmdir d17/d21/d29 39 2024-04-06T03:40:30.500 INFO:tasks.workunit.client.1.smithi136.stdout:1/195: dread d12/d1b/f32 [0,4194304] 0 2024-04-06T03:40:30.502 INFO:tasks.workunit.client.0.smithi094.stdout:6/187: dwrite dc/d11/d1b/f31 [0,4194304] 0 2024-04-06T03:40:30.504 INFO:tasks.workunit.client.1.smithi136.stdout:3/144: unlink d0/d4/l27 0 2024-04-06T03:40:30.504 INFO:tasks.workunit.client.1.smithi136.stdout:8/203: dread d3/da/f19 [0,4194304] 0 2024-04-06T03:40:30.510 INFO:tasks.workunit.client.1.smithi136.stdout:6/165: symlink d2/de/d3d/l3e 0 2024-04-06T03:40:30.511 INFO:tasks.workunit.client.1.smithi136.stdout:0/164: dread d1/fc [0,4194304] 0 2024-04-06T03:40:30.511 INFO:tasks.workunit.client.1.smithi136.stdout:0/165: truncate d1/d6/fd 4496266 0 2024-04-06T03:40:30.513 INFO:tasks.workunit.client.1.smithi136.stdout:8/204: dread d3/f21 [0,4194304] 0 2024-04-06T03:40:30.515 INFO:tasks.workunit.client.1.smithi136.stdout:1/196: mkdir d12/d3b 0 2024-04-06T03:40:30.515 INFO:tasks.workunit.client.1.smithi136.stdout:1/197: chown d12/d1b/d22 654252431 1 2024-04-06T03:40:30.518 INFO:tasks.workunit.client.1.smithi136.stdout:3/145: symlink d0/d4/d12/d2d/l2f 0 2024-04-06T03:40:30.521 INFO:tasks.workunit.client.1.smithi136.stdout:2/285: dwrite d3/d9/d10/f22 [0,4194304] 0 2024-04-06T03:40:30.524 INFO:tasks.workunit.client.1.smithi136.stdout:8/205: symlink d3/da/d26/l3b 0 2024-04-06T03:40:30.524 INFO:tasks.workunit.client.0.smithi094.stdout:6/188: symlink dc/d11/d22/l47 0 2024-04-06T03:40:30.526 INFO:tasks.workunit.client.1.smithi136.stdout:1/198: unlink c2 0 2024-04-06T03:40:30.527 INFO:tasks.workunit.client.1.smithi136.stdout:7/172: dwrite d5/d21/f39 [0,4194304] 0 2024-04-06T03:40:30.529 INFO:tasks.workunit.client.1.smithi136.stdout:2/286: dread d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:30.529 INFO:tasks.workunit.client.1.smithi136.stdout:2/287: stat d3/d9 0 2024-04-06T03:40:30.529 INFO:tasks.workunit.client.1.smithi136.stdout:2/288: creat d3/f64 x:0 0 0 2024-04-06T03:40:30.529 INFO:tasks.workunit.client.1.smithi136.stdout:2/289: stat d3/d15/d19/d43/d54 0 2024-04-06T03:40:30.529 INFO:tasks.workunit.client.1.smithi136.stdout:2/290: creat d3/d9/d10/d18/d4b/f65 x:0 0 0 2024-04-06T03:40:30.529 INFO:tasks.workunit.client.0.smithi094.stdout:6/189: creat dc/d11/d1b/d24/d46/f48 x:0 0 0 2024-04-06T03:40:30.534 INFO:tasks.workunit.client.0.smithi094.stdout:6/190: symlink dc/l49 0 2024-04-06T03:40:30.536 INFO:tasks.workunit.client.1.smithi136.stdout:1/199: mknod d12/d1b/c3c 0 2024-04-06T03:40:30.537 INFO:tasks.workunit.client.1.smithi136.stdout:3/146: dwrite d0/f1e [0,4194304] 0 2024-04-06T03:40:30.539 INFO:tasks.workunit.client.1.smithi136.stdout:6/166: dwrite d2/f5 [0,4194304] 0 2024-04-06T03:40:30.539 INFO:tasks.workunit.client.1.smithi136.stdout:6/167: write d2/f21 [1261030,107194] 0 2024-04-06T03:40:30.539 INFO:tasks.workunit.client.1.smithi136.stdout:7/173: truncate d5/df/f10 3636517 0 2024-04-06T03:40:30.539 INFO:tasks.workunit.client.1.smithi136.stdout:9/177: dwrite d17/d21/d29/f2a [0,4194304] 0 2024-04-06T03:40:30.545 INFO:tasks.workunit.client.0.smithi094.stdout:5/166: sync 2024-04-06T03:40:30.549 INFO:tasks.workunit.client.1.smithi136.stdout:8/206: dwrite d3/f2a [0,4194304] 0 2024-04-06T03:40:30.549 INFO:tasks.workunit.client.1.smithi136.stdout:8/207: chown d3/f4 47797 1 2024-04-06T03:40:30.549 INFO:tasks.workunit.client.1.smithi136.stdout:1/200: link d12/f24 d12/d1b/d1e/d23/f3d 0 2024-04-06T03:40:30.549 INFO:tasks.workunit.client.1.smithi136.stdout:1/201: stat d12/c19 0 2024-04-06T03:40:30.550 INFO:tasks.workunit.client.0.smithi094.stdout:2/275: write d0/d18/f35 [4374351,128765] 0 2024-04-06T03:40:30.556 INFO:tasks.workunit.client.1.smithi136.stdout:6/168: creat d2/d7/d13/f3f x:0 0 0 2024-04-06T03:40:30.557 INFO:tasks.workunit.client.0.smithi094.stdout:4/284: rmdir d0/d15 39 2024-04-06T03:40:30.559 INFO:tasks.workunit.client.0.smithi094.stdout:6/191: dwrite dc/d11/d1b/d24/f2e [0,4194304] 0 2024-04-06T03:40:30.560 INFO:tasks.workunit.client.1.smithi136.stdout:1/202: dread d12/d1b/d1e/d23/f31 [0,4194304] 0 2024-04-06T03:40:30.561 INFO:tasks.workunit.client.1.smithi136.stdout:2/291: dwrite d3/d9/d10/d18/d4b/f5d [0,4194304] 0 2024-04-06T03:40:30.561 INFO:tasks.workunit.client.0.smithi094.stdout:2/276: rmdir d0/d8/d27/d2b 39 2024-04-06T03:40:30.561 INFO:tasks.workunit.client.0.smithi094.stdout:2/277: truncate d0/dd/d3d/d49/f53 634198 0 2024-04-06T03:40:30.562 INFO:tasks.workunit.client.0.smithi094.stdout:2/278: write d0/dd/d3d/d49/f53 [475631,16032] 0 2024-04-06T03:40:30.562 INFO:tasks.workunit.client.1.smithi136.stdout:7/174: mkdir d5/df/d12/d1b/d3a 0 2024-04-06T03:40:30.562 INFO:tasks.workunit.client.1.smithi136.stdout:7/175: creat d5/df/d12/d1b/f3b x:0 0 0 2024-04-06T03:40:30.562 INFO:tasks.workunit.client.1.smithi136.stdout:7/176: dread - d5/df/d11/f38 zero size 2024-04-06T03:40:30.564 INFO:tasks.workunit.client.0.smithi094.stdout:4/285: link d0/df/d13/d4a/l4c d0/l5f 0 2024-04-06T03:40:30.565 INFO:tasks.workunit.client.0.smithi094.stdout:4/286: write d0/f44 [37225,9595] 0 2024-04-06T03:40:30.565 INFO:tasks.workunit.client.0.smithi094.stdout:4/287: write d0/d4/fb [220438,85447] 0 2024-04-06T03:40:30.566 INFO:tasks.workunit.client.1.smithi136.stdout:3/147: dwrite d0/f5 [0,4194304] 0 2024-04-06T03:40:30.572 INFO:tasks.workunit.client.0.smithi094.stdout:5/167: dwrite db/d19/d2c/f2d [0,4194304] 0 2024-04-06T03:40:30.572 INFO:tasks.workunit.client.1.smithi136.stdout:9/178: creat d17/d21/d29/f3a x:0 0 0 2024-04-06T03:40:30.575 INFO:tasks.workunit.client.0.smithi094.stdout:6/192: mkdir dc/d3a/d4a 0 2024-04-06T03:40:30.576 INFO:tasks.workunit.client.1.smithi136.stdout:6/169: dwrite d2/f3c [0,4194304] 0 2024-04-06T03:40:30.577 INFO:tasks.workunit.client.0.smithi094.stdout:2/279: mknod d0/dd/d3d/d49/c54 0 2024-04-06T03:40:30.578 INFO:tasks.workunit.client.1.smithi136.stdout:8/208: dwrite d3/f10 [0,4194304] 0 2024-04-06T03:40:30.578 INFO:tasks.workunit.client.1.smithi136.stdout:8/209: read d3/de/f27 [469643,63410] 0 2024-04-06T03:40:30.578 INFO:tasks.workunit.client.1.smithi136.stdout:8/210: fdatasync f1 0 2024-04-06T03:40:30.579 INFO:tasks.workunit.client.0.smithi094.stdout:4/288: symlink d0/df/d13/d4a/l60 0 2024-04-06T03:40:30.582 INFO:tasks.workunit.client.1.smithi136.stdout:1/203: unlink d12/d1b/f32 0 2024-04-06T03:40:30.586 INFO:tasks.workunit.client.1.smithi136.stdout:4/158: rename d1d/d31/d36 to d1d/d31/d3e 0 2024-04-06T03:40:30.586 INFO:tasks.workunit.client.1.smithi136.stdout:4/159: creat d1d/d31/f3f x:0 0 0 2024-04-06T03:40:30.586 INFO:tasks.workunit.client.1.smithi136.stdout:4/160: readlink l6 0 2024-04-06T03:40:30.586 INFO:tasks.workunit.client.1.smithi136.stdout:4/161: fsync d1d/f2b 0 2024-04-06T03:40:30.586 INFO:tasks.workunit.client.1.smithi136.stdout:2/292: dread d3/d9/f12 [0,4194304] 0 2024-04-06T03:40:30.590 INFO:tasks.workunit.client.0.smithi094.stdout:6/193: creat dc/d11/f4b x:0 0 0 2024-04-06T03:40:30.593 INFO:tasks.workunit.client.0.smithi094.stdout:2/280: creat d0/d18/f55 x:0 0 0 2024-04-06T03:40:30.593 INFO:tasks.workunit.client.0.smithi094.stdout:2/281: stat d0/dd/d3d/d49 0 2024-04-06T03:40:30.596 INFO:tasks.workunit.client.1.smithi136.stdout:9/179: mkdir d17/d20/d25/d3b 0 2024-04-06T03:40:30.596 INFO:tasks.workunit.client.0.smithi094.stdout:4/289: mkdir d0/d15/d3a/d61 0 2024-04-06T03:40:30.596 INFO:tasks.workunit.client.0.smithi094.stdout:4/290: write d0/f44 [452628,110557] 0 2024-04-06T03:40:30.598 INFO:tasks.workunit.client.1.smithi136.stdout:6/170: mkdir d2/d7/d40 0 2024-04-06T03:40:30.599 INFO:tasks.workunit.client.1.smithi136.stdout:5/234: dread d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:30.600 INFO:tasks.workunit.client.1.smithi136.stdout:5/235: readlink d1/l9 0 2024-04-06T03:40:30.600 INFO:tasks.workunit.client.0.smithi094.stdout:6/194: unlink dc/ff 0 2024-04-06T03:40:30.600 INFO:tasks.workunit.client.0.smithi094.stdout:6/195: write dc/d11/f20 [4306536,113272] 0 2024-04-06T03:40:30.603 INFO:tasks.workunit.client.1.smithi136.stdout:7/177: dwrite d5/df/d11/f26 [0,4194304] 0 2024-04-06T03:40:30.605 INFO:tasks.workunit.client.1.smithi136.stdout:9/180: dread d17/d21/d29/f2a [0,4194304] 0 2024-04-06T03:40:30.606 INFO:tasks.workunit.client.0.smithi094.stdout:2/282: dread d0/d8/d1d/f38 [0,4194304] 0 2024-04-06T03:40:30.606 INFO:tasks.workunit.client.0.smithi094.stdout:5/168: dwrite db/fc [0,4194304] 0 2024-04-06T03:40:30.606 INFO:tasks.workunit.client.0.smithi094.stdout:2/283: write d0/d8/d27/d2b/f52 [620767,37376] 0 2024-04-06T03:40:30.608 INFO:tasks.workunit.client.0.smithi094.stdout:4/291: truncate d0/df/d1e/d34/f39 566055 0 2024-04-06T03:40:30.608 INFO:tasks.workunit.client.0.smithi094.stdout:2/284: write d0/d8/d1d/f38 [205976,84234] 0 2024-04-06T03:40:30.609 INFO:tasks.workunit.client.1.smithi136.stdout:2/293: dwrite d3/d9/d10/d18/d1d/f35 [0,4194304] 0 2024-04-06T03:40:30.611 INFO:tasks.workunit.client.1.smithi136.stdout:1/204: dwrite f3 [0,4194304] 0 2024-04-06T03:40:30.611 INFO:tasks.workunit.client.1.smithi136.stdout:1/205: fdatasync d12/d1b/d1e/d23/f31 0 2024-04-06T03:40:30.613 INFO:tasks.workunit.client.1.smithi136.stdout:0/166: truncate d1/d6/d2c/f20 4008869 0 2024-04-06T03:40:30.615 INFO:tasks.workunit.client.0.smithi094.stdout:7/223: sync 2024-04-06T03:40:30.616 INFO:tasks.workunit.client.0.smithi094.stdout:3/248: sync 2024-04-06T03:40:30.616 INFO:tasks.workunit.client.0.smithi094.stdout:7/224: write d2/d6/fa [839058,60705] 0 2024-04-06T03:40:30.616 INFO:tasks.workunit.client.0.smithi094.stdout:7/225: creat d2/d6/dc/dd/f3e x:0 0 0 2024-04-06T03:40:30.617 INFO:tasks.workunit.client.1.smithi136.stdout:9/181: write d17/d19/f27 [2851273,105566] 0 2024-04-06T03:40:30.620 INFO:tasks.workunit.client.1.smithi136.stdout:8/211: unlink d3/de/d18/f33 0 2024-04-06T03:40:30.623 INFO:tasks.workunit.client.0.smithi094.stdout:3/249: read d9/d10/f22 [33317,124988] 0 2024-04-06T03:40:30.626 INFO:tasks.workunit.client.1.smithi136.stdout:4/162: link d1d/f22 d1d/d31/f40 0 2024-04-06T03:40:30.627 INFO:tasks.workunit.client.1.smithi136.stdout:4/163: readlink d1d/d2f/l3c 0 2024-04-06T03:40:30.629 INFO:tasks.workunit.client.0.smithi094.stdout:6/196: symlink dc/d11/l4c 0 2024-04-06T03:40:30.629 INFO:tasks.workunit.client.0.smithi094.stdout:6/197: creat dc/d11/d1b/d24/d46/f4d x:0 0 0 2024-04-06T03:40:30.632 INFO:tasks.workunit.client.1.smithi136.stdout:6/171: mkdir d2/d25/d41 0 2024-04-06T03:40:30.632 INFO:tasks.workunit.client.1.smithi136.stdout:0/167: dwrite d1/d6/f19 [0,4194304] 0 2024-04-06T03:40:30.633 INFO:tasks.workunit.client.1.smithi136.stdout:4/164: dread f8 [0,4194304] 0 2024-04-06T03:40:30.639 INFO:tasks.workunit.client.1.smithi136.stdout:7/178: link d5/df/d12/d1b/f29 d5/d21/f3c 0 2024-04-06T03:40:30.641 INFO:tasks.workunit.client.0.smithi094.stdout:7/226: truncate d2/d6/f8 975757 0 2024-04-06T03:40:30.641 INFO:tasks.workunit.client.1.smithi136.stdout:7/179: write d5/df/d11/d22/f28 [258098,105800] 0 2024-04-06T03:40:30.641 INFO:tasks.workunit.client.1.smithi136.stdout:7/180: write f1 [50260,33682] 0 2024-04-06T03:40:30.643 INFO:tasks.workunit.client.0.smithi094.stdout:3/250: mkdir d9/d1e/d26/d54 0 2024-04-06T03:40:30.644 INFO:tasks.workunit.client.0.smithi094.stdout:5/169: rmdir db/d19/d24/d28 39 2024-04-06T03:40:30.644 INFO:tasks.workunit.client.0.smithi094.stdout:5/170: chown db/d19/l22 0 1 2024-04-06T03:40:30.644 INFO:tasks.workunit.client.0.smithi094.stdout:5/171: chown db/f23 14447128 1 2024-04-06T03:40:30.648 INFO:tasks.workunit.client.0.smithi094.stdout:4/292: getdents d0/df/d13 0 2024-04-06T03:40:30.650 INFO:tasks.workunit.client.0.smithi094.stdout:7/227: rmdir d2/d6/dc/dd/df 39 2024-04-06T03:40:30.651 INFO:tasks.workunit.client.0.smithi094.stdout:7/228: fsync d2/d6/dc/d12/d29/f34 0 2024-04-06T03:40:30.653 INFO:tasks.workunit.client.0.smithi094.stdout:3/251: unlink d9/d10/d27/f32 0 2024-04-06T03:40:30.656 INFO:tasks.workunit.client.0.smithi094.stdout:5/172: dwrite db/d19/f1f [0,4194304] 0 2024-04-06T03:40:30.656 INFO:tasks.workunit.client.0.smithi094.stdout:5/173: fsync db/d19/f41 0 2024-04-06T03:40:30.656 INFO:tasks.workunit.client.0.smithi094.stdout:4/293: creat d0/d15/d3a/d3e/d46/f62 x:0 0 0 2024-04-06T03:40:30.656 INFO:tasks.workunit.client.0.smithi094.stdout:4/294: read d0/f29 [246848,114218] 0 2024-04-06T03:40:30.657 INFO:tasks.workunit.client.0.smithi094.stdout:7/229: mkdir d2/d6/dc/d12/d29/d3f 0 2024-04-06T03:40:30.659 INFO:tasks.workunit.client.0.smithi094.stdout:3/252: mkdir d9/d1e/d26/d55 0 2024-04-06T03:40:30.663 INFO:tasks.workunit.client.0.smithi094.stdout:3/253: mknod d9/d20/d30/c56 0 2024-04-06T03:40:30.666 INFO:tasks.workunit.client.0.smithi094.stdout:3/254: link d9/d20/d2d/c49 d9/c57 0 2024-04-06T03:40:30.666 INFO:tasks.workunit.client.0.smithi094.stdout:3/255: chown d9/d1d/c31 354 1 2024-04-06T03:40:30.666 INFO:tasks.workunit.client.0.smithi094.stdout:3/256: stat d9/d1e/d26/l41 0 2024-04-06T03:40:30.670 INFO:tasks.workunit.client.0.smithi094.stdout:3/257: link d9/d48/l4c d9/d10/l58 0 2024-04-06T03:40:30.671 INFO:tasks.workunit.client.1.smithi136.stdout:2/294: symlink d3/d9/d10/d18/d1d/l66 0 2024-04-06T03:40:30.673 INFO:tasks.workunit.client.0.smithi094.stdout:7/230: dwrite d2/f1f [4194304,4194304] 0 2024-04-06T03:40:30.674 INFO:tasks.workunit.client.1.smithi136.stdout:1/206: symlink d12/l3e 0 2024-04-06T03:40:30.677 INFO:tasks.workunit.client.0.smithi094.stdout:7/231: unlink d2/d6/dc/d12/d29/f34 0 2024-04-06T03:40:30.678 INFO:tasks.workunit.client.1.smithi136.stdout:4/165: fsync d1d/d31/f40 0 2024-04-06T03:40:30.679 INFO:tasks.workunit.client.1.smithi136.stdout:8/212: creat d3/da/d26/d35/f3c x:0 0 0 2024-04-06T03:40:30.683 INFO:tasks.workunit.client.1.smithi136.stdout:5/236: rmdir d1/dc/d1c 39 2024-04-06T03:40:30.685 INFO:tasks.workunit.client.0.smithi094.stdout:1/208: sync 2024-04-06T03:40:30.685 INFO:tasks.workunit.client.0.smithi094.stdout:0/247: sync 2024-04-06T03:40:30.685 INFO:tasks.workunit.client.0.smithi094.stdout:0/248: truncate d1/f23 5196154 0 2024-04-06T03:40:30.685 INFO:tasks.workunit.client.0.smithi094.stdout:0/249: truncate d1/d9/dd/d18/d1e/d22/d35/f52 1044874 0 2024-04-06T03:40:30.686 INFO:tasks.workunit.client.1.smithi136.stdout:7/181: mkdir d5/df/d3d 0 2024-04-06T03:40:30.686 INFO:tasks.workunit.client.0.smithi094.stdout:9/163: sync 2024-04-06T03:40:30.686 INFO:tasks.workunit.client.0.smithi094.stdout:9/164: fsync d2/dc/d21/d26/f29 0 2024-04-06T03:40:30.686 INFO:tasks.workunit.client.0.smithi094.stdout:8/226: sync 2024-04-06T03:40:30.690 INFO:tasks.workunit.client.0.smithi094.stdout:1/209: creat d0/d1/f4c x:0 0 0 2024-04-06T03:40:30.691 INFO:tasks.workunit.client.0.smithi094.stdout:8/227: write d18/f1c [1838687,8467] 0 2024-04-06T03:40:30.692 INFO:tasks.workunit.client.1.smithi136.stdout:2/295: rmdir d3/d9/d10 39 2024-04-06T03:40:30.692 INFO:tasks.workunit.client.1.smithi136.stdout:2/296: truncate d3/d15/d19/f60 205565 0 2024-04-06T03:40:30.694 INFO:tasks.workunit.client.0.smithi094.stdout:7/232: dread d2/d6/f9 [0,4194304] 0 2024-04-06T03:40:30.694 INFO:tasks.workunit.client.0.smithi094.stdout:3/258: dwrite d9/d10/f2b [0,4194304] 0 2024-04-06T03:40:30.694 INFO:tasks.workunit.client.0.smithi094.stdout:7/233: write d2/d6/dc/d12/d29/d35/f3d [696331,40426] 0 2024-04-06T03:40:30.694 INFO:tasks.workunit.client.0.smithi094.stdout:3/259: write d9/d1e/d26/f28 [755522,129223] 0 2024-04-06T03:40:30.695 INFO:tasks.workunit.client.0.smithi094.stdout:9/165: write d2/dc/d10/f12 [2816833,37609] 0 2024-04-06T03:40:30.695 INFO:tasks.workunit.client.0.smithi094.stdout:6/198: dwrite dc/d11/d1b/d24/f25 [0,4194304] 0 2024-04-06T03:40:30.696 INFO:tasks.workunit.client.0.smithi094.stdout:0/250: creat d1/d9/dd/d18/d1a/f54 x:0 0 0 2024-04-06T03:40:30.697 INFO:tasks.workunit.client.0.smithi094.stdout:7/234: dread d2/d6/dc/dd/f31 [0,4194304] 0 2024-04-06T03:40:30.697 INFO:tasks.workunit.client.0.smithi094.stdout:7/235: fsync d2/d6/dc/dd/df/f1a 0 2024-04-06T03:40:30.697 INFO:tasks.workunit.client.0.smithi094.stdout:7/236: fdatasync d2/d6/dc/f15 0 2024-04-06T03:40:30.697 INFO:tasks.workunit.client.0.smithi094.stdout:7/237: chown d2/d6/dc/d12/f1c 1676267 1 2024-04-06T03:40:30.700 INFO:tasks.workunit.client.1.smithi136.stdout:4/166: dwrite d1d/d31/f33 [0,4194304] 0 2024-04-06T03:40:30.705 INFO:tasks.workunit.client.1.smithi136.stdout:8/213: dwrite d3/de/f3a [0,4194304] 0 2024-04-06T03:40:30.705 INFO:tasks.workunit.client.1.smithi136.stdout:8/214: chown d3/de 135 1 2024-04-06T03:40:30.705 INFO:tasks.workunit.client.1.smithi136.stdout:8/215: fdatasync f1 0 2024-04-06T03:40:30.711 INFO:tasks.workunit.client.1.smithi136.stdout:8/216: dread d3/f4 [0,4194304] 0 2024-04-06T03:40:30.711 INFO:tasks.workunit.client.1.smithi136.stdout:8/217: readlink d3/da/l25 0 2024-04-06T03:40:30.824 INFO:tasks.workunit.client.1.smithi136.stdout:1/207: creat d12/d1b/d28/f3f x:0 0 0 2024-04-06T03:40:30.825 INFO:tasks.workunit.client.1.smithi136.stdout:1/208: stat fb 0 2024-04-06T03:40:30.826 INFO:tasks.workunit.client.0.smithi094.stdout:1/210: rename d0/d1/f4c to d0/d1/d16/d24/d30/d38/f4d 0 2024-04-06T03:40:30.828 INFO:tasks.workunit.client.1.smithi136.stdout:5/237: symlink d1/l48 0 2024-04-06T03:40:30.829 INFO:tasks.workunit.client.0.smithi094.stdout:8/228: mknod d18/d2c/d37/d48/c50 0 2024-04-06T03:40:30.831 INFO:tasks.workunit.client.1.smithi136.stdout:7/182: link d5/df/d11/d22/f2f d5/f3e 0 2024-04-06T03:40:30.834 INFO:tasks.workunit.client.1.smithi136.stdout:2/297: creat d3/d45/f67 x:0 0 0 2024-04-06T03:40:30.835 INFO:tasks.workunit.client.1.smithi136.stdout:4/167: chown ld 3 1 2024-04-06T03:40:30.838 INFO:tasks.workunit.client.0.smithi094.stdout:3/260: link d9/d1d/f4a d9/d1e/f59 0 2024-04-06T03:40:30.844 INFO:tasks.workunit.client.0.smithi094.stdout:1/211: dwrite d0/f19 [4194304,4194304] 0 2024-04-06T03:40:30.844 INFO:tasks.workunit.client.0.smithi094.stdout:1/212: chown d0/d7/f26 32 1 2024-04-06T03:40:30.846 INFO:tasks.workunit.client.0.smithi094.stdout:6/199: unlink dc/d11/d22/f30 0 2024-04-06T03:40:30.846 INFO:tasks.workunit.client.0.smithi094.stdout:6/200: readlink dc/l26 0 2024-04-06T03:40:30.848 INFO:tasks.workunit.client.0.smithi094.stdout:9/166: creat d2/dc/d10/d2a/d2d/f3b x:0 0 0 2024-04-06T03:40:30.849 INFO:tasks.workunit.client.1.smithi136.stdout:8/218: mkdir d3/da/d26/d35/d3d 0 2024-04-06T03:40:30.850 INFO:tasks.workunit.client.0.smithi094.stdout:0/251: truncate d1/d9/dd/d18/d1a/f4d 5054957 0 2024-04-06T03:40:30.850 INFO:tasks.workunit.client.0.smithi094.stdout:0/252: stat d1/d9/dd/d18/d1d/l39 0 2024-04-06T03:40:30.850 INFO:tasks.workunit.client.1.smithi136.stdout:5/238: dwrite d1/dc/d1c/f36 [0,4194304] 0 2024-04-06T03:40:30.850 INFO:tasks.workunit.client.1.smithi136.stdout:5/239: creat d1/dc/d10/f49 x:0 0 0 2024-04-06T03:40:30.852 INFO:tasks.workunit.client.0.smithi094.stdout:7/238: symlink d2/d6/l40 0 2024-04-06T03:40:30.852 INFO:tasks.workunit.client.0.smithi094.stdout:7/239: rename d2 to d2/d6/dc/dd/df/d41 22 2024-04-06T03:40:30.854 INFO:tasks.workunit.client.1.smithi136.stdout:1/209: mknod d12/c40 0 2024-04-06T03:40:30.856 INFO:tasks.workunit.client.1.smithi136.stdout:7/183: creat d5/df/d3d/f3f x:0 0 0 2024-04-06T03:40:30.856 INFO:tasks.workunit.client.1.smithi136.stdout:7/184: write d5/f3e [81900,42114] 0 2024-04-06T03:40:30.856 INFO:tasks.workunit.client.1.smithi136.stdout:7/185: truncate d5/df/d12/f37 1158674 0 2024-04-06T03:40:30.858 INFO:tasks.workunit.client.1.smithi136.stdout:2/298: link d3/d15/d19/f39 d3/f68 0 2024-04-06T03:40:30.858 INFO:tasks.workunit.client.1.smithi136.stdout:2/299: read - d3/f64 zero size 2024-04-06T03:40:30.858 INFO:tasks.workunit.client.1.smithi136.stdout:2/300: chown d3/d15/f30 97145 1 2024-04-06T03:40:30.860 INFO:tasks.workunit.client.1.smithi136.stdout:4/168: link d1d/d31/f33 d1d/d31/f41 0 2024-04-06T03:40:30.862 INFO:tasks.workunit.client.1.smithi136.stdout:8/219: symlink d3/da/l3e 0 2024-04-06T03:40:30.862 INFO:tasks.workunit.client.1.smithi136.stdout:8/220: chown d3/de/d18/c22 255365850 1 2024-04-06T03:40:30.868 INFO:tasks.workunit.client.1.smithi136.stdout:3/148: sync 2024-04-06T03:40:30.868 INFO:tasks.workunit.client.0.smithi094.stdout:7/240: dwrite d2/d6/dc/dd/df/f2d [0,4194304] 0 2024-04-06T03:40:30.869 INFO:tasks.workunit.client.0.smithi094.stdout:8/229: mknod d18/d2c/d37/c51 0 2024-04-06T03:40:30.870 INFO:tasks.workunit.client.1.smithi136.stdout:5/240: link d1/dc/d15/l2d d1/dc/d15/l4a 0 2024-04-06T03:40:30.871 INFO:tasks.workunit.client.1.smithi136.stdout:8/221: dread d3/f9 [0,4194304] 0 2024-04-06T03:40:30.872 INFO:tasks.workunit.client.0.smithi094.stdout:3/261: creat d9/d10/d27/d3a/f5a x:0 0 0 2024-04-06T03:40:30.872 INFO:tasks.workunit.client.0.smithi094.stdout:3/262: chown d9/c14 1488665616 1 2024-04-06T03:40:30.872 INFO:tasks.workunit.client.0.smithi094.stdout:3/263: write d9/d1d/f45 [695840,13422] 0 2024-04-06T03:40:30.873 INFO:tasks.workunit.client.1.smithi136.stdout:1/210: rename c4 to d12/d1b/d1e/d1f/c41 0 2024-04-06T03:40:30.876 INFO:tasks.workunit.client.0.smithi094.stdout:1/213: creat d0/d1/d16/d3f/f4e x:0 0 0 2024-04-06T03:40:30.878 INFO:tasks.workunit.client.1.smithi136.stdout:7/186: unlink d5/f3e 0 2024-04-06T03:40:30.879 INFO:tasks.workunit.client.0.smithi094.stdout:6/201: mkdir dc/d11/d1b/d4e 0 2024-04-06T03:40:30.882 INFO:tasks.workunit.client.1.smithi136.stdout:2/301: mknod d3/d4a/c69 0 2024-04-06T03:40:30.886 INFO:tasks.workunit.client.0.smithi094.stdout:9/167: link d2/dc/d10/f1b d2/dc/d21/d26/f3c 0 2024-04-06T03:40:30.886 INFO:tasks.workunit.client.1.smithi136.stdout:2/302: dread d3/d15/d19/f1b [0,4194304] 0 2024-04-06T03:40:30.887 INFO:tasks.workunit.client.1.smithi136.stdout:2/303: write d3/f64 [590456,82128] 0 2024-04-06T03:40:30.887 INFO:tasks.workunit.client.1.smithi136.stdout:2/304: write d3/f68 [1168124,55662] 0 2024-04-06T03:40:30.888 INFO:tasks.workunit.client.1.smithi136.stdout:5/241: mkdir d1/d16/d4b 0 2024-04-06T03:40:30.889 INFO:tasks.workunit.client.0.smithi094.stdout:3/264: dwrite d9/d10/d27/f38 [0,4194304] 0 2024-04-06T03:40:30.890 INFO:tasks.workunit.client.0.smithi094.stdout:0/253: creat d1/f55 x:0 0 0 2024-04-06T03:40:30.892 INFO:tasks.workunit.client.1.smithi136.stdout:8/222: symlink d3/da/d26/d35/l3f 0 2024-04-06T03:40:30.892 INFO:tasks.workunit.client.1.smithi136.stdout:8/223: stat d3/de/l1a 0 2024-04-06T03:40:30.893 INFO:tasks.workunit.client.0.smithi094.stdout:7/241: truncate d2/fb 1168578 0 2024-04-06T03:40:30.895 INFO:tasks.workunit.client.1.smithi136.stdout:9/182: sync 2024-04-06T03:40:30.897 INFO:tasks.workunit.client.1.smithi136.stdout:1/211: creat d12/d1b/d1e/d27/f42 x:0 0 0 2024-04-06T03:40:30.901 INFO:tasks.workunit.client.0.smithi094.stdout:1/214: dwrite d0/d1/f10 [0,4194304] 0 2024-04-06T03:40:30.901 INFO:tasks.workunit.client.0.smithi094.stdout:1/215: chown d0/d1/d16/d24 57 1 2024-04-06T03:40:30.902 INFO:tasks.workunit.client.1.smithi136.stdout:2/305: creat d3/d9/d10/f6a x:0 0 0 2024-04-06T03:40:30.902 INFO:tasks.workunit.client.1.smithi136.stdout:2/306: dread - d3/d9/d10/d18/f23 zero size 2024-04-06T03:40:30.902 INFO:tasks.workunit.client.1.smithi136.stdout:2/307: dread - d3/d15/f30 zero size 2024-04-06T03:40:30.903 INFO:tasks.workunit.client.0.smithi094.stdout:6/202: dwrite dc/f17 [0,4194304] 0 2024-04-06T03:40:30.903 INFO:tasks.workunit.client.1.smithi136.stdout:7/187: dwrite d5/df/d3d/f3f [0,4194304] 0 2024-04-06T03:40:30.904 INFO:tasks.workunit.client.1.smithi136.stdout:6/172: sync 2024-04-06T03:40:30.904 INFO:tasks.workunit.client.1.smithi136.stdout:0/168: sync 2024-04-06T03:40:30.904 INFO:tasks.workunit.client.1.smithi136.stdout:7/188: read d5/df/d12/d1b/f2d [335951,90862] 0 2024-04-06T03:40:30.905 INFO:tasks.workunit.client.1.smithi136.stdout:8/224: dread d3/da/d26/d35/f38 [0,4194304] 0 2024-04-06T03:40:30.906 INFO:tasks.workunit.client.1.smithi136.stdout:8/225: fdatasync d3/da/d26/d35/f38 0 2024-04-06T03:40:30.906 INFO:tasks.workunit.client.1.smithi136.stdout:5/242: unlink d1/l9 0 2024-04-06T03:40:30.907 INFO:tasks.workunit.client.0.smithi094.stdout:8/230: mkdir d18/d34/d52 0 2024-04-06T03:40:30.907 INFO:tasks.workunit.client.0.smithi094.stdout:8/231: truncate d18/f36 216201 0 2024-04-06T03:40:30.908 INFO:tasks.workunit.client.0.smithi094.stdout:9/168: mknod d2/dc/d21/c3d 0 2024-04-06T03:40:30.909 INFO:tasks.workunit.client.1.smithi136.stdout:9/183: truncate f13 3532150 0 2024-04-06T03:40:30.910 INFO:tasks.workunit.client.1.smithi136.stdout:0/169: dread d1/f4 [0,4194304] 0 2024-04-06T03:40:30.913 INFO:tasks.workunit.client.0.smithi094.stdout:7/242: dwrite d2/f23 [0,4194304] 0 2024-04-06T03:40:30.914 INFO:tasks.workunit.client.1.smithi136.stdout:1/212: creat d12/d1b/d22/f43 x:0 0 0 2024-04-06T03:40:30.914 INFO:tasks.workunit.client.1.smithi136.stdout:1/213: dread - d12/d1b/f35 zero size 2024-04-06T03:40:30.914 INFO:tasks.workunit.client.1.smithi136.stdout:1/214: write d12/d1b/d28/f3f [902854,27997] 0 2024-04-06T03:40:30.917 INFO:tasks.workunit.client.0.smithi094.stdout:1/216: write d0/d7/d1a/f1e [2023048,46143] 0 2024-04-06T03:40:30.917 INFO:tasks.workunit.client.0.smithi094.stdout:3/265: mknod d9/d10/d27/d3a/c5b 0 2024-04-06T03:40:30.919 INFO:tasks.workunit.client.1.smithi136.stdout:6/173: dread d2/de/f2e [0,4194304] 0 2024-04-06T03:40:30.920 INFO:tasks.workunit.client.1.smithi136.stdout:2/308: unlink d3/d45/f63 0 2024-04-06T03:40:30.920 INFO:tasks.workunit.client.0.smithi094.stdout:6/203: write dc/d11/d1b/d24/f25 [943654,61275] 0 2024-04-06T03:40:30.922 INFO:tasks.workunit.client.0.smithi094.stdout:0/254: symlink d1/d9/dd/d18/d32/d42/l56 0 2024-04-06T03:40:30.922 INFO:tasks.workunit.client.0.smithi094.stdout:3/266: dread d9/d1d/f25 [0,4194304] 0 2024-04-06T03:40:30.922 INFO:tasks.workunit.client.0.smithi094.stdout:3/267: chown d9 8 1 2024-04-06T03:40:30.922 INFO:tasks.workunit.client.0.smithi094.stdout:1/217: fdatasync d0/d7/d1a/f1e 0 2024-04-06T03:40:30.922 INFO:tasks.workunit.client.1.smithi136.stdout:2/309: write d3/d15/d19/f39 [391688,72289] 0 2024-04-06T03:40:30.924 INFO:tasks.workunit.client.0.smithi094.stdout:9/169: dwrite d2/d2c/d33/d37/f3a [0,4194304] 0 2024-04-06T03:40:30.928 INFO:tasks.workunit.client.0.smithi094.stdout:8/232: unlink d18/d2c/f33 0 2024-04-06T03:40:30.928 INFO:tasks.workunit.client.0.smithi094.stdout:8/233: stat d18/f19 0 2024-04-06T03:40:30.928 INFO:tasks.workunit.client.1.smithi136.stdout:7/189: dwrite f1 [0,4194304] 0 2024-04-06T03:40:30.929 INFO:tasks.workunit.client.1.smithi136.stdout:8/226: symlink d3/d2d/l40 0 2024-04-06T03:40:30.930 INFO:tasks.workunit.client.1.smithi136.stdout:8/227: fdatasync d3/de/f27 0 2024-04-06T03:40:30.931 INFO:tasks.workunit.client.1.smithi136.stdout:5/243: getdents d1/dc/d10/d1e 0 2024-04-06T03:40:30.934 INFO:tasks.workunit.client.1.smithi136.stdout:4/169: rename l18 to d1d/l42 0 2024-04-06T03:40:30.935 INFO:tasks.workunit.client.0.smithi094.stdout:7/243: truncate d2/d6/f9 1977106 0 2024-04-06T03:40:30.937 INFO:tasks.workunit.client.1.smithi136.stdout:9/184: mkdir d17/d20/d25/d3c 0 2024-04-06T03:40:30.937 INFO:tasks.workunit.client.1.smithi136.stdout:9/185: dread - d17/d20/d25/f31 zero size 2024-04-06T03:40:30.946 INFO:tasks.workunit.client.1.smithi136.stdout:6/174: mknod d2/d7/d26/d30/c42 0 2024-04-06T03:40:30.947 INFO:tasks.workunit.client.0.smithi094.stdout:1/218: write d0/d4/fc [623077,61127] 0 2024-04-06T03:40:30.947 INFO:tasks.workunit.client.0.smithi094.stdout:0/255: dwrite d1/d9/dd/f3b [0,4194304] 0 2024-04-06T03:40:30.947 INFO:tasks.workunit.client.0.smithi094.stdout:6/204: mknod dc/d11/d22/c4f 0 2024-04-06T03:40:30.947 INFO:tasks.workunit.client.1.smithi136.stdout:8/228: dread d3/f21 [0,4194304] 0 2024-04-06T03:40:30.947 INFO:tasks.workunit.client.1.smithi136.stdout:8/229: chown d3/de/d18/c30 5076 1 2024-04-06T03:40:30.947 INFO:tasks.workunit.client.1.smithi136.stdout:0/170: dwrite d1/d6/d2c/d1a/d1c/f31 [0,4194304] 0 2024-04-06T03:40:30.947 INFO:tasks.workunit.client.1.smithi136.stdout:0/171: chown d1/d6 1929838 1 2024-04-06T03:40:30.947 INFO:tasks.workunit.client.0.smithi094.stdout:0/256: read d1/d9/dd/d18/d1e/d22/f37 [672321,39758] 0 2024-04-06T03:40:30.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/257: write d1/d9/dd/d18/d1a/f4d [4186478,61117] 0 2024-04-06T03:40:30.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/258: write d1/d9/dd/f1c [4491227,24581] 0 2024-04-06T03:40:30.949 INFO:tasks.workunit.client.1.smithi136.stdout:2/310: link d3/d9/d10/d18/d1d/d3c/f51 d3/d9/d10/f6b 0 2024-04-06T03:40:30.951 INFO:tasks.workunit.client.0.smithi094.stdout:3/268: rmdir d9/d20/d3e/d52 0 2024-04-06T03:40:30.955 INFO:tasks.workunit.client.1.smithi136.stdout:7/190: symlink d5/df/d11/d22/l40 0 2024-04-06T03:40:30.955 INFO:tasks.workunit.client.1.smithi136.stdout:7/191: write d5/df/d12/f37 [1477185,59431] 0 2024-04-06T03:40:30.955 INFO:tasks.workunit.client.1.smithi136.stdout:7/192: creat d5/df/f41 x:0 0 0 2024-04-06T03:40:30.955 INFO:tasks.workunit.client.1.smithi136.stdout:7/193: write d5/df/d1d/f1f [943680,93252] 0 2024-04-06T03:40:30.955 INFO:tasks.workunit.client.1.smithi136.stdout:7/194: getdents d5/df/d11/d22/d2a 0 2024-04-06T03:40:30.956 INFO:tasks.workunit.client.1.smithi136.stdout:5/244: mknod d1/c4c 0 2024-04-06T03:40:30.957 INFO:tasks.workunit.client.0.smithi094.stdout:9/170: mknod d2/dc/c3e 0 2024-04-06T03:40:30.957 INFO:tasks.workunit.client.0.smithi094.stdout:9/171: stat d2/d2c/f36 0 2024-04-06T03:40:30.957 INFO:tasks.workunit.client.0.smithi094.stdout:9/172: stat d2/dc/l28 0 2024-04-06T03:40:30.957 INFO:tasks.workunit.client.1.smithi136.stdout:9/186: dwrite f11 [0,4194304] 0 2024-04-06T03:40:30.959 INFO:tasks.workunit.client.1.smithi136.stdout:3/149: rename d0/f25 to d0/d4/f30 0 2024-04-06T03:40:30.960 INFO:tasks.workunit.client.0.smithi094.stdout:8/234: creat d18/d3c/d49/f53 x:0 0 0 2024-04-06T03:40:30.961 INFO:tasks.workunit.client.1.smithi136.stdout:1/215: rmdir d12/d1b/d1e/d27 39 2024-04-06T03:40:30.962 INFO:tasks.workunit.client.1.smithi136.stdout:1/216: dread - d12/d1b/f29 zero size 2024-04-06T03:40:30.962 INFO:tasks.workunit.client.1.smithi136.stdout:4/170: dwrite d1d/f29 [0,4194304] 0 2024-04-06T03:40:30.962 INFO:tasks.workunit.client.1.smithi136.stdout:4/171: truncate d1d/d31/f3f 1048226 0 2024-04-06T03:40:30.963 INFO:tasks.workunit.client.0.smithi094.stdout:7/244: creat d2/d6/dc/dd/df/f42 x:0 0 0 2024-04-06T03:40:30.965 INFO:tasks.workunit.client.0.smithi094.stdout:0/259: dread d1/f2c [0,4194304] 0 2024-04-06T03:40:30.966 INFO:tasks.workunit.client.1.smithi136.stdout:6/175: rmdir d2/d25/d41 0 2024-04-06T03:40:30.966 INFO:tasks.workunit.client.1.smithi136.stdout:6/176: write d2/f9 [3150806,88107] 0 2024-04-06T03:40:30.969 INFO:tasks.workunit.client.1.smithi136.stdout:4/172: dread f7 [0,4194304] 0 2024-04-06T03:40:30.975 INFO:tasks.workunit.client.0.smithi094.stdout:1/219: dwrite d0/d7/f46 [0,4194304] 0 2024-04-06T03:40:30.976 INFO:tasks.workunit.client.1.smithi136.stdout:3/150: dread d0/d4/f22 [0,4194304] 0 2024-04-06T03:40:30.976 INFO:tasks.workunit.client.1.smithi136.stdout:7/195: dwrite d5/d21/f24 [0,4194304] 0 2024-04-06T03:40:30.978 INFO:tasks.workunit.client.0.smithi094.stdout:6/205: mkdir dc/d11/d22/d50 0 2024-04-06T03:40:30.978 INFO:tasks.workunit.client.0.smithi094.stdout:6/206: truncate dc/d11/f4b 937347 0 2024-04-06T03:40:30.980 INFO:tasks.workunit.client.1.smithi136.stdout:8/230: truncate d3/da/fb 537080 0 2024-04-06T03:40:30.981 INFO:tasks.workunit.client.1.smithi136.stdout:6/177: dread d2/f5 [0,4194304] 0 2024-04-06T03:40:30.981 INFO:tasks.workunit.client.1.smithi136.stdout:6/178: write d2/de/f2e [4686086,123124] 0 2024-04-06T03:40:30.981 INFO:tasks.workunit.client.1.smithi136.stdout:4/173: dread d1d/f29 [0,4194304] 0 2024-04-06T03:40:30.982 INFO:tasks.workunit.client.1.smithi136.stdout:0/172: creat d1/d6/d2c/d1a/f39 x:0 0 0 2024-04-06T03:40:30.985 INFO:tasks.workunit.client.0.smithi094.stdout:3/269: creat d9/d48/f5c x:0 0 0 2024-04-06T03:40:30.988 INFO:tasks.workunit.client.1.smithi136.stdout:3/151: dread d0/f23 [0,4194304] 0 2024-04-06T03:40:30.989 INFO:tasks.workunit.client.0.smithi094.stdout:1/220: dread d0/d1/f10 [0,4194304] 0 2024-04-06T03:40:30.989 INFO:tasks.workunit.client.0.smithi094.stdout:9/173: link d2/fe d2/dc/d21/d23/f3f 0 2024-04-06T03:40:30.989 INFO:tasks.workunit.client.0.smithi094.stdout:9/174: stat d2/dc/c24 0 2024-04-06T03:40:30.994 INFO:tasks.workunit.client.1.smithi136.stdout:0/173: dread d1/d6/fd [0,4194304] 0 2024-04-06T03:40:30.996 INFO:tasks.workunit.client.1.smithi136.stdout:7/196: dwrite d5/df/f41 [0,4194304] 0 2024-04-06T03:40:30.996 INFO:tasks.workunit.client.1.smithi136.stdout:5/245: truncate d1/dc/d1c/f36 3841211 0 2024-04-06T03:40:31.000 INFO:tasks.workunit.client.0.smithi094.stdout:0/260: rename d1/d9/f41 to d1/d9/dd/d18/d1d/f57 0 2024-04-06T03:40:31.000 INFO:tasks.workunit.client.0.smithi094.stdout:9/175: dread d2/dc/f18 [4194304,4194304] 0 2024-04-06T03:40:31.006 INFO:tasks.workunit.client.1.smithi136.stdout:4/174: dwrite d1d/d2f/f32 [0,4194304] 0 2024-04-06T03:40:31.006 INFO:tasks.workunit.client.1.smithi136.stdout:4/175: dread - d1d/f25 zero size 2024-04-06T03:40:31.006 INFO:tasks.workunit.client.1.smithi136.stdout:4/176: chown d1d/d31/f40 276061 1 2024-04-06T03:40:31.007 INFO:tasks.workunit.client.1.smithi136.stdout:9/187: mkdir d17/d20/d25/d3d 0 2024-04-06T03:40:31.008 INFO:tasks.workunit.client.0.smithi094.stdout:1/221: dwrite d0/d1/d16/d24/d30/f33 [0,4194304] 0 2024-04-06T03:40:31.008 INFO:tasks.workunit.client.0.smithi094.stdout:1/222: stat d0/d1/d16/d24/d30/f33 0 2024-04-06T03:40:31.009 INFO:tasks.workunit.client.1.smithi136.stdout:0/174: dwrite d1/d6/d2c/d1a/f39 [0,4194304] 0 2024-04-06T03:40:31.010 INFO:tasks.workunit.client.1.smithi136.stdout:1/217: mkdir d12/d3b/d44 0 2024-04-06T03:40:31.010 INFO:tasks.workunit.client.1.smithi136.stdout:1/218: truncate d12/d1b/d28/f3f 1399597 0 2024-04-06T03:40:31.024 INFO:tasks.workunit.client.0.smithi094.stdout:6/207: creat dc/d3a/d4a/f51 x:0 0 0 2024-04-06T03:40:31.025 INFO:tasks.workunit.client.0.smithi094.stdout:6/208: fsync dc/d11/d1b/f31 0 2024-04-06T03:40:31.026 INFO:tasks.workunit.client.1.smithi136.stdout:8/231: creat d3/da/f41 x:0 0 0 2024-04-06T03:40:31.027 INFO:tasks.workunit.client.0.smithi094.stdout:3/270: truncate d9/d20/f34 3413691 0 2024-04-06T03:40:31.028 INFO:tasks.workunit.client.1.smithi136.stdout:4/177: dwrite d1d/f25 [0,4194304] 0 2024-04-06T03:40:31.029 INFO:tasks.workunit.client.1.smithi136.stdout:6/179: truncate d2/de/f27 1500884 0 2024-04-06T03:40:31.029 INFO:tasks.workunit.client.1.smithi136.stdout:6/180: fdatasync d2/f5 0 2024-04-06T03:40:31.029 INFO:tasks.workunit.client.1.smithi136.stdout:6/181: stat d2/d7/d37 0 2024-04-06T03:40:31.029 INFO:tasks.workunit.client.1.smithi136.stdout:9/188: dwrite d17/d20/d25/f26 [0,4194304] 0 2024-04-06T03:40:31.031 INFO:tasks.workunit.client.0.smithi094.stdout:1/223: dwrite d0/d1/d16/d3f/f4e [0,4194304] 0 2024-04-06T03:40:31.031 INFO:tasks.workunit.client.1.smithi136.stdout:2/311: getdents d3/d15 0 2024-04-06T03:40:31.033 INFO:tasks.workunit.client.0.smithi094.stdout:6/209: dread dc/d3a/f41 [0,4194304] 0 2024-04-06T03:40:31.035 INFO:tasks.workunit.client.1.smithi136.stdout:3/152: creat d0/d4/d12/d19/d2c/f31 x:0 0 0 2024-04-06T03:40:31.035 INFO:tasks.workunit.client.1.smithi136.stdout:3/153: dread - d0/fb zero size 2024-04-06T03:40:31.035 INFO:tasks.workunit.client.1.smithi136.stdout:9/189: dread d17/d21/d29/f36 [0,4194304] 0 2024-04-06T03:40:31.035 INFO:tasks.workunit.client.1.smithi136.stdout:9/190: dread - d17/d19/f1e zero size 2024-04-06T03:40:31.046 INFO:tasks.workunit.client.1.smithi136.stdout:7/197: mknod d5/df/d12/d1b/c42 0 2024-04-06T03:40:31.048 INFO:tasks.workunit.client.1.smithi136.stdout:5/246: chown d1/dc/d10/c46 992 1 2024-04-06T03:40:31.049 INFO:tasks.workunit.client.1.smithi136.stdout:5/247: truncate d1/dc/d10/f11 4366876 0 2024-04-06T03:40:31.049 INFO:tasks.workunit.client.0.smithi094.stdout:8/235: rmdir d18 39 2024-04-06T03:40:31.050 INFO:tasks.workunit.client.1.smithi136.stdout:9/191: write d17/d21/d29/f2a [760921,36597] 0 2024-04-06T03:40:31.051 INFO:tasks.workunit.client.0.smithi094.stdout:0/261: creat d1/d9/dd/d18/d1e/d22/d35/f58 x:0 0 0 2024-04-06T03:40:31.051 INFO:tasks.workunit.client.0.smithi094.stdout:0/262: write d1/f2c [3252454,41171] 0 2024-04-06T03:40:31.051 INFO:tasks.workunit.client.0.smithi094.stdout:0/263: stat d1/d9/dd/d18/d1e/d22/d35 0 2024-04-06T03:40:31.052 INFO:tasks.workunit.client.1.smithi136.stdout:7/198: dread d5/df/f10 [0,4194304] 0 2024-04-06T03:40:31.055 INFO:tasks.workunit.client.0.smithi094.stdout:3/271: symlink d9/d10/l5d 0 2024-04-06T03:40:31.055 INFO:tasks.workunit.client.0.smithi094.stdout:3/272: readlink d9/d48/l4c 0 2024-04-06T03:40:31.055 INFO:tasks.workunit.client.0.smithi094.stdout:3/273: chown d9/f19 0 1 2024-04-06T03:40:31.056 INFO:tasks.workunit.client.0.smithi094.stdout:4/295: sync 2024-04-06T03:40:31.056 INFO:tasks.workunit.client.0.smithi094.stdout:4/296: read - d0/df/d13/f58 zero size 2024-04-06T03:40:31.056 INFO:tasks.workunit.client.0.smithi094.stdout:5/174: sync 2024-04-06T03:40:31.056 INFO:tasks.workunit.client.0.smithi094.stdout:5/175: chown db/f2a 308806 1 2024-04-06T03:40:31.056 INFO:tasks.workunit.client.0.smithi094.stdout:2/285: sync 2024-04-06T03:40:31.057 INFO:tasks.workunit.client.1.smithi136.stdout:0/175: mknod d1/d2e/c3a 0 2024-04-06T03:40:31.057 INFO:tasks.workunit.client.1.smithi136.stdout:0/176: stat d1/d6/d2c/c25 0 2024-04-06T03:40:31.059 INFO:tasks.workunit.client.1.smithi136.stdout:3/154: dwrite d0/d4/d12/f15 [0,4194304] 0 2024-04-06T03:40:31.059 INFO:tasks.workunit.client.1.smithi136.stdout:3/155: chown d0/d4/d12 26020 1 2024-04-06T03:40:31.062 INFO:tasks.workunit.client.1.smithi136.stdout:1/219: link d12/c19 d12/d1b/d22/c45 0 2024-04-06T03:40:31.064 INFO:tasks.workunit.client.0.smithi094.stdout:4/297: dread d0/d4/fd [0,4194304] 0 2024-04-06T03:40:31.065 INFO:tasks.workunit.client.1.smithi136.stdout:9/192: write d17/d19/f35 [276401,61639] 0 2024-04-06T03:40:31.065 INFO:tasks.workunit.client.1.smithi136.stdout:2/312: dwrite d3/d9/d10/d18/f48 [0,4194304] 0 2024-04-06T03:40:31.065 INFO:tasks.workunit.client.1.smithi136.stdout:2/313: chown d3/d15/d19/d43/d54 45621013 1 2024-04-06T03:40:31.065 INFO:tasks.workunit.client.1.smithi136.stdout:8/232: stat d3/de/d18/c30 0 2024-04-06T03:40:31.065 INFO:tasks.workunit.client.1.smithi136.stdout:8/233: fdatasync d3/da/fd 0 2024-04-06T03:40:31.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/224: creat d0/d4/d1b/d2f/d3c/d48/f4f x:0 0 0 2024-04-06T03:40:31.072 INFO:tasks.workunit.client.1.smithi136.stdout:9/193: dread d17/d19/f27 [0,4194304] 0 2024-04-06T03:40:31.072 INFO:tasks.workunit.client.1.smithi136.stdout:1/220: dread fb [0,4194304] 0 2024-04-06T03:40:31.073 INFO:tasks.workunit.client.1.smithi136.stdout:1/221: readlink d12/d1b/l1c 0 2024-04-06T03:40:31.073 INFO:tasks.workunit.client.1.smithi136.stdout:1/222: dread - d12/d1b/d22/f43 zero size 2024-04-06T03:40:31.073 INFO:tasks.workunit.client.0.smithi094.stdout:6/210: creat dc/d11/d1b/f52 x:0 0 0 2024-04-06T03:40:31.075 INFO:tasks.workunit.client.0.smithi094.stdout:2/286: dwrite d0/dd/d3d/d49/f53 [0,4194304] 0 2024-04-06T03:40:31.076 INFO:tasks.workunit.client.0.smithi094.stdout:8/236: unlink l9 0 2024-04-06T03:40:31.078 INFO:tasks.workunit.client.1.smithi136.stdout:4/178: creat d1d/d2f/f43 x:0 0 0 2024-04-06T03:40:31.078 INFO:tasks.workunit.client.1.smithi136.stdout:4/179: chown fc 27 1 2024-04-06T03:40:31.078 INFO:tasks.workunit.client.1.smithi136.stdout:4/180: fdatasync d1d/d31/f3f 0 2024-04-06T03:40:31.078 INFO:tasks.workunit.client.1.smithi136.stdout:4/181: creat d1d/d31/f44 x:0 0 0 2024-04-06T03:40:31.078 INFO:tasks.workunit.client.1.smithi136.stdout:4/182: dread - d1d/f38 zero size 2024-04-06T03:40:31.079 INFO:tasks.workunit.client.1.smithi136.stdout:7/199: dwrite d5/fd [4194304,4194304] 0 2024-04-06T03:40:31.079 INFO:tasks.workunit.client.0.smithi094.stdout:0/264: truncate d1/d10/f2e 4085314 0 2024-04-06T03:40:31.079 INFO:tasks.workunit.client.0.smithi094.stdout:0/265: write d1/d9/dd/d18/d1e/d22/f49 [720133,33834] 0 2024-04-06T03:40:31.080 INFO:tasks.workunit.client.0.smithi094.stdout:0/266: dread d1/d9/dd/d18/d1e/d22/d35/f52 [0,4194304] 0 2024-04-06T03:40:31.081 INFO:tasks.workunit.client.1.smithi136.stdout:6/182: rmdir d2/d7/d37 0 2024-04-06T03:40:31.083 INFO:tasks.workunit.client.0.smithi094.stdout:9/176: symlink d2/l40 0 2024-04-06T03:40:31.083 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:30 smithi094 ceph-mon[29713]: Reconfiguring node-exporter.smithi094 (dependencies changed)... 2024-04-06T03:40:31.083 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:30 smithi094 ceph-mon[29713]: Reconfiguring daemon node-exporter.smithi094 on smithi094 2024-04-06T03:40:31.083 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:30 smithi094 ceph-mon[29713]: pgmap v15: 65 pgs: 65 active+clean; 514 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 26 MiB/s rd, 39 MiB/s wr, 611 op/s 2024-04-06T03:40:31.085 INFO:tasks.workunit.client.0.smithi094.stdout:3/274: symlink d9/d1e/d26/l5e 0 2024-04-06T03:40:31.086 INFO:tasks.workunit.client.1.smithi136.stdout:2/314: dwrite d3/d9/d10/d18/d4b/f65 [0,4194304] 0 2024-04-06T03:40:31.088 INFO:tasks.workunit.client.1.smithi136.stdout:7/200: write d5/d21/f24 [2147002,7226] 0 2024-04-06T03:40:31.089 INFO:tasks.workunit.client.1.smithi136.stdout:7/201: creat d5/d21/f43 x:0 0 0 2024-04-06T03:40:31.089 INFO:tasks.workunit.client.1.smithi136.stdout:7/202: creat d5/df/d3d/f44 x:0 0 0 2024-04-06T03:40:31.089 INFO:tasks.workunit.client.0.smithi094.stdout:5/176: symlink db/d19/d24/d28/l42 0 2024-04-06T03:40:31.090 INFO:tasks.workunit.client.0.smithi094.stdout:8/237: write f13 [1579996,91880] 0 2024-04-06T03:40:31.090 INFO:tasks.workunit.client.0.smithi094.stdout:8/238: fsync f14 0 2024-04-06T03:40:31.091 INFO:tasks.workunit.client.1.smithi136.stdout:5/248: getdents d1/dc/d1c/d32 0 2024-04-06T03:40:31.092 INFO:tasks.workunit.client.0.smithi094.stdout:4/298: symlink d0/d15/d3a/d3e/d46/l63 0 2024-04-06T03:40:31.093 INFO:tasks.workunit.client.0.smithi094.stdout:2/287: dwrite d0/f1 [4194304,4194304] 0 2024-04-06T03:40:31.094 INFO:tasks.workunit.client.0.smithi094.stdout:2/288: truncate d0/d18/f55 29879 0 2024-04-06T03:40:31.094 INFO:tasks.workunit.client.1.smithi136.stdout:9/194: dread d17/d19/f2f [0,4194304] 0 2024-04-06T03:40:31.095 INFO:tasks.workunit.client.0.smithi094.stdout:1/225: mkdir d0/d1/d16/d40/d50 0 2024-04-06T03:40:31.095 INFO:tasks.workunit.client.0.smithi094.stdout:1/226: dread - d0/d1/d16/f3b zero size 2024-04-06T03:40:31.096 INFO:tasks.workunit.client.0.smithi094.stdout:1/227: chown d0/d7/c1f 272359219 1 2024-04-06T03:40:31.096 INFO:tasks.workunit.client.0.smithi094.stdout:1/228: readlink d0/d7/d2d/l22 0 2024-04-06T03:40:31.096 INFO:tasks.workunit.client.0.smithi094.stdout:1/229: fdatasync d0/d1/d27/f3d 0 2024-04-06T03:40:31.097 INFO:tasks.workunit.client.1.smithi136.stdout:0/177: link d1/d6/l22 d1/d6/d2c/l3b 0 2024-04-06T03:40:31.097 INFO:tasks.workunit.client.1.smithi136.stdout:0/178: stat d1/d6/f11 0 2024-04-06T03:40:31.098 INFO:tasks.workunit.client.0.smithi094.stdout:6/211: mknod dc/d11/d1b/d24/c53 0 2024-04-06T03:40:31.098 INFO:tasks.workunit.client.0.smithi094.stdout:6/212: write fa [4928294,53266] 0 2024-04-06T03:40:31.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:30 smithi136 ceph-mon[36436]: Reconfiguring node-exporter.smithi094 (dependencies changed)... 2024-04-06T03:40:31.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:30 smithi136 ceph-mon[36436]: Reconfiguring daemon node-exporter.smithi094 on smithi094 2024-04-06T03:40:31.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:30 smithi136 ceph-mon[36436]: pgmap v15: 65 pgs: 65 active+clean; 514 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 26 MiB/s rd, 39 MiB/s wr, 611 op/s 2024-04-06T03:40:31.105 INFO:tasks.workunit.client.0.smithi094.stdout:3/275: dwrite d9/d1d/f42 [0,4194304] 0 2024-04-06T03:40:31.108 INFO:tasks.workunit.client.1.smithi136.stdout:1/223: creat d12/d1b/d1e/d27/d37/f46 x:0 0 0 2024-04-06T03:40:31.110 INFO:tasks.workunit.client.0.smithi094.stdout:9/177: unlink d2/dc/d10/l1c 0 2024-04-06T03:40:31.110 INFO:tasks.workunit.client.0.smithi094.stdout:9/178: creat d2/d2c/f41 x:0 0 0 2024-04-06T03:40:31.110 INFO:tasks.workunit.client.1.smithi136.stdout:2/315: dwrite d3/d9/d10/d18/d1d/f35 [4194304,4194304] 0 2024-04-06T03:40:31.110 INFO:tasks.workunit.client.1.smithi136.stdout:2/316: fsync d3/d9/d10/d18/f2a 0 2024-04-06T03:40:31.112 INFO:tasks.workunit.client.0.smithi094.stdout:5/177: creat db/d40/f43 x:0 0 0 2024-04-06T03:40:31.113 INFO:tasks.workunit.client.1.smithi136.stdout:4/183: mknod d1d/d2f/c45 0 2024-04-06T03:40:31.115 INFO:tasks.workunit.client.1.smithi136.stdout:6/183: link d2/de/c1d d2/d7/d26/d30/c43 0 2024-04-06T03:40:31.117 INFO:tasks.workunit.client.0.smithi094.stdout:8/239: mkdir d18/d3c/d54 0 2024-04-06T03:40:31.118 INFO:tasks.workunit.client.1.smithi136.stdout:2/317: dread d3/d15/f36 [0,4194304] 0 2024-04-06T03:40:31.118 INFO:tasks.workunit.client.0.smithi094.stdout:6/213: dwrite dc/d11/d1b/d24/f2e [4194304,4194304] 0 2024-04-06T03:40:31.119 INFO:tasks.workunit.client.0.smithi094.stdout:4/299: rename d0/d4/f51 to d0/df/d13/d4a/f64 0 2024-04-06T03:40:31.123 INFO:tasks.workunit.client.0.smithi094.stdout:1/230: symlink d0/d1/d16/d40/d50/l51 0 2024-04-06T03:40:31.124 INFO:tasks.workunit.client.0.smithi094.stdout:1/231: truncate d0/d1/d16/d24/d30/f32 1220605 0 2024-04-06T03:40:31.127 INFO:tasks.workunit.client.1.smithi136.stdout:7/203: symlink d5/df/d1d/l45 0 2024-04-06T03:40:31.127 INFO:tasks.workunit.client.1.smithi136.stdout:7/204: chown d5/l34 346 1 2024-04-06T03:40:31.127 INFO:tasks.workunit.client.1.smithi136.stdout:7/205: creat d5/f46 x:0 0 0 2024-04-06T03:40:31.130 INFO:tasks.workunit.client.0.smithi094.stdout:3/276: truncate d9/f3f 2148803 0 2024-04-06T03:40:31.130 INFO:tasks.workunit.client.0.smithi094.stdout:3/277: creat d9/f5f x:0 0 0 2024-04-06T03:40:31.133 INFO:tasks.workunit.client.1.smithi136.stdout:9/195: link d17/d20/l22 d17/d21/d29/d34/l3e 0 2024-04-06T03:40:31.136 INFO:tasks.workunit.client.0.smithi094.stdout:0/267: getdents d1/d9/dd/d18/d1a 0 2024-04-06T03:40:31.136 INFO:tasks.workunit.client.0.smithi094.stdout:0/268: write d1/d9/dd/d18/d1a/f54 [476056,62289] 0 2024-04-06T03:40:31.138 INFO:tasks.workunit.client.1.smithi136.stdout:0/179: symlink d1/d6/d2c/d1a/l3c 0 2024-04-06T03:40:31.138 INFO:tasks.workunit.client.1.smithi136.stdout:0/180: dread - d1/d6/d2c/d1a/f1d zero size 2024-04-06T03:40:31.141 INFO:tasks.workunit.client.1.smithi136.stdout:3/156: getdents d0/d4 0 2024-04-06T03:40:31.141 INFO:tasks.workunit.client.1.smithi136.stdout:3/157: creat d0/f32 x:0 0 0 2024-04-06T03:40:31.142 INFO:tasks.workunit.client.0.smithi094.stdout:9/179: link d2/dc/d21/l22 d2/d2c/d33/l42 0 2024-04-06T03:40:31.144 INFO:tasks.workunit.client.1.smithi136.stdout:8/234: getdents d3/da/d26/d35 0 2024-04-06T03:40:31.144 INFO:tasks.workunit.client.1.smithi136.stdout:8/235: fsync d3/da/d26/f28 0 2024-04-06T03:40:31.144 INFO:tasks.workunit.client.1.smithi136.stdout:8/236: creat d3/da/f42 x:0 0 0 2024-04-06T03:40:31.144 INFO:tasks.workunit.client.1.smithi136.stdout:8/237: readlink d3/l15 0 2024-04-06T03:40:31.145 INFO:tasks.workunit.client.0.smithi094.stdout:3/278: dwrite d9/d10/f16 [4194304,4194304] 0 2024-04-06T03:40:31.145 INFO:tasks.workunit.client.1.smithi136.stdout:1/224: mknod d12/d1b/d28/c47 0 2024-04-06T03:40:31.147 INFO:tasks.workunit.client.0.smithi094.stdout:6/214: mkdir dc/d3a/d4a/d54 0 2024-04-06T03:40:31.147 INFO:tasks.workunit.client.0.smithi094.stdout:6/215: creat dc/d11/d1b/d24/d46/f55 x:0 0 0 2024-04-06T03:40:31.151 INFO:tasks.workunit.client.0.smithi094.stdout:5/178: rename db/f1d to db/d19/d30/f44 0 2024-04-06T03:40:31.151 INFO:tasks.workunit.client.0.smithi094.stdout:5/179: creat db/d19/f45 x:0 0 0 2024-04-06T03:40:31.151 INFO:tasks.workunit.client.0.smithi094.stdout:5/180: write db/d19/d24/f37 [229844,117693] 0 2024-04-06T03:40:31.152 INFO:tasks.workunit.client.1.smithi136.stdout:4/184: symlink d1d/d31/d3e/l46 0 2024-04-06T03:40:31.153 INFO:tasks.workunit.client.0.smithi094.stdout:4/300: creat d0/d15/d3a/d3e/d50/f65 x:0 0 0 2024-04-06T03:40:31.154 INFO:tasks.workunit.client.0.smithi094.stdout:5/181: dread db/f2f [0,4194304] 0 2024-04-06T03:40:31.155 INFO:tasks.workunit.client.0.smithi094.stdout:6/216: dread fa [0,4194304] 0 2024-04-06T03:40:31.155 INFO:tasks.workunit.client.0.smithi094.stdout:6/217: creat dc/d3a/d4a/f56 x:0 0 0 2024-04-06T03:40:31.155 INFO:tasks.workunit.client.0.smithi094.stdout:6/218: creat dc/d11/d22/f57 x:0 0 0 2024-04-06T03:40:31.156 INFO:tasks.workunit.client.1.smithi136.stdout:6/184: unlink d2/de/c16 0 2024-04-06T03:40:31.156 INFO:tasks.workunit.client.1.smithi136.stdout:6/185: chown d2/d7/d26/d30 6497147 1 2024-04-06T03:40:31.157 INFO:tasks.workunit.client.0.smithi094.stdout:2/289: mknod d0/c56 0 2024-04-06T03:40:31.162 INFO:tasks.workunit.client.0.smithi094.stdout:5/182: dread db/fc [0,4194304] 0 2024-04-06T03:40:31.163 INFO:tasks.workunit.client.0.smithi094.stdout:4/301: dread d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:31.169 INFO:tasks.workunit.client.0.smithi094.stdout:6/219: dwrite dc/f28 [0,4194304] 0 2024-04-06T03:40:31.170 INFO:tasks.workunit.client.0.smithi094.stdout:0/269: creat d1/d9/dd/d18/d2d/f59 x:0 0 0 2024-04-06T03:40:31.174 INFO:tasks.workunit.client.1.smithi136.stdout:2/318: creat d3/d15/f6c x:0 0 0 2024-04-06T03:40:31.174 INFO:tasks.workunit.client.1.smithi136.stdout:7/206: rename d5/df/d12/d1b/f3b to d5/df/d12/d1b/f47 0 2024-04-06T03:40:31.175 INFO:tasks.workunit.client.1.smithi136.stdout:9/196: mkdir d17/d21/d32/d3f 0 2024-04-06T03:40:31.176 INFO:tasks.workunit.client.0.smithi094.stdout:4/302: dwrite d0/d4/fb [0,4194304] 0 2024-04-06T03:40:31.178 INFO:tasks.workunit.client.1.smithi136.stdout:2/319: fsync d3/f68 0 2024-04-06T03:40:31.184 INFO:tasks.workunit.client.0.smithi094.stdout:0/270: dread d1/d9/dd/f1c [0,4194304] 0 2024-04-06T03:40:31.184 INFO:tasks.workunit.client.0.smithi094.stdout:0/271: write d1/d9/dd/d18/d32/d42/f4a [418244,124013] 0 2024-04-06T03:40:31.184 INFO:tasks.workunit.client.0.smithi094.stdout:0/272: write d1/d9/dd/d18/d2d/f59 [623164,51116] 0 2024-04-06T03:40:31.184 INFO:tasks.workunit.client.0.smithi094.stdout:0/273: stat d1/d9/dd/d18/d1e/d22 0 2024-04-06T03:40:31.184 INFO:tasks.workunit.client.0.smithi094.stdout:0/274: chown d1/d9/dd/d18/d1e 5 1 2024-04-06T03:40:31.193 INFO:tasks.workunit.client.0.smithi094.stdout:4/303: dwrite d0/f22 [0,4194304] 0 2024-04-06T03:40:31.194 INFO:tasks.workunit.client.1.smithi136.stdout:9/197: dwrite fa [0,4194304] 0 2024-04-06T03:40:31.197 INFO:tasks.workunit.client.0.smithi094.stdout:9/180: link d2/dc/d10/f1b d2/dc/d21/f43 0 2024-04-06T03:40:31.197 INFO:tasks.workunit.client.0.smithi094.stdout:9/181: truncate d2/d2c/d33/d37/f3a 4881280 0 2024-04-06T03:40:31.199 INFO:tasks.workunit.client.0.smithi094.stdout:3/279: symlink d9/d20/l60 0 2024-04-06T03:40:31.200 INFO:tasks.workunit.client.0.smithi094.stdout:3/280: dread d9/d1d/f39 [0,4194304] 0 2024-04-06T03:40:31.202 INFO:tasks.workunit.client.0.smithi094.stdout:8/240: rename d18/d2c/c42 to d18/d3c/d54/c55 0 2024-04-06T03:40:31.202 INFO:tasks.workunit.client.0.smithi094.stdout:8/241: fsync f11 0 2024-04-06T03:40:31.202 INFO:tasks.workunit.client.0.smithi094.stdout:8/242: dread - d18/d34/f3b zero size 2024-04-06T03:40:31.203 INFO:tasks.workunit.client.1.smithi136.stdout:8/238: creat d3/da/d26/d35/d39/f43 x:0 0 0 2024-04-06T03:40:31.203 INFO:tasks.workunit.client.1.smithi136.stdout:1/225: mkdir d12/d1b/d28/d48 0 2024-04-06T03:40:31.203 INFO:tasks.workunit.client.1.smithi136.stdout:8/239: chown d3/da/d26/d35/c20 853 1 2024-04-06T03:40:31.203 INFO:tasks.workunit.client.1.smithi136.stdout:6/186: rename d2/d7/d13/f3f to d2/d7/d13/f44 0 2024-04-06T03:40:31.203 INFO:tasks.workunit.client.1.smithi136.stdout:5/249: rename d1/d1f/l37 to d1/l4d 0 2024-04-06T03:40:31.203 INFO:tasks.workunit.client.1.smithi136.stdout:2/320: unlink d3/d9/c1a 0 2024-04-06T03:40:31.203 INFO:tasks.workunit.client.1.smithi136.stdout:2/321: write d3/f64 [1332769,18352] 0 2024-04-06T03:40:31.204 INFO:tasks.workunit.client.1.smithi136.stdout:2/322: chown d3/d15/d19/c38 297 1 2024-04-06T03:40:31.204 INFO:tasks.workunit.client.1.smithi136.stdout:2/323: fsync d3/f61 0 2024-04-06T03:40:31.209 INFO:tasks.workunit.client.1.smithi136.stdout:9/198: mknod d17/d21/d32/c40 0 2024-04-06T03:40:31.210 INFO:tasks.workunit.client.0.smithi094.stdout:1/232: rmdir d0/d4/d1b 39 2024-04-06T03:40:31.213 INFO:tasks.workunit.client.1.smithi136.stdout:0/181: rename d1/d6/f11 to d1/d6/d2c/f3d 0 2024-04-06T03:40:31.214 INFO:tasks.workunit.client.1.smithi136.stdout:0/182: readlink d1/d6/lf 0 2024-04-06T03:40:31.214 INFO:tasks.workunit.client.1.smithi136.stdout:0/183: chown d1/d6 506426 1 2024-04-06T03:40:31.214 INFO:tasks.workunit.client.1.smithi136.stdout:0/184: chown d1/d6/d2c 98 1 2024-04-06T03:40:31.214 INFO:tasks.workunit.client.1.smithi136.stdout:0/185: chown d1/d6/d2c/d1a/f2a 0 1 2024-04-06T03:40:31.214 INFO:tasks.workunit.client.1.smithi136.stdout:0/186: readlink d1/d6/d2c/d1a/l35 0 2024-04-06T03:40:31.214 INFO:tasks.workunit.client.1.smithi136.stdout:0/187: write d1/fc [4866041,109183] 0 2024-04-06T03:40:31.215 INFO:tasks.workunit.client.1.smithi136.stdout:6/187: mknod d2/d25/c45 0 2024-04-06T03:40:31.215 INFO:tasks.workunit.client.1.smithi136.stdout:6/188: chown d2/de/c36 80 1 2024-04-06T03:40:31.216 INFO:tasks.workunit.client.0.smithi094.stdout:6/220: creat dc/d3a/d4a/d54/f58 x:0 0 0 2024-04-06T03:40:31.218 INFO:tasks.workunit.client.1.smithi136.stdout:5/250: mknod d1/dc/d10/d1e/c4e 0 2024-04-06T03:40:31.219 INFO:tasks.workunit.client.1.smithi136.stdout:2/324: dwrite d3/d45/f67 [0,4194304] 0 2024-04-06T03:40:31.219 INFO:tasks.workunit.client.1.smithi136.stdout:1/226: dwrite d12/d1b/d1e/d1f/f30 [0,4194304] 0 2024-04-06T03:40:31.219 INFO:tasks.workunit.client.1.smithi136.stdout:2/325: fdatasync d3/d15/d19/f60 0 2024-04-06T03:40:31.219 INFO:tasks.workunit.client.1.smithi136.stdout:1/227: write d12/f2c [655016,26852] 0 2024-04-06T03:40:31.220 INFO:tasks.workunit.client.1.smithi136.stdout:2/326: write d3/d9/d10/d18/d1d/d3c/f40 [174951,100564] 0 2024-04-06T03:40:31.221 INFO:tasks.workunit.client.1.smithi136.stdout:9/199: creat d17/d21/d32/d3f/f41 x:0 0 0 2024-04-06T03:40:31.222 INFO:tasks.workunit.client.1.smithi136.stdout:9/200: readlink d17/d19/l38 0 2024-04-06T03:40:31.226 INFO:tasks.workunit.client.1.smithi136.stdout:3/158: rename d0/f5 to d0/d4/d12/d2d/f33 0 2024-04-06T03:40:31.229 INFO:tasks.workunit.client.1.smithi136.stdout:0/188: mkdir d1/d6/d2c/d3e 0 2024-04-06T03:40:31.229 INFO:tasks.workunit.client.1.smithi136.stdout:0/189: chown d1/f7 0 1 2024-04-06T03:40:31.230 INFO:tasks.workunit.client.0.smithi094.stdout:0/275: mknod d1/d9/dd/d18/d1e/c5a 0 2024-04-06T03:40:31.231 INFO:tasks.workunit.client.0.smithi094.stdout:0/276: dread - d1/d9/dd/d18/d1a/f46 zero size 2024-04-06T03:40:31.231 INFO:tasks.workunit.client.1.smithi136.stdout:2/327: write d3/d9/d10/d18/d4b/f65 [4041009,45667] 0 2024-04-06T03:40:31.232 INFO:tasks.workunit.client.1.smithi136.stdout:2/328: read d3/f64 [686956,106981] 0 2024-04-06T03:40:31.234 INFO:tasks.workunit.client.1.smithi136.stdout:1/228: dwrite d12/d1b/d22/f43 [0,4194304] 0 2024-04-06T03:40:31.235 INFO:tasks.workunit.client.1.smithi136.stdout:9/201: dread d17/d20/d25/f26 [0,4194304] 0 2024-04-06T03:40:31.235 INFO:tasks.workunit.client.0.smithi094.stdout:6/221: dwrite dc/d11/d22/f44 [0,4194304] 0 2024-04-06T03:40:31.239 INFO:tasks.workunit.client.0.smithi094.stdout:9/182: creat d2/d2c/d33/f44 x:0 0 0 2024-04-06T03:40:31.239 INFO:tasks.workunit.client.0.smithi094.stdout:9/183: dread - d2/d2c/f41 zero size 2024-04-06T03:40:31.243 INFO:tasks.workunit.client.0.smithi094.stdout:6/222: dread dc/d11/d1b/f38 [0,4194304] 0 2024-04-06T03:40:31.244 INFO:tasks.workunit.client.0.smithi094.stdout:6/223: stat dc/d11/l2a 0 2024-04-06T03:40:31.244 INFO:tasks.workunit.client.1.smithi136.stdout:4/185: rename d1d/l2e to d1d/d31/l47 0 2024-04-06T03:40:31.246 INFO:tasks.workunit.client.0.smithi094.stdout:3/281: mknod d9/d10/d27/d3a/c61 0 2024-04-06T03:40:31.246 INFO:tasks.workunit.client.0.smithi094.stdout:3/282: readlink d9/d20/l24 0 2024-04-06T03:40:31.247 INFO:tasks.workunit.client.1.smithi136.stdout:3/159: getdents d0/d4/d12/d19/d1b 0 2024-04-06T03:40:31.247 INFO:tasks.workunit.client.1.smithi136.stdout:3/160: write d0/d4/fd [5568915,103633] 0 2024-04-06T03:40:31.247 INFO:tasks.workunit.client.1.smithi136.stdout:3/161: read - d0/fb zero size 2024-04-06T03:40:31.249 INFO:tasks.workunit.client.1.smithi136.stdout:0/190: creat d1/d2e/f3f x:0 0 0 2024-04-06T03:40:31.250 INFO:tasks.workunit.client.0.smithi094.stdout:0/277: symlink d1/d9/dd/d18/d1a/l5b 0 2024-04-06T03:40:31.252 INFO:tasks.workunit.client.0.smithi094.stdout:6/224: dread dc/d11/d1b/d24/f25 [0,4194304] 0 2024-04-06T03:40:31.253 INFO:tasks.workunit.client.0.smithi094.stdout:9/184: dwrite d2/f14 [4194304,4194304] 0 2024-04-06T03:40:31.254 INFO:tasks.workunit.client.1.smithi136.stdout:5/251: creat d1/d16/d4b/f4f x:0 0 0 2024-04-06T03:40:31.256 INFO:tasks.workunit.client.1.smithi136.stdout:2/329: unlink d3/d9/f12 0 2024-04-06T03:40:31.256 INFO:tasks.workunit.client.1.smithi136.stdout:2/330: chown d3/d4a 191602456 1 2024-04-06T03:40:31.256 INFO:tasks.workunit.client.1.smithi136.stdout:2/331: write d3/d9/d10/d18/d4b/f5d [4677784,46960] 0 2024-04-06T03:40:31.262 INFO:tasks.workunit.client.0.smithi094.stdout:4/304: link d0/df/d1e/f25 d0/df/f66 0 2024-04-06T03:40:31.262 INFO:tasks.workunit.client.0.smithi094.stdout:4/305: fdatasync d0/d4/f21 0 2024-04-06T03:40:31.264 INFO:tasks.workunit.client.1.smithi136.stdout:0/191: write d1/d6/f12 [1585460,66730] 0 2024-04-06T03:40:31.265 INFO:tasks.workunit.client.1.smithi136.stdout:9/202: creat d17/f42 x:0 0 0 2024-04-06T03:40:31.266 INFO:tasks.workunit.client.1.smithi136.stdout:9/203: write d17/d21/d29/f36 [2964422,23488] 0 2024-04-06T03:40:31.268 INFO:tasks.workunit.client.1.smithi136.stdout:7/207: rename d5/df/d11/d22/f2f to d5/df/d11/d22/d2a/f48 0 2024-04-06T03:40:31.268 INFO:tasks.workunit.client.1.smithi136.stdout:7/208: stat d5/df/d3d/f44 0 2024-04-06T03:40:31.268 INFO:tasks.workunit.client.1.smithi136.stdout:7/209: chown d5/fd 51055 1 2024-04-06T03:40:31.268 INFO:tasks.workunit.client.1.smithi136.stdout:5/252: dwrite d1/dc/d1c/d32/f38 [0,4194304] 0 2024-04-06T03:40:31.268 INFO:tasks.workunit.client.1.smithi136.stdout:5/253: write d1/dc/d10/f49 [799988,98354] 0 2024-04-06T03:40:31.270 INFO:tasks.workunit.client.1.smithi136.stdout:3/162: link d0/d4/d12/d19/f29 d0/d4/d12/d19/d2e/f34 0 2024-04-06T03:40:31.274 INFO:tasks.workunit.client.0.smithi094.stdout:3/283: mkdir d9/d1e/d26/d55/d62 0 2024-04-06T03:40:31.274 INFO:tasks.workunit.client.0.smithi094.stdout:0/278: dwrite d1/f55 [0,4194304] 0 2024-04-06T03:40:31.275 INFO:tasks.workunit.client.0.smithi094.stdout:9/185: dwrite d2/dc/d21/d26/f29 [0,4194304] 0 2024-04-06T03:40:31.275 INFO:tasks.workunit.client.1.smithi136.stdout:2/332: mknod d3/d9/d10/c6d 0 2024-04-06T03:40:31.277 INFO:tasks.workunit.client.0.smithi094.stdout:2/290: rename d0/d18/f35 to d0/f57 0 2024-04-06T03:40:31.281 INFO:tasks.workunit.client.1.smithi136.stdout:9/204: mkdir d17/d21/d32/d43 0 2024-04-06T03:40:31.281 INFO:tasks.workunit.client.1.smithi136.stdout:9/205: creat d17/d21/d32/d3f/f44 x:0 0 0 2024-04-06T03:40:31.284 INFO:tasks.workunit.client.0.smithi094.stdout:3/284: write d9/fd [386615,126148] 0 2024-04-06T03:40:31.284 INFO:tasks.workunit.client.0.smithi094.stdout:3/285: write d9/d20/f34 [3931647,124692] 0 2024-04-06T03:40:31.285 INFO:tasks.workunit.client.1.smithi136.stdout:8/240: rename d3/da/c24 to d3/de/c44 0 2024-04-06T03:40:31.285 INFO:tasks.workunit.client.0.smithi094.stdout:3/286: read d9/d20/d30/f37 [330611,44000] 0 2024-04-06T03:40:31.287 INFO:tasks.workunit.client.0.smithi094.stdout:9/186: dwrite d2/dc/d21/f43 [4194304,4194304] 0 2024-04-06T03:40:31.289 INFO:tasks.workunit.client.1.smithi136.stdout:7/210: creat d5/df/d11/d22/d2a/f49 x:0 0 0 2024-04-06T03:40:31.292 INFO:tasks.workunit.client.0.smithi094.stdout:0/279: getdents d1/d9/dd/d18/d1a 0 2024-04-06T03:40:31.292 INFO:tasks.workunit.client.0.smithi094.stdout:0/280: fsync d1/d9/dd/d18/d1a/f4d 0 2024-04-06T03:40:31.292 INFO:tasks.workunit.client.0.smithi094.stdout:0/281: chown d1/d9/f20 16331 1 2024-04-06T03:40:31.293 INFO:tasks.workunit.client.1.smithi136.stdout:5/254: symlink d1/dc/l50 0 2024-04-06T03:40:31.293 INFO:tasks.workunit.client.1.smithi136.stdout:5/255: chown d1/dc/d1c/f2c 120357115 1 2024-04-06T03:40:31.293 INFO:tasks.workunit.client.0.smithi094.stdout:0/282: dread d1/d9/dd/d18/f31 [0,4194304] 0 2024-04-06T03:40:31.293 INFO:tasks.workunit.client.0.smithi094.stdout:3/287: dread d9/d1d/f3c [0,4194304] 0 2024-04-06T03:40:31.302 INFO:tasks.workunit.client.0.smithi094.stdout:8/243: rename l10 to d18/d30/l56 0 2024-04-06T03:40:31.302 INFO:tasks.workunit.client.0.smithi094.stdout:8/244: read f11 [4504711,45750] 0 2024-04-06T03:40:31.303 INFO:tasks.workunit.client.1.smithi136.stdout:9/206: symlink d17/d21/d29/d34/l45 0 2024-04-06T03:40:31.303 INFO:tasks.workunit.client.1.smithi136.stdout:5/256: write d1/dc/d1c/f36 [583546,105119] 0 2024-04-06T03:40:31.304 INFO:tasks.workunit.client.0.smithi094.stdout:2/291: mknod d0/dd/d1b/c58 0 2024-04-06T03:40:31.304 INFO:tasks.workunit.client.0.smithi094.stdout:2/292: chown d0/d8/d27/d2b/c32 11806870 1 2024-04-06T03:40:31.306 INFO:tasks.workunit.client.0.smithi094.stdout:0/283: dwrite d1/d9/dd/d18/d1a/f4d [0,4194304] 0 2024-04-06T03:40:31.307 INFO:tasks.workunit.client.1.smithi136.stdout:6/189: rename d2/de/f2e to d2/de/d3d/f46 0 2024-04-06T03:40:31.310 INFO:tasks.workunit.client.0.smithi094.stdout:6/225: rmdir dc/d11/d22 39 2024-04-06T03:40:31.310 INFO:tasks.workunit.client.0.smithi094.stdout:6/226: truncate dc/d11/d22/f57 84490 0 2024-04-06T03:40:31.310 INFO:tasks.workunit.client.0.smithi094.stdout:6/227: read dc/d11/d1b/d24/f32 [448201,63923] 0 2024-04-06T03:40:31.315 INFO:tasks.workunit.client.1.smithi136.stdout:6/190: dread d2/de/f27 [0,4194304] 0 2024-04-06T03:40:31.315 INFO:tasks.workunit.client.1.smithi136.stdout:6/191: stat d2/d7 0 2024-04-06T03:40:31.316 INFO:tasks.workunit.client.0.smithi094.stdout:4/306: mkdir d0/d4/d67 0 2024-04-06T03:40:31.316 INFO:tasks.workunit.client.0.smithi094.stdout:0/284: write d1/d9/dd/d18/d1a/f4d [783632,9392] 0 2024-04-06T03:40:31.316 INFO:tasks.workunit.client.1.smithi136.stdout:7/211: mkdir d5/df/d12/d1b/d3a/d4a 0 2024-04-06T03:40:31.320 INFO:tasks.workunit.client.1.smithi136.stdout:5/257: dread d1/f40 [0,4194304] 0 2024-04-06T03:40:31.320 INFO:tasks.workunit.client.1.smithi136.stdout:5/258: write d1/dc/d1c/f34 [694713,104718] 0 2024-04-06T03:40:31.320 INFO:tasks.workunit.client.1.smithi136.stdout:5/259: write d1/f40 [2538378,30164] 0 2024-04-06T03:40:31.321 INFO:tasks.workunit.client.1.smithi136.stdout:1/229: rename d12/l3e to d12/d1b/l49 0 2024-04-06T03:40:31.322 INFO:tasks.workunit.client.1.smithi136.stdout:6/192: dread d2/de/f27 [0,4194304] 0 2024-04-06T03:40:31.324 INFO:tasks.workunit.client.1.smithi136.stdout:8/241: getdents d3 0 2024-04-06T03:40:31.327 INFO:tasks.workunit.client.0.smithi094.stdout:1/233: rename d0/d1/d27/f3d to d0/d7/f52 0 2024-04-06T03:40:31.328 INFO:tasks.workunit.client.1.smithi136.stdout:5/260: write d1/dc/d1c/d32/f38 [184348,126206] 0 2024-04-06T03:40:31.328 INFO:tasks.workunit.client.1.smithi136.stdout:5/261: write d1/dc/d1c/f36 [1317419,88086] 0 2024-04-06T03:40:31.328 INFO:tasks.workunit.client.0.smithi094.stdout:6/228: dwrite dc/d11/d22/f34 [0,4194304] 0 2024-04-06T03:40:31.328 INFO:tasks.workunit.client.0.smithi094.stdout:6/229: truncate dc/d3a/d4a/f51 513335 0 2024-04-06T03:40:31.331 INFO:tasks.workunit.client.0.smithi094.stdout:8/245: unlink d18/c1e 0 2024-04-06T03:40:31.332 INFO:tasks.workunit.client.1.smithi136.stdout:9/207: dwrite fd [0,4194304] 0 2024-04-06T03:40:31.333 INFO:tasks.workunit.client.1.smithi136.stdout:4/186: rename d1d/d2f/f39 to d1d/d31/f48 0 2024-04-06T03:40:31.336 INFO:tasks.workunit.client.1.smithi136.stdout:1/230: dread d12/d1b/d1e/d1f/f30 [0,4194304] 0 2024-04-06T03:40:31.336 INFO:tasks.workunit.client.1.smithi136.stdout:8/242: mknod d3/da/d26/d35/d3d/c45 0 2024-04-06T03:40:31.336 INFO:tasks.workunit.client.1.smithi136.stdout:7/212: rmdir d5/df/d11/d22/d2a 39 2024-04-06T03:40:31.339 INFO:tasks.workunit.client.1.smithi136.stdout:5/262: symlink d1/dc/d10/d1e/l51 0 2024-04-06T03:40:31.342 INFO:tasks.workunit.client.1.smithi136.stdout:9/208: dread f4 [0,4194304] 0 2024-04-06T03:40:31.342 INFO:tasks.workunit.client.1.smithi136.stdout:9/209: chown d17/d20/d28/l2d 3703 1 2024-04-06T03:40:31.343 INFO:tasks.workunit.client.1.smithi136.stdout:0/192: rename d1/d6/f12 to d1/d6/d2c/f40 0 2024-04-06T03:40:31.345 INFO:tasks.workunit.client.1.smithi136.stdout:8/243: symlink d3/da/d26/d35/l46 0 2024-04-06T03:40:31.345 INFO:tasks.workunit.client.1.smithi136.stdout:8/244: fdatasync d3/da/f19 0 2024-04-06T03:40:31.345 INFO:tasks.workunit.client.1.smithi136.stdout:8/245: fdatasync d3/de/f3a 0 2024-04-06T03:40:31.345 INFO:tasks.workunit.client.1.smithi136.stdout:4/187: dread d1d/d31/f33 [0,4194304] 0 2024-04-06T03:40:31.345 INFO:tasks.workunit.client.1.smithi136.stdout:6/193: dwrite d2/de/f35 [0,4194304] 0 2024-04-06T03:40:31.345 INFO:tasks.workunit.client.1.smithi136.stdout:1/231: write d12/d1b/d1e/d23/f31 [4098029,39277] 0 2024-04-06T03:40:31.345 INFO:tasks.workunit.client.1.smithi136.stdout:1/232: chown d12/d1b/d1e 339351 1 2024-04-06T03:40:31.346 INFO:tasks.workunit.client.1.smithi136.stdout:4/188: read d1d/f2b [5094,36177] 0 2024-04-06T03:40:31.346 INFO:tasks.workunit.client.1.smithi136.stdout:5/263: mkdir d1/d1f/d52 0 2024-04-06T03:40:31.349 INFO:tasks.workunit.client.1.smithi136.stdout:3/163: rename d0/d4/d12/d19/d1b to d0/d4/d12/d35 0 2024-04-06T03:40:31.349 INFO:tasks.workunit.client.1.smithi136.stdout:3/164: write d0/d4/f30 [637034,13452] 0 2024-04-06T03:40:31.351 INFO:tasks.workunit.client.0.smithi094.stdout:4/307: truncate d0/f22 589563 0 2024-04-06T03:40:31.351 INFO:tasks.workunit.client.0.smithi094.stdout:4/308: readlink d0/l5f 0 2024-04-06T03:40:31.351 INFO:tasks.workunit.client.0.smithi094.stdout:0/285: creat d1/d9/dd/d18/d1e/d22/d35/f5c x:0 0 0 2024-04-06T03:40:31.352 INFO:tasks.workunit.client.0.smithi094.stdout:9/187: mknod d2/c45 0 2024-04-06T03:40:31.360 INFO:tasks.workunit.client.1.smithi136.stdout:3/165: read d0/d4/d12/f20 [3696210,39637] 0 2024-04-06T03:40:31.360 INFO:tasks.workunit.client.1.smithi136.stdout:3/166: write d0/d4/d12/d19/d2e/f34 [883300,22038] 0 2024-04-06T03:40:31.360 INFO:tasks.workunit.client.1.smithi136.stdout:3/167: fdatasync d0/d4/f22 0 2024-04-06T03:40:31.360 INFO:tasks.workunit.client.1.smithi136.stdout:3/168: write d0/d4/d12/f15 [4970757,102935] 0 2024-04-06T03:40:31.360 INFO:tasks.workunit.client.0.smithi094.stdout:3/288: rename d9/d10/l1a to d9/l63 0 2024-04-06T03:40:31.361 INFO:tasks.workunit.client.1.smithi136.stdout:9/210: read fd [2177549,28532] 0 2024-04-06T03:40:31.361 INFO:tasks.workunit.client.1.smithi136.stdout:9/211: write d17/d21/d32/d3f/f41 [390360,111823] 0 2024-04-06T03:40:31.361 INFO:tasks.workunit.client.1.smithi136.stdout:9/212: chown d17/d21/d32/c40 641500 1 2024-04-06T03:40:31.361 INFO:tasks.workunit.client.0.smithi094.stdout:0/286: write d1/d9/dd/f3b [4094268,24149] 0 2024-04-06T03:40:31.362 INFO:tasks.workunit.client.0.smithi094.stdout:3/289: dread d9/d1d/f25 [0,4194304] 0 2024-04-06T03:40:31.363 INFO:tasks.workunit.client.0.smithi094.stdout:1/234: stat d0/d1/l8 0 2024-04-06T03:40:31.363 INFO:tasks.workunit.client.0.smithi094.stdout:1/235: creat d0/d1/d16/f53 x:0 0 0 2024-04-06T03:40:31.364 INFO:tasks.workunit.client.1.smithi136.stdout:7/213: dwrite d5/fd [4194304,4194304] 0 2024-04-06T03:40:31.364 INFO:tasks.workunit.client.1.smithi136.stdout:7/214: fdatasync d5/df/d12/d1b/f30 0 2024-04-06T03:40:31.364 INFO:tasks.workunit.client.1.smithi136.stdout:7/215: chown d5/df/d12/d1b/d3a/d4a 143606943 1 2024-04-06T03:40:31.364 INFO:tasks.workunit.client.1.smithi136.stdout:7/216: fsync d5/df/d11/d22/d2a/f48 0 2024-04-06T03:40:31.364 INFO:tasks.workunit.client.1.smithi136.stdout:7/217: read - d5/df/d11/f38 zero size 2024-04-06T03:40:31.364 INFO:tasks.workunit.client.1.smithi136.stdout:3/169: write d0/f1e [1498981,21621] 0 2024-04-06T03:40:31.364 INFO:tasks.workunit.client.1.smithi136.stdout:3/170: chown d0/d4/d12/d2d/f33 151226685 1 2024-04-06T03:40:31.365 INFO:tasks.workunit.client.1.smithi136.stdout:8/246: symlink d3/da/d26/d35/l47 0 2024-04-06T03:40:31.365 INFO:tasks.workunit.client.1.smithi136.stdout:8/247: write d3/de/ff [2098834,84699] 0 2024-04-06T03:40:31.369 INFO:tasks.workunit.client.1.smithi136.stdout:1/233: mknod d12/d1b/c4a 0 2024-04-06T03:40:31.371 INFO:tasks.workunit.client.1.smithi136.stdout:5/264: rmdir d1 39 2024-04-06T03:40:31.372 INFO:tasks.workunit.client.0.smithi094.stdout:5/183: sync 2024-04-06T03:40:31.372 INFO:tasks.workunit.client.0.smithi094.stdout:7/245: sync 2024-04-06T03:40:31.373 INFO:tasks.workunit.client.0.smithi094.stdout:5/184: chown db/d19/d24 0 1 2024-04-06T03:40:31.373 INFO:tasks.workunit.client.0.smithi094.stdout:7/246: chown d2/d6/dc/dd 611475482 1 2024-04-06T03:40:31.376 INFO:tasks.workunit.client.1.smithi136.stdout:2/333: rename l2 to d3/d9/d10/d18/l6e 0 2024-04-06T03:40:31.377 INFO:tasks.workunit.client.0.smithi094.stdout:8/246: symlink d18/d34/d52/l57 0 2024-04-06T03:40:31.378 INFO:tasks.workunit.client.1.smithi136.stdout:0/193: rmdir d1/d6/d2c/d1a/d1c 39 2024-04-06T03:40:31.379 INFO:tasks.workunit.client.1.smithi136.stdout:8/248: dwrite d3/de/f29 [0,4194304] 0 2024-04-06T03:40:31.380 INFO:tasks.workunit.client.0.smithi094.stdout:4/309: creat d0/df/d13/f68 x:0 0 0 2024-04-06T03:40:31.380 INFO:tasks.workunit.client.0.smithi094.stdout:4/310: truncate d0/d15/d3a/f43 822699 0 2024-04-06T03:40:31.380 INFO:tasks.workunit.client.1.smithi136.stdout:7/218: dwrite d5/df/d1d/f1f [0,4194304] 0 2024-04-06T03:40:31.382 INFO:tasks.workunit.client.0.smithi094.stdout:9/188: mkdir d2/dc/d46 0 2024-04-06T03:40:31.385 INFO:tasks.workunit.client.0.smithi094.stdout:2/293: rename d0/l3 to d0/d8/d27/l59 0 2024-04-06T03:40:31.385 INFO:tasks.workunit.client.0.smithi094.stdout:2/294: dread - d0/dd/d19/f4c zero size 2024-04-06T03:40:31.385 INFO:tasks.workunit.client.0.smithi094.stdout:2/295: write d0/dd/d19/d31/f3a [1631223,88853] 0 2024-04-06T03:40:31.389 INFO:tasks.workunit.client.0.smithi094.stdout:7/247: dread d2/d6/dc/d12/d29/d35/f39 [0,4194304] 0 2024-04-06T03:40:31.392 INFO:tasks.workunit.client.0.smithi094.stdout:0/287: unlink d1/d9/dd/d18/d1e/d22/c48 0 2024-04-06T03:40:31.398 INFO:tasks.workunit.client.0.smithi094.stdout:3/290: mkdir d9/d10/d27/d3a/d64 0 2024-04-06T03:40:31.400 INFO:tasks.workunit.client.1.smithi136.stdout:0/194: dwrite d1/d6/d2c/f20 [0,4194304] 0 2024-04-06T03:40:31.403 INFO:tasks.workunit.client.0.smithi094.stdout:7/248: dread d2/d6/dc/f2b [0,4194304] 0 2024-04-06T03:40:31.403 INFO:tasks.workunit.client.0.smithi094.stdout:7/249: dread - d2/d6/dc/dd/f3b zero size 2024-04-06T03:40:31.404 INFO:tasks.workunit.client.0.smithi094.stdout:1/236: link d0/f19 d0/d4/d1b/d2f/f54 0 2024-04-06T03:40:31.404 INFO:tasks.workunit.client.0.smithi094.stdout:1/237: write d0/d7/d1a/f1e [911528,99735] 0 2024-04-06T03:40:31.405 INFO:tasks.workunit.client.0.smithi094.stdout:6/230: getdents dc/d3a/d4a 0 2024-04-06T03:40:31.406 INFO:tasks.workunit.client.0.smithi094.stdout:9/189: dwrite d2/dc/d10/d2a/d2d/f3b [0,4194304] 0 2024-04-06T03:40:31.409 INFO:tasks.workunit.client.0.smithi094.stdout:5/185: unlink db/d19/d24/f38 0 2024-04-06T03:40:31.409 INFO:tasks.workunit.client.0.smithi094.stdout:5/186: truncate db/d19/d30/f44 1338806 0 2024-04-06T03:40:31.411 INFO:tasks.workunit.client.0.smithi094.stdout:2/296: dwrite d0/d8/d27/d2b/f2c [0,4194304] 0 2024-04-06T03:40:31.413 INFO:tasks.workunit.client.1.smithi136.stdout:0/195: dwrite d1/d6/d2c/f27 [0,4194304] 0 2024-04-06T03:40:31.413 INFO:tasks.workunit.client.1.smithi136.stdout:0/196: stat d1/d2e/c37 0 2024-04-06T03:40:31.413 INFO:tasks.workunit.client.1.smithi136.stdout:0/197: chown d1/d6/d2c/d1a/d1c/l24 116923304 1 2024-04-06T03:40:31.418 INFO:tasks.workunit.client.0.smithi094.stdout:0/288: mkdir d1/d9/dd/d18/d5d 0 2024-04-06T03:40:31.418 INFO:tasks.workunit.client.0.smithi094.stdout:0/289: readlink d1/d9/dd/d18/d1d/l39 0 2024-04-06T03:40:31.422 INFO:tasks.workunit.client.1.smithi136.stdout:0/198: dread d1/d6/f1f [0,4194304] 0 2024-04-06T03:40:31.425 INFO:tasks.workunit.client.0.smithi094.stdout:3/291: creat d9/d1e/d26/d55/f65 x:0 0 0 2024-04-06T03:40:31.425 INFO:tasks.workunit.client.0.smithi094.stdout:3/292: chown d9/d20/d30/f4d 987699 1 2024-04-06T03:40:31.425 INFO:tasks.workunit.client.0.smithi094.stdout:3/293: write d9/d10/f22 [5011879,5784] 0 2024-04-06T03:40:31.426 INFO:tasks.workunit.client.1.smithi136.stdout:9/213: mkdir d17/d21/d29/d34/d46 0 2024-04-06T03:40:31.426 INFO:tasks.workunit.client.1.smithi136.stdout:9/214: write d17/d19/f2f [3978124,7744] 0 2024-04-06T03:40:31.429 INFO:tasks.workunit.client.0.smithi094.stdout:7/250: mknod d2/d6/dc/dd/c43 0 2024-04-06T03:40:31.429 INFO:tasks.workunit.client.0.smithi094.stdout:7/251: dread - d2/d6/dc/dd/f3b zero size 2024-04-06T03:40:31.432 INFO:tasks.workunit.client.0.smithi094.stdout:1/238: creat d0/d4/d1b/f55 x:0 0 0 2024-04-06T03:40:31.434 INFO:tasks.workunit.client.0.smithi094.stdout:3/294: write d9/d1d/f42 [2316559,18736] 0 2024-04-06T03:40:31.434 INFO:tasks.workunit.client.0.smithi094.stdout:3/295: chown d9/c14 12 1 2024-04-06T03:40:31.435 INFO:tasks.workunit.client.0.smithi094.stdout:6/231: symlink dc/d11/d1b/d24/d46/l59 0 2024-04-06T03:40:31.436 INFO:tasks.workunit.client.0.smithi094.stdout:0/290: dwrite d1/d9/f20 [0,4194304] 0 2024-04-06T03:40:31.439 INFO:tasks.workunit.client.0.smithi094.stdout:7/252: dread d2/d6/dc/d12/f1c [0,4194304] 0 2024-04-06T03:40:31.439 INFO:tasks.workunit.client.0.smithi094.stdout:8/247: dwrite d18/f1a [0,4194304] 0 2024-04-06T03:40:31.439 INFO:tasks.workunit.client.0.smithi094.stdout:6/232: dread dc/d11/d1b/d24/f32 [0,4194304] 0 2024-04-06T03:40:31.443 INFO:tasks.workunit.client.0.smithi094.stdout:9/190: mkdir d2/dc/d21/d23/d47 0 2024-04-06T03:40:31.445 INFO:tasks.workunit.client.0.smithi094.stdout:5/187: mknod db/d19/d30/c46 0 2024-04-06T03:40:31.447 INFO:tasks.workunit.client.0.smithi094.stdout:1/239: dwrite d0/d4/fc [0,4194304] 0 2024-04-06T03:40:31.447 INFO:tasks.workunit.client.0.smithi094.stdout:2/297: readlink d0/d18/l29 0 2024-04-06T03:40:31.449 INFO:tasks.workunit.client.1.smithi136.stdout:3/171: symlink d0/d4/d12/d19/l36 0 2024-04-06T03:40:31.451 INFO:tasks.workunit.client.0.smithi094.stdout:4/311: rmdir d0/d4 39 2024-04-06T03:40:31.458 INFO:tasks.workunit.client.0.smithi094.stdout:3/296: symlink d9/d48/l66 0 2024-04-06T03:40:31.459 INFO:tasks.workunit.client.0.smithi094.stdout:5/188: read db/d19/d21/f26 [1964231,91064] 0 2024-04-06T03:40:31.463 INFO:tasks.workunit.client.0.smithi094.stdout:1/240: dwrite d0/d1/d16/d24/d30/d38/f4d [0,4194304] 0 2024-04-06T03:40:31.465 INFO:tasks.workunit.client.0.smithi094.stdout:8/248: truncate d18/f2d 1006401 0 2024-04-06T03:40:31.466 INFO:tasks.workunit.client.0.smithi094.stdout:9/191: dwrite d2/dc/d10/d2a/d2d/f3b [0,4194304] 0 2024-04-06T03:40:31.466 INFO:tasks.workunit.client.0.smithi094.stdout:9/192: dread - d2/d2c/f41 zero size 2024-04-06T03:40:31.467 INFO:tasks.workunit.client.1.smithi136.stdout:1/234: creat d12/d3b/f4b x:0 0 0 2024-04-06T03:40:31.467 INFO:tasks.workunit.client.1.smithi136.stdout:1/235: fsync d12/d1b/d1e/d23/f3d 0 2024-04-06T03:40:31.467 INFO:tasks.workunit.client.1.smithi136.stdout:1/236: chown d12/d1b/d1e/d27 3 1 2024-04-06T03:40:31.467 INFO:tasks.workunit.client.1.smithi136.stdout:1/237: creat d12/d3b/f4c x:0 0 0 2024-04-06T03:40:31.467 INFO:tasks.workunit.client.1.smithi136.stdout:1/238: write d12/d1b/f29 [773485,88755] 0 2024-04-06T03:40:31.468 INFO:tasks.workunit.client.0.smithi094.stdout:7/253: truncate d2/d6/dc/dd/df/f2d 2151697 0 2024-04-06T03:40:31.471 INFO:tasks.workunit.client.1.smithi136.stdout:2/334: mknod d3/d9/d10/d18/d1d/d3c/c6f 0 2024-04-06T03:40:31.471 INFO:tasks.workunit.client.1.smithi136.stdout:2/335: read - d3/d9/d10/f6b zero size 2024-04-06T03:40:31.472 INFO:tasks.workunit.client.0.smithi094.stdout:3/297: dread d9/fc [0,4194304] 0 2024-04-06T03:40:31.472 INFO:tasks.workunit.client.0.smithi094.stdout:3/298: readlink d9/l13 0 2024-04-06T03:40:31.475 INFO:tasks.workunit.client.0.smithi094.stdout:5/189: dwrite db/d19/f3d [0,4194304] 0 2024-04-06T03:40:31.475 INFO:tasks.workunit.client.0.smithi094.stdout:7/254: read d2/d6/dc/d12/d29/d35/f39 [2325942,124062] 0 2024-04-06T03:40:31.475 INFO:tasks.workunit.client.0.smithi094.stdout:7/255: fdatasync d2/d6/fa 0 2024-04-06T03:40:31.477 INFO:tasks.workunit.client.0.smithi094.stdout:2/298: rmdir d0/d18/d4d 0 2024-04-06T03:40:31.479 INFO:tasks.workunit.client.1.smithi136.stdout:8/249: mkdir d3/da/d26/d35/d3d/d48 0 2024-04-06T03:40:31.481 INFO:tasks.workunit.client.0.smithi094.stdout:4/312: symlink d0/d4/d67/l69 0 2024-04-06T03:40:31.483 INFO:tasks.workunit.client.1.smithi136.stdout:7/219: rename d5/df/d11/l15 to d5/df/d12/l4b 0 2024-04-06T03:40:31.490 INFO:tasks.workunit.client.0.smithi094.stdout:1/241: mknod d0/d1/d16/d3f/c56 0 2024-04-06T03:40:31.490 INFO:tasks.workunit.client.0.smithi094.stdout:1/242: chown d0/d7/d2d/l44 983542 1 2024-04-06T03:40:31.490 INFO:tasks.workunit.client.0.smithi094.stdout:1/243: fsync d0/d7/f52 0 2024-04-06T03:40:31.491 INFO:tasks.workunit.client.0.smithi094.stdout:5/190: dwrite db/d19/d30/f44 [0,4194304] 0 2024-04-06T03:40:31.492 INFO:tasks.workunit.client.0.smithi094.stdout:8/249: write d18/d34/f3d [329997,117588] 0 2024-04-06T03:40:31.492 INFO:tasks.workunit.client.1.smithi136.stdout:8/250: dread d3/f9 [0,4194304] 0 2024-04-06T03:40:31.492 INFO:tasks.workunit.client.1.smithi136.stdout:0/199: rmdir d1/d6/d2c/d1a/d1c 39 2024-04-06T03:40:31.493 INFO:tasks.workunit.client.1.smithi136.stdout:0/200: chown d1/d6/l2d 702548 1 2024-04-06T03:40:31.493 INFO:tasks.workunit.client.1.smithi136.stdout:0/201: truncate d1/d2e/f3f 606515 0 2024-04-06T03:40:31.494 INFO:tasks.workunit.client.1.smithi136.stdout:9/215: link d17/d20/d25/c2c d17/d20/d25/c47 0 2024-04-06T03:40:31.494 INFO:tasks.workunit.client.1.smithi136.stdout:9/216: write d17/d21/d29/f2a [470005,295] 0 2024-04-06T03:40:31.495 INFO:tasks.workunit.client.1.smithi136.stdout:9/217: fsync fa 0 2024-04-06T03:40:31.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/193: mknod d2/dc/d10/d2a/c48 0 2024-04-06T03:40:31.498 INFO:tasks.workunit.client.0.smithi094.stdout:4/313: dwrite d0/d4/fb [0,4194304] 0 2024-04-06T03:40:31.498 INFO:tasks.workunit.client.0.smithi094.stdout:4/314: creat d0/d15/d3a/d3e/d46/f6a x:0 0 0 2024-04-06T03:40:31.498 INFO:tasks.workunit.client.0.smithi094.stdout:4/315: dread - d0/d15/d3a/d3e/d46/f62 zero size 2024-04-06T03:40:31.500 INFO:tasks.workunit.client.0.smithi094.stdout:6/233: symlink dc/d11/l5a 0 2024-04-06T03:40:31.500 INFO:tasks.workunit.client.0.smithi094.stdout:5/191: write db/d19/f3d [429597,45070] 0 2024-04-06T03:40:31.502 INFO:tasks.workunit.client.1.smithi136.stdout:9/218: fdatasync d17/d19/f2f 0 2024-04-06T03:40:31.502 INFO:tasks.workunit.client.1.smithi136.stdout:9/219: truncate d17/d19/f1e 979911 0 2024-04-06T03:40:31.503 INFO:tasks.workunit.client.0.smithi094.stdout:3/299: creat d9/d20/d2d/f67 x:0 0 0 2024-04-06T03:40:31.510 INFO:tasks.workunit.client.0.smithi094.stdout:7/256: mknod d2/d6/dc/d12/d29/d35/c44 0 2024-04-06T03:40:31.512 INFO:tasks.workunit.client.0.smithi094.stdout:8/250: dwrite fd [0,4194304] 0 2024-04-06T03:40:31.512 INFO:tasks.workunit.client.1.smithi136.stdout:3/172: mkdir d0/d4/d12/d2d/d37 0 2024-04-06T03:40:31.514 INFO:tasks.workunit.client.0.smithi094.stdout:2/299: symlink d0/dd/d3d/d49/l5a 0 2024-04-06T03:40:31.516 INFO:tasks.workunit.client.0.smithi094.stdout:6/234: dwrite dc/fd [0,4194304] 0 2024-04-06T03:40:31.516 INFO:tasks.workunit.client.0.smithi094.stdout:6/235: write dc/d11/d22/f57 [939470,37591] 0 2024-04-06T03:40:31.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/244: mkdir d0/d1/d16/d40/d50/d57 0 2024-04-06T03:40:31.517 INFO:tasks.workunit.client.0.smithi094.stdout:1/245: fsync d0/d4/f39 0 2024-04-06T03:40:31.521 INFO:tasks.workunit.client.1.smithi136.stdout:5/265: mknod d1/c53 0 2024-04-06T03:40:31.524 INFO:tasks.workunit.client.1.smithi136.stdout:2/336: mkdir d3/d9/d3b/d70 0 2024-04-06T03:40:31.526 INFO:tasks.workunit.client.1.smithi136.stdout:7/220: symlink d5/df/d12/d1b/d3a/d4a/l4c 0 2024-04-06T03:40:31.527 INFO:tasks.workunit.client.0.smithi094.stdout:9/194: symlink d2/d2c/d33/l49 0 2024-04-06T03:40:31.529 INFO:tasks.workunit.client.1.smithi136.stdout:8/251: symlink d3/da/l49 0 2024-04-06T03:40:31.529 INFO:tasks.workunit.client.0.smithi094.stdout:7/257: dwrite d2/d6/f22 [0,4194304] 0 2024-04-06T03:40:31.532 INFO:tasks.workunit.client.1.smithi136.stdout:0/202: link d1/d6/d2c/l3b d1/d6/d2c/l41 0 2024-04-06T03:40:31.535 INFO:tasks.workunit.client.1.smithi136.stdout:9/220: creat d17/d20/f48 x:0 0 0 2024-04-06T03:40:31.535 INFO:tasks.workunit.client.1.smithi136.stdout:9/221: chown l14 323497 1 2024-04-06T03:40:31.536 INFO:tasks.workunit.client.0.smithi094.stdout:4/316: symlink d0/d15/d3a/d3e/d50/l6b 0 2024-04-06T03:40:31.536 INFO:tasks.workunit.client.1.smithi136.stdout:4/189: sync 2024-04-06T03:40:31.536 INFO:tasks.workunit.client.1.smithi136.stdout:6/194: sync 2024-04-06T03:40:31.536 INFO:tasks.workunit.client.1.smithi136.stdout:4/190: write f1b [708564,78039] 0 2024-04-06T03:40:31.537 INFO:tasks.workunit.client.1.smithi136.stdout:4/191: stat d1d/d31/d3e 0 2024-04-06T03:40:31.537 INFO:tasks.workunit.client.0.smithi094.stdout:6/236: dwrite dc/f12 [0,4194304] 0 2024-04-06T03:40:31.538 INFO:tasks.workunit.client.1.smithi136.stdout:3/173: mknod d0/d4/d12/d2d/c38 0 2024-04-06T03:40:31.538 INFO:tasks.workunit.client.1.smithi136.stdout:3/174: truncate d0/f1e 4253133 0 2024-04-06T03:40:31.539 INFO:tasks.workunit.client.0.smithi094.stdout:4/317: dread d0/f22 [0,4194304] 0 2024-04-06T03:40:31.542 INFO:tasks.workunit.client.1.smithi136.stdout:5/266: dwrite d1/d16/d4b/f4f [0,4194304] 0 2024-04-06T03:40:31.543 INFO:tasks.workunit.client.1.smithi136.stdout:2/337: symlink d3/d45/d5f/l71 0 2024-04-06T03:40:31.543 INFO:tasks.workunit.client.1.smithi136.stdout:9/222: write fe [837649,129867] 0 2024-04-06T03:40:31.543 INFO:tasks.workunit.client.0.smithi094.stdout:8/251: truncate d18/d30/f3e 335243 0 2024-04-06T03:40:31.544 INFO:tasks.workunit.client.1.smithi136.stdout:7/221: mkdir d5/df/d11/d22/d4d 0 2024-04-06T03:40:31.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/300: creat d0/d8/d4f/f5b x:0 0 0 2024-04-06T03:40:31.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/301: readlink d0/d18/d3f/l46 0 2024-04-06T03:40:31.545 INFO:tasks.workunit.client.0.smithi094.stdout:2/302: write d0/dd/d1b/f3e [278847,92402] 0 2024-04-06T03:40:31.546 INFO:tasks.workunit.client.1.smithi136.stdout:0/203: write d1/d6/d2c/f27 [1720688,74157] 0 2024-04-06T03:40:31.546 INFO:tasks.workunit.client.1.smithi136.stdout:6/195: mkdir d2/de/d3d/d47 0 2024-04-06T03:40:31.547 INFO:tasks.workunit.client.1.smithi136.stdout:8/252: dwrite d3/de/f29 [0,4194304] 0 2024-04-06T03:40:31.547 INFO:tasks.workunit.client.1.smithi136.stdout:4/192: creat d1d/f49 x:0 0 0 2024-04-06T03:40:31.547 INFO:tasks.workunit.client.0.smithi094.stdout:9/195: dwrite d2/dc/d21/f43 [4194304,4194304] 0 2024-04-06T03:40:31.549 INFO:tasks.workunit.client.0.smithi094.stdout:1/246: mknod d0/d4/d1b/d2f/d4b/c58 0 2024-04-06T03:40:31.552 INFO:tasks.workunit.client.1.smithi136.stdout:3/175: rmdir d0/d4 39 2024-04-06T03:40:31.552 INFO:tasks.workunit.client.1.smithi136.stdout:3/176: chown d0/d4/d12/f15 200 1 2024-04-06T03:40:31.554 INFO:tasks.workunit.client.1.smithi136.stdout:3/177: write d0/d4/d12/f1a [53306,77400] 0 2024-04-06T03:40:31.554 INFO:tasks.workunit.client.1.smithi136.stdout:3/178: getdents d0/d4/d12/d2d/d37 0 2024-04-06T03:40:31.554 INFO:tasks.workunit.client.1.smithi136.stdout:3/179: write d0/f2 [1001328,56729] 0 2024-04-06T03:40:31.558 INFO:tasks.workunit.client.0.smithi094.stdout:7/258: unlink d2/d6/dc/d12/f1c 0 2024-04-06T03:40:31.558 INFO:tasks.workunit.client.0.smithi094.stdout:7/259: chown d2/d6/dc/f2b 0 1 2024-04-06T03:40:31.559 INFO:tasks.workunit.client.1.smithi136.stdout:5/267: readlink d1/l48 0 2024-04-06T03:40:31.560 INFO:tasks.workunit.client.1.smithi136.stdout:0/204: dread d1/d6/fd [0,4194304] 0 2024-04-06T03:40:31.561 INFO:tasks.workunit.client.1.smithi136.stdout:0/205: creat d1/d2e/f42 x:0 0 0 2024-04-06T03:40:31.561 INFO:tasks.workunit.client.0.smithi094.stdout:9/196: write d2/d2c/d33/d37/f3a [2360718,4213] 0 2024-04-06T03:40:31.561 INFO:tasks.workunit.client.0.smithi094.stdout:9/197: stat d2/dc/f18 0 2024-04-06T03:40:31.562 INFO:tasks.workunit.client.1.smithi136.stdout:2/338: getdents d3/d15/d19/d43 0 2024-04-06T03:40:31.563 INFO:tasks.workunit.client.0.smithi094.stdout:2/303: dwrite d0/dd/d1b/f1c [0,4194304] 0 2024-04-06T03:40:31.565 INFO:tasks.workunit.client.1.smithi136.stdout:5/268: dread d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:31.565 INFO:tasks.workunit.client.1.smithi136.stdout:5/269: write d1/dc/d1c/d32/f38 [283991,111061] 0 2024-04-06T03:40:31.565 INFO:tasks.workunit.client.1.smithi136.stdout:5/270: chown d1/dc/d15/c3d 6 1 2024-04-06T03:40:31.566 INFO:tasks.workunit.client.1.smithi136.stdout:9/223: mkdir d17/d20/d28/d49 0 2024-04-06T03:40:31.568 INFO:tasks.workunit.client.0.smithi094.stdout:6/237: mknod dc/d11/d1b/d4e/c5b 0 2024-04-06T03:40:31.570 INFO:tasks.workunit.client.0.smithi094.stdout:1/247: dwrite d0/d1/d16/d24/d30/f33 [0,4194304] 0 2024-04-06T03:40:31.570 INFO:tasks.workunit.client.1.smithi136.stdout:7/222: link d5/df/d12/c18 d5/df/d11/d22/c4e 0 2024-04-06T03:40:31.571 INFO:tasks.workunit.client.1.smithi136.stdout:1/239: truncate d12/d1b/f29 773974 0 2024-04-06T03:40:31.571 INFO:tasks.workunit.client.0.smithi094.stdout:3/300: rmdir d9/d10/d27/d3a 39 2024-04-06T03:40:31.574 INFO:tasks.workunit.client.1.smithi136.stdout:6/196: unlink d2/de/f15 0 2024-04-06T03:40:31.575 INFO:tasks.workunit.client.0.smithi094.stdout:4/318: getdents d0/d15/d3a/d3e/d49 0 2024-04-06T03:40:31.575 INFO:tasks.workunit.client.0.smithi094.stdout:4/319: truncate d0/d4/f21 1859087 0 2024-04-06T03:40:31.578 INFO:tasks.workunit.client.1.smithi136.stdout:8/253: mknod d3/d2d/c4a 0 2024-04-06T03:40:31.579 INFO:tasks.workunit.client.1.smithi136.stdout:8/254: creat d3/de/f4b x:0 0 0 2024-04-06T03:40:31.579 INFO:tasks.workunit.client.1.smithi136.stdout:8/255: write d3/da/d26/d35/d39/f43 [989210,93637] 0 2024-04-06T03:40:31.579 INFO:tasks.workunit.client.1.smithi136.stdout:8/256: chown d3/de/c34 847 1 2024-04-06T03:40:31.580 INFO:tasks.workunit.client.1.smithi136.stdout:1/240: dread d12/d1b/d22/f43 [0,4194304] 0 2024-04-06T03:40:31.582 INFO:tasks.workunit.client.1.smithi136.stdout:4/193: link ld d1d/d31/d3e/l4a 0 2024-04-06T03:40:31.583 INFO:tasks.workunit.client.0.smithi094.stdout:8/252: creat d18/d34/f58 x:0 0 0 2024-04-06T03:40:31.585 INFO:tasks.workunit.client.0.smithi094.stdout:7/260: symlink d2/d6/l45 0 2024-04-06T03:40:31.586 INFO:tasks.workunit.client.1.smithi136.stdout:3/180: dwrite d0/d4/fe [0,4194304] 0 2024-04-06T03:40:31.586 INFO:tasks.workunit.client.1.smithi136.stdout:3/181: read - d0/f13 zero size 2024-04-06T03:40:31.586 INFO:tasks.workunit.client.1.smithi136.stdout:6/197: dread d2/fd [0,4194304] 0 2024-04-06T03:40:31.587 INFO:tasks.workunit.client.0.smithi094.stdout:9/198: dwrite d2/fb [4194304,4194304] 0 2024-04-06T03:40:31.590 INFO:tasks.workunit.client.1.smithi136.stdout:2/339: symlink d3/d9/d3b/l72 0 2024-04-06T03:40:31.590 INFO:tasks.workunit.client.1.smithi136.stdout:2/340: creat d3/d15/f73 x:0 0 0 2024-04-06T03:40:31.590 INFO:tasks.workunit.client.1.smithi136.stdout:2/341: truncate d3/d15/d19/f39 1509713 0 2024-04-06T03:40:31.592 INFO:tasks.workunit.client.0.smithi094.stdout:7/261: write d2/d6/dc/f15 [4154990,8868] 0 2024-04-06T03:40:31.593 INFO:tasks.workunit.client.1.smithi136.stdout:5/271: truncate d1/d16/f1b 2012934 0 2024-04-06T03:40:31.596 INFO:tasks.workunit.client.0.smithi094.stdout:9/199: dread d2/d2c/d33/d37/f3a [0,4194304] 0 2024-04-06T03:40:31.597 INFO:tasks.workunit.client.1.smithi136.stdout:9/224: symlink d17/l4a 0 2024-04-06T03:40:31.597 INFO:tasks.workunit.client.1.smithi136.stdout:9/225: truncate d17/d20/f48 589139 0 2024-04-06T03:40:31.598 INFO:tasks.workunit.client.0.smithi094.stdout:6/238: rmdir dc/d3a/d4a 39 2024-04-06T03:40:31.598 INFO:tasks.workunit.client.0.smithi094.stdout:6/239: chown dc/l3d 739572 1 2024-04-06T03:40:31.603 INFO:tasks.workunit.client.0.smithi094.stdout:1/248: mknod d0/d1/d16/d24/d30/c59 0 2024-04-06T03:40:31.604 INFO:tasks.workunit.client.0.smithi094.stdout:1/249: fsync d0/d1/d16/d24/d30/d38/f4d 0 2024-04-06T03:40:31.604 INFO:tasks.workunit.client.0.smithi094.stdout:1/250: write d0/d1/d16/f3a [4858932,26581] 0 2024-04-06T03:40:31.604 INFO:tasks.workunit.client.0.smithi094.stdout:1/251: chown d0/d4/d1b/d2f/d3c 1022 1 2024-04-06T03:40:31.604 INFO:tasks.workunit.client.0.smithi094.stdout:1/252: dread - d0/d4/d1b/f55 zero size 2024-04-06T03:40:31.605 INFO:tasks.workunit.client.0.smithi094.stdout:4/320: dwrite d0/d15/f40 [0,4194304] 0 2024-04-06T03:40:31.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/301: link d9/d20/f3d d9/d1e/d26/d55/d62/f68 0 2024-04-06T03:40:31.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/302: fdatasync d9/d48/f5c 0 2024-04-06T03:40:31.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/303: creat d9/d20/d30/f69 x:0 0 0 2024-04-06T03:40:31.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/304: read - d9/d1e/f59 zero size 2024-04-06T03:40:31.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/305: truncate d9/d1d/f39 413925 0 2024-04-06T03:40:31.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/306: dread - d9/d1e/d26/d55/d62/f68 zero size 2024-04-06T03:40:31.610 INFO:tasks.workunit.client.0.smithi094.stdout:3/307: dread - d9/d1d/f36 zero size 2024-04-06T03:40:31.611 INFO:tasks.workunit.client.0.smithi094.stdout:7/262: dwrite d2/d6/f8 [0,4194304] 0 2024-04-06T03:40:31.614 INFO:tasks.workunit.client.1.smithi136.stdout:2/342: dwrite d3/d9/d10/f22 [0,4194304] 0 2024-04-06T03:40:31.615 INFO:tasks.workunit.client.1.smithi136.stdout:2/343: creat d3/d9/d10/d18/f74 x:0 0 0 2024-04-06T03:40:31.615 INFO:tasks.workunit.client.1.smithi136.stdout:7/223: mknod d5/df/d11/d22/d4d/c4f 0 2024-04-06T03:40:31.625 INFO:tasks.workunit.client.1.smithi136.stdout:8/257: unlink d3/de/l1a 0 2024-04-06T03:40:31.626 INFO:tasks.workunit.client.1.smithi136.stdout:1/241: mkdir d12/d1b/d1e/d4d 0 2024-04-06T03:40:31.627 INFO:tasks.workunit.client.1.smithi136.stdout:1/242: creat d12/d1b/d1e/d27/d37/f4e x:0 0 0 2024-04-06T03:40:31.627 INFO:tasks.workunit.client.1.smithi136.stdout:1/243: fsync fb 0 2024-04-06T03:40:31.629 INFO:tasks.workunit.client.0.smithi094.stdout:6/240: dwrite dc/d3a/f41 [0,4194304] 0 2024-04-06T03:40:31.629 INFO:tasks.workunit.client.1.smithi136.stdout:4/194: mknod d1d/d31/c4b 0 2024-04-06T03:40:31.629 INFO:tasks.workunit.client.1.smithi136.stdout:4/195: chown d1d/d31/d3e 161074247 1 2024-04-06T03:40:31.629 INFO:tasks.workunit.client.1.smithi136.stdout:4/196: creat d1d/d31/f4c x:0 0 0 2024-04-06T03:40:31.630 INFO:tasks.workunit.client.0.smithi094.stdout:2/304: rmdir d0/dd 39 2024-04-06T03:40:31.630 INFO:tasks.workunit.client.0.smithi094.stdout:8/253: dwrite d18/f22 [0,4194304] 0 2024-04-06T03:40:31.630 INFO:tasks.workunit.client.0.smithi094.stdout:8/254: chown d18/d30/c32 9670828 1 2024-04-06T03:40:31.630 INFO:tasks.workunit.client.0.smithi094.stdout:8/255: write d18/d30/f4b [346314,126215] 0 2024-04-06T03:40:31.632 INFO:tasks.workunit.client.1.smithi136.stdout:3/182: creat d0/d4/d12/d2d/f39 x:0 0 0 2024-04-06T03:40:31.635 INFO:tasks.workunit.client.0.smithi094.stdout:9/200: rename d2/fb to d2/d2c/d33/f4a 0 2024-04-06T03:40:31.639 INFO:tasks.workunit.client.1.smithi136.stdout:4/197: dread f14 [0,4194304] 0 2024-04-06T03:40:31.639 INFO:tasks.workunit.client.1.smithi136.stdout:4/198: write d1d/f2b [61584,117694] 0 2024-04-06T03:40:31.640 INFO:tasks.workunit.client.1.smithi136.stdout:8/258: dwrite f1 [0,4194304] 0 2024-04-06T03:40:31.644 INFO:tasks.workunit.client.1.smithi136.stdout:4/199: dread d1d/f29 [0,4194304] 0 2024-04-06T03:40:31.646 INFO:tasks.workunit.client.1.smithi136.stdout:3/183: dwrite d0/f1e [0,4194304] 0 2024-04-06T03:40:31.649 INFO:tasks.workunit.client.1.smithi136.stdout:5/272: getdents d1/dc/d15 0 2024-04-06T03:40:31.653 INFO:tasks.workunit.client.1.smithi136.stdout:9/226: mkdir d17/d20/d25/d3d/d4b 0 2024-04-06T03:40:31.653 INFO:tasks.workunit.client.1.smithi136.stdout:9/227: stat d17/d21/d32/d43 0 2024-04-06T03:40:31.658 INFO:tasks.workunit.client.1.smithi136.stdout:5/273: dread d1/d1f/f21 [0,4194304] 0 2024-04-06T03:40:31.658 INFO:tasks.workunit.client.1.smithi136.stdout:5/274: fsync d1/dc/d1c/d32/f38 0 2024-04-06T03:40:31.658 INFO:tasks.workunit.client.1.smithi136.stdout:5/275: chown d1/dc/d1c/d32 19737323 1 2024-04-06T03:40:31.659 INFO:tasks.workunit.client.1.smithi136.stdout:8/259: dwrite d3/de/f29 [0,4194304] 0 2024-04-06T03:40:31.661 INFO:tasks.workunit.client.1.smithi136.stdout:2/344: mknod d3/d9/d10/d18/c75 0 2024-04-06T03:40:31.661 INFO:tasks.workunit.client.1.smithi136.stdout:2/345: chown d3/d4a/c33 150 1 2024-04-06T03:40:31.662 INFO:tasks.workunit.client.1.smithi136.stdout:7/224: mkdir d5/df/d11/d22/d2c/d50 0 2024-04-06T03:40:31.663 INFO:tasks.workunit.client.1.smithi136.stdout:9/228: dread d17/d19/f27 [0,4194304] 0 2024-04-06T03:40:31.665 INFO:tasks.workunit.client.1.smithi136.stdout:3/184: dwrite d0/d4/f22 [4194304,4194304] 0 2024-04-06T03:40:31.666 INFO:tasks.workunit.client.1.smithi136.stdout:1/244: rename d12/d1b/d1e/d23/l25 to d12/d1b/d1e/l4f 0 2024-04-06T03:40:31.671 INFO:tasks.workunit.client.0.smithi094.stdout:7/263: mknod d2/d6/dc/dd/df/d36/c46 0 2024-04-06T03:40:31.672 INFO:tasks.workunit.client.1.smithi136.stdout:4/200: unlink d1d/d31/c37 0 2024-04-06T03:40:31.674 INFO:tasks.workunit.client.0.smithi094.stdout:6/241: mknod dc/d3a/c5c 0 2024-04-06T03:40:31.676 INFO:tasks.workunit.client.0.smithi094.stdout:2/305: symlink d0/dd/d1b/l5c 0 2024-04-06T03:40:31.679 INFO:tasks.workunit.client.0.smithi094.stdout:8/256: dread d18/f36 [0,4194304] 0 2024-04-06T03:40:31.679 INFO:tasks.workunit.client.0.smithi094.stdout:8/257: write d18/d30/f3e [335069,21576] 0 2024-04-06T03:40:31.680 INFO:tasks.workunit.client.0.smithi094.stdout:8/258: write d18/f36 [149327,59872] 0 2024-04-06T03:40:31.682 INFO:tasks.workunit.client.0.smithi094.stdout:4/321: rename d0/d15/d3a/d3e/d49/f4d to d0/d4/d4b/f6c 0 2024-04-06T03:40:31.682 INFO:tasks.workunit.client.0.smithi094.stdout:4/322: write d0/d15/d3a/d3e/d46/f6a [998540,79913] 0 2024-04-06T03:40:31.683 INFO:tasks.workunit.client.0.smithi094.stdout:6/242: read dc/d11/d1b/f2f [1264306,88248] 0 2024-04-06T03:40:31.684 INFO:tasks.workunit.client.0.smithi094.stdout:9/201: symlink d2/dc/d21/d23/l4b 0 2024-04-06T03:40:31.685 INFO:tasks.workunit.client.1.smithi136.stdout:5/276: rmdir d1/dc/d1c 39 2024-04-06T03:40:31.687 INFO:tasks.workunit.client.0.smithi094.stdout:1/253: rmdir d0/d4 39 2024-04-06T03:40:31.689 INFO:tasks.workunit.client.0.smithi094.stdout:9/202: dread d2/d2c/d33/d37/f3a [0,4194304] 0 2024-04-06T03:40:31.689 INFO:tasks.workunit.client.0.smithi094.stdout:9/203: read d2/dc/d21/d26/f3c [1751033,61647] 0 2024-04-06T03:40:31.689 INFO:tasks.workunit.client.0.smithi094.stdout:9/204: fsync d2/d2c/f36 0 2024-04-06T03:40:31.692 INFO:tasks.workunit.client.1.smithi136.stdout:8/260: link d3/f4 d3/de/f4c 0 2024-04-06T03:40:31.694 INFO:tasks.workunit.client.1.smithi136.stdout:7/225: symlink d5/l51 0 2024-04-06T03:40:31.696 INFO:tasks.workunit.client.1.smithi136.stdout:9/229: truncate d17/d20/d25/f26 169582 0 2024-04-06T03:40:31.696 INFO:tasks.workunit.client.1.smithi136.stdout:9/230: chown d17/d20/d25/f26 544704023 1 2024-04-06T03:40:31.703 INFO:tasks.workunit.client.0.smithi094.stdout:7/264: read d2/f1f [1470894,97988] 0 2024-04-06T03:40:31.703 INFO:tasks.workunit.client.1.smithi136.stdout:2/346: rename d3/d4a/c69 to d3/d9/d3b/c76 0 2024-04-06T03:40:31.703 INFO:tasks.workunit.client.1.smithi136.stdout:2/347: write d3/d15/d19/f60 [305737,108681] 0 2024-04-06T03:40:31.706 INFO:tasks.workunit.client.1.smithi136.stdout:1/245: creat d12/d1b/d28/d48/f50 x:0 0 0 2024-04-06T03:40:31.710 INFO:tasks.workunit.client.0.smithi094.stdout:2/306: creat d0/d8/d27/d2b/f5d x:0 0 0 2024-04-06T03:40:31.712 INFO:tasks.workunit.client.0.smithi094.stdout:9/205: dwrite d2/d2c/f41 [0,4194304] 0 2024-04-06T03:40:31.712 INFO:tasks.workunit.client.0.smithi094.stdout:2/307: dread d0/dd/ff [0,4194304] 0 2024-04-06T03:40:31.713 INFO:tasks.workunit.client.1.smithi136.stdout:5/277: chown d1/ca 14 1 2024-04-06T03:40:31.715 INFO:tasks.workunit.client.1.smithi136.stdout:0/206: sync 2024-04-06T03:40:31.715 INFO:tasks.workunit.client.1.smithi136.stdout:2/348: write d3/d9/d10/f22 [4090275,43874] 0 2024-04-06T03:40:31.716 INFO:tasks.workunit.client.1.smithi136.stdout:8/261: link d3/da/fb d3/f4d 0 2024-04-06T03:40:31.717 INFO:tasks.workunit.client.1.smithi136.stdout:8/262: stat d3/da/d26/d35/f3c 0 2024-04-06T03:40:31.717 INFO:tasks.workunit.client.0.smithi094.stdout:9/206: dread d2/dc/f18 [4194304,4194304] 0 2024-04-06T03:40:31.717 INFO:tasks.workunit.client.0.smithi094.stdout:9/207: write d2/dc/d21/d26/f29 [4816808,53314] 0 2024-04-06T03:40:31.718 INFO:tasks.workunit.client.1.smithi136.stdout:9/231: dwrite d17/d19/f35 [0,4194304] 0 2024-04-06T03:40:31.719 INFO:tasks.workunit.client.1.smithi136.stdout:9/232: chown d17/d21/d29/f2a 31191 1 2024-04-06T03:40:31.719 INFO:tasks.workunit.client.1.smithi136.stdout:9/233: getdents d17/d21/d32/d43 0 2024-04-06T03:40:31.719 INFO:tasks.workunit.client.0.smithi094.stdout:4/323: unlink d0/df/d13/f18 0 2024-04-06T03:40:31.721 INFO:tasks.workunit.client.1.smithi136.stdout:7/226: link d5/df/f41 d5/f52 0 2024-04-06T03:40:31.724 INFO:tasks.workunit.client.0.smithi094.stdout:6/243: truncate dc/d11/d1b/f31 1606884 0 2024-04-06T03:40:31.725 INFO:tasks.workunit.client.0.smithi094.stdout:2/308: write d0/f5 [4777216,28919] 0 2024-04-06T03:40:31.726 INFO:tasks.workunit.client.1.smithi136.stdout:1/246: truncate d12/f16 1388218 0 2024-04-06T03:40:31.726 INFO:tasks.workunit.client.1.smithi136.stdout:1/247: write d12/d1b/f29 [827893,51561] 0 2024-04-06T03:40:31.727 INFO:tasks.workunit.client.1.smithi136.stdout:5/278: fdatasync d1/f4 0 2024-04-06T03:40:31.731 INFO:tasks.workunit.client.1.smithi136.stdout:0/207: mkdir d1/d2e/d43 0 2024-04-06T03:40:31.732 INFO:tasks.workunit.client.1.smithi136.stdout:2/349: dwrite d3/d15/f36 [4194304,4194304] 0 2024-04-06T03:40:31.733 INFO:tasks.workunit.client.0.smithi094.stdout:7/265: mknod d2/d6/dc/d12/d29/d3f/c47 0 2024-04-06T03:40:31.734 INFO:tasks.workunit.client.0.smithi094.stdout:7/266: chown d2/d6/dc/dd/f3e 45175194 1 2024-04-06T03:40:31.736 INFO:tasks.workunit.client.1.smithi136.stdout:9/234: truncate f11 3206824 0 2024-04-06T03:40:31.738 INFO:tasks.workunit.client.0.smithi094.stdout:8/259: mknod d18/d2c/d37/d48/c59 0 2024-04-06T03:40:31.738 INFO:tasks.workunit.client.0.smithi094.stdout:8/260: truncate d18/d2c/f2e 1837962 0 2024-04-06T03:40:31.745 INFO:tasks.workunit.client.0.smithi094.stdout:8/261: read f11 [2950958,61727] 0 2024-04-06T03:40:31.745 INFO:tasks.workunit.client.1.smithi136.stdout:1/248: creat d12/d1b/d1e/d4d/f51 x:0 0 0 2024-04-06T03:40:31.746 INFO:tasks.workunit.client.1.smithi136.stdout:5/279: dwrite d1/d1f/f21 [0,4194304] 0 2024-04-06T03:40:31.747 INFO:tasks.workunit.client.0.smithi094.stdout:9/208: rename d2/f4 to d2/d2c/d33/f4c 0 2024-04-06T03:40:31.747 INFO:tasks.workunit.client.0.smithi094.stdout:9/209: creat d2/dc/d10/d2a/d2d/f4d x:0 0 0 2024-04-06T03:40:31.748 INFO:tasks.workunit.client.1.smithi136.stdout:8/263: dwrite d3/f10 [0,4194304] 0 2024-04-06T03:40:31.748 INFO:tasks.workunit.client.1.smithi136.stdout:8/264: fsync d3/da/d26/d35/d39/f43 0 2024-04-06T03:40:31.748 INFO:tasks.workunit.client.1.smithi136.stdout:0/208: creat d1/d6/d2c/d1a/d36/f44 x:0 0 0 2024-04-06T03:40:31.751 INFO:tasks.workunit.client.0.smithi094.stdout:4/324: getdents d0/d15 0 2024-04-06T03:40:31.753 INFO:tasks.workunit.client.0.smithi094.stdout:6/244: mknod dc/d11/d1b/d24/c5d 0 2024-04-06T03:40:31.753 INFO:tasks.workunit.client.0.smithi094.stdout:6/245: write dc/d11/d1b/f1f [347840,91243] 0 2024-04-06T03:40:31.756 INFO:tasks.workunit.client.1.smithi136.stdout:0/209: write d1/d6/f19 [154106,4629] 0 2024-04-06T03:40:31.764 INFO:tasks.workunit.client.1.smithi136.stdout:5/280: mknod d1/d16/c54 0 2024-04-06T03:40:31.764 INFO:tasks.workunit.client.1.smithi136.stdout:5/281: fdatasync d1/dc/d1c/f2c 0 2024-04-06T03:40:31.767 INFO:tasks.workunit.client.0.smithi094.stdout:8/262: chown d18/f27 290241 1 2024-04-06T03:40:31.767 INFO:tasks.workunit.client.1.smithi136.stdout:1/249: dwrite d12/f38 [0,4194304] 0 2024-04-06T03:40:31.773 INFO:tasks.workunit.client.0.smithi094.stdout:2/309: rename d0/d8/d1d/l2f to d0/d8/d4f/l5e 0 2024-04-06T03:40:31.780 INFO:tasks.workunit.client.0.smithi094.stdout:8/263: dread f14 [4194304,4194304] 0 2024-04-06T03:40:31.780 INFO:tasks.workunit.client.0.smithi094.stdout:8/264: write d18/d2c/f41 [808415,130750] 0 2024-04-06T03:40:31.781 INFO:tasks.workunit.client.0.smithi094.stdout:8/265: write d18/d2c/f2e [260995,9147] 0 2024-04-06T03:40:31.781 INFO:tasks.workunit.client.0.smithi094.stdout:8/266: write d18/f27 [1451919,84674] 0 2024-04-06T03:40:31.781 INFO:tasks.workunit.client.0.smithi094.stdout:8/267: fsync d18/f1a 0 2024-04-06T03:40:31.782 INFO:tasks.workunit.client.0.smithi094.stdout:9/210: truncate d2/d2c/d33/d37/f3a 193738 0 2024-04-06T03:40:31.783 INFO:tasks.workunit.client.1.smithi136.stdout:8/265: rmdir d3/da/d26/d35 39 2024-04-06T03:40:31.783 INFO:tasks.workunit.client.1.smithi136.stdout:0/210: dwrite d1/fc [4194304,4194304] 0 2024-04-06T03:40:31.783 INFO:tasks.workunit.client.1.smithi136.stdout:0/211: chown d1/d6/d2c/d1a/d1c/f31 2222362 1 2024-04-06T03:40:31.784 INFO:tasks.workunit.client.0.smithi094.stdout:6/246: dwrite dc/d11/d1b/f2f [0,4194304] 0 2024-04-06T03:40:31.784 INFO:tasks.workunit.client.0.smithi094.stdout:4/325: dwrite d0/d4/f14 [4194304,4194304] 0 2024-04-06T03:40:31.785 INFO:tasks.workunit.client.0.smithi094.stdout:6/247: write dc/d11/d22/f57 [127460,125491] 0 2024-04-06T03:40:31.785 INFO:tasks.workunit.client.0.smithi094.stdout:6/248: creat dc/d3a/f5e x:0 0 0 2024-04-06T03:40:31.786 INFO:tasks.workunit.client.0.smithi094.stdout:2/310: creat d0/d18/d3f/f5f x:0 0 0 2024-04-06T03:40:31.788 INFO:tasks.workunit.client.1.smithi136.stdout:1/250: dwrite d12/d1b/d28/f3f [0,4194304] 0 2024-04-06T03:40:31.789 INFO:tasks.workunit.client.1.smithi136.stdout:2/350: rmdir d3/d15 39 2024-04-06T03:40:31.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/211: creat d2/dc/d46/f4e x:0 0 0 2024-04-06T03:40:31.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/212: fsync d2/f30 0 2024-04-06T03:40:31.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/213: readlink d2/dc/l28 0 2024-04-06T03:40:31.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/214: stat d2/dc/d21/d23 0 2024-04-06T03:40:31.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/215: fsync d2/dc/d21/d26/f29 0 2024-04-06T03:40:31.793 INFO:tasks.workunit.client.1.smithi136.stdout:0/212: write d1/d6/d2c/d1a/d1c/f31 [471665,98708] 0 2024-04-06T03:40:31.794 INFO:tasks.workunit.client.0.smithi094.stdout:6/249: write dc/f28 [2004977,79488] 0 2024-04-06T03:40:31.794 INFO:tasks.workunit.client.0.smithi094.stdout:6/250: fdatasync fa 0 2024-04-06T03:40:31.798 INFO:tasks.workunit.client.1.smithi136.stdout:7/227: rename d5/d21 to d5/df/d53 0 2024-04-06T03:40:31.799 INFO:tasks.workunit.client.1.smithi136.stdout:8/266: dwrite d3/da/d26/d35/f38 [0,4194304] 0 2024-04-06T03:40:31.800 INFO:tasks.workunit.client.0.smithi094.stdout:4/326: truncate d0/d4/f9 5644235 0 2024-04-06T03:40:31.800 INFO:tasks.workunit.client.1.smithi136.stdout:2/351: dread d3/d9/d10/d18/d1d/f35 [4194304,4194304] 0 2024-04-06T03:40:31.802 INFO:tasks.workunit.client.0.smithi094.stdout:2/311: creat d0/d8/d1d/f60 x:0 0 0 2024-04-06T03:40:31.802 INFO:tasks.workunit.client.0.smithi094.stdout:2/312: chown d0/dd/d1b/c51 0 1 2024-04-06T03:40:31.803 INFO:tasks.workunit.client.1.smithi136.stdout:7/228: dread d5/df/d11/d22/f28 [0,4194304] 0 2024-04-06T03:40:31.804 INFO:tasks.workunit.client.0.smithi094.stdout:8/268: truncate d18/f20 3162201 0 2024-04-06T03:40:31.809 INFO:tasks.workunit.client.1.smithi136.stdout:6/198: sync 2024-04-06T03:40:31.809 INFO:tasks.workunit.client.1.smithi136.stdout:3/185: sync 2024-04-06T03:40:31.812 INFO:tasks.workunit.client.0.smithi094.stdout:6/251: unlink cb 0 2024-04-06T03:40:31.812 INFO:tasks.workunit.client.0.smithi094.stdout:6/252: read - dc/d11/d1b/d24/d46/f4d zero size 2024-04-06T03:40:31.813 INFO:tasks.workunit.client.1.smithi136.stdout:1/251: mknod d12/d1b/d1e/d1f/c52 0 2024-04-06T03:40:31.813 INFO:tasks.workunit.client.1.smithi136.stdout:2/352: dwrite d3/d9/d10/d18/d1d/d3c/f51 [0,4194304] 0 2024-04-06T03:40:31.814 INFO:tasks.workunit.client.1.smithi136.stdout:2/353: write d3/d15/d19/f39 [390090,21150] 0 2024-04-06T03:40:31.816 INFO:tasks.workunit.client.1.smithi136.stdout:9/235: rename d17/d20/l22 to d17/d20/d25/d3b/l4c 0 2024-04-06T03:40:31.816 INFO:tasks.workunit.client.1.smithi136.stdout:4/201: sync 2024-04-06T03:40:31.817 INFO:tasks.workunit.client.1.smithi136.stdout:8/267: mknod d3/da/d26/c4e 0 2024-04-06T03:40:31.818 INFO:tasks.workunit.client.0.smithi094.stdout:4/327: dread d0/d4/f6 [0,4194304] 0 2024-04-06T03:40:31.818 INFO:tasks.workunit.client.0.smithi094.stdout:4/328: readlink d0/l5f 0 2024-04-06T03:40:31.820 INFO:tasks.workunit.client.1.smithi136.stdout:7/229: mkdir d5/df/d1d/d54 0 2024-04-06T03:40:31.823 INFO:tasks.workunit.client.0.smithi094.stdout:2/313: rename d0/d18/l4b to d0/dd/d19/d31/l61 0 2024-04-06T03:40:31.824 INFO:tasks.workunit.client.1.smithi136.stdout:3/186: mkdir d0/d4/d12/d3a 0 2024-04-06T03:40:31.826 INFO:tasks.workunit.client.1.smithi136.stdout:1/252: mkdir d12/d53 0 2024-04-06T03:40:31.830 INFO:tasks.workunit.client.1.smithi136.stdout:6/199: read d2/d25/f17 [1340752,15413] 0 2024-04-06T03:40:31.831 INFO:tasks.workunit.client.1.smithi136.stdout:2/354: mkdir d3/d4a/d77 0 2024-04-06T03:40:31.832 INFO:tasks.workunit.client.0.smithi094.stdout:9/216: rmdir d2/dc/d21/d23/d2f 39 2024-04-06T03:40:31.835 INFO:tasks.workunit.client.0.smithi094.stdout:2/314: creat d0/dd/f62 x:0 0 0 2024-04-06T03:40:31.836 INFO:tasks.workunit.client.1.smithi136.stdout:9/236: mknod d17/d20/d28/d49/c4d 0 2024-04-06T03:40:31.836 INFO:tasks.workunit.client.0.smithi094.stdout:8/269: link d18/d2c/d37/d48/c50 d18/d3c/d49/c5a 0 2024-04-06T03:40:31.838 INFO:tasks.workunit.client.0.smithi094.stdout:9/217: unlink d2/dc/d10/f1b 0 2024-04-06T03:40:31.840 INFO:tasks.workunit.client.1.smithi136.stdout:8/268: link d3/f4d d3/da/d26/d35/d39/f4f 0 2024-04-06T03:40:31.840 INFO:tasks.workunit.client.1.smithi136.stdout:8/269: chown d3/f10 3792 1 2024-04-06T03:40:31.840 INFO:tasks.workunit.client.0.smithi094.stdout:4/329: unlink d0/df/d13/f68 0 2024-04-06T03:40:31.843 INFO:tasks.workunit.client.0.smithi094.stdout:2/315: rename d0/dd/l12 to d0/d8/l63 0 2024-04-06T03:40:31.845 INFO:tasks.workunit.client.1.smithi136.stdout:3/187: mkdir d0/d4/d12/d19/d2e/d3b 0 2024-04-06T03:40:31.850 INFO:tasks.workunit.client.0.smithi094.stdout:8/270: creat d18/d2c/d37/d48/f5b x:0 0 0 2024-04-06T03:40:31.851 INFO:tasks.workunit.client.1.smithi136.stdout:1/253: creat d12/d3b/d44/f54 x:0 0 0 2024-04-06T03:40:31.851 INFO:tasks.workunit.client.1.smithi136.stdout:1/254: write d12/d1b/d1e/d4d/f51 [521274,33107] 0 2024-04-06T03:40:31.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/330: mknod d0/c6d 0 2024-04-06T03:40:31.857 INFO:tasks.workunit.client.1.smithi136.stdout:2/355: unlink d3/d9/d3b/l72 0 2024-04-06T03:40:31.858 INFO:tasks.workunit.client.0.smithi094.stdout:9/218: dwrite d2/d2c/d33/f4a [4194304,4194304] 0 2024-04-06T03:40:31.859 INFO:tasks.workunit.client.1.smithi136.stdout:8/270: dwrite d3/da/d26/d35/d39/f43 [0,4194304] 0 2024-04-06T03:40:31.860 INFO:tasks.workunit.client.1.smithi136.stdout:5/282: rename d1/l43 to d1/d16/d4b/l55 0 2024-04-06T03:40:31.861 INFO:tasks.workunit.client.0.smithi094.stdout:4/331: rename d0/d15/d3a/f54 to d0/d23/f6e 0 2024-04-06T03:40:31.862 INFO:tasks.workunit.client.0.smithi094.stdout:4/332: chown d0/l28 259943516 1 2024-04-06T03:40:31.862 INFO:tasks.workunit.client.0.smithi094.stdout:4/333: fsync d0/d4/fd 0 2024-04-06T03:40:31.862 INFO:tasks.workunit.client.1.smithi136.stdout:9/237: creat d17/d20/d25/d3b/f4e x:0 0 0 2024-04-06T03:40:31.862 INFO:tasks.workunit.client.1.smithi136.stdout:9/238: creat d17/d21/d32/f4f x:0 0 0 2024-04-06T03:40:31.862 INFO:tasks.workunit.client.1.smithi136.stdout:9/239: chown d17/l4a 844 1 2024-04-06T03:40:31.863 INFO:tasks.workunit.client.0.smithi094.stdout:2/316: link d0/d18/f22 d0/f64 0 2024-04-06T03:40:31.864 INFO:tasks.workunit.client.1.smithi136.stdout:6/200: dread d2/f3c [0,4194304] 0 2024-04-06T03:40:31.864 INFO:tasks.workunit.client.1.smithi136.stdout:4/202: mkdir d1d/d4d 0 2024-04-06T03:40:31.864 INFO:tasks.workunit.client.1.smithi136.stdout:4/203: read - d1d/d31/f4c zero size 2024-04-06T03:40:31.865 INFO:tasks.workunit.client.0.smithi094.stdout:8/271: mkdir d18/d5c 0 2024-04-06T03:40:31.866 INFO:tasks.workunit.client.1.smithi136.stdout:7/230: rmdir d5/df/d1d 39 2024-04-06T03:40:31.867 INFO:tasks.workunit.client.0.smithi094.stdout:9/219: write d2/f14 [5401142,65876] 0 2024-04-06T03:40:31.872 INFO:tasks.workunit.client.0.smithi094.stdout:4/334: readlink d0/df/d1e/l2e 0 2024-04-06T03:40:31.873 INFO:tasks.workunit.client.0.smithi094.stdout:8/272: chown d18/f25 203529887 1 2024-04-06T03:40:31.876 INFO:tasks.workunit.client.0.smithi094.stdout:4/335: symlink d0/d15/d3a/l6f 0 2024-04-06T03:40:31.879 INFO:tasks.workunit.client.1.smithi136.stdout:2/356: creat d3/d45/d5f/f78 x:0 0 0 2024-04-06T03:40:31.880 INFO:tasks.workunit.client.0.smithi094.stdout:9/220: mknod d2/c4f 0 2024-04-06T03:40:31.880 INFO:tasks.workunit.client.0.smithi094.stdout:9/221: write d2/dc/d10/f12 [1765207,8836] 0 2024-04-06T03:40:31.882 INFO:tasks.workunit.client.0.smithi094.stdout:2/317: dwrite d0/f5 [8388608,4194304] 0 2024-04-06T03:40:31.886 INFO:tasks.workunit.client.0.smithi094.stdout:8/273: rename d18/c29 to d18/d2c/c5d 0 2024-04-06T03:40:31.886 INFO:tasks.workunit.client.0.smithi094.stdout:8/274: chown d18/d3c 38506619 1 2024-04-06T03:40:31.886 INFO:tasks.workunit.client.0.smithi094.stdout:8/275: chown d18/d3c/d49/c5a 4299 1 2024-04-06T03:40:31.886 INFO:tasks.workunit.client.0.smithi094.stdout:8/276: truncate d18/d30/f40 454576 0 2024-04-06T03:40:31.888 INFO:tasks.workunit.client.1.smithi136.stdout:8/271: creat d3/da/d26/f50 x:0 0 0 2024-04-06T03:40:31.888 INFO:tasks.workunit.client.1.smithi136.stdout:8/272: stat d3/da/d26/d35/d3d/d48 0 2024-04-06T03:40:31.889 INFO:tasks.workunit.client.0.smithi094.stdout:9/222: creat d2/dc/d21/f50 x:0 0 0 2024-04-06T03:40:31.890 INFO:tasks.workunit.client.1.smithi136.stdout:9/240: dwrite f4 [0,4194304] 0 2024-04-06T03:40:31.892 INFO:tasks.workunit.client.1.smithi136.stdout:0/213: rename d1/f4 to d1/d6/f45 0 2024-04-06T03:40:31.892 INFO:tasks.workunit.client.1.smithi136.stdout:0/214: fsync d1/d6/d2c/d1a/f28 0 2024-04-06T03:40:31.892 INFO:tasks.workunit.client.1.smithi136.stdout:0/215: fdatasync d1/d6/f45 0 2024-04-06T03:40:31.893 INFO:tasks.workunit.client.1.smithi136.stdout:0/216: chown d1/f29 48964 1 2024-04-06T03:40:31.893 INFO:tasks.workunit.client.1.smithi136.stdout:0/217: creat d1/d2e/f46 x:0 0 0 2024-04-06T03:40:31.893 INFO:tasks.workunit.client.0.smithi094.stdout:4/336: truncate d0/df/d13/f37 3757481 0 2024-04-06T03:40:31.893 INFO:tasks.workunit.client.0.smithi094.stdout:4/337: chown d0/df/f1b 300 1 2024-04-06T03:40:31.893 INFO:tasks.workunit.client.1.smithi136.stdout:7/231: dwrite d5/df/d11/f26 [4194304,4194304] 0 2024-04-06T03:40:31.894 INFO:tasks.workunit.client.0.smithi094.stdout:8/277: read d18/f20 [745395,58017] 0 2024-04-06T03:40:31.894 INFO:tasks.workunit.client.0.smithi094.stdout:8/278: chown d18/d2c/f2e 45527 1 2024-04-06T03:40:31.896 INFO:tasks.workunit.client.1.smithi136.stdout:5/283: unlink d1/l48 0 2024-04-06T03:40:31.897 INFO:tasks.workunit.client.0.smithi094.stdout:9/223: fsync d2/dc/d21/d26/f3c 0 2024-04-06T03:40:31.898 INFO:tasks.workunit.client.1.smithi136.stdout:8/273: dread d3/f10 [0,4194304] 0 2024-04-06T03:40:31.898 INFO:tasks.workunit.client.1.smithi136.stdout:8/274: chown d3/de/d18/c22 3901422 1 2024-04-06T03:40:31.899 INFO:tasks.workunit.client.1.smithi136.stdout:8/275: dread - d3/da/f42 zero size 2024-04-06T03:40:31.899 INFO:tasks.workunit.client.1.smithi136.stdout:8/276: readlink d3/da/l25 0 2024-04-06T03:40:31.899 INFO:tasks.workunit.client.1.smithi136.stdout:8/277: write d3/de/d18/f1e [670378,89129] 0 2024-04-06T03:40:31.904 INFO:tasks.workunit.client.0.smithi094.stdout:8/279: read fd [1679079,36837] 0 2024-04-06T03:40:31.906 INFO:tasks.workunit.client.1.smithi136.stdout:4/204: mknod d1d/d4d/c4e 0 2024-04-06T03:40:31.907 INFO:tasks.workunit.client.0.smithi094.stdout:2/318: dwrite d0/d8/d1d/f38 [0,4194304] 0 2024-04-06T03:40:31.907 INFO:tasks.workunit.client.1.smithi136.stdout:5/284: write d1/f4 [1959850,120697] 0 2024-04-06T03:40:31.907 INFO:tasks.workunit.client.0.smithi094.stdout:4/338: link d0/d15/d3a/l6f d0/df/d13/d4a/l70 0 2024-04-06T03:40:31.907 INFO:tasks.workunit.client.0.smithi094.stdout:4/339: truncate d0/d4/fb 5090116 0 2024-04-06T03:40:31.908 INFO:tasks.workunit.client.0.smithi094.stdout:4/340: readlink d0/l38 0 2024-04-06T03:40:31.908 INFO:tasks.workunit.client.1.smithi136.stdout:7/232: dwrite d5/df/d53/f24 [0,4194304] 0 2024-04-06T03:40:31.910 INFO:tasks.workunit.client.1.smithi136.stdout:1/255: rmdir d12/d3b/d44 39 2024-04-06T03:40:31.914 INFO:tasks.workunit.client.0.smithi094.stdout:8/280: read d18/f1c [1354674,11599] 0 2024-04-06T03:40:31.914 INFO:tasks.workunit.client.1.smithi136.stdout:5/285: dread d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:31.915 INFO:tasks.workunit.client.1.smithi136.stdout:2/357: mknod d3/d9/d3b/c79 0 2024-04-06T03:40:31.915 INFO:tasks.workunit.client.1.smithi136.stdout:2/358: fdatasync d3/d9/d10/d18/d1d/f20 0 2024-04-06T03:40:31.915 INFO:tasks.workunit.client.1.smithi136.stdout:5/286: dread d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:31.915 INFO:tasks.workunit.client.1.smithi136.stdout:9/241: truncate d17/d19/f2f 1274065 0 2024-04-06T03:40:31.916 INFO:tasks.workunit.client.1.smithi136.stdout:5/287: dread d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:31.916 INFO:tasks.workunit.client.1.smithi136.stdout:5/288: fdatasync d1/dc/d10/d1e/f2f 0 2024-04-06T03:40:31.917 INFO:tasks.workunit.client.1.smithi136.stdout:0/218: mknod d1/d6/d2c/c47 0 2024-04-06T03:40:31.920 INFO:tasks.workunit.client.1.smithi136.stdout:9/242: write fb [1610619,63682] 0 2024-04-06T03:40:31.923 INFO:tasks.workunit.client.0.smithi094.stdout:9/224: unlink d2/dc/d21/f50 0 2024-04-06T03:40:31.923 INFO:tasks.workunit.client.0.smithi094.stdout:9/225: chown d2/d2c/d33/f4c 50 1 2024-04-06T03:40:31.923 INFO:tasks.workunit.client.0.smithi094.stdout:9/226: write d2/f5 [2350807,24616] 0 2024-04-06T03:40:31.923 INFO:tasks.workunit.client.0.smithi094.stdout:9/227: fdatasync d2/f30 0 2024-04-06T03:40:31.929 INFO:tasks.workunit.client.1.smithi136.stdout:8/278: rename d3/da/d26/d35/l47 to d3/d2d/l51 0 2024-04-06T03:40:31.930 INFO:tasks.workunit.client.0.smithi094.stdout:9/228: dread d2/d2c/d33/f4c [0,4194304] 0 2024-04-06T03:40:31.931 INFO:tasks.workunit.client.1.smithi136.stdout:4/205: creat d1d/f4f x:0 0 0 2024-04-06T03:40:31.931 INFO:tasks.workunit.client.1.smithi136.stdout:5/289: dwrite d1/dc/d10/d1e/f2f [4194304,4194304] 0 2024-04-06T03:40:31.932 INFO:tasks.workunit.client.0.smithi094.stdout:2/319: rename d0/d18 to d0/d18/d65 22 2024-04-06T03:40:31.934 INFO:tasks.workunit.client.1.smithi136.stdout:7/233: creat d5/df/d1d/f55 x:0 0 0 2024-04-06T03:40:31.934 INFO:tasks.workunit.client.1.smithi136.stdout:7/234: chown d5/df/d3d/f3f 148553 1 2024-04-06T03:40:31.934 INFO:tasks.workunit.client.1.smithi136.stdout:7/235: chown d5/df/d1d/f1f 260381558 1 2024-04-06T03:40:31.935 INFO:tasks.workunit.client.0.smithi094.stdout:4/341: mknod d0/df/d13/d4a/c71 0 2024-04-06T03:40:31.936 INFO:tasks.workunit.client.1.smithi136.stdout:2/359: symlink d3/d15/d19/d43/d54/l7a 0 2024-04-06T03:40:31.936 INFO:tasks.workunit.client.1.smithi136.stdout:2/360: stat d3/d45/l59 0 2024-04-06T03:40:31.936 INFO:tasks.workunit.client.0.smithi094.stdout:8/281: truncate d18/d34/f38 3583633 0 2024-04-06T03:40:31.938 INFO:tasks.workunit.client.1.smithi136.stdout:0/219: creat d1/d6/d2c/d1a/d1c/f48 x:0 0 0 2024-04-06T03:40:31.939 INFO:tasks.workunit.client.1.smithi136.stdout:0/220: creat d1/d6/d2c/d1a/d36/f49 x:0 0 0 2024-04-06T03:40:31.939 INFO:tasks.workunit.client.1.smithi136.stdout:0/221: readlink d1/d2e/l34 0 2024-04-06T03:40:31.939 INFO:tasks.workunit.client.1.smithi136.stdout:0/222: chown d1/d6/d2c/c47 141817454 1 2024-04-06T03:40:31.942 INFO:tasks.workunit.client.0.smithi094.stdout:4/342: dread d0/df/d13/f37 [0,4194304] 0 2024-04-06T03:40:31.942 INFO:tasks.workunit.client.1.smithi136.stdout:3/188: sync 2024-04-06T03:40:31.944 INFO:tasks.workunit.client.1.smithi136.stdout:7/236: dread d5/fd [4194304,4194304] 0 2024-04-06T03:40:31.944 INFO:tasks.workunit.client.1.smithi136.stdout:6/201: rename d2/d7/f20 to d2/d7/d13/f48 0 2024-04-06T03:40:31.949 INFO:tasks.workunit.client.1.smithi136.stdout:8/279: unlink d3/da/d26/d35/c23 0 2024-04-06T03:40:31.953 INFO:tasks.workunit.client.0.smithi094.stdout:2/320: mknod d0/dd/c66 0 2024-04-06T03:40:31.956 INFO:tasks.workunit.client.1.smithi136.stdout:7/237: write d5/df/d12/d1b/f30 [3850473,37332] 0 2024-04-06T03:40:31.956 INFO:tasks.workunit.client.1.smithi136.stdout:7/238: fdatasync d5/f46 0 2024-04-06T03:40:31.956 INFO:tasks.workunit.client.1.smithi136.stdout:7/239: creat d5/df/d11/d22/f56 x:0 0 0 2024-04-06T03:40:31.956 INFO:tasks.workunit.client.1.smithi136.stdout:7/240: creat d5/df/d1d/f57 x:0 0 0 2024-04-06T03:40:31.957 INFO:tasks.workunit.client.1.smithi136.stdout:4/206: symlink d1d/d31/l50 0 2024-04-06T03:40:31.958 INFO:tasks.workunit.client.0.smithi094.stdout:0/291: sync 2024-04-06T03:40:31.958 INFO:tasks.workunit.client.0.smithi094.stdout:0/292: readlink d1/d9/dd/d18/d1e/l26 0 2024-04-06T03:40:31.958 INFO:tasks.workunit.client.0.smithi094.stdout:5/192: sync 2024-04-06T03:40:31.958 INFO:tasks.workunit.client.0.smithi094.stdout:5/193: write db/d19/f3d [773853,57803] 0 2024-04-06T03:40:31.958 INFO:tasks.workunit.client.0.smithi094.stdout:3/308: sync 2024-04-06T03:40:31.960 INFO:tasks.workunit.client.0.smithi094.stdout:8/282: symlink d18/d3c/d49/l5e 0 2024-04-06T03:40:31.962 INFO:tasks.workunit.client.1.smithi136.stdout:2/361: dwrite d3/f61 [0,4194304] 0 2024-04-06T03:40:31.964 INFO:tasks.workunit.client.1.smithi136.stdout:8/280: read d3/de/f3a [490594,71584] 0 2024-04-06T03:40:31.964 INFO:tasks.workunit.client.1.smithi136.stdout:5/290: creat d1/dc/d1c/f56 x:0 0 0 2024-04-06T03:40:31.964 INFO:tasks.workunit.client.1.smithi136.stdout:5/291: stat d1/dc/d1c/d32/f38 0 2024-04-06T03:40:31.964 INFO:tasks.workunit.client.1.smithi136.stdout:5/292: creat d1/d16/d4b/f57 x:0 0 0 2024-04-06T03:40:31.964 INFO:tasks.workunit.client.1.smithi136.stdout:5/293: write d1/dc/d10/f49 [1193053,114477] 0 2024-04-06T03:40:31.964 INFO:tasks.workunit.client.1.smithi136.stdout:5/294: fdatasync d1/d16/d4b/f57 0 2024-04-06T03:40:31.964 INFO:tasks.workunit.client.1.smithi136.stdout:5/295: chown d1/d1f/d52 425681 1 2024-04-06T03:40:31.968 INFO:tasks.workunit.client.1.smithi136.stdout:0/223: dwrite d1/f7 [0,4194304] 0 2024-04-06T03:40:31.969 INFO:tasks.workunit.client.0.smithi094.stdout:0/293: dread d1/f2c [0,4194304] 0 2024-04-06T03:40:31.969 INFO:tasks.workunit.client.0.smithi094.stdout:0/294: creat d1/d9/f5e x:0 0 0 2024-04-06T03:40:31.970 INFO:tasks.workunit.client.1.smithi136.stdout:5/296: write d1/dc/d1c/f2c [914980,89821] 0 2024-04-06T03:40:31.970 INFO:tasks.workunit.client.1.smithi136.stdout:3/189: mkdir d0/d4/d12/d35/d3c 0 2024-04-06T03:40:31.971 INFO:tasks.workunit.client.0.smithi094.stdout:9/229: getdents d2/dc/d21 0 2024-04-06T03:40:31.971 INFO:tasks.workunit.client.0.smithi094.stdout:9/230: dread - d2/f30 zero size 2024-04-06T03:40:31.972 INFO:tasks.workunit.client.1.smithi136.stdout:6/202: chown d2/l19 86927629 1 2024-04-06T03:40:31.972 INFO:tasks.workunit.client.0.smithi094.stdout:8/283: symlink d18/d34/l5f 0 2024-04-06T03:40:31.972 INFO:tasks.workunit.client.0.smithi094.stdout:3/309: read d9/d10/d27/f38 [1041054,11841] 0 2024-04-06T03:40:31.975 INFO:tasks.workunit.client.1.smithi136.stdout:9/243: dwrite d17/d19/f1f [0,4194304] 0 2024-04-06T03:40:31.977 INFO:tasks.workunit.client.1.smithi136.stdout:8/281: write d3/da/d26/d35/d39/f43 [2738572,94966] 0 2024-04-06T03:40:31.977 INFO:tasks.workunit.client.1.smithi136.stdout:1/256: rename d12/d1b/d28/l2f to d12/l55 0 2024-04-06T03:40:31.977 INFO:tasks.workunit.client.0.smithi094.stdout:2/321: dwrite d0/dd/d19/d31/f3a [0,4194304] 0 2024-04-06T03:40:31.978 INFO:tasks.workunit.client.0.smithi094.stdout:5/194: dwrite db/d19/d2c/f2d [0,4194304] 0 2024-04-06T03:40:31.978 INFO:tasks.workunit.client.0.smithi094.stdout:5/195: stat db/d19/d24/d2e/f3a 0 2024-04-06T03:40:31.978 INFO:tasks.workunit.client.0.smithi094.stdout:5/196: dread - db/d40/f43 zero size 2024-04-06T03:40:31.980 INFO:tasks.workunit.client.0.smithi094.stdout:0/295: creat d1/d9/dd/d18/d32/f5f x:0 0 0 2024-04-06T03:40:31.982 INFO:tasks.workunit.client.0.smithi094.stdout:4/343: truncate d0/df/d1e/f24 6739242 0 2024-04-06T03:40:31.982 INFO:tasks.workunit.client.0.smithi094.stdout:4/344: readlink d0/df/d13/d4a/l60 0 2024-04-06T03:40:31.984 INFO:tasks.workunit.client.0.smithi094.stdout:8/284: rmdir d18 39 2024-04-06T03:40:31.987 INFO:tasks.workunit.client.1.smithi136.stdout:7/241: mkdir d5/df/d12/d1b/d58 0 2024-04-06T03:40:31.988 INFO:tasks.workunit.client.0.smithi094.stdout:3/310: rename d9/d1d/f25 to d9/d1d/f6a 0 2024-04-06T03:40:31.990 INFO:tasks.workunit.client.0.smithi094.stdout:9/231: dwrite d2/d2c/d33/d37/f3a [0,4194304] 0 2024-04-06T03:40:31.991 INFO:tasks.workunit.client.0.smithi094.stdout:0/296: dread d1/f2c [0,4194304] 0 2024-04-06T03:40:31.995 INFO:tasks.workunit.client.1.smithi136.stdout:9/244: dwrite fd [0,4194304] 0 2024-04-06T03:40:31.996 INFO:tasks.workunit.client.1.smithi136.stdout:2/362: mknod d3/c7b 0 2024-04-06T03:40:31.997 INFO:tasks.workunit.client.0.smithi094.stdout:2/322: truncate d0/d8/d27/d2b/f2c 950095 0 2024-04-06T03:40:32.000 INFO:tasks.workunit.client.1.smithi136.stdout:1/257: dwrite d12/d1b/d28/f3f [0,4194304] 0 2024-04-06T03:40:32.000 INFO:tasks.workunit.client.1.smithi136.stdout:7/242: dread d5/df/d11/f1c [0,4194304] 0 2024-04-06T03:40:32.002 INFO:tasks.workunit.client.0.smithi094.stdout:5/197: symlink db/d19/d24/d2e/l47 0 2024-04-06T03:40:32.002 INFO:tasks.workunit.client.0.smithi094.stdout:5/198: dread - db/f29 zero size 2024-04-06T03:40:32.004 INFO:tasks.workunit.client.1.smithi136.stdout:0/224: mkdir d1/d2e/d43/d4a 0 2024-04-06T03:40:32.005 INFO:tasks.workunit.client.0.smithi094.stdout:4/345: truncate d0/d4/f14 2732045 0 2024-04-06T03:40:32.005 INFO:tasks.workunit.client.0.smithi094.stdout:4/346: read - d0/d23/f6e zero size 2024-04-06T03:40:32.005 INFO:tasks.workunit.client.0.smithi094.stdout:4/347: truncate d0/d15/d3a/f43 1404106 0 2024-04-06T03:40:32.006 INFO:tasks.workunit.client.1.smithi136.stdout:0/225: dread d1/d6/f45 [0,4194304] 0 2024-04-06T03:40:32.006 INFO:tasks.workunit.client.1.smithi136.stdout:0/226: readlink d1/d2e/l2f 0 2024-04-06T03:40:32.006 INFO:tasks.workunit.client.1.smithi136.stdout:0/227: creat d1/d6/d2c/d1a/f4b x:0 0 0 2024-04-06T03:40:32.006 INFO:tasks.workunit.client.1.smithi136.stdout:0/228: write d1/d6/d2c/d1a/d1c/f23 [1960642,115708] 0 2024-04-06T03:40:32.006 INFO:tasks.workunit.client.1.smithi136.stdout:0/229: readlink d1/le 0 2024-04-06T03:40:32.006 INFO:tasks.workunit.client.1.smithi136.stdout:0/230: readlink d1/le 0 2024-04-06T03:40:32.007 INFO:tasks.workunit.client.1.smithi136.stdout:0/231: write d1/d6/d2c/d1a/f28 [915306,81610] 0 2024-04-06T03:40:32.007 INFO:tasks.workunit.client.1.smithi136.stdout:0/232: write d1/d2e/f42 [721969,76314] 0 2024-04-06T03:40:32.007 INFO:tasks.workunit.client.1.smithi136.stdout:3/190: symlink d0/d4/d12/d2d/l3d 0 2024-04-06T03:40:32.008 INFO:tasks.workunit.client.0.smithi094.stdout:8/285: unlink f3 0 2024-04-06T03:40:32.009 INFO:tasks.workunit.client.1.smithi136.stdout:6/203: fsync d2/f5 0 2024-04-06T03:40:32.009 INFO:tasks.workunit.client.1.smithi136.stdout:6/204: chown d2/d7/l23 3 1 2024-04-06T03:40:32.010 INFO:tasks.workunit.client.1.smithi136.stdout:7/243: write d5/f52 [2695700,4316] 0 2024-04-06T03:40:32.010 INFO:tasks.workunit.client.1.smithi136.stdout:7/244: chown d5/df/d3d/f3f 277528 1 2024-04-06T03:40:32.011 INFO:tasks.workunit.client.0.smithi094.stdout:3/311: mkdir d9/d48/d6b 0 2024-04-06T03:40:32.011 INFO:tasks.workunit.client.0.smithi094.stdout:3/312: write d9/d1e/d26/f28 [1517630,69715] 0 2024-04-06T03:40:32.013 INFO:tasks.workunit.client.1.smithi136.stdout:4/207: rename ff to d1d/d2f/f51 0 2024-04-06T03:40:32.014 INFO:tasks.workunit.client.1.smithi136.stdout:2/363: dwrite d3/d9/d10/d18/d4b/f65 [0,4194304] 0 2024-04-06T03:40:32.016 INFO:tasks.workunit.client.1.smithi136.stdout:1/258: mknod d12/d53/c56 0 2024-04-06T03:40:32.020 INFO:tasks.workunit.client.1.smithi136.stdout:3/191: creat d0/d4/d12/d3a/f3e x:0 0 0 2024-04-06T03:40:32.022 INFO:tasks.workunit.client.1.smithi136.stdout:0/233: dwrite d1/d6/d2c/d1a/d1c/f23 [0,4194304] 0 2024-04-06T03:40:32.023 INFO:tasks.workunit.client.0.smithi094.stdout:9/232: rmdir d2/dc/d10/d2a 39 2024-04-06T03:40:32.023 INFO:tasks.workunit.client.0.smithi094.stdout:9/233: chown d2/dc/d10/d2a/c48 23129 1 2024-04-06T03:40:32.023 INFO:tasks.workunit.client.0.smithi094.stdout:9/234: chown d2/f5 582252899 1 2024-04-06T03:40:32.023 INFO:tasks.workunit.client.0.smithi094.stdout:9/235: chown d2/d2c/d33/f44 1 1 2024-04-06T03:40:32.025 INFO:tasks.workunit.client.1.smithi136.stdout:7/245: rmdir d5/df/d11/d22/d2c/d50 0 2024-04-06T03:40:32.025 INFO:tasks.workunit.client.0.smithi094.stdout:0/297: mkdir d1/d9/dd/d18/d5d/d60 0 2024-04-06T03:40:32.028 INFO:tasks.workunit.client.1.smithi136.stdout:5/297: rename d1/dc/d10/l30 to d1/dc/d1c/l58 0 2024-04-06T03:40:32.032 INFO:tasks.workunit.client.1.smithi136.stdout:4/208: creat d1d/f52 x:0 0 0 2024-04-06T03:40:32.032 INFO:tasks.workunit.client.1.smithi136.stdout:4/209: creat d1d/d2f/f53 x:0 0 0 2024-04-06T03:40:32.035 INFO:tasks.workunit.client.1.smithi136.stdout:2/364: symlink d3/d9/d10/d18/d1d/d3c/d58/l7c 0 2024-04-06T03:40:32.035 INFO:tasks.workunit.client.1.smithi136.stdout:2/365: truncate d3/d9/d10/d18/d1d/d3c/f40 1531006 0 2024-04-06T03:40:32.035 INFO:tasks.workunit.client.1.smithi136.stdout:2/366: chown d3/d9/d10/c3a 185 1 2024-04-06T03:40:32.036 INFO:tasks.workunit.client.0.smithi094.stdout:2/323: mkdir d0/dd/d19/d31/d67 0 2024-04-06T03:40:32.037 INFO:tasks.workunit.client.1.smithi136.stdout:1/259: creat d12/d3b/d44/f57 x:0 0 0 2024-04-06T03:40:32.038 INFO:tasks.workunit.client.1.smithi136.stdout:3/192: dwrite d0/d4/d12/d2d/f33 [0,4194304] 0 2024-04-06T03:40:32.039 INFO:tasks.workunit.client.1.smithi136.stdout:3/193: fsync d0/f2 0 2024-04-06T03:40:32.039 INFO:tasks.workunit.client.1.smithi136.stdout:3/194: chown d0/f1e 1755049832 1 2024-04-06T03:40:32.039 INFO:tasks.workunit.client.1.smithi136.stdout:3/195: write d0/d4/d12/d19/f29 [637055,17452] 0 2024-04-06T03:40:32.041 INFO:tasks.workunit.client.1.smithi136.stdout:2/367: read d3/d9/d10/d18/f2a [1705111,5864] 0 2024-04-06T03:40:32.042 INFO:tasks.workunit.client.0.smithi094.stdout:5/199: creat db/d19/d24/d2e/f48 x:0 0 0 2024-04-06T03:40:32.042 INFO:tasks.workunit.client.0.smithi094.stdout:5/200: write db/d19/d24/d28/f3f [646681,9099] 0 2024-04-06T03:40:32.042 INFO:tasks.workunit.client.0.smithi094.stdout:5/201: creat db/d19/d21/f49 x:0 0 0 2024-04-06T03:40:32.042 INFO:tasks.workunit.client.0.smithi094.stdout:5/202: stat db/d19/d24/d2e/f36 0 2024-04-06T03:40:32.044 INFO:tasks.workunit.client.1.smithi136.stdout:6/205: link d2/d7/d26/d30/c42 d2/de/c49 0 2024-04-06T03:40:32.047 INFO:tasks.workunit.client.0.smithi094.stdout:4/348: unlink d0/f29 0 2024-04-06T03:40:32.047 INFO:tasks.workunit.client.0.smithi094.stdout:4/349: creat d0/d15/d3a/d3e/d46/f72 x:0 0 0 2024-04-06T03:40:32.049 INFO:tasks.workunit.client.0.smithi094.stdout:8/286: truncate f14 4332982 0 2024-04-06T03:40:32.050 INFO:tasks.workunit.client.0.smithi094.stdout:1/254: sync 2024-04-06T03:40:32.050 INFO:tasks.workunit.client.0.smithi094.stdout:7/267: sync 2024-04-06T03:40:32.050 INFO:tasks.workunit.client.0.smithi094.stdout:1/255: truncate d0/d4/f9 4475968 0 2024-04-06T03:40:32.051 INFO:tasks.workunit.client.0.smithi094.stdout:7/268: creat d2/d6/dc/dd/f48 x:0 0 0 2024-04-06T03:40:32.051 INFO:tasks.workunit.client.1.smithi136.stdout:7/246: mkdir d5/df/d12/d59 0 2024-04-06T03:40:32.051 INFO:tasks.workunit.client.0.smithi094.stdout:3/313: mkdir d9/d1e/d26/d6c 0 2024-04-06T03:40:32.053 INFO:tasks.workunit.client.1.smithi136.stdout:8/282: rename d3/da/d26/d35/l2b to d3/da/d26/d35/d39/l52 0 2024-04-06T03:40:32.053 INFO:tasks.workunit.client.1.smithi136.stdout:8/283: getdents d3/da/d26/d35/d3d/d48 0 2024-04-06T03:40:32.054 INFO:tasks.workunit.client.0.smithi094.stdout:9/236: write d2/dc/d21/d26/f3c [3378424,130632] 0 2024-04-06T03:40:32.055 INFO:tasks.workunit.client.0.smithi094.stdout:9/237: write d2/d2c/d33/f4c [4359642,14365] 0 2024-04-06T03:40:32.057 INFO:tasks.workunit.client.1.smithi136.stdout:5/298: mknod d1/dc/d10/c59 0 2024-04-06T03:40:32.058 INFO:tasks.workunit.client.0.smithi094.stdout:0/298: truncate d1/f55 223773 0 2024-04-06T03:40:32.059 INFO:tasks.workunit.client.0.smithi094.stdout:6/253: sync 2024-04-06T03:40:32.065 INFO:tasks.workunit.client.1.smithi136.stdout:4/210: symlink d1d/d4d/l54 0 2024-04-06T03:40:32.070 INFO:tasks.workunit.client.1.smithi136.stdout:3/196: symlink d0/d4/d12/l3f 0 2024-04-06T03:40:32.070 INFO:tasks.workunit.client.1.smithi136.stdout:3/197: read d0/f1 [1358623,68232] 0 2024-04-06T03:40:32.071 INFO:tasks.workunit.client.0.smithi094.stdout:5/203: mknod db/d19/d21/c4a 0 2024-04-06T03:40:32.071 INFO:tasks.workunit.client.0.smithi094.stdout:5/204: chown db/d40 639522 1 2024-04-06T03:40:32.078 INFO:tasks.workunit.client.1.smithi136.stdout:7/247: dwrite d5/df/f10 [0,4194304] 0 2024-04-06T03:40:32.080 INFO:tasks.workunit.client.0.smithi094.stdout:4/350: creat d0/d15/d3a/d3e/d49/f73 x:0 0 0 2024-04-06T03:40:32.083 INFO:tasks.workunit.client.0.smithi094.stdout:8/287: mkdir d18/d2c/d37/d60 0 2024-04-06T03:40:32.085 INFO:tasks.workunit.client.0.smithi094.stdout:4/351: dread d0/df/f66 [4194304,4194304] 0 2024-04-06T03:40:32.086 INFO:tasks.workunit.client.0.smithi094.stdout:4/352: readlink d0/df/d1e/l2e 0 2024-04-06T03:40:32.087 INFO:tasks.workunit.client.1.smithi136.stdout:3/198: dwrite d0/d4/d12/f15 [0,4194304] 0 2024-04-06T03:40:32.088 INFO:tasks.workunit.client.1.smithi136.stdout:5/299: dwrite d1/d16/d17/f44 [0,4194304] 0 2024-04-06T03:40:32.088 INFO:tasks.workunit.client.1.smithi136.stdout:8/284: dwrite d3/fc [0,4194304] 0 2024-04-06T03:40:32.089 INFO:tasks.workunit.client.0.smithi094.stdout:1/256: rmdir d0/d1/d16/d40/d50/d57 0 2024-04-06T03:40:32.093 INFO:tasks.workunit.client.0.smithi094.stdout:4/353: write d0/f8 [1382898,77803] 0 2024-04-06T03:40:32.093 INFO:tasks.workunit.client.0.smithi094.stdout:4/354: creat d0/d15/d3a/d3e/d49/f74 x:0 0 0 2024-04-06T03:40:32.094 INFO:tasks.workunit.client.0.smithi094.stdout:4/355: truncate d0/f22 1044089 0 2024-04-06T03:40:32.094 INFO:tasks.workunit.client.1.smithi136.stdout:7/248: dwrite d5/f52 [0,4194304] 0 2024-04-06T03:40:32.094 INFO:tasks.workunit.client.1.smithi136.stdout:7/249: write d5/df/d1d/f1f [5168191,90569] 0 2024-04-06T03:40:32.094 INFO:tasks.workunit.client.0.smithi094.stdout:7/269: mknod d2/d6/dc/d12/d3a/c49 0 2024-04-06T03:40:32.095 INFO:tasks.workunit.client.0.smithi094.stdout:7/270: creat d2/d6/dc/f4a x:0 0 0 2024-04-06T03:40:32.095 INFO:tasks.workunit.client.0.smithi094.stdout:3/314: symlink d9/d48/l6d 0 2024-04-06T03:40:32.097 INFO:tasks.workunit.client.0.smithi094.stdout:9/238: mkdir d2/dc/d21/d23/d47/d51 0 2024-04-06T03:40:32.102 INFO:tasks.workunit.client.0.smithi094.stdout:2/324: rmdir d0/dd 39 2024-04-06T03:40:32.108 INFO:tasks.workunit.client.1.smithi136.stdout:7/250: dwrite d5/df/d1d/f57 [0,4194304] 0 2024-04-06T03:40:32.109 INFO:tasks.workunit.client.0.smithi094.stdout:5/205: symlink db/d19/d21/l4b 0 2024-04-06T03:40:32.111 INFO:tasks.workunit.client.0.smithi094.stdout:8/288: symlink d18/d3c/d49/l61 0 2024-04-06T03:40:32.114 INFO:tasks.workunit.client.0.smithi094.stdout:1/257: link d0/c4a d0/d1/d16/d24/d30/d38/c5a 0 2024-04-06T03:40:32.124 INFO:tasks.workunit.client.1.smithi136.stdout:6/206: mknod d2/d7/d13/c4a 0 2024-04-06T03:40:32.124 INFO:tasks.workunit.client.0.smithi094.stdout:7/271: mknod d2/d6/dc/d12/c4b 0 2024-04-06T03:40:32.125 INFO:tasks.workunit.client.1.smithi136.stdout:0/234: rmdir d1/d6/d2c/d1a/d1c 39 2024-04-06T03:40:32.126 INFO:tasks.workunit.client.0.smithi094.stdout:3/315: mkdir d9/d1e/d6e 0 2024-04-06T03:40:32.128 INFO:tasks.workunit.client.1.smithi136.stdout:9/245: rename d17/d21/d32/c40 to d17/d20/d25/c50 0 2024-04-06T03:40:32.128 INFO:tasks.workunit.client.1.smithi136.stdout:9/246: fsync f11 0 2024-04-06T03:40:32.131 INFO:tasks.workunit.client.0.smithi094.stdout:9/239: link d2/d2c/d33/f4a d2/dc/d21/d23/d47/d51/f52 0 2024-04-06T03:40:32.131 INFO:tasks.workunit.client.0.smithi094.stdout:9/240: truncate d2/d2c/d33/f4c 5901039 0 2024-04-06T03:40:32.132 INFO:tasks.workunit.client.1.smithi136.stdout:4/211: chown d1d/f20 112 1 2024-04-06T03:40:32.132 INFO:tasks.workunit.client.0.smithi094.stdout:0/299: rmdir d1/d9/dd/d18/d32/d42 39 2024-04-06T03:40:32.133 INFO:tasks.workunit.client.1.smithi136.stdout:6/207: write d2/f5 [1540065,109293] 0 2024-04-06T03:40:32.133 INFO:tasks.workunit.client.1.smithi136.stdout:6/208: readlink d2/d7/l1f 0 2024-04-06T03:40:32.133 INFO:tasks.workunit.client.1.smithi136.stdout:6/209: dread - d2/d25/f3b zero size 2024-04-06T03:40:32.135 INFO:tasks.workunit.client.0.smithi094.stdout:4/356: dwrite d0/df/d13/f58 [0,4194304] 0 2024-04-06T03:40:32.135 INFO:tasks.workunit.client.0.smithi094.stdout:4/357: fsync d0/d4/d4b/f6c 0 2024-04-06T03:40:32.137 INFO:tasks.workunit.client.0.smithi094.stdout:2/325: truncate d0/dd/d3d/d49/f53 2875297 0 2024-04-06T03:40:32.139 INFO:tasks.workunit.client.1.smithi136.stdout:3/199: link d0/f1e d0/d4/d12/d19/d2c/f40 0 2024-04-06T03:40:32.141 INFO:tasks.workunit.client.0.smithi094.stdout:0/300: write d1/d10/f2e [819660,29397] 0 2024-04-06T03:40:32.141 INFO:tasks.workunit.client.0.smithi094.stdout:0/301: chown d1/d9/dd/d18/d32/d42/f4a 30824 1 2024-04-06T03:40:32.142 INFO:tasks.workunit.client.0.smithi094.stdout:5/206: symlink db/d19/d30/l4c 0 2024-04-06T03:40:32.143 INFO:tasks.workunit.client.0.smithi094.stdout:8/289: mkdir d18/d2c/d62 0 2024-04-06T03:40:32.145 INFO:tasks.workunit.client.0.smithi094.stdout:1/258: unlink d0/d1/d16/d3f/f4e 0 2024-04-06T03:40:32.147 INFO:tasks.workunit.client.0.smithi094.stdout:7/272: mknod d2/c4c 0 2024-04-06T03:40:32.147 INFO:tasks.workunit.client.1.smithi136.stdout:9/247: dwrite d17/d20/f48 [0,4194304] 0 2024-04-06T03:40:32.147 INFO:tasks.workunit.client.1.smithi136.stdout:9/248: fsync d17/d19/f1e 0 2024-04-06T03:40:32.148 INFO:tasks.workunit.client.1.smithi136.stdout:5/300: symlink d1/d16/d4b/l5a 0 2024-04-06T03:40:32.149 INFO:tasks.workunit.client.0.smithi094.stdout:3/316: symlink d9/d1e/d26/d54/l6f 0 2024-04-06T03:40:32.151 INFO:tasks.workunit.client.1.smithi136.stdout:8/285: mkdir d3/da/d26/d35/d53 0 2024-04-06T03:40:32.151 INFO:tasks.workunit.client.1.smithi136.stdout:8/286: stat d3/de/f4b 0 2024-04-06T03:40:32.154 INFO:tasks.workunit.client.0.smithi094.stdout:5/207: dread db/f11 [0,4194304] 0 2024-04-06T03:40:32.156 INFO:tasks.workunit.client.0.smithi094.stdout:9/241: mknod d2/dc/d10/c53 0 2024-04-06T03:40:32.157 INFO:tasks.workunit.client.1.smithi136.stdout:8/287: dread d3/da/fd [0,4194304] 0 2024-04-06T03:40:32.157 INFO:tasks.workunit.client.1.smithi136.stdout:8/288: fdatasync d3/de/f4c 0 2024-04-06T03:40:32.157 INFO:tasks.workunit.client.1.smithi136.stdout:9/249: read f11 [4998,91895] 0 2024-04-06T03:40:32.157 INFO:tasks.workunit.client.1.smithi136.stdout:5/301: read d1/d16/d17/f1a [920478,19737] 0 2024-04-06T03:40:32.160 INFO:tasks.workunit.client.0.smithi094.stdout:3/317: write d9/d1e/d26/f2c [2586261,22646] 0 2024-04-06T03:40:32.160 INFO:tasks.workunit.client.0.smithi094.stdout:3/318: creat d9/d10/f70 x:0 0 0 2024-04-06T03:40:32.164 INFO:tasks.workunit.client.0.smithi094.stdout:4/358: creat d0/d15/d3a/f75 x:0 0 0 2024-04-06T03:40:32.165 INFO:tasks.workunit.client.1.smithi136.stdout:7/251: symlink d5/df/d12/d59/l5a 0 2024-04-06T03:40:32.166 INFO:tasks.workunit.client.1.smithi136.stdout:9/250: dread f13 [0,4194304] 0 2024-04-06T03:40:32.167 INFO:tasks.workunit.client.0.smithi094.stdout:2/326: symlink d0/dd/d3d/d49/l68 0 2024-04-06T03:40:32.168 INFO:tasks.workunit.client.1.smithi136.stdout:0/235: getdents d1/d6 0 2024-04-06T03:40:32.168 INFO:tasks.workunit.client.1.smithi136.stdout:0/236: readlink d1/d2e/l2f 0 2024-04-06T03:40:32.169 INFO:tasks.workunit.client.0.smithi094.stdout:8/290: write d18/f1b [1967385,50415] 0 2024-04-06T03:40:32.170 INFO:tasks.workunit.client.1.smithi136.stdout:8/289: dwrite d3/f4d [0,4194304] 0 2024-04-06T03:40:32.170 INFO:tasks.workunit.client.1.smithi136.stdout:2/368: rename d3/d9/d3b/c76 to d3/d15/c7d 0 2024-04-06T03:40:32.172 INFO:tasks.workunit.client.0.smithi094.stdout:1/259: link d0/d1/l21 d0/d1/d16/d40/d50/l5b 0 2024-04-06T03:40:32.173 INFO:tasks.workunit.client.1.smithi136.stdout:4/212: mkdir d1d/d31/d3e/d55 0 2024-04-06T03:40:32.173 INFO:tasks.workunit.client.1.smithi136.stdout:4/213: write d1d/f38 [334299,112250] 0 2024-04-06T03:40:32.175 INFO:tasks.workunit.client.0.smithi094.stdout:7/273: creat d2/d6/dc/dd/df/d36/f4d x:0 0 0 2024-04-06T03:40:32.175 INFO:tasks.workunit.client.0.smithi094.stdout:7/274: chown d2/c3 168152 1 2024-04-06T03:40:32.175 INFO:tasks.workunit.client.0.smithi094.stdout:7/275: creat d2/d6/dc/dd/f4e x:0 0 0 2024-04-06T03:40:32.175 INFO:tasks.workunit.client.0.smithi094.stdout:7/276: write d2/d6/dc/d12/f2a [262162,123386] 0 2024-04-06T03:40:32.175 INFO:tasks.workunit.client.0.smithi094.stdout:7/277: fsync d2/d6/dc/dd/df/f42 0 2024-04-06T03:40:32.177 INFO:tasks.workunit.client.1.smithi136.stdout:1/260: sync 2024-04-06T03:40:32.177 INFO:tasks.workunit.client.1.smithi136.stdout:6/210: mkdir d2/d7/d4b 0 2024-04-06T03:40:32.177 INFO:tasks.workunit.client.1.smithi136.stdout:6/211: chown d2/d25/f3b 2 1 2024-04-06T03:40:32.180 INFO:tasks.workunit.client.1.smithi136.stdout:3/200: truncate d0/f23 1246560 0 2024-04-06T03:40:32.181 INFO:tasks.workunit.client.1.smithi136.stdout:8/290: write d3/f9 [1687183,41253] 0 2024-04-06T03:40:32.182 INFO:tasks.workunit.client.1.smithi136.stdout:8/291: chown d3/da/d26 7 1 2024-04-06T03:40:32.182 INFO:tasks.workunit.client.0.smithi094.stdout:9/242: mknod d2/dc/c54 0 2024-04-06T03:40:32.182 INFO:tasks.workunit.client.0.smithi094.stdout:1/260: read d0/d1/f10 [622726,37773] 0 2024-04-06T03:40:32.188 INFO:tasks.workunit.client.1.smithi136.stdout:5/302: creat d1/dc/f5b x:0 0 0 2024-04-06T03:40:32.189 INFO:tasks.workunit.client.0.smithi094.stdout:3/319: link d9/d48/l66 d9/d1e/d26/d54/l71 0 2024-04-06T03:40:32.191 INFO:tasks.workunit.client.1.smithi136.stdout:7/252: mkdir d5/df/d12/d59/d5b 0 2024-04-06T03:40:32.192 INFO:tasks.workunit.client.0.smithi094.stdout:4/359: creat d0/d4/f76 x:0 0 0 2024-04-06T03:40:32.193 INFO:tasks.workunit.client.0.smithi094.stdout:4/360: chown d0/df/d1e/d34/c3b 51546 1 2024-04-06T03:40:32.194 INFO:tasks.workunit.client.1.smithi136.stdout:9/251: mkdir d17/d21/d32/d43/d51 0 2024-04-06T03:40:32.194 INFO:tasks.workunit.client.1.smithi136.stdout:9/252: write d17/f42 [575500,5597] 0 2024-04-06T03:40:32.196 INFO:tasks.workunit.client.0.smithi094.stdout:2/327: rename d0/d18/d3f/l46 to d0/dd/d3d/d49/l69 0 2024-04-06T03:40:32.196 INFO:tasks.workunit.client.0.smithi094.stdout:2/328: fsync d0/dd/d1b/f1c 0 2024-04-06T03:40:32.197 INFO:tasks.workunit.client.1.smithi136.stdout:1/261: dwrite d12/f24 [0,4194304] 0 2024-04-06T03:40:32.198 INFO:tasks.workunit.client.1.smithi136.stdout:0/237: creat d1/f4c x:0 0 0 2024-04-06T03:40:32.199 INFO:tasks.workunit.client.1.smithi136.stdout:7/253: dread d5/df/d11/f26 [4194304,4194304] 0 2024-04-06T03:40:32.200 INFO:tasks.workunit.client.0.smithi094.stdout:7/278: dwrite d2/f11 [0,4194304] 0 2024-04-06T03:40:32.200 INFO:tasks.workunit.client.0.smithi094.stdout:8/291: creat d18/f63 x:0 0 0 2024-04-06T03:40:32.200 INFO:tasks.workunit.client.0.smithi094.stdout:1/261: dwrite d0/d1/d16/d24/d30/d38/f4d [0,4194304] 0 2024-04-06T03:40:32.200 INFO:tasks.workunit.client.0.smithi094.stdout:1/262: creat d0/d1/d16/d3f/f5c x:0 0 0 2024-04-06T03:40:32.203 INFO:tasks.workunit.client.0.smithi094.stdout:4/361: dread d0/df/f66 [0,4194304] 0 2024-04-06T03:40:32.203 INFO:tasks.workunit.client.0.smithi094.stdout:4/362: write d0/d4/f21 [1916332,61813] 0 2024-04-06T03:40:32.204 INFO:tasks.workunit.client.1.smithi136.stdout:4/214: mkdir d1d/d31/d3e/d56 0 2024-04-06T03:40:32.206 INFO:tasks.workunit.client.0.smithi094.stdout:5/208: getdents db/d19 0 2024-04-06T03:40:32.207 INFO:tasks.workunit.client.1.smithi136.stdout:6/212: creat d2/d7/d26/d30/f4c x:0 0 0 2024-04-06T03:40:32.207 INFO:tasks.workunit.client.0.smithi094.stdout:7/279: dread d2/d6/f9 [0,4194304] 0 2024-04-06T03:40:32.207 INFO:tasks.workunit.client.0.smithi094.stdout:7/280: write d2/d6/dc/d12/d29/d35/f3d [1545862,104387] 0 2024-04-06T03:40:32.207 INFO:tasks.workunit.client.0.smithi094.stdout:7/281: chown d2/d6/dc/dd/c20 2018922 1 2024-04-06T03:40:32.207 INFO:tasks.workunit.client.0.smithi094.stdout:7/282: chown d2/d6/dc/dd/f31 110696810 1 2024-04-06T03:40:32.208 INFO:tasks.workunit.client.1.smithi136.stdout:6/213: write d2/d7/f2c [763874,81407] 0 2024-04-06T03:40:32.209 INFO:tasks.workunit.client.0.smithi094.stdout:1/263: dread d0/d4/fc [0,4194304] 0 2024-04-06T03:40:32.209 INFO:tasks.workunit.client.0.smithi094.stdout:1/264: fsync d0/d1/d16/d24/d30/f33 0 2024-04-06T03:40:32.209 INFO:tasks.workunit.client.0.smithi094.stdout:1/265: creat d0/f5d x:0 0 0 2024-04-06T03:40:32.209 INFO:tasks.workunit.client.0.smithi094.stdout:3/320: rmdir d9/d1e/d26/d54 39 2024-04-06T03:40:32.210 INFO:tasks.workunit.client.1.smithi136.stdout:3/201: mknod d0/d4/d12/d35/d3c/c41 0 2024-04-06T03:40:32.210 INFO:tasks.workunit.client.1.smithi136.stdout:3/202: fsync d0/d4/f18 0 2024-04-06T03:40:32.212 INFO:tasks.workunit.client.0.smithi094.stdout:9/243: rename d2/dc/d10/d2a/d2d/f4d to d2/d2c/d33/d37/f55 0 2024-04-06T03:40:32.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/329: dwrite d0/dd/d19/d31/f42 [0,4194304] 0 2024-04-06T03:40:32.215 INFO:tasks.workunit.client.1.smithi136.stdout:8/292: creat d3/da/d26/d35/d3d/d48/f54 x:0 0 0 2024-04-06T03:40:32.218 INFO:tasks.workunit.client.0.smithi094.stdout:8/292: unlink d18/d2c/f3a 0 2024-04-06T03:40:32.218 INFO:tasks.workunit.client.0.smithi094.stdout:8/293: chown d18/f1b 649638806 1 2024-04-06T03:40:32.219 INFO:tasks.workunit.client.1.smithi136.stdout:3/203: dread d0/d4/d12/f15 [0,4194304] 0 2024-04-06T03:40:32.220 INFO:tasks.workunit.client.1.smithi136.stdout:9/253: unlink d17/d21/d29/f36 0 2024-04-06T03:40:32.220 INFO:tasks.workunit.client.1.smithi136.stdout:9/254: chown d17/d21/d32/d3f/f44 65518834 1 2024-04-06T03:40:32.220 INFO:tasks.workunit.client.1.smithi136.stdout:9/255: write d17/d19/f1e [1916102,110261] 0 2024-04-06T03:40:32.222 INFO:tasks.workunit.client.1.smithi136.stdout:1/262: rename d12/d1b/d1e/d23/f31 to d12/d1b/d28/d48/f58 0 2024-04-06T03:40:32.223 INFO:tasks.workunit.client.0.smithi094.stdout:4/363: mkdir d0/d77 0 2024-04-06T03:40:32.224 INFO:tasks.workunit.client.1.smithi136.stdout:0/238: mknod d1/d2e/d43/d4a/c4d 0 2024-04-06T03:40:32.225 INFO:tasks.workunit.client.0.smithi094.stdout:5/209: mknod db/d19/c4d 0 2024-04-06T03:40:32.226 INFO:tasks.workunit.client.1.smithi136.stdout:7/254: mkdir d5/df/d1d/d54/d5c 0 2024-04-06T03:40:32.227 INFO:tasks.workunit.client.1.smithi136.stdout:2/369: rmdir d3/d45/d5f 39 2024-04-06T03:40:32.230 INFO:tasks.workunit.client.1.smithi136.stdout:4/215: rmdir d1d/d4d 39 2024-04-06T03:40:32.230 INFO:tasks.workunit.client.1.smithi136.stdout:4/216: dread - d1d/f22 zero size 2024-04-06T03:40:32.230 INFO:tasks.workunit.client.1.smithi136.stdout:4/217: chown d1d/f26 3620 1 2024-04-06T03:40:32.231 INFO:tasks.workunit.client.0.smithi094.stdout:1/266: getdents d0/d7/d1a 0 2024-04-06T03:40:32.232 INFO:tasks.workunit.client.1.smithi136.stdout:6/214: write d2/f9 [27885,49495] 0 2024-04-06T03:40:32.235 INFO:tasks.workunit.client.0.smithi094.stdout:3/321: mknod d9/d10/d27/d3a/c72 0 2024-04-06T03:40:32.237 INFO:tasks.workunit.client.1.smithi136.stdout:8/293: link f1 d3/d2d/f55 0 2024-04-06T03:40:32.238 INFO:tasks.workunit.client.1.smithi136.stdout:8/294: creat d3/de/f56 x:0 0 0 2024-04-06T03:40:32.238 INFO:tasks.workunit.client.1.smithi136.stdout:8/295: write d3/de/f4b [753810,77546] 0 2024-04-06T03:40:32.241 INFO:tasks.workunit.client.0.smithi094.stdout:9/244: creat d2/dc/f56 x:0 0 0 2024-04-06T03:40:32.244 INFO:tasks.workunit.client.1.smithi136.stdout:5/303: symlink d1/l5c 0 2024-04-06T03:40:32.244 INFO:tasks.workunit.client.1.smithi136.stdout:5/304: chown d1/dc/d15/l4a 383903807 1 2024-04-06T03:40:32.246 INFO:tasks.workunit.client.1.smithi136.stdout:4/218: dwrite d1d/f52 [0,4194304] 0 2024-04-06T03:40:32.246 INFO:tasks.workunit.client.1.smithi136.stdout:4/219: creat d1d/d2f/f57 x:0 0 0 2024-04-06T03:40:32.246 INFO:tasks.workunit.client.1.smithi136.stdout:4/220: dread - d1d/d31/f44 zero size 2024-04-06T03:40:32.246 INFO:tasks.workunit.client.0.smithi094.stdout:2/330: symlink d0/d8/d27/d2b/l6a 0 2024-04-06T03:40:32.248 INFO:tasks.workunit.client.1.smithi136.stdout:3/204: symlink d0/d4/d12/d19/d2e/l42 0 2024-04-06T03:40:32.248 INFO:tasks.workunit.client.1.smithi136.stdout:3/205: chown d0/d4/d12/d3a 25265 1 2024-04-06T03:40:32.249 INFO:tasks.workunit.client.0.smithi094.stdout:8/294: creat d18/f64 x:0 0 0 2024-04-06T03:40:32.249 INFO:tasks.workunit.client.0.smithi094.stdout:9/245: dread d2/d2c/d33/d37/f3a [0,4194304] 0 2024-04-06T03:40:32.250 INFO:tasks.workunit.client.1.smithi136.stdout:9/256: creat d17/d20/d25/d3d/d4b/f52 x:0 0 0 2024-04-06T03:40:32.251 INFO:tasks.workunit.client.0.smithi094.stdout:4/364: symlink d0/df/d13/d4a/l78 0 2024-04-06T03:40:32.253 INFO:tasks.workunit.client.1.smithi136.stdout:1/263: symlink d12/d1b/d1e/d23/l59 0 2024-04-06T03:40:32.253 INFO:tasks.workunit.client.1.smithi136.stdout:1/264: readlink d12/d1b/l49 0 2024-04-06T03:40:32.256 INFO:tasks.workunit.client.1.smithi136.stdout:7/255: stat d5/df/d12/d1b/c31 0 2024-04-06T03:40:32.256 INFO:tasks.workunit.client.1.smithi136.stdout:7/256: chown d5/df/d53/f43 148244 1 2024-04-06T03:40:32.258 INFO:tasks.workunit.client.0.smithi094.stdout:1/267: unlink d0/d7/d1a/f1e 0 2024-04-06T03:40:32.258 INFO:tasks.workunit.client.0.smithi094.stdout:1/268: read d0/d7/f26 [452183,84372] 0 2024-04-06T03:40:32.258 INFO:tasks.workunit.client.0.smithi094.stdout:1/269: truncate d0/d1/d16/f3b 813570 0 2024-04-06T03:40:32.258 INFO:tasks.workunit.client.0.smithi094.stdout:2/331: write d0/f64 [1827036,46209] 0 2024-04-06T03:40:32.260 INFO:tasks.workunit.client.1.smithi136.stdout:2/370: mkdir d3/d4a/d77/d7e 0 2024-04-06T03:40:32.262 INFO:tasks.workunit.client.0.smithi094.stdout:3/322: rename d9/l63 to d9/d48/d6b/l73 0 2024-04-06T03:40:32.262 INFO:tasks.workunit.client.0.smithi094.stdout:3/323: truncate d9/d20/d30/f4d 435907 0 2024-04-06T03:40:32.265 INFO:tasks.workunit.client.0.smithi094.stdout:8/295: creat d18/d34/d52/f65 x:0 0 0 2024-04-06T03:40:32.266 INFO:tasks.workunit.client.1.smithi136.stdout:3/206: read d0/d4/fd [1473699,2290] 0 2024-04-06T03:40:32.269 INFO:tasks.workunit.client.1.smithi136.stdout:5/305: dwrite d1/d16/d4b/f57 [0,4194304] 0 2024-04-06T03:40:32.269 INFO:tasks.workunit.client.1.smithi136.stdout:5/306: getdents d1/d1f/d52 0 2024-04-06T03:40:32.270 INFO:tasks.workunit.client.1.smithi136.stdout:9/257: dread d17/d19/f1f [0,4194304] 0 2024-04-06T03:40:32.270 INFO:tasks.workunit.client.1.smithi136.stdout:9/258: chown d17/d21/d32/d3f 400614 1 2024-04-06T03:40:32.274 INFO:tasks.workunit.client.0.smithi094.stdout:4/365: truncate d0/df/d13/f5a 205732 0 2024-04-06T03:40:32.278 INFO:tasks.workunit.client.0.smithi094.stdout:5/210: mknod db/d19/d24/c4e 0 2024-04-06T03:40:32.278 INFO:tasks.workunit.client.0.smithi094.stdout:5/211: read - db/f2a zero size 2024-04-06T03:40:32.279 INFO:tasks.workunit.client.0.smithi094.stdout:5/212: write db/f1e [4619982,127411] 0 2024-04-06T03:40:32.280 INFO:tasks.workunit.client.0.smithi094.stdout:7/283: truncate d2/d6/dc/d12/f2a 122187 0 2024-04-06T03:40:32.284 INFO:tasks.workunit.client.0.smithi094.stdout:1/270: mknod d0/d4/d1b/d2f/d4b/c5e 0 2024-04-06T03:40:32.284 INFO:tasks.workunit.client.0.smithi094.stdout:1/271: write d0/d1/d16/d24/d30/f32 [1230344,126972] 0 2024-04-06T03:40:32.284 INFO:tasks.workunit.client.0.smithi094.stdout:1/272: write d0/d4/d1b/d2f/d3c/d48/f4f [206671,38752] 0 2024-04-06T03:40:32.293 INFO:tasks.workunit.client.1.smithi136.stdout:6/215: dwrite d2/f21 [0,4194304] 0 2024-04-06T03:40:32.295 INFO:tasks.workunit.client.0.smithi094.stdout:3/324: dwrite d9/d10/f70 [0,4194304] 0 2024-04-06T03:40:32.295 INFO:tasks.workunit.client.0.smithi094.stdout:3/325: dread - d9/d20/f50 zero size 2024-04-06T03:40:32.295 INFO:tasks.workunit.client.0.smithi094.stdout:3/326: creat d9/d1e/d26/d55/f74 x:0 0 0 2024-04-06T03:40:32.295 INFO:tasks.workunit.client.0.smithi094.stdout:3/327: dread - d9/d20/d30/f69 zero size 2024-04-06T03:40:32.297 INFO:tasks.workunit.client.0.smithi094.stdout:7/284: write d2/d6/dc/d12/f2a [495306,47171] 0 2024-04-06T03:40:32.300 INFO:tasks.workunit.client.0.smithi094.stdout:2/332: dwrite d0/d8/f11 [12582912,4194304] 0 2024-04-06T03:40:32.303 INFO:tasks.workunit.client.1.smithi136.stdout:1/265: mkdir d12/d1b/d1e/d1f/d5a 0 2024-04-06T03:40:32.304 INFO:tasks.workunit.client.1.smithi136.stdout:0/239: mkdir d1/d4e 0 2024-04-06T03:40:32.305 INFO:tasks.workunit.client.1.smithi136.stdout:7/257: creat d5/df/d11/d22/d2a/f5d x:0 0 0 2024-04-06T03:40:32.307 INFO:tasks.workunit.client.1.smithi136.stdout:2/371: mknod d3/d9/c7f 0 2024-04-06T03:40:32.309 INFO:tasks.workunit.client.1.smithi136.stdout:3/207: mkdir d0/d4/d12/d3a/d43 0 2024-04-06T03:40:32.309 INFO:tasks.workunit.client.1.smithi136.stdout:3/208: write d0/d4/f18 [397633,49742] 0 2024-04-06T03:40:32.311 INFO:tasks.workunit.client.0.smithi094.stdout:9/246: rmdir d2/dc 39 2024-04-06T03:40:32.312 INFO:tasks.workunit.client.0.smithi094.stdout:8/296: dwrite d18/d2c/f2e [0,4194304] 0 2024-04-06T03:40:32.312 INFO:tasks.workunit.client.1.smithi136.stdout:9/259: dwrite d17/d19/f2f [0,4194304] 0 2024-04-06T03:40:32.312 INFO:tasks.workunit.client.1.smithi136.stdout:9/260: fsync d17/d19/f1e 0 2024-04-06T03:40:32.313 INFO:tasks.workunit.client.0.smithi094.stdout:4/366: creat d0/d4/d67/f79 x:0 0 0 2024-04-06T03:40:32.313 INFO:tasks.workunit.client.0.smithi094.stdout:4/367: fdatasync d0/df/f1b 0 2024-04-06T03:40:32.313 INFO:tasks.workunit.client.0.smithi094.stdout:4/368: write d0/f44 [482409,120814] 0 2024-04-06T03:40:32.313 INFO:tasks.workunit.client.0.smithi094.stdout:1/273: dwrite d0/d4/d1b/d2f/f54 [0,4194304] 0 2024-04-06T03:40:32.314 INFO:tasks.workunit.client.1.smithi136.stdout:6/216: mknod d2/de/d3d/d47/c4d 0 2024-04-06T03:40:32.317 INFO:tasks.workunit.client.1.smithi136.stdout:7/258: truncate d5/df/d3d/f3f 658775 0 2024-04-06T03:40:32.317 INFO:tasks.workunit.client.1.smithi136.stdout:7/259: dread - d5/df/d11/f2b zero size 2024-04-06T03:40:32.317 INFO:tasks.workunit.client.1.smithi136.stdout:1/266: dwrite d12/f39 [0,4194304] 0 2024-04-06T03:40:32.317 INFO:tasks.workunit.client.1.smithi136.stdout:1/267: chown d12/d1b/l49 5 1 2024-04-06T03:40:32.319 INFO:tasks.workunit.client.0.smithi094.stdout:8/297: dread d18/f27 [0,4194304] 0 2024-04-06T03:40:32.319 INFO:tasks.workunit.client.0.smithi094.stdout:8/298: fsync fd 0 2024-04-06T03:40:32.320 INFO:tasks.workunit.client.1.smithi136.stdout:2/372: link d3/d9/d10/d18/c75 d3/d15/d19/d4e/c80 0 2024-04-06T03:40:32.320 INFO:tasks.workunit.client.1.smithi136.stdout:0/240: write d1/d6/d2c/d1a/d1c/f21 [1559280,43083] 0 2024-04-06T03:40:32.325 INFO:tasks.workunit.client.1.smithi136.stdout:9/261: link d17/d20/c24 d17/d20/d25/d3d/c53 0 2024-04-06T03:40:32.327 INFO:tasks.workunit.client.1.smithi136.stdout:9/262: write d17/d21/d32/d3f/f41 [214221,46045] 0 2024-04-06T03:40:32.327 INFO:tasks.workunit.client.1.smithi136.stdout:3/209: write d0/d4/d12/d19/d2c/f40 [5706281,105318] 0 2024-04-06T03:40:32.328 INFO:tasks.workunit.client.1.smithi136.stdout:7/260: unlink d5/df/d53/f24 0 2024-04-06T03:40:32.330 INFO:tasks.workunit.client.1.smithi136.stdout:2/373: truncate d3/d9/d10/d18/d1d/f20 1946744 0 2024-04-06T03:40:32.330 INFO:tasks.workunit.client.1.smithi136.stdout:2/374: creat d3/d9/d10/d18/f81 x:0 0 0 2024-04-06T03:40:32.333 INFO:tasks.workunit.client.1.smithi136.stdout:1/268: write d12/d1b/d1e/d1f/f30 [2083020,65889] 0 2024-04-06T03:40:32.333 INFO:tasks.workunit.client.1.smithi136.stdout:9/263: creat d17/d20/d28/d49/f54 x:0 0 0 2024-04-06T03:40:32.335 INFO:tasks.workunit.client.1.smithi136.stdout:3/210: creat d0/d4/d12/d3a/d43/f44 x:0 0 0 2024-04-06T03:40:32.335 INFO:tasks.workunit.client.1.smithi136.stdout:3/211: dread - d0/f32 zero size 2024-04-06T03:40:32.336 INFO:tasks.workunit.client.1.smithi136.stdout:3/212: chown d0/d4/d12/d2d 0 1 2024-04-06T03:40:32.336 INFO:tasks.workunit.client.0.smithi094.stdout:3/328: mkdir d9/d1e/d6e/d75 0 2024-04-06T03:40:32.338 INFO:tasks.workunit.client.1.smithi136.stdout:7/261: unlink d5/df/d12/d1b/c42 0 2024-04-06T03:40:32.338 INFO:tasks.workunit.client.1.smithi136.stdout:7/262: write d5/df/d11/d22/f56 [66798,71555] 0 2024-04-06T03:40:32.339 INFO:tasks.workunit.client.1.smithi136.stdout:7/263: readlink d5/l1e 0 2024-04-06T03:40:32.342 INFO:tasks.workunit.client.0.smithi094.stdout:7/285: rename d2/d6/dc/dd/df/l38 to d2/d6/dc/d12/d3a/l4f 0 2024-04-06T03:40:32.342 INFO:tasks.workunit.client.0.smithi094.stdout:7/286: chown d2/d6/f9 45 1 2024-04-06T03:40:32.343 INFO:tasks.workunit.client.1.smithi136.stdout:0/241: dwrite d1/d6/d2c/d1a/d1c/f23 [0,4194304] 0 2024-04-06T03:40:32.344 INFO:tasks.workunit.client.0.smithi094.stdout:4/369: dwrite d0/df/f19 [0,4194304] 0 2024-04-06T03:40:32.344 INFO:tasks.workunit.client.1.smithi136.stdout:6/217: dwrite d2/d7/f2c [0,4194304] 0 2024-04-06T03:40:32.344 INFO:tasks.workunit.client.1.smithi136.stdout:6/218: fdatasync d2/f21 0 2024-04-06T03:40:32.345 INFO:tasks.workunit.client.1.smithi136.stdout:2/375: creat d3/d45/d5f/f82 x:0 0 0 2024-04-06T03:40:32.345 INFO:tasks.workunit.client.1.smithi136.stdout:2/376: dread - d3/d9/d10/f6a zero size 2024-04-06T03:40:32.345 INFO:tasks.workunit.client.1.smithi136.stdout:2/377: stat d3/d45/d5f/f78 0 2024-04-06T03:40:32.345 INFO:tasks.workunit.client.1.smithi136.stdout:2/378: readlink d3/d9/d10/l5b 0 2024-04-06T03:40:32.348 INFO:tasks.workunit.client.1.smithi136.stdout:3/213: creat d0/d4/f45 x:0 0 0 2024-04-06T03:40:32.349 INFO:tasks.workunit.client.1.smithi136.stdout:2/379: dread d3/d15/d19/f1b [0,4194304] 0 2024-04-06T03:40:32.349 INFO:tasks.workunit.client.1.smithi136.stdout:2/380: fdatasync d3/d9/d10/d18/f2a 0 2024-04-06T03:40:32.351 INFO:tasks.workunit.client.1.smithi136.stdout:7/264: creat d5/df/d12/d59/f5e x:0 0 0 2024-04-06T03:40:32.356 INFO:tasks.workunit.client.0.smithi094.stdout:1/274: mkdir d0/d4/d37/d5f 0 2024-04-06T03:40:32.359 INFO:tasks.workunit.client.0.smithi094.stdout:5/213: symlink db/d19/d30/l4f 0 2024-04-06T03:40:32.359 INFO:tasks.workunit.client.1.smithi136.stdout:9/264: dwrite fe [0,4194304] 0 2024-04-06T03:40:32.360 INFO:tasks.workunit.client.1.smithi136.stdout:9/265: fsync f16 0 2024-04-06T03:40:32.362 INFO:tasks.workunit.client.1.smithi136.stdout:7/265: write d5/df/d53/f39 [1462024,57752] 0 2024-04-06T03:40:32.362 INFO:tasks.workunit.client.1.smithi136.stdout:1/269: dwrite d12/d1b/d1e/d27/d37/f46 [0,4194304] 0 2024-04-06T03:40:32.363 INFO:tasks.workunit.client.0.smithi094.stdout:8/299: mkdir d18/d3c/d54/d66 0 2024-04-06T03:40:32.364 INFO:tasks.workunit.client.0.smithi094.stdout:9/247: dread d2/dc/f18 [0,4194304] 0 2024-04-06T03:40:32.364 INFO:tasks.workunit.client.0.smithi094.stdout:9/248: fsync d2/fe 0 2024-04-06T03:40:32.364 INFO:tasks.workunit.client.0.smithi094.stdout:9/249: chown d2/dc/c24 6757 1 2024-04-06T03:40:32.365 INFO:tasks.workunit.client.1.smithi136.stdout:0/242: dwrite d1/d6/d2c/f20 [0,4194304] 0 2024-04-06T03:40:32.365 INFO:tasks.workunit.client.0.smithi094.stdout:4/370: dwrite d0/df/d13/d4a/f64 [0,4194304] 0 2024-04-06T03:40:32.368 INFO:tasks.workunit.client.0.smithi094.stdout:3/329: rmdir d9/d10/d27/d3a/d64 0 2024-04-06T03:40:32.371 INFO:tasks.workunit.client.0.smithi094.stdout:7/287: link d2/d6/l18 d2/d6/dc/d12/l50 0 2024-04-06T03:40:32.372 INFO:tasks.workunit.client.0.smithi094.stdout:4/371: write d0/d4/f21 [775587,44210] 0 2024-04-06T03:40:32.372 INFO:tasks.workunit.client.0.smithi094.stdout:4/372: truncate d0/d4/f21 2907109 0 2024-04-06T03:40:32.376 INFO:tasks.workunit.client.0.smithi094.stdout:2/333: rename d0/d8/l1f to d0/d8/d4f/l6b 0 2024-04-06T03:40:32.376 INFO:tasks.workunit.client.0.smithi094.stdout:2/334: dread - d0/d8/d1d/f60 zero size 2024-04-06T03:40:32.383 INFO:tasks.workunit.client.1.smithi136.stdout:0/243: dwrite d1/d6/d2c/d1a/d36/f49 [0,4194304] 0 2024-04-06T03:40:32.383 INFO:tasks.workunit.client.0.smithi094.stdout:7/288: write d2/d6/dc/dd/df/f1a [166929,105089] 0 2024-04-06T03:40:32.383 INFO:tasks.workunit.client.0.smithi094.stdout:8/300: creat d18/f67 x:0 0 0 2024-04-06T03:40:32.383 INFO:tasks.workunit.client.0.smithi094.stdout:7/289: dread d2/d6/dc/dd/f31 [0,4194304] 0 2024-04-06T03:40:32.383 INFO:tasks.workunit.client.0.smithi094.stdout:2/335: dread d0/dd/d3d/d49/f53 [0,4194304] 0 2024-04-06T03:40:32.387 INFO:tasks.workunit.client.1.smithi136.stdout:2/381: dread d3/d9/d10/d18/d1d/f20 [0,4194304] 0 2024-04-06T03:40:32.387 INFO:tasks.workunit.client.1.smithi136.stdout:2/382: creat d3/f83 x:0 0 0 2024-04-06T03:40:32.388 INFO:tasks.workunit.client.0.smithi094.stdout:9/250: rmdir d2/dc/d21/d26 39 2024-04-06T03:40:32.390 INFO:tasks.workunit.client.1.smithi136.stdout:6/219: symlink d2/d7/d26/l4e 0 2024-04-06T03:40:32.390 INFO:tasks.workunit.client.1.smithi136.stdout:6/220: truncate d2/f5 6166323 0 2024-04-06T03:40:32.390 INFO:tasks.workunit.client.1.smithi136.stdout:6/221: fsync d2/de/f10 0 2024-04-06T03:40:32.393 INFO:tasks.workunit.client.1.smithi136.stdout:3/214: mkdir d0/d4/d12/d35/d3c/d46 0 2024-04-06T03:40:32.393 INFO:tasks.workunit.client.1.smithi136.stdout:3/215: stat d0/d4/d12/d2d/l2f 0 2024-04-06T03:40:32.393 INFO:tasks.workunit.client.1.smithi136.stdout:3/216: write d0/d4/f45 [778991,121738] 0 2024-04-06T03:40:32.393 INFO:tasks.workunit.client.1.smithi136.stdout:3/217: fsync d0/d4/d12/f15 0 2024-04-06T03:40:32.393 INFO:tasks.workunit.client.1.smithi136.stdout:3/218: write d0/d4/d12/d35/f24 [672295,65586] 0 2024-04-06T03:40:32.395 INFO:tasks.workunit.client.1.smithi136.stdout:9/266: unlink d17/f42 0 2024-04-06T03:40:32.395 INFO:tasks.workunit.client.0.smithi094.stdout:3/330: getdents d9/d1e/d26/d54 0 2024-04-06T03:40:32.399 INFO:tasks.workunit.client.1.smithi136.stdout:7/266: mknod d5/df/d11/d22/d4d/c5f 0 2024-04-06T03:40:32.400 INFO:tasks.workunit.client.0.smithi094.stdout:5/214: mknod db/d19/d21/c50 0 2024-04-06T03:40:32.400 INFO:tasks.workunit.client.0.smithi094.stdout:5/215: dread - db/d19/d24/d2e/f36 zero size 2024-04-06T03:40:32.405 INFO:tasks.workunit.client.0.smithi094.stdout:2/336: creat d0/dd/d19/d31/d67/f6c x:0 0 0 2024-04-06T03:40:32.408 INFO:tasks.workunit.client.0.smithi094.stdout:9/251: creat d2/dc/d21/d23/d2f/f57 x:0 0 0 2024-04-06T03:40:32.408 INFO:tasks.workunit.client.0.smithi094.stdout:9/252: chown d2/dc/d10 3990185 1 2024-04-06T03:40:32.414 INFO:tasks.workunit.client.0.smithi094.stdout:4/373: mknod d0/d4/c7a 0 2024-04-06T03:40:32.414 INFO:tasks.workunit.client.0.smithi094.stdout:8/301: dwrite d18/d30/f4b [0,4194304] 0 2024-04-06T03:40:32.415 INFO:tasks.workunit.client.1.smithi136.stdout:1/270: mknod d12/d1b/d1e/d27/d37/c5b 0 2024-04-06T03:40:32.416 INFO:tasks.workunit.client.0.smithi094.stdout:7/290: dwrite d2/f1f [4194304,4194304] 0 2024-04-06T03:40:32.416 INFO:tasks.workunit.client.0.smithi094.stdout:7/291: read d2/d6/dc/dd/f28 [369508,58722] 0 2024-04-06T03:40:32.416 INFO:tasks.workunit.client.0.smithi094.stdout:7/292: fdatasync d2/d6/dc/dd/f3b 0 2024-04-06T03:40:32.416 INFO:tasks.workunit.client.0.smithi094.stdout:7/293: dread - d2/d6/dc/dd/f4e zero size 2024-04-06T03:40:32.416 INFO:tasks.workunit.client.0.smithi094.stdout:7/294: read d2/d6/dc/dd/f28 [748723,85327] 0 2024-04-06T03:40:32.416 INFO:tasks.workunit.client.0.smithi094.stdout:7/295: chown d2/d6/dc/dd/df/c1d 29 1 2024-04-06T03:40:32.416 INFO:tasks.workunit.client.0.smithi094.stdout:9/253: link d2/dc/d21/c3d d2/d2c/c58 0 2024-04-06T03:40:32.417 INFO:tasks.workunit.client.1.smithi136.stdout:7/267: write d5/df/f10 [3375267,86088] 0 2024-04-06T03:40:32.417 INFO:tasks.workunit.client.1.smithi136.stdout:6/222: dwrite d2/d25/f38 [0,4194304] 0 2024-04-06T03:40:32.417 INFO:tasks.workunit.client.1.smithi136.stdout:7/268: write d5/df/d3d/f44 [247953,39758] 0 2024-04-06T03:40:32.417 INFO:tasks.workunit.client.1.smithi136.stdout:7/269: truncate d5/f2e 1165774 0 2024-04-06T03:40:32.417 INFO:tasks.workunit.client.0.smithi094.stdout:4/374: fdatasync d0/d4/f21 0 2024-04-06T03:40:32.417 INFO:tasks.workunit.client.0.smithi094.stdout:4/375: truncate d0/d15/d3a/d3e/d50/f5e 130035 0 2024-04-06T03:40:32.420 INFO:tasks.workunit.client.1.smithi136.stdout:0/244: mknod d1/d6/d2c/c4f 0 2024-04-06T03:40:32.421 INFO:tasks.workunit.client.1.smithi136.stdout:8/296: sync 2024-04-06T03:40:32.422 INFO:tasks.workunit.client.0.smithi094.stdout:8/302: chown d18/f19 104252359 1 2024-04-06T03:40:32.423 INFO:tasks.workunit.client.1.smithi136.stdout:2/383: truncate d3/d9/d10/d18/d4b/f65 1697545 0 2024-04-06T03:40:32.423 INFO:tasks.workunit.client.1.smithi136.stdout:2/384: fsync d3/d9/d10/d18/d1d/d3c/f40 0 2024-04-06T03:40:32.423 INFO:tasks.workunit.client.1.smithi136.stdout:2/385: write d3/d15/f36 [2481554,41456] 0 2024-04-06T03:40:32.423 INFO:tasks.workunit.client.1.smithi136.stdout:2/386: chown d3/d9/d10/d18/l1c 11 1 2024-04-06T03:40:32.424 INFO:tasks.workunit.client.0.smithi094.stdout:2/337: dwrite d0/d18/f24 [0,4194304] 0 2024-04-06T03:40:32.425 INFO:tasks.workunit.client.0.smithi094.stdout:7/296: mkdir d2/d6/dc/d12/d51 0 2024-04-06T03:40:32.427 INFO:tasks.workunit.client.1.smithi136.stdout:3/219: mknod d0/d4/d12/d2d/c47 0 2024-04-06T03:40:32.428 INFO:tasks.workunit.client.0.smithi094.stdout:9/254: mknod d2/dc/c59 0 2024-04-06T03:40:32.429 INFO:tasks.workunit.client.1.smithi136.stdout:9/267: mkdir d17/d20/d25/d3c/d55 0 2024-04-06T03:40:32.431 INFO:tasks.workunit.client.0.smithi094.stdout:4/376: creat d0/d15/d3a/d3e/f7b x:0 0 0 2024-04-06T03:40:32.431 INFO:tasks.workunit.client.0.smithi094.stdout:4/377: write d0/d15/d3a/d3e/d49/f74 [268418,75500] 0 2024-04-06T03:40:32.431 INFO:tasks.workunit.client.1.smithi136.stdout:4/221: sync 2024-04-06T03:40:32.432 INFO:tasks.workunit.client.1.smithi136.stdout:5/307: sync 2024-04-06T03:40:32.432 INFO:tasks.workunit.client.1.smithi136.stdout:4/222: chown d1d/f2b 40607 1 2024-04-06T03:40:32.432 INFO:tasks.workunit.client.1.smithi136.stdout:4/223: write d1d/d31/f44 [204691,70667] 0 2024-04-06T03:40:32.432 INFO:tasks.workunit.client.1.smithi136.stdout:1/271: unlink cf 0 2024-04-06T03:40:32.435 INFO:tasks.workunit.client.0.smithi094.stdout:3/331: rename d9/d48 to d9/d20/d76 0 2024-04-06T03:40:32.436 INFO:tasks.workunit.client.0.smithi094.stdout:5/216: rename db/d19/d24 to db/d19/d24/d51 22 2024-04-06T03:40:32.446 INFO:tasks.workunit.client.1.smithi136.stdout:6/223: rename d2/d7/d40 to d2/d7/d26/d30/d4f 0 2024-04-06T03:40:32.447 INFO:tasks.workunit.client.1.smithi136.stdout:1/272: write d12/d1b/d1e/d23/f3d [444639,4572] 0 2024-04-06T03:40:32.447 INFO:tasks.workunit.client.1.smithi136.stdout:8/297: dwrite d3/de/f4c [0,4194304] 0 2024-04-06T03:40:32.447 INFO:tasks.workunit.client.0.smithi094.stdout:8/303: dwrite d18/f63 [0,4194304] 0 2024-04-06T03:40:32.448 INFO:tasks.workunit.client.0.smithi094.stdout:8/304: chown d18/d34/d52/f65 192025908 1 2024-04-06T03:40:32.448 INFO:tasks.workunit.client.0.smithi094.stdout:8/305: write d18/d30/f40 [671663,76965] 0 2024-04-06T03:40:32.448 INFO:tasks.workunit.client.0.smithi094.stdout:8/306: fsync f17 0 2024-04-06T03:40:32.449 INFO:tasks.workunit.client.0.smithi094.stdout:2/338: unlink d0/dd/d19/d31/l61 0 2024-04-06T03:40:32.451 INFO:tasks.workunit.client.1.smithi136.stdout:7/270: rmdir d5/df/d12/d59/d5b 0 2024-04-06T03:40:32.453 INFO:tasks.workunit.client.0.smithi094.stdout:9/255: mkdir d2/d2c/d33/d5a 0 2024-04-06T03:40:32.453 INFO:tasks.workunit.client.0.smithi094.stdout:9/256: write d2/fe [1788257,121409] 0 2024-04-06T03:40:32.454 INFO:tasks.workunit.client.0.smithi094.stdout:9/257: stat d2/dc/d21/d23/d47 0 2024-04-06T03:40:32.454 INFO:tasks.workunit.client.0.smithi094.stdout:9/258: chown d2/dc/d21/d23/d2f/l38 250 1 2024-04-06T03:40:32.454 INFO:tasks.workunit.client.0.smithi094.stdout:9/259: chown d2/dc/d21/d23/l4b 85327 1 2024-04-06T03:40:32.454 INFO:tasks.workunit.client.0.smithi094.stdout:9/260: fdatasync d2/d2c/d33/d37/f3a 0 2024-04-06T03:40:32.454 INFO:tasks.workunit.client.1.smithi136.stdout:1/273: write d12/d1b/d22/f43 [330010,2524] 0 2024-04-06T03:40:32.454 INFO:tasks.workunit.client.1.smithi136.stdout:0/245: creat d1/d2e/d43/f50 x:0 0 0 2024-04-06T03:40:32.455 INFO:tasks.workunit.client.0.smithi094.stdout:4/378: creat d0/d4/f7c x:0 0 0 2024-04-06T03:40:32.455 INFO:tasks.workunit.client.0.smithi094.stdout:4/379: readlink d0/d15/d3a/l4f 0 2024-04-06T03:40:32.455 INFO:tasks.workunit.client.1.smithi136.stdout:3/220: dwrite d0/d4/d12/f20 [4194304,4194304] 0 2024-04-06T03:40:32.457 INFO:tasks.workunit.client.1.smithi136.stdout:2/387: creat d3/d15/d19/d43/d54/f84 x:0 0 0 2024-04-06T03:40:32.458 INFO:tasks.workunit.client.0.smithi094.stdout:3/332: dwrite d9/d1e/d26/d55/f65 [0,4194304] 0 2024-04-06T03:40:32.459 INFO:tasks.workunit.client.0.smithi094.stdout:7/297: rename d2/d6/dc/c19 to d2/d6/dc/d12/d29/c52 0 2024-04-06T03:40:32.460 INFO:tasks.workunit.client.1.smithi136.stdout:9/268: mkdir d17/d21/d32/d43/d56 0 2024-04-06T03:40:32.461 INFO:tasks.workunit.client.0.smithi094.stdout:5/217: getdents db/d19/d24/d28 0 2024-04-06T03:40:32.465 INFO:tasks.workunit.client.1.smithi136.stdout:0/246: write d1/d6/d2c/f20 [1672076,16325] 0 2024-04-06T03:40:32.465 INFO:tasks.workunit.client.1.smithi136.stdout:5/308: dwrite d1/dc/d10/d1e/f2f [4194304,4194304] 0 2024-04-06T03:40:32.465 INFO:tasks.workunit.client.1.smithi136.stdout:0/247: write d1/d6/d2c/d1a/d36/f49 [4672032,40730] 0 2024-04-06T03:40:32.465 INFO:tasks.workunit.client.1.smithi136.stdout:5/309: write d1/d16/d4b/f4f [4513048,118580] 0 2024-04-06T03:40:32.465 INFO:tasks.workunit.client.1.smithi136.stdout:5/310: truncate d1/dc/d15/d20/f41 383029 0 2024-04-06T03:40:32.466 INFO:tasks.workunit.client.1.smithi136.stdout:5/311: dread d1/dc/d15/d20/f41 [0,4194304] 0 2024-04-06T03:40:32.466 INFO:tasks.workunit.client.1.smithi136.stdout:5/312: chown d1/dc/f5b 611 1 2024-04-06T03:40:32.466 INFO:tasks.workunit.client.1.smithi136.stdout:5/313: write d1/dc/d10/f11 [5105242,86040] 0 2024-04-06T03:40:32.466 INFO:tasks.workunit.client.0.smithi094.stdout:3/333: write d9/d10/f70 [1495816,113510] 0 2024-04-06T03:40:32.470 INFO:tasks.workunit.client.0.smithi094.stdout:9/261: dwrite d2/dc/d21/d23/d2f/f34 [0,4194304] 0 2024-04-06T03:40:32.471 INFO:tasks.workunit.client.0.smithi094.stdout:8/307: write d18/f4d [889246,16231] 0 2024-04-06T03:40:32.472 INFO:tasks.workunit.client.0.smithi094.stdout:5/218: write db/f12 [1133778,94229] 0 2024-04-06T03:40:32.472 INFO:tasks.workunit.client.0.smithi094.stdout:5/219: creat db/d19/d21/f52 x:0 0 0 2024-04-06T03:40:32.473 INFO:tasks.workunit.client.0.smithi094.stdout:2/339: symlink d0/dd/d1b/l6d 0 2024-04-06T03:40:32.473 INFO:tasks.workunit.client.0.smithi094.stdout:2/340: read - d0/dd/f62 zero size 2024-04-06T03:40:32.473 INFO:tasks.workunit.client.0.smithi094.stdout:2/341: stat d0/dd/d1b/c58 0 2024-04-06T03:40:32.473 INFO:tasks.workunit.client.0.smithi094.stdout:2/342: chown d0/dd/c66 10190483 1 2024-04-06T03:40:32.476 INFO:tasks.workunit.client.1.smithi136.stdout:6/224: rename d2/d7/d13/f48 to d2/d7/d26/d30/f50 0 2024-04-06T03:40:32.477 INFO:tasks.workunit.client.0.smithi094.stdout:4/380: mknod d0/df/d57/c7d 0 2024-04-06T03:40:32.477 INFO:tasks.workunit.client.1.smithi136.stdout:6/225: read d2/d7/f12 [138426,69285] 0 2024-04-06T03:40:32.480 INFO:tasks.workunit.client.0.smithi094.stdout:9/262: write d2/dc/d10/d2a/d2d/f3b [1861650,126604] 0 2024-04-06T03:40:32.483 INFO:tasks.workunit.client.0.smithi094.stdout:8/308: dwrite d18/d2c/f47 [0,4194304] 0 2024-04-06T03:40:32.483 INFO:tasks.workunit.client.0.smithi094.stdout:8/309: readlink d18/d2c/l45 0 2024-04-06T03:40:32.483 INFO:tasks.workunit.client.0.smithi094.stdout:8/310: readlink d18/d34/l5f 0 2024-04-06T03:40:32.483 INFO:tasks.workunit.client.0.smithi094.stdout:8/311: write d18/f67 [221171,128441] 0 2024-04-06T03:40:32.484 INFO:tasks.workunit.client.0.smithi094.stdout:7/298: mkdir d2/d6/dc/d12/d51/d53 0 2024-04-06T03:40:32.484 INFO:tasks.workunit.client.0.smithi094.stdout:7/299: creat d2/d6/dc/d12/d29/d35/f54 x:0 0 0 2024-04-06T03:40:32.488 INFO:tasks.workunit.client.0.smithi094.stdout:3/334: symlink d9/d20/d76/l77 0 2024-04-06T03:40:32.488 INFO:tasks.workunit.client.1.smithi136.stdout:6/226: dread d2/d7/f2c [0,4194304] 0 2024-04-06T03:40:32.488 INFO:tasks.workunit.client.0.smithi094.stdout:8/312: dread d18/f20 [0,4194304] 0 2024-04-06T03:40:32.492 INFO:tasks.workunit.client.1.smithi136.stdout:7/271: symlink d5/df/d12/d1b/d3a/d4a/l60 0 2024-04-06T03:40:32.494 INFO:tasks.workunit.client.1.smithi136.stdout:1/274: symlink d12/d53/l5c 0 2024-04-06T03:40:32.499 INFO:tasks.workunit.client.1.smithi136.stdout:9/269: mkdir d17/d21/d32/d43/d51/d57 0 2024-04-06T03:40:32.505 INFO:tasks.workunit.client.1.smithi136.stdout:0/248: rmdir d1/d6/d2c/d3e 0 2024-04-06T03:40:32.507 INFO:tasks.workunit.client.1.smithi136.stdout:5/314: mknod d1/dc/d15/c5d 0 2024-04-06T03:40:32.509 INFO:tasks.workunit.client.1.smithi136.stdout:8/298: rename d3/da/f19 to d3/da/d26/d35/f57 0 2024-04-06T03:40:32.509 INFO:tasks.workunit.client.1.smithi136.stdout:8/299: readlink d3/l15 0 2024-04-06T03:40:32.518 INFO:tasks.workunit.client.0.smithi094.stdout:5/220: truncate db/d19/d24/f37 2821610 0 2024-04-06T03:40:32.518 INFO:tasks.workunit.client.1.smithi136.stdout:5/315: dread d1/d16/d4b/f57 [0,4194304] 0 2024-04-06T03:40:32.518 INFO:tasks.workunit.client.1.smithi136.stdout:5/316: dread - d1/dc/f5b zero size 2024-04-06T03:40:32.520 INFO:tasks.workunit.client.0.smithi094.stdout:2/343: creat d0/dd/d19/d31/f6e x:0 0 0 2024-04-06T03:40:32.520 INFO:tasks.workunit.client.0.smithi094.stdout:2/344: chown d0/d8/d4f/l6b 60358261 1 2024-04-06T03:40:32.521 INFO:tasks.workunit.client.1.smithi136.stdout:1/275: mknod d12/d1b/c5d 0 2024-04-06T03:40:32.521 INFO:tasks.workunit.client.1.smithi136.stdout:1/276: creat d12/d1b/d28/f5e x:0 0 0 2024-04-06T03:40:32.521 INFO:tasks.workunit.client.1.smithi136.stdout:1/277: truncate d12/d1b/d28/f5e 216342 0 2024-04-06T03:40:32.522 INFO:tasks.workunit.client.1.smithi136.stdout:1/278: write d12/d1b/d1e/d27/d37/f4e [122662,33137] 0 2024-04-06T03:40:32.522 INFO:tasks.workunit.client.0.smithi094.stdout:4/381: chown d0/df/d13/f35 64491 1 2024-04-06T03:40:32.522 INFO:tasks.workunit.client.1.smithi136.stdout:5/317: dread d1/dc/d1c/f34 [0,4194304] 0 2024-04-06T03:40:32.523 INFO:tasks.workunit.client.1.smithi136.stdout:3/221: truncate d0/d4/d12/d35/f24 270754 0 2024-04-06T03:40:32.524 INFO:tasks.workunit.client.1.smithi136.stdout:0/249: dwrite d1/d6/d2c/f27 [0,4194304] 0 2024-04-06T03:40:32.524 INFO:tasks.workunit.client.1.smithi136.stdout:0/250: write d1/f18 [718305,3448] 0 2024-04-06T03:40:32.524 INFO:tasks.workunit.client.1.smithi136.stdout:9/270: dwrite d17/d21/d29/f3a [0,4194304] 0 2024-04-06T03:40:32.524 INFO:tasks.workunit.client.1.smithi136.stdout:9/271: dread - d17/d20/d25/f31 zero size 2024-04-06T03:40:32.524 INFO:tasks.workunit.client.1.smithi136.stdout:9/272: dread - d17/d20/d25/d3d/d4b/f52 zero size 2024-04-06T03:40:32.526 INFO:tasks.workunit.client.1.smithi136.stdout:2/388: truncate d3/d9/fe 1658791 0 2024-04-06T03:40:32.526 INFO:tasks.workunit.client.1.smithi136.stdout:2/389: write d3/f68 [356950,111452] 0 2024-04-06T03:40:32.526 INFO:tasks.workunit.client.1.smithi136.stdout:2/390: write d3/d45/d5f/f78 [199867,66650] 0 2024-04-06T03:40:32.526 INFO:tasks.workunit.client.0.smithi094.stdout:9/263: mkdir d2/dc/d21/d23/d5b 0 2024-04-06T03:40:32.527 INFO:tasks.workunit.client.0.smithi094.stdout:9/264: write d2/d2c/d33/f4c [1815073,24878] 0 2024-04-06T03:40:32.527 INFO:tasks.workunit.client.0.smithi094.stdout:0/302: sync 2024-04-06T03:40:32.529 INFO:tasks.workunit.client.0.smithi094.stdout:6/254: sync 2024-04-06T03:40:32.532 INFO:tasks.workunit.client.1.smithi136.stdout:7/272: rename d5/df/d12/d1b/f29 to d5/df/d11/d22/f61 0 2024-04-06T03:40:32.533 INFO:tasks.workunit.client.0.smithi094.stdout:7/300: rmdir d2/d6 39 2024-04-06T03:40:32.535 INFO:tasks.workunit.client.0.smithi094.stdout:3/335: link d9/d20/d76/d6b/l73 d9/d20/d76/d6b/l78 0 2024-04-06T03:40:32.536 INFO:tasks.workunit.client.0.smithi094.stdout:3/336: fsync d9/d20/f50 0 2024-04-06T03:40:32.536 INFO:tasks.workunit.client.1.smithi136.stdout:5/318: symlink d1/d16/d4b/l5e 0 2024-04-06T03:40:32.537 INFO:tasks.workunit.client.0.smithi094.stdout:8/313: mkdir d18/d30/d68 0 2024-04-06T03:40:32.538 INFO:tasks.workunit.client.1.smithi136.stdout:3/222: mknod d0/d4/d12/d35/d3c/d46/c48 0 2024-04-06T03:40:32.539 INFO:tasks.workunit.client.1.smithi136.stdout:3/223: write d0/f32 [155811,23012] 0 2024-04-06T03:40:32.539 INFO:tasks.workunit.client.1.smithi136.stdout:2/391: dwrite d3/d9/d10/d18/f81 [0,4194304] 0 2024-04-06T03:40:32.543 INFO:tasks.workunit.client.1.smithi136.stdout:0/251: mknod d1/d4e/c51 0 2024-04-06T03:40:32.545 INFO:tasks.workunit.client.0.smithi094.stdout:5/221: unlink c8 0 2024-04-06T03:40:32.545 INFO:tasks.workunit.client.0.smithi094.stdout:5/222: chown db/ce 1 1 2024-04-06T03:40:32.548 INFO:tasks.workunit.client.1.smithi136.stdout:9/273: symlink d17/d20/d28/d49/l58 0 2024-04-06T03:40:32.554 INFO:tasks.workunit.client.1.smithi136.stdout:8/300: rename d3/da/d26/d35/d39/f43 to d3/da/d26/d35/d3d/d48/f58 0 2024-04-06T03:40:32.554 INFO:tasks.workunit.client.1.smithi136.stdout:8/301: fdatasync d3/de/d18/f1e 0 2024-04-06T03:40:32.557 INFO:tasks.workunit.client.0.smithi094.stdout:9/265: rename d2/f5 to d2/d2c/d33/f5c 0 2024-04-06T03:40:32.557 INFO:tasks.workunit.client.0.smithi094.stdout:9/266: fdatasync d2/d2c/f41 0 2024-04-06T03:40:32.558 INFO:tasks.workunit.client.1.smithi136.stdout:7/273: getdents d5/df/d11/d22/d4d 0 2024-04-06T03:40:32.558 INFO:tasks.workunit.client.1.smithi136.stdout:7/274: fsync d5/df/d11/f2b 0 2024-04-06T03:40:32.558 INFO:tasks.workunit.client.1.smithi136.stdout:7/275: truncate d5/df/d53/f3c 4907630 0 2024-04-06T03:40:32.560 INFO:tasks.workunit.client.0.smithi094.stdout:0/303: mknod d1/d9/dd/d18/d5d/c61 0 2024-04-06T03:40:32.562 INFO:tasks.workunit.client.0.smithi094.stdout:6/255: symlink dc/d3a/d4a/l5f 0 2024-04-06T03:40:32.565 INFO:tasks.workunit.client.1.smithi136.stdout:3/224: unlink d0/d4/d12/d19/l36 0 2024-04-06T03:40:32.566 INFO:tasks.workunit.client.0.smithi094.stdout:3/337: symlink d9/l79 0 2024-04-06T03:40:32.566 INFO:tasks.workunit.client.1.smithi136.stdout:7/276: dread d5/df/d11/f14 [0,4194304] 0 2024-04-06T03:40:32.567 INFO:tasks.workunit.client.0.smithi094.stdout:9/267: dread d2/dc/d21/d23/d2f/f34 [0,4194304] 0 2024-04-06T03:40:32.567 INFO:tasks.workunit.client.0.smithi094.stdout:9/268: dread - d2/f30 zero size 2024-04-06T03:40:32.567 INFO:tasks.workunit.client.0.smithi094.stdout:9/269: write d2/fe [56886,43653] 0 2024-04-06T03:40:32.568 INFO:tasks.workunit.client.0.smithi094.stdout:8/314: link d18/d2c/d37/d48/f5b d18/d3c/d54/d66/f69 0 2024-04-06T03:40:32.568 INFO:tasks.workunit.client.0.smithi094.stdout:8/315: read - d18/f64 zero size 2024-04-06T03:40:32.571 INFO:tasks.workunit.client.0.smithi094.stdout:5/223: link db/d19/d24/d2e/f48 db/d19/d24/f53 0 2024-04-06T03:40:32.572 INFO:tasks.workunit.client.0.smithi094.stdout:2/345: getdents d0/dd/d19/d31 0 2024-04-06T03:40:32.573 INFO:tasks.workunit.client.0.smithi094.stdout:2/346: chown d0/d8/d27/l36 242 1 2024-04-06T03:40:32.573 INFO:tasks.workunit.client.1.smithi136.stdout:7/277: dread d5/df/d12/d1b/f30 [0,4194304] 0 2024-04-06T03:40:32.573 INFO:tasks.workunit.client.1.smithi136.stdout:9/274: dwrite d17/d21/d29/f3a [0,4194304] 0 2024-04-06T03:40:32.573 INFO:tasks.workunit.client.1.smithi136.stdout:7/278: creat d5/df/f62 x:0 0 0 2024-04-06T03:40:32.574 INFO:tasks.workunit.client.0.smithi094.stdout:4/382: symlink d0/df/d1e/l7e 0 2024-04-06T03:40:32.578 INFO:tasks.workunit.client.0.smithi094.stdout:7/301: rename d2/d6/dc/d12/c25 to d2/c55 0 2024-04-06T03:40:32.580 INFO:tasks.workunit.client.1.smithi136.stdout:3/225: dwrite d0/d4/d12/f20 [0,4194304] 0 2024-04-06T03:40:32.580 INFO:tasks.workunit.client.1.smithi136.stdout:3/226: fsync d0/d4/d12/f1a 0 2024-04-06T03:40:32.581 INFO:tasks.workunit.client.0.smithi094.stdout:0/304: rmdir d1/d9/dd/d18/d1e/d22/d35 39 2024-04-06T03:40:32.583 INFO:tasks.workunit.client.0.smithi094.stdout:6/256: mknod dc/c60 0 2024-04-06T03:40:32.584 INFO:tasks.workunit.client.0.smithi094.stdout:3/338: mknod d9/d20/d76/c7a 0 2024-04-06T03:40:32.587 INFO:tasks.workunit.client.0.smithi094.stdout:8/316: mknod d18/d2c/d62/c6a 0 2024-04-06T03:40:32.588 INFO:tasks.workunit.client.1.smithi136.stdout:9/275: dwrite f11 [0,4194304] 0 2024-04-06T03:40:32.588 INFO:tasks.workunit.client.1.smithi136.stdout:0/252: rmdir d1/d4e 39 2024-04-06T03:40:32.591 INFO:tasks.workunit.client.0.smithi094.stdout:4/383: creat d0/df/d1e/f7f x:0 0 0 2024-04-06T03:40:32.591 INFO:tasks.workunit.client.0.smithi094.stdout:4/384: chown d0/d15/d3a/d3e/d49 491314 1 2024-04-06T03:40:32.594 INFO:tasks.workunit.client.0.smithi094.stdout:9/270: rename d2/l1a to d2/dc/d21/l5d 0 2024-04-06T03:40:32.596 INFO:tasks.workunit.client.1.smithi136.stdout:8/302: symlink d3/da/d26/l59 0 2024-04-06T03:40:32.597 INFO:tasks.workunit.client.0.smithi094.stdout:7/302: mkdir d2/d6/dc/d12/d3a/d56 0 2024-04-06T03:40:32.598 INFO:tasks.workunit.client.1.smithi136.stdout:8/303: dread d3/de/f4b [0,4194304] 0 2024-04-06T03:40:32.598 INFO:tasks.workunit.client.1.smithi136.stdout:8/304: write d3/da/d26/d35/f3c [754128,130535] 0 2024-04-06T03:40:32.599 INFO:tasks.workunit.client.1.smithi136.stdout:8/305: fsync d3/de/f4b 0 2024-04-06T03:40:32.599 INFO:tasks.workunit.client.0.smithi094.stdout:0/305: mknod d1/d9/c62 0 2024-04-06T03:40:32.600 INFO:tasks.workunit.client.0.smithi094.stdout:0/306: dread d1/d9/dd/d18/d32/d42/f4a [0,4194304] 0 2024-04-06T03:40:32.600 INFO:tasks.workunit.client.0.smithi094.stdout:0/307: fdatasync d1/d9/dd/d18/d1e/d22/f49 0 2024-04-06T03:40:32.603 INFO:tasks.workunit.client.1.smithi136.stdout:0/253: dwrite d1/d2e/f42 [0,4194304] 0 2024-04-06T03:40:32.603 INFO:tasks.workunit.client.0.smithi094.stdout:6/257: mkdir dc/d11/d22/d50/d61 0 2024-04-06T03:40:32.604 INFO:tasks.workunit.client.0.smithi094.stdout:3/339: mknod d9/d1e/d6e/c7b 0 2024-04-06T03:40:32.605 INFO:tasks.workunit.client.1.smithi136.stdout:7/279: truncate d5/fd 7673386 0 2024-04-06T03:40:32.605 INFO:tasks.workunit.client.1.smithi136.stdout:7/280: read - d5/f32 zero size 2024-04-06T03:40:32.606 INFO:tasks.workunit.client.0.smithi094.stdout:2/347: rmdir d0/d18/d3f 39 2024-04-06T03:40:32.606 INFO:tasks.workunit.client.1.smithi136.stdout:3/227: stat d0/l28 0 2024-04-06T03:40:32.609 INFO:tasks.workunit.client.0.smithi094.stdout:4/385: stat d0/df/d13/d4a/l4c 0 2024-04-06T03:40:32.611 INFO:tasks.workunit.client.1.smithi136.stdout:9/276: symlink d17/d20/l59 0 2024-04-06T03:40:32.611 INFO:tasks.workunit.client.1.smithi136.stdout:9/277: write fb [2401928,97005] 0 2024-04-06T03:40:32.612 INFO:tasks.workunit.client.0.smithi094.stdout:5/224: rename db/d19/d24/d2e/l47 to db/d19/d24/l54 0 2024-04-06T03:40:32.615 INFO:tasks.workunit.client.0.smithi094.stdout:7/303: mkdir d2/d6/dc/d12/d29/d35/d57 0 2024-04-06T03:40:32.617 INFO:tasks.workunit.client.1.smithi136.stdout:5/319: rmdir d1/d1f/d52 0 2024-04-06T03:40:32.617 INFO:tasks.workunit.client.1.smithi136.stdout:5/320: fdatasync d1/dc/d1c/f34 0 2024-04-06T03:40:32.618 INFO:tasks.workunit.client.1.smithi136.stdout:5/321: fdatasync d1/f4 0 2024-04-06T03:40:32.620 INFO:tasks.workunit.client.1.smithi136.stdout:8/306: symlink d3/d2d/l5a 0 2024-04-06T03:40:32.623 INFO:tasks.workunit.client.1.smithi136.stdout:8/307: dread d3/da/fd [0,4194304] 0 2024-04-06T03:40:32.624 INFO:tasks.workunit.client.0.smithi094.stdout:2/348: dwrite d0/dd/d1b/f3e [0,4194304] 0 2024-04-06T03:40:32.624 INFO:tasks.workunit.client.0.smithi094.stdout:2/349: write d0/d18/f55 [379508,1300] 0 2024-04-06T03:40:32.624 INFO:tasks.workunit.client.0.smithi094.stdout:2/350: fdatasync d0/d18/f24 0 2024-04-06T03:40:32.625 INFO:tasks.workunit.client.0.smithi094.stdout:0/308: mkdir d1/d9/dd/d63 0 2024-04-06T03:40:32.628 INFO:tasks.workunit.client.0.smithi094.stdout:6/258: symlink dc/l62 0 2024-04-06T03:40:32.630 INFO:tasks.workunit.client.1.smithi136.stdout:6/227: sync 2024-04-06T03:40:32.631 INFO:tasks.workunit.client.1.smithi136.stdout:4/224: sync 2024-04-06T03:40:32.631 INFO:tasks.workunit.client.1.smithi136.stdout:4/225: chown d1d/d2f/f57 0 1 2024-04-06T03:40:32.631 INFO:tasks.workunit.client.1.smithi136.stdout:4/226: creat d1d/d31/f58 x:0 0 0 2024-04-06T03:40:32.632 INFO:tasks.workunit.client.0.smithi094.stdout:7/304: dwrite d2/d6/dc/dd/f4e [0,4194304] 0 2024-04-06T03:40:32.632 INFO:tasks.workunit.client.0.smithi094.stdout:3/340: mkdir d9/d10/d27/d7c 0 2024-04-06T03:40:32.632 INFO:tasks.workunit.client.0.smithi094.stdout:2/351: write d0/f57 [2530324,92050] 0 2024-04-06T03:40:32.634 INFO:tasks.workunit.client.0.smithi094.stdout:1/275: sync 2024-04-06T03:40:32.636 INFO:tasks.workunit.client.0.smithi094.stdout:2/352: dread d0/dd/d3d/d49/f53 [0,4194304] 0 2024-04-06T03:40:32.640 INFO:tasks.workunit.client.1.smithi136.stdout:0/254: symlink d1/d4e/l52 0 2024-04-06T03:40:32.640 INFO:tasks.workunit.client.1.smithi136.stdout:0/255: chown d1/d6/d2c/d1a/d1c/f21 27004373 1 2024-04-06T03:40:32.644 INFO:tasks.workunit.client.1.smithi136.stdout:3/228: mkdir d0/d4/d12/d19/d2e/d3b/d49 0 2024-04-06T03:40:32.645 INFO:tasks.workunit.client.0.smithi094.stdout:7/305: rmdir d2/d6/dc/d12/d51/d53 0 2024-04-06T03:40:32.646 INFO:tasks.workunit.client.1.smithi136.stdout:1/279: rename d12/d1b/d1e/d1f to d12/d3b/d44/d5f 0 2024-04-06T03:40:32.649 INFO:tasks.workunit.client.0.smithi094.stdout:3/341: truncate d9/d1d/f42 3034557 0 2024-04-06T03:40:32.651 INFO:tasks.workunit.client.1.smithi136.stdout:6/228: mknod d2/de/d3d/c51 0 2024-04-06T03:40:32.655 INFO:tasks.workunit.client.0.smithi094.stdout:8/317: rename d18/d34/f38 to d18/f6b 0 2024-04-06T03:40:32.657 INFO:tasks.workunit.client.1.smithi136.stdout:1/280: dread f3 [0,4194304] 0 2024-04-06T03:40:32.657 INFO:tasks.workunit.client.0.smithi094.stdout:3/342: dread d9/f15 [0,4194304] 0 2024-04-06T03:40:32.658 INFO:tasks.workunit.client.1.smithi136.stdout:4/227: dwrite f1a [0,4194304] 0 2024-04-06T03:40:32.659 INFO:tasks.workunit.client.0.smithi094.stdout:2/353: symlink d0/d8/d1d/l6f 0 2024-04-06T03:40:32.659 INFO:tasks.workunit.client.1.smithi136.stdout:2/392: sync 2024-04-06T03:40:32.660 INFO:tasks.workunit.client.1.smithi136.stdout:2/393: creat d3/d9/d10/d18/d4b/f85 x:0 0 0 2024-04-06T03:40:32.660 INFO:tasks.workunit.client.1.smithi136.stdout:7/281: rmdir d5/df/d12/d1b 39 2024-04-06T03:40:32.661 INFO:tasks.workunit.client.0.smithi094.stdout:7/306: mknod d2/d6/dc/d12/d3a/d56/c58 0 2024-04-06T03:40:32.662 INFO:tasks.workunit.client.1.smithi136.stdout:3/229: rename d0/d4/d12/d3a/d43/f44 to d0/d4/d12/d2d/f4a 0 2024-04-06T03:40:32.663 INFO:tasks.workunit.client.1.smithi136.stdout:3/230: creat d0/d4/d12/d3a/f4b x:0 0 0 2024-04-06T03:40:32.663 INFO:tasks.workunit.client.1.smithi136.stdout:3/231: truncate d0/d4/fe 4964609 0 2024-04-06T03:40:32.665 INFO:tasks.workunit.client.0.smithi094.stdout:8/318: symlink d18/d2c/d62/l6c 0 2024-04-06T03:40:32.666 INFO:tasks.workunit.client.1.smithi136.stdout:6/229: mknod d2/de/d3d/d47/c52 0 2024-04-06T03:40:32.666 INFO:tasks.workunit.client.1.smithi136.stdout:6/230: truncate d2/d7/f12 976483 0 2024-04-06T03:40:32.669 INFO:tasks.workunit.client.0.smithi094.stdout:0/309: rename d1/d9/dd/d18/d1e/d22 to d1/d9/dd/d18/d1e/d22/d64 22 2024-04-06T03:40:32.669 INFO:tasks.workunit.client.0.smithi094.stdout:0/310: chown d1/d9/dd/d18/d1a 23320 1 2024-04-06T03:40:32.669 INFO:tasks.workunit.client.0.smithi094.stdout:0/311: chown d1/d9/l3f 3 1 2024-04-06T03:40:32.669 INFO:tasks.workunit.client.0.smithi094.stdout:4/386: rename d0/df/d13/d4a/f64 to d0/d77/f80 0 2024-04-06T03:40:32.670 INFO:tasks.workunit.client.1.smithi136.stdout:0/256: mkdir d1/d53 0 2024-04-06T03:40:32.671 INFO:tasks.workunit.client.1.smithi136.stdout:1/281: mkdir d12/d3b/d60 0 2024-04-06T03:40:32.674 INFO:tasks.workunit.client.0.smithi094.stdout:2/354: mkdir d0/d8/d4f/d70 0 2024-04-06T03:40:32.674 INFO:tasks.workunit.client.0.smithi094.stdout:2/355: fdatasync d0/d18/f24 0 2024-04-06T03:40:32.675 INFO:tasks.workunit.client.1.smithi136.stdout:7/282: dread d5/df/d53/f3c [0,4194304] 0 2024-04-06T03:40:32.675 INFO:tasks.workunit.client.1.smithi136.stdout:7/283: truncate d5/df/d11/d22/d2a/f5d 494695 0 2024-04-06T03:40:32.675 INFO:tasks.workunit.client.1.smithi136.stdout:7/284: truncate d5/df/d3d/f44 1007498 0 2024-04-06T03:40:32.675 INFO:tasks.workunit.client.1.smithi136.stdout:7/285: dread - d5/df/f62 zero size 2024-04-06T03:40:32.675 INFO:tasks.workunit.client.1.smithi136.stdout:7/286: dread - d5/df/f62 zero size 2024-04-06T03:40:32.675 INFO:tasks.workunit.client.1.smithi136.stdout:0/257: dread d1/d6/d2c/d1a/f2a [0,4194304] 0 2024-04-06T03:40:32.675 INFO:tasks.workunit.client.1.smithi136.stdout:0/258: dread - d1/d6/d2c/d1a/d36/f44 zero size 2024-04-06T03:40:32.676 INFO:tasks.workunit.client.1.smithi136.stdout:2/394: dread d3/d9/d10/d18/d4b/f5d [0,4194304] 0 2024-04-06T03:40:32.676 INFO:tasks.workunit.client.1.smithi136.stdout:2/395: chown d3/d9/d10/d18/d1d/d3c/d58/l7c 7717 1 2024-04-06T03:40:32.677 INFO:tasks.workunit.client.0.smithi094.stdout:7/307: unlink d2/d6/dc/dd/df/f42 0 2024-04-06T03:40:32.677 INFO:tasks.workunit.client.0.smithi094.stdout:7/308: creat d2/d6/dc/d12/d29/d35/f59 x:0 0 0 2024-04-06T03:40:32.678 INFO:tasks.workunit.client.1.smithi136.stdout:3/232: symlink d0/d4/d12/d19/d2e/d3b/d49/l4c 0 2024-04-06T03:40:32.678 INFO:tasks.workunit.client.1.smithi136.stdout:3/233: fdatasync d0/d4/d12/f20 0 2024-04-06T03:40:32.679 INFO:tasks.workunit.client.1.smithi136.stdout:1/282: truncate d12/d1b/d1e/d27/d37/f4e 139874 0 2024-04-06T03:40:32.680 INFO:tasks.workunit.client.0.smithi094.stdout:0/312: read d1/d9/fc [499070,65617] 0 2024-04-06T03:40:32.680 INFO:tasks.workunit.client.0.smithi094.stdout:0/313: write d1/f55 [813346,15779] 0 2024-04-06T03:40:32.681 INFO:tasks.workunit.client.0.smithi094.stdout:4/387: mkdir d0/d15/d3a/d61/d81 0 2024-04-06T03:40:32.682 INFO:tasks.workunit.client.1.smithi136.stdout:7/287: link d5/df/d1d/f55 d5/df/d3d/f63 0 2024-04-06T03:40:32.684 INFO:tasks.workunit.client.0.smithi094.stdout:1/276: link d0/d7/d1a/c2b d0/d7/d1a/c60 0 2024-04-06T03:40:32.684 INFO:tasks.workunit.client.0.smithi094.stdout:1/277: truncate d0/d1/d27/f2e 532080 0 2024-04-06T03:40:32.684 INFO:tasks.workunit.client.0.smithi094.stdout:1/278: write d0/d1/f10 [3798161,16718] 0 2024-04-06T03:40:32.684 INFO:tasks.workunit.client.0.smithi094.stdout:1/279: stat d0/d1/d16/f53 0 2024-04-06T03:40:32.686 INFO:tasks.workunit.client.0.smithi094.stdout:3/343: getdents d9/d20 0 2024-04-06T03:40:32.686 INFO:tasks.workunit.client.0.smithi094.stdout:3/344: readlink d9/d20/l24 0 2024-04-06T03:40:32.688 INFO:tasks.workunit.client.1.smithi136.stdout:4/228: dwrite d1d/f25 [0,4194304] 0 2024-04-06T03:40:32.688 INFO:tasks.workunit.client.1.smithi136.stdout:4/229: chown d1d/d31/c34 23655 1 2024-04-06T03:40:32.688 INFO:tasks.workunit.client.1.smithi136.stdout:2/396: creat d3/d4a/d77/d7e/f86 x:0 0 0 2024-04-06T03:40:32.690 INFO:tasks.workunit.client.0.smithi094.stdout:2/356: truncate d0/d18/f55 337268 0 2024-04-06T03:40:32.690 INFO:tasks.workunit.client.0.smithi094.stdout:2/357: fsync d0/f5 0 2024-04-06T03:40:32.690 INFO:tasks.workunit.client.0.smithi094.stdout:5/225: dwrite db/d19/f3d [0,4194304] 0 2024-04-06T03:40:32.691 INFO:tasks.workunit.client.0.smithi094.stdout:8/319: dwrite d18/d34/f58 [0,4194304] 0 2024-04-06T03:40:32.691 INFO:tasks.workunit.client.1.smithi136.stdout:6/231: dwrite d2/de/d3d/f46 [4194304,4194304] 0 2024-04-06T03:40:32.691 INFO:tasks.workunit.client.1.smithi136.stdout:6/232: fdatasync d2/f21 0 2024-04-06T03:40:32.695 INFO:tasks.workunit.client.0.smithi094.stdout:7/309: mkdir d2/d6/dc/dd/d5a 0 2024-04-06T03:40:32.696 INFO:tasks.workunit.client.0.smithi094.stdout:7/310: truncate d2/d6/dc/dd/f2f 1461708 0 2024-04-06T03:40:32.696 INFO:tasks.workunit.client.0.smithi094.stdout:7/311: truncate d2/d6/dc/f4a 440780 0 2024-04-06T03:40:32.697 INFO:tasks.workunit.client.0.smithi094.stdout:0/314: creat d1/d9/dd/d18/d5d/f65 x:0 0 0 2024-04-06T03:40:32.701 INFO:tasks.workunit.client.0.smithi094.stdout:8/320: dwrite d18/d3c/d54/d66/f69 [0,4194304] 0 2024-04-06T03:40:32.702 INFO:tasks.workunit.client.0.smithi094.stdout:8/321: read - d18/f64 zero size 2024-04-06T03:40:32.705 INFO:tasks.workunit.client.1.smithi136.stdout:3/234: mknod d0/d4/d12/c4d 0 2024-04-06T03:40:32.706 INFO:tasks.workunit.client.1.smithi136.stdout:3/235: dread - d0/d4/d12/d2d/f39 zero size 2024-04-06T03:40:32.706 INFO:tasks.workunit.client.0.smithi094.stdout:1/280: creat d0/d7/d2d/f61 x:0 0 0 2024-04-06T03:40:32.706 INFO:tasks.workunit.client.0.smithi094.stdout:1/281: write d0/d4/d1b/d2f/d3c/d48/f4f [510318,59833] 0 2024-04-06T03:40:32.707 INFO:tasks.workunit.client.1.smithi136.stdout:1/283: creat d12/d53/f61 x:0 0 0 2024-04-06T03:40:32.708 INFO:tasks.workunit.client.0.smithi094.stdout:3/345: getdents d9/d20/d76/d6b 0 2024-04-06T03:40:32.708 INFO:tasks.workunit.client.0.smithi094.stdout:3/346: read - d9/d20/d2d/f67 zero size 2024-04-06T03:40:32.710 INFO:tasks.workunit.client.1.smithi136.stdout:7/288: creat d5/df/d12/d1b/d3a/d4a/f64 x:0 0 0 2024-04-06T03:40:32.711 INFO:tasks.workunit.client.0.smithi094.stdout:2/358: truncate d0/dd/d1b/f3e 966555 0 2024-04-06T03:40:32.712 INFO:tasks.workunit.client.1.smithi136.stdout:0/259: rmdir d1/d6/d2c/d1a/d36 39 2024-04-06T03:40:32.717 INFO:tasks.workunit.client.0.smithi094.stdout:7/312: dwrite d2/d6/dc/f2b [0,4194304] 0 2024-04-06T03:40:32.718 INFO:tasks.workunit.client.1.smithi136.stdout:2/397: mkdir d3/d9/d3b/d87 0 2024-04-06T03:40:32.722 INFO:tasks.workunit.client.0.smithi094.stdout:2/359: dread d0/d8/d27/d2b/f2d [0,4194304] 0 2024-04-06T03:40:32.722 INFO:tasks.workunit.client.1.smithi136.stdout:6/233: mkdir d2/d7/d4b/d53 0 2024-04-06T03:40:32.723 INFO:tasks.workunit.client.1.smithi136.stdout:6/234: chown d2/de/f27 11436 1 2024-04-06T03:40:32.723 INFO:tasks.workunit.client.1.smithi136.stdout:6/235: truncate d2/de/f27 1983331 0 2024-04-06T03:40:32.725 INFO:tasks.workunit.client.1.smithi136.stdout:3/236: creat d0/d4/d12/d35/d3c/f4e x:0 0 0 2024-04-06T03:40:32.725 INFO:tasks.workunit.client.1.smithi136.stdout:3/237: write d0/f1 [1656711,71480] 0 2024-04-06T03:40:32.730 INFO:tasks.workunit.client.1.smithi136.stdout:7/289: link d5/l34 d5/df/d12/d59/l65 0 2024-04-06T03:40:32.735 INFO:tasks.workunit.client.1.smithi136.stdout:3/238: symlink d0/l4f 0 2024-04-06T03:40:32.736 INFO:tasks.workunit.client.0.smithi094.stdout:4/388: rmdir d0/d15/d3a/d3e 39 2024-04-06T03:40:32.738 INFO:tasks.workunit.client.1.smithi136.stdout:7/290: mknod d5/c66 0 2024-04-06T03:40:32.739 INFO:tasks.workunit.client.1.smithi136.stdout:3/239: truncate d0/d4/d12/d2d/f33 1112169 0 2024-04-06T03:40:32.741 INFO:tasks.workunit.client.0.smithi094.stdout:8/322: getdents d18/d2c/d62 0 2024-04-06T03:40:32.741 INFO:tasks.workunit.client.0.smithi094.stdout:3/347: dwrite d9/d10/f2b [0,4194304] 0 2024-04-06T03:40:32.741 INFO:tasks.workunit.client.0.smithi094.stdout:3/348: dread - d9/d1e/d26/d55/f74 zero size 2024-04-06T03:40:32.742 INFO:tasks.workunit.client.1.smithi136.stdout:0/260: dwrite d1/d2e/f46 [0,4194304] 0 2024-04-06T03:40:32.742 INFO:tasks.workunit.client.1.smithi136.stdout:0/261: write d1/d6/d2c/d1a/d1c/f48 [175461,70916] 0 2024-04-06T03:40:32.742 INFO:tasks.workunit.client.0.smithi094.stdout:1/282: mknod d0/d4/d37/c62 0 2024-04-06T03:40:32.745 INFO:tasks.workunit.client.0.smithi094.stdout:7/313: link d2/d6/dc/c2c d2/d6/dc/c5b 0 2024-04-06T03:40:32.746 INFO:tasks.workunit.client.1.smithi136.stdout:6/236: dwrite d2/d25/f3b [0,4194304] 0 2024-04-06T03:40:32.747 INFO:tasks.workunit.client.0.smithi094.stdout:5/226: dwrite db/d19/d2c/f2d [4194304,4194304] 0 2024-04-06T03:40:32.748 INFO:tasks.workunit.client.1.smithi136.stdout:2/398: dwrite d3/d15/f30 [0,4194304] 0 2024-04-06T03:40:32.748 INFO:tasks.workunit.client.1.smithi136.stdout:2/399: chown d3/d9/fe 1090022 1 2024-04-06T03:40:32.748 INFO:tasks.workunit.client.0.smithi094.stdout:2/360: mkdir d0/d8/d71 0 2024-04-06T03:40:32.748 INFO:tasks.workunit.client.0.smithi094.stdout:2/361: dread - d0/d8/d1d/f60 zero size 2024-04-06T03:40:32.748 INFO:tasks.workunit.client.0.smithi094.stdout:2/362: chown d0/d8/d27/d2b/f2c 3197824 1 2024-04-06T03:40:32.749 INFO:tasks.workunit.client.1.smithi136.stdout:1/284: dwrite d12/f2c [0,4194304] 0 2024-04-06T03:40:32.749 INFO:tasks.workunit.client.1.smithi136.stdout:1/285: chown d12/d3b/d60 99149 1 2024-04-06T03:40:32.749 INFO:tasks.workunit.client.1.smithi136.stdout:1/286: truncate d12/d1b/d28/d48/f50 340641 0 2024-04-06T03:40:32.749 INFO:tasks.workunit.client.1.smithi136.stdout:1/287: fdatasync d12/d1b/d28/f3f 0 2024-04-06T03:40:32.749 INFO:tasks.workunit.client.1.smithi136.stdout:1/288: write d12/f16 [1360522,113688] 0 2024-04-06T03:40:32.749 INFO:tasks.workunit.client.0.smithi094.stdout:0/315: getdents d1/d9/dd/d18/d1e/d22 0 2024-04-06T03:40:32.750 INFO:tasks.workunit.client.1.smithi136.stdout:7/291: write d5/df/d53/f3c [1073914,15735] 0 2024-04-06T03:40:32.751 INFO:tasks.workunit.client.1.smithi136.stdout:0/262: creat d1/f54 x:0 0 0 2024-04-06T03:40:32.758 INFO:tasks.workunit.client.1.smithi136.stdout:6/237: creat d2/d7/d26/f54 x:0 0 0 2024-04-06T03:40:32.758 INFO:tasks.workunit.client.1.smithi136.stdout:6/238: chown d2/d7/d26/d30/l2b 2 1 2024-04-06T03:40:32.762 INFO:tasks.workunit.client.0.smithi094.stdout:4/389: mkdir d0/df/d13/d82 0 2024-04-06T03:40:32.766 INFO:tasks.workunit.client.1.smithi136.stdout:2/400: rmdir d3/d9/d10/d18/d1d/d37 0 2024-04-06T03:40:32.766 INFO:tasks.workunit.client.1.smithi136.stdout:2/401: read d3/d15/d19/f1b [75839,118355] 0 2024-04-06T03:40:32.766 INFO:tasks.workunit.client.1.smithi136.stdout:2/402: chown d3/d15/d19/d43/d54/f84 160609412 1 2024-04-06T03:40:32.771 INFO:tasks.workunit.client.0.smithi094.stdout:3/349: dread d9/d10/f2b [0,4194304] 0 2024-04-06T03:40:32.774 INFO:tasks.workunit.client.1.smithi136.stdout:1/289: write d12/f16 [1019424,96580] 0 2024-04-06T03:40:32.774 INFO:tasks.workunit.client.1.smithi136.stdout:1/290: dread - d12/d53/f61 zero size 2024-04-06T03:40:32.778 INFO:tasks.workunit.client.0.smithi094.stdout:2/363: read d0/d8/d1d/f38 [3047663,51015] 0 2024-04-06T03:40:32.779 INFO:tasks.workunit.client.0.smithi094.stdout:4/390: dread d0/d4/f21 [0,4194304] 0 2024-04-06T03:40:32.781 INFO:tasks.workunit.client.1.smithi136.stdout:3/240: dwrite d0/d4/d12/d35/f24 [0,4194304] 0 2024-04-06T03:40:32.781 INFO:tasks.workunit.client.1.smithi136.stdout:3/241: chown d0/d4/l14 8282 1 2024-04-06T03:40:32.781 INFO:tasks.workunit.client.1.smithi136.stdout:3/242: creat d0/d4/d12/d19/f50 x:0 0 0 2024-04-06T03:40:32.782 INFO:tasks.workunit.client.0.smithi094.stdout:8/323: dwrite f17 [0,4194304] 0 2024-04-06T03:40:32.782 INFO:tasks.workunit.client.1.smithi136.stdout:6/239: dwrite d2/de/f2f [0,4194304] 0 2024-04-06T03:40:32.782 INFO:tasks.workunit.client.1.smithi136.stdout:6/240: dread - d2/d7/f31 zero size 2024-04-06T03:40:32.783 INFO:tasks.workunit.client.1.smithi136.stdout:0/263: dwrite d1/d6/d2c/d1a/f39 [4194304,4194304] 0 2024-04-06T03:40:32.787 INFO:tasks.workunit.client.0.smithi094.stdout:1/283: dwrite d0/d4/d1b/d2f/f54 [4194304,4194304] 0 2024-04-06T03:40:32.788 INFO:tasks.workunit.client.0.smithi094.stdout:4/391: dread d0/df/f66 [0,4194304] 0 2024-04-06T03:40:32.789 INFO:tasks.workunit.client.0.smithi094.stdout:4/392: chown d0/df/f19 1550474 1 2024-04-06T03:40:32.789 INFO:tasks.workunit.client.0.smithi094.stdout:4/393: creat d0/f83 x:0 0 0 2024-04-06T03:40:32.791 INFO:tasks.workunit.client.0.smithi094.stdout:2/364: dwrite d0/d18/f24 [4194304,4194304] 0 2024-04-06T03:40:32.794 INFO:tasks.workunit.client.1.smithi136.stdout:3/243: dread d0/f1e [4194304,4194304] 0 2024-04-06T03:40:32.794 INFO:tasks.workunit.client.1.smithi136.stdout:3/244: truncate d0/d4/d12/d3a/f3e 507916 0 2024-04-06T03:40:32.805 INFO:tasks.workunit.client.0.smithi094.stdout:1/284: dwrite d0/d4/f9 [4194304,4194304] 0 2024-04-06T03:40:32.843 INFO:tasks.workunit.client.0.smithi094.stdout:5/227: rmdir db 39 2024-04-06T03:40:32.848 INFO:tasks.workunit.client.1.smithi136.stdout:7/292: mknod d5/df/d12/d1b/d58/c67 0 2024-04-06T03:40:32.850 INFO:tasks.workunit.client.1.smithi136.stdout:2/403: creat d3/d9/d3b/d87/f88 x:0 0 0 2024-04-06T03:40:32.853 INFO:tasks.workunit.client.0.smithi094.stdout:8/324: rename d18/f36 to d18/d2c/d62/f6d 0 2024-04-06T03:40:32.854 INFO:tasks.workunit.client.1.smithi136.stdout:1/291: creat d12/d3b/d44/d5f/d5a/f62 x:0 0 0 2024-04-06T03:40:32.854 INFO:tasks.workunit.client.1.smithi136.stdout:9/278: sync 2024-04-06T03:40:32.859 INFO:tasks.workunit.client.0.smithi094.stdout:2/365: getdents d0/dd/d19 0 2024-04-06T03:40:32.862 INFO:tasks.workunit.client.0.smithi094.stdout:8/325: dread d18/d30/f4b [0,4194304] 0 2024-04-06T03:40:32.862 INFO:tasks.workunit.client.0.smithi094.stdout:8/326: write d18/d3c/d54/d66/f69 [4853724,63657] 0 2024-04-06T03:40:32.871 INFO:tasks.workunit.client.0.smithi094.stdout:1/285: symlink d0/d7/d2d/l63 0 2024-04-06T03:40:32.871 INFO:tasks.workunit.client.0.smithi094.stdout:1/286: dread - d0/f5d zero size 2024-04-06T03:40:32.871 INFO:tasks.workunit.client.0.smithi094.stdout:1/287: chown d0/d1/f10 100232 1 2024-04-06T03:40:32.872 INFO:tasks.workunit.client.1.smithi136.stdout:3/245: mkdir d0/d4/d12/d3a/d51 0 2024-04-06T03:40:32.873 INFO:tasks.workunit.client.1.smithi136.stdout:5/322: sync 2024-04-06T03:40:32.873 INFO:tasks.workunit.client.1.smithi136.stdout:8/308: sync 2024-04-06T03:40:32.875 INFO:tasks.workunit.client.0.smithi094.stdout:5/228: fdatasync db/d19/f41 0 2024-04-06T03:40:32.876 INFO:tasks.workunit.client.0.smithi094.stdout:5/229: fdatasync db/d40/f43 0 2024-04-06T03:40:32.877 INFO:tasks.workunit.client.1.smithi136.stdout:7/293: mkdir d5/df/d53/d68 0 2024-04-06T03:40:32.878 INFO:tasks.workunit.client.0.smithi094.stdout:3/350: getdents d9/d1e/d26/d55 0 2024-04-06T03:40:32.878 INFO:tasks.workunit.client.0.smithi094.stdout:3/351: chown d9/d1e/c44 7752 1 2024-04-06T03:40:32.879 INFO:tasks.workunit.client.1.smithi136.stdout:2/404: creat d3/d9/f89 x:0 0 0 2024-04-06T03:40:32.879 INFO:tasks.workunit.client.0.smithi094.stdout:1/288: write d0/d4/d1b/d2f/f54 [7125317,54490] 0 2024-04-06T03:40:32.880 INFO:tasks.workunit.client.0.smithi094.stdout:4/394: link d0/df/d1e/c2f d0/df/d1e/d34/c84 0 2024-04-06T03:40:32.881 INFO:tasks.workunit.client.0.smithi094.stdout:1/289: dread d0/d1/d16/f3b [0,4194304] 0 2024-04-06T03:40:32.882 INFO:tasks.workunit.client.1.smithi136.stdout:1/292: creat d12/d3b/d44/d5f/f63 x:0 0 0 2024-04-06T03:40:32.883 INFO:tasks.workunit.client.0.smithi094.stdout:2/366: mkdir d0/dd/d1b/d72 0 2024-04-06T03:40:32.884 INFO:tasks.workunit.client.1.smithi136.stdout:9/279: symlink d17/d21/d29/d34/d46/l5a 0 2024-04-06T03:40:32.884 INFO:tasks.workunit.client.1.smithi136.stdout:9/280: dread - d17/d20/d25/d3b/f4e zero size 2024-04-06T03:40:32.886 INFO:tasks.workunit.client.1.smithi136.stdout:0/264: link d1/f54 d1/d4e/f55 0 2024-04-06T03:40:32.886 INFO:tasks.workunit.client.1.smithi136.stdout:0/265: write d1/d6/fb [3471955,105540] 0 2024-04-06T03:40:32.886 INFO:tasks.workunit.client.0.smithi094.stdout:8/327: creat d18/d2c/f6e x:0 0 0 2024-04-06T03:40:32.889 INFO:tasks.workunit.client.0.smithi094.stdout:5/230: creat db/d40/f55 x:0 0 0 2024-04-06T03:40:32.889 INFO:tasks.workunit.client.0.smithi094.stdout:5/231: chown db/fc 7057645 1 2024-04-06T03:40:32.890 INFO:tasks.workunit.client.1.smithi136.stdout:8/309: link d3/da/d26/f50 d3/da/d26/d35/d53/f5b 0 2024-04-06T03:40:32.892 INFO:tasks.workunit.client.0.smithi094.stdout:3/352: mkdir d9/d1d/d7d 0 2024-04-06T03:40:32.893 INFO:tasks.workunit.client.1.smithi136.stdout:5/323: link d1/dc/d15/l2d d1/dc/d1c/l5f 0 2024-04-06T03:40:32.899 INFO:tasks.workunit.client.0.smithi094.stdout:2/367: link d0/d8/d27/d2b/f2d d0/d8/d1d/f73 0 2024-04-06T03:40:32.899 INFO:tasks.workunit.client.1.smithi136.stdout:2/405: dwrite d3/d9/d10/d18/f23 [0,4194304] 0 2024-04-06T03:40:32.901 INFO:tasks.workunit.client.1.smithi136.stdout:7/294: mkdir d5/df/d11/d22/d4d/d69 0 2024-04-06T03:40:32.903 INFO:tasks.workunit.client.1.smithi136.stdout:1/293: truncate f10 4967064 0 2024-04-06T03:40:32.904 INFO:tasks.workunit.client.0.smithi094.stdout:4/395: dwrite d0/d4/f76 [0,4194304] 0 2024-04-06T03:40:32.904 INFO:tasks.workunit.client.0.smithi094.stdout:5/232: mknod db/d19/d24/d28/c56 0 2024-04-06T03:40:32.905 INFO:tasks.workunit.client.1.smithi136.stdout:9/281: getdents d17/d20/d25/d3d 0 2024-04-06T03:40:32.906 INFO:tasks.workunit.client.0.smithi094.stdout:1/290: mkdir d0/d7/d1a/d64 0 2024-04-06T03:40:32.907 INFO:tasks.workunit.client.0.smithi094.stdout:4/396: dread d0/d4/f9 [4194304,4194304] 0 2024-04-06T03:40:32.907 INFO:tasks.workunit.client.0.smithi094.stdout:4/397: write d0/d4/fd [3183489,102828] 0 2024-04-06T03:40:32.907 INFO:tasks.workunit.client.0.smithi094.stdout:4/398: fdatasync d0/f22 0 2024-04-06T03:40:32.909 INFO:tasks.workunit.client.0.smithi094.stdout:3/353: dwrite d9/f5f [0,4194304] 0 2024-04-06T03:40:32.910 INFO:tasks.workunit.client.1.smithi136.stdout:0/266: mkdir d1/d6/d2c/d56 0 2024-04-06T03:40:32.910 INFO:tasks.workunit.client.1.smithi136.stdout:0/267: write d1/d2e/f42 [5154488,102988] 0 2024-04-06T03:40:32.912 INFO:tasks.workunit.client.1.smithi136.stdout:3/246: rmdir d0/d4/d12/d19/d2c 39 2024-04-06T03:40:32.915 INFO:tasks.workunit.client.1.smithi136.stdout:4/230: sync 2024-04-06T03:40:32.916 INFO:tasks.workunit.client.0.smithi094.stdout:1/291: write d0/d7/f46 [2455961,50963] 0 2024-04-06T03:40:32.916 INFO:tasks.workunit.client.0.smithi094.stdout:1/292: creat d0/d1/f65 x:0 0 0 2024-04-06T03:40:32.917 INFO:tasks.workunit.client.1.smithi136.stdout:9/282: read f11 [84431,23720] 0 2024-04-06T03:40:32.917 INFO:tasks.workunit.client.0.smithi094.stdout:2/368: link d0/dd/d1b/l6d d0/d8/d4f/d70/l74 0 2024-04-06T03:40:32.918 INFO:tasks.workunit.client.1.smithi136.stdout:8/310: mknod d3/da/d26/d35/d53/c5c 0 2024-04-06T03:40:32.919 INFO:tasks.workunit.client.1.smithi136.stdout:7/295: dwrite f1 [0,4194304] 0 2024-04-06T03:40:32.919 INFO:tasks.workunit.client.0.smithi094.stdout:8/328: rename d18/c1f to d18/d3c/d54/d66/c6f 0 2024-04-06T03:40:32.920 INFO:tasks.workunit.client.1.smithi136.stdout:5/324: chown d1/c4c 195838846 1 2024-04-06T03:40:32.922 INFO:tasks.workunit.client.0.smithi094.stdout:1/293: dread d0/d4/d1b/d2f/d3c/d48/f4f [0,4194304] 0 2024-04-06T03:40:32.922 INFO:tasks.workunit.client.0.smithi094.stdout:1/294: fsync d0/d4/f9 0 2024-04-06T03:40:32.922 INFO:tasks.workunit.client.0.smithi094.stdout:1/295: stat d0/d4/d1b/d2f/d3c/d48/f4f 0 2024-04-06T03:40:32.924 INFO:tasks.workunit.client.0.smithi094.stdout:8/329: dread d18/f25 [0,4194304] 0 2024-04-06T03:40:32.924 INFO:tasks.workunit.client.0.smithi094.stdout:8/330: fsync f13 0 2024-04-06T03:40:32.925 INFO:tasks.workunit.client.0.smithi094.stdout:5/233: rmdir db/d19/d2c 39 2024-04-06T03:40:32.925 INFO:tasks.workunit.client.0.smithi094.stdout:5/234: write db/d40/f55 [867341,82662] 0 2024-04-06T03:40:32.925 INFO:tasks.workunit.client.0.smithi094.stdout:5/235: fdatasync db/d19/d24/f37 0 2024-04-06T03:40:32.925 INFO:tasks.workunit.client.0.smithi094.stdout:5/236: truncate db/d19/d24/f37 3081691 0 2024-04-06T03:40:32.926 INFO:tasks.workunit.client.0.smithi094.stdout:4/399: rmdir d0/d15/d3a/d3e/d49 39 2024-04-06T03:40:32.926 INFO:tasks.workunit.client.0.smithi094.stdout:4/400: fsync d0/df/d1e/f7f 0 2024-04-06T03:40:32.930 INFO:tasks.workunit.client.1.smithi136.stdout:8/311: dread d3/f4 [0,4194304] 0 2024-04-06T03:40:32.930 INFO:tasks.workunit.client.1.smithi136.stdout:8/312: chown d3/da/d26/d35/l46 107019 1 2024-04-06T03:40:32.930 INFO:tasks.workunit.client.1.smithi136.stdout:8/313: chown d3/da/d26/f28 696930224 1 2024-04-06T03:40:32.938 INFO:tasks.workunit.client.1.smithi136.stdout:1/294: rename d12/l1a to d12/d53/l64 0 2024-04-06T03:40:32.938 INFO:tasks.workunit.client.1.smithi136.stdout:1/295: write f10 [642340,11025] 0 2024-04-06T03:40:32.938 INFO:tasks.workunit.client.1.smithi136.stdout:1/296: chown d12/f24 41334 1 2024-04-06T03:40:32.938 INFO:tasks.workunit.client.1.smithi136.stdout:6/241: sync 2024-04-06T03:40:32.938 INFO:tasks.workunit.client.1.smithi136.stdout:1/297: fsync d12/d3b/d44/f57 0 2024-04-06T03:40:32.938 INFO:tasks.workunit.client.1.smithi136.stdout:8/314: dread d3/de/f4b [0,4194304] 0 2024-04-06T03:40:32.938 INFO:tasks.workunit.client.1.smithi136.stdout:3/247: dwrite d0/d4/d12/f20 [0,4194304] 0 2024-04-06T03:40:32.939 INFO:tasks.workunit.client.0.smithi094.stdout:3/354: mkdir d9/d10/d7e 0 2024-04-06T03:40:32.939 INFO:tasks.workunit.client.0.smithi094.stdout:3/355: readlink d9/d20/d76/d6b/l73 0 2024-04-06T03:40:32.939 INFO:tasks.workunit.client.0.smithi094.stdout:3/356: chown d9 1567 1 2024-04-06T03:40:32.939 INFO:tasks.workunit.client.0.smithi094.stdout:8/331: dwrite d18/d30/f3e [0,4194304] 0 2024-04-06T03:40:32.939 INFO:tasks.workunit.client.0.smithi094.stdout:8/332: write f1 [3574674,3865] 0 2024-04-06T03:40:32.940 INFO:tasks.workunit.client.0.smithi094.stdout:5/237: dread db/d19/d21/f26 [0,4194304] 0 2024-04-06T03:40:32.941 INFO:tasks.workunit.client.1.smithi136.stdout:0/268: unlink d1/d6/d2c/d1a/f2a 0 2024-04-06T03:40:32.942 INFO:tasks.workunit.client.1.smithi136.stdout:4/231: mknod d1d/d31/d3e/d55/c59 0 2024-04-06T03:40:32.943 INFO:tasks.workunit.client.1.smithi136.stdout:4/232: write d1d/f22 [375592,29129] 0 2024-04-06T03:40:32.943 INFO:tasks.workunit.client.1.smithi136.stdout:9/283: stat d17/d19/l33 0 2024-04-06T03:40:32.943 INFO:tasks.workunit.client.1.smithi136.stdout:9/284: fsync d17/d21/d32/d3f/f41 0 2024-04-06T03:40:32.943 INFO:tasks.workunit.client.1.smithi136.stdout:9/285: write f16 [721681,16100] 0 2024-04-06T03:40:32.943 INFO:tasks.workunit.client.1.smithi136.stdout:9/286: creat d17/d21/d32/d3f/f5b x:0 0 0 2024-04-06T03:40:32.943 INFO:tasks.workunit.client.1.smithi136.stdout:9/287: read fb [801451,50688] 0 2024-04-06T03:40:32.945 INFO:tasks.workunit.client.1.smithi136.stdout:7/296: symlink d5/df/d12/l6a 0 2024-04-06T03:40:32.945 INFO:tasks.workunit.client.1.smithi136.stdout:7/297: write d5/df/d12/d1b/f30 [1739109,96305] 0 2024-04-06T03:40:32.946 INFO:tasks.workunit.client.0.smithi094.stdout:2/369: mknod d0/d8/d4f/c75 0 2024-04-06T03:40:32.946 INFO:tasks.workunit.client.0.smithi094.stdout:2/370: truncate d0/dd/d19/f4c 880244 0 2024-04-06T03:40:32.947 INFO:tasks.workunit.client.1.smithi136.stdout:5/325: symlink d1/dc/d1c/d32/l60 0 2024-04-06T03:40:32.947 INFO:tasks.workunit.client.0.smithi094.stdout:1/296: mknod d0/d1/c66 0 2024-04-06T03:40:32.948 INFO:tasks.workunit.client.1.smithi136.stdout:2/406: rmdir d3/d45/d5f 39 2024-04-06T03:40:32.948 INFO:tasks.workunit.client.1.smithi136.stdout:1/298: write d12/f38 [2219817,77459] 0 2024-04-06T03:40:32.949 INFO:tasks.workunit.client.1.smithi136.stdout:3/248: read d0/d4/d12/d35/f24 [1364433,26336] 0 2024-04-06T03:40:32.949 INFO:tasks.workunit.client.0.smithi094.stdout:4/401: creat d0/d15/d3a/d3e/d46/f85 x:0 0 0 2024-04-06T03:40:32.955 INFO:tasks.workunit.client.0.smithi094.stdout:3/357: mknod d9/d1e/d26/d55/d62/c7f 0 2024-04-06T03:40:32.955 INFO:tasks.workunit.client.0.smithi094.stdout:4/402: dread d0/d4/f6 [0,4194304] 0 2024-04-06T03:40:32.955 INFO:tasks.workunit.client.1.smithi136.stdout:6/242: symlink d2/de/d3d/l55 0 2024-04-06T03:40:32.955 INFO:tasks.workunit.client.1.smithi136.stdout:8/315: dwrite d3/f4d [0,4194304] 0 2024-04-06T03:40:32.956 INFO:tasks.workunit.client.1.smithi136.stdout:4/233: creat d1d/d31/d3e/f5a x:0 0 0 2024-04-06T03:40:32.956 INFO:tasks.workunit.client.1.smithi136.stdout:4/234: chown d1d/d31/f48 104845658 1 2024-04-06T03:40:32.956 INFO:tasks.workunit.client.1.smithi136.stdout:4/235: fdatasync f7 0 2024-04-06T03:40:32.956 INFO:tasks.workunit.client.1.smithi136.stdout:4/236: chown d1d/d31/f3a 296085606 1 2024-04-06T03:40:32.956 INFO:tasks.workunit.client.1.smithi136.stdout:0/269: write d1/d2e/f46 [1650418,33121] 0 2024-04-06T03:40:32.956 INFO:tasks.workunit.client.1.smithi136.stdout:9/288: symlink d17/d21/d32/d3f/l5c 0 2024-04-06T03:40:32.957 INFO:tasks.workunit.client.1.smithi136.stdout:7/298: mkdir d5/df/d11/d22/d6b 0 2024-04-06T03:40:32.957 INFO:tasks.workunit.client.1.smithi136.stdout:7/299: read - d5/df/d11/d22/d2a/f49 zero size 2024-04-06T03:40:32.964 INFO:tasks.workunit.client.1.smithi136.stdout:3/249: dwrite d0/d4/d12/d19/d2c/f31 [0,4194304] 0 2024-04-06T03:40:32.964 INFO:tasks.workunit.client.1.smithi136.stdout:3/250: write d0/f1 [119014,119448] 0 2024-04-06T03:40:32.964 INFO:tasks.workunit.client.1.smithi136.stdout:3/251: chown d0/d4/d12 27214 1 2024-04-06T03:40:32.965 INFO:tasks.workunit.client.1.smithi136.stdout:3/252: write d0/d4/f22 [9182391,101357] 0 2024-04-06T03:40:32.965 INFO:tasks.workunit.client.0.smithi094.stdout:3/358: dread d9/f19 [0,4194304] 0 2024-04-06T03:40:32.965 INFO:tasks.workunit.client.1.smithi136.stdout:0/270: dread d1/d6/d2c/d1a/d1c/f31 [0,4194304] 0 2024-04-06T03:40:32.965 INFO:tasks.workunit.client.1.smithi136.stdout:0/271: chown d1/d6/d2c/l41 1 1 2024-04-06T03:40:32.971 INFO:tasks.workunit.client.0.smithi094.stdout:3/359: dread d9/d10/f22 [0,4194304] 0 2024-04-06T03:40:32.971 INFO:tasks.workunit.client.0.smithi094.stdout:3/360: chown d9/fc 55798418 1 2024-04-06T03:40:32.973 INFO:tasks.workunit.client.1.smithi136.stdout:9/289: dwrite d17/d21/d32/f37 [0,4194304] 0 2024-04-06T03:40:32.983 INFO:tasks.workunit.client.1.smithi136.stdout:6/243: link d2/d7/f12 d2/d25/f56 0 2024-04-06T03:40:32.984 INFO:tasks.workunit.client.0.smithi094.stdout:9/271: sync 2024-04-06T03:40:32.984 INFO:tasks.workunit.client.1.smithi136.stdout:5/326: dwrite d1/f4 [0,4194304] 0 2024-04-06T03:40:32.985 INFO:tasks.workunit.client.0.smithi094.stdout:8/333: rename d18/d3c/d54 to d18/d2c/d37/d60/d70 0 2024-04-06T03:40:32.986 INFO:tasks.workunit.client.1.smithi136.stdout:8/316: symlink d3/d2d/l5d 0 2024-04-06T03:40:32.988 INFO:tasks.workunit.client.0.smithi094.stdout:5/238: mkdir db/d19/d2c/d57 0 2024-04-06T03:40:32.988 INFO:tasks.workunit.client.1.smithi136.stdout:8/317: dread d3/de/d18/f1e [0,4194304] 0 2024-04-06T03:40:32.988 INFO:tasks.workunit.client.1.smithi136.stdout:8/318: chown d3/de/f29 114591 1 2024-04-06T03:40:32.988 INFO:tasks.workunit.client.1.smithi136.stdout:8/319: truncate d3/de/f56 661039 0 2024-04-06T03:40:32.988 INFO:tasks.workunit.client.1.smithi136.stdout:8/320: fdatasync d3/de/f56 0 2024-04-06T03:40:32.990 INFO:tasks.workunit.client.1.smithi136.stdout:4/237: rename f1a to d1d/d4d/f5b 0 2024-04-06T03:40:32.991 INFO:tasks.workunit.client.0.smithi094.stdout:1/297: mkdir d0/d4/d1b/d2f/d3c/d48/d67 0 2024-04-06T03:40:32.993 INFO:tasks.workunit.client.0.smithi094.stdout:4/403: creat d0/df/d57/f86 x:0 0 0 2024-04-06T03:40:32.994 INFO:tasks.workunit.client.1.smithi136.stdout:7/300: creat d5/df/d12/d1b/d3a/f6c x:0 0 0 2024-04-06T03:40:32.995 INFO:tasks.workunit.client.0.smithi094.stdout:3/361: creat d9/d1e/d26/d55/d62/f80 x:0 0 0 2024-04-06T03:40:32.998 INFO:tasks.workunit.client.0.smithi094.stdout:9/272: mknod d2/dc/c5e 0 2024-04-06T03:40:32.999 INFO:tasks.workunit.client.0.smithi094.stdout:6/259: sync 2024-04-06T03:40:32.999 INFO:tasks.workunit.client.0.smithi094.stdout:0/316: sync 2024-04-06T03:40:33.001 INFO:tasks.workunit.client.0.smithi094.stdout:8/334: mknod d18/d2c/c71 0 2024-04-06T03:40:33.004 INFO:tasks.workunit.client.0.smithi094.stdout:5/239: creat db/d19/d2c/d57/f58 x:0 0 0 2024-04-06T03:40:33.004 INFO:tasks.workunit.client.0.smithi094.stdout:7/314: sync 2024-04-06T03:40:33.004 INFO:tasks.workunit.client.0.smithi094.stdout:7/315: write d2/d6/dc/d12/d29/d35/f39 [414877,92778] 0 2024-04-06T03:40:33.006 INFO:tasks.workunit.client.0.smithi094.stdout:9/273: dread d2/d2c/d33/f5c [0,4194304] 0 2024-04-06T03:40:33.007 INFO:tasks.workunit.client.0.smithi094.stdout:1/298: symlink d0/l68 0 2024-04-06T03:40:33.009 INFO:tasks.workunit.client.1.smithi136.stdout:9/290: truncate fb 1436543 0 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:4/404: creat d0/d15/d3a/d61/d81/f87 x:0 0 0 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:4/405: creat d0/df/d1e/d34/f88 x:0 0 0 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:4/406: creat d0/d15/d3a/d61/d81/f89 x:0 0 0 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:4/407: read - d0/d15/d3a/d61/d81/f87 zero size 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:4/408: chown d0/d23/f32 4238 1 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:4/409: write d0/d15/d3a/d3e/f7b [878885,56183] 0 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:3/362: dwrite d9/d10/f3b [0,4194304] 0 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:4/410: creat d0/df/d1e/d34/f8a x:0 0 0 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:3/363: truncate d9/d20/d30/f37 1788324 0 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:3/364: write d9/d20/d30/f69 [683440,27166] 0 2024-04-06T03:40:33.012 INFO:tasks.workunit.client.0.smithi094.stdout:3/365: truncate d9/d1d/f36 685671 0 2024-04-06T03:40:33.014 INFO:tasks.workunit.client.0.smithi094.stdout:0/317: rmdir d1/d9/dd/d18/d1a 39 2024-04-06T03:40:33.014 INFO:tasks.workunit.client.0.smithi094.stdout:0/318: read d1/d9/dd/d18/d1a/f4d [406062,76989] 0 2024-04-06T03:40:33.015 INFO:tasks.workunit.client.0.smithi094.stdout:6/260: dwrite dc/d11/f4b [0,4194304] 0 2024-04-06T03:40:33.016 INFO:tasks.workunit.client.0.smithi094.stdout:8/335: write d18/f21 [136731,21474] 0 2024-04-06T03:40:33.018 INFO:tasks.workunit.client.1.smithi136.stdout:2/407: getdents d3/d9 0 2024-04-06T03:40:33.019 INFO:tasks.workunit.client.0.smithi094.stdout:5/240: mknod db/d19/d2c/d57/c59 0 2024-04-06T03:40:33.023 INFO:tasks.workunit.client.1.smithi136.stdout:1/299: rmdir d12/d53 39 2024-04-06T03:40:33.025 INFO:tasks.workunit.client.0.smithi094.stdout:9/274: unlink d2/d2c/d33/l42 0 2024-04-06T03:40:33.025 INFO:tasks.workunit.client.0.smithi094.stdout:9/275: stat d2/dc/d21/d23/l4b 0 2024-04-06T03:40:33.025 INFO:tasks.workunit.client.0.smithi094.stdout:9/276: creat d2/dc/d21/d23/d2f/f5f x:0 0 0 2024-04-06T03:40:33.026 INFO:tasks.workunit.client.0.smithi094.stdout:1/299: creat d0/d4/d1b/d2f/d3c/d48/f69 x:0 0 0 2024-04-06T03:40:33.032 INFO:tasks.workunit.client.0.smithi094.stdout:8/336: dwrite d18/f6b [0,4194304] 0 2024-04-06T03:40:33.034 INFO:tasks.workunit.client.1.smithi136.stdout:6/244: unlink d2/de/d3d/c51 0 2024-04-06T03:40:33.034 INFO:tasks.workunit.client.0.smithi094.stdout:7/316: dwrite d2/d6/dc/f4a [0,4194304] 0 2024-04-06T03:40:33.034 INFO:tasks.workunit.client.0.smithi094.stdout:7/317: chown d2/d6/f24 164922 1 2024-04-06T03:40:33.034 INFO:tasks.workunit.client.0.smithi094.stdout:7/318: stat d2/d6/dc/dd/f2f 0 2024-04-06T03:40:33.036 INFO:tasks.workunit.client.1.smithi136.stdout:5/327: creat d1/f61 x:0 0 0 2024-04-06T03:40:33.036 INFO:tasks.workunit.client.1.smithi136.stdout:5/328: fsync d1/d16/f1b 0 2024-04-06T03:40:33.038 INFO:tasks.workunit.client.1.smithi136.stdout:8/321: creat d3/f5e x:0 0 0 2024-04-06T03:40:33.038 INFO:tasks.workunit.client.0.smithi094.stdout:1/300: dwrite d0/d4/d1b/d2f/d3c/d48/f69 [0,4194304] 0 2024-04-06T03:40:33.040 INFO:tasks.workunit.client.0.smithi094.stdout:1/301: dread d0/d4/d1b/d2f/d3c/d48/f4f [0,4194304] 0 2024-04-06T03:40:33.041 INFO:tasks.workunit.client.1.smithi136.stdout:4/238: link f1c d1d/d31/d3e/d55/f5c 0 2024-04-06T03:40:33.041 INFO:tasks.workunit.client.0.smithi094.stdout:4/411: chown d0/df/d13/f36 11 1 2024-04-06T03:40:33.042 INFO:tasks.workunit.client.0.smithi094.stdout:4/412: fdatasync d0/df/f66 0 2024-04-06T03:40:33.044 INFO:tasks.workunit.client.0.smithi094.stdout:3/366: rename d9/d10/f3b to d9/d10/f81 0 2024-04-06T03:40:33.046 INFO:tasks.workunit.client.1.smithi136.stdout:7/301: mkdir d5/d6d 0 2024-04-06T03:40:33.047 INFO:tasks.workunit.client.0.smithi094.stdout:0/319: symlink d1/d10/l66 0 2024-04-06T03:40:33.049 INFO:tasks.workunit.client.1.smithi136.stdout:9/291: creat d17/d20/d25/d3d/f5d x:0 0 0 2024-04-06T03:40:33.050 INFO:tasks.workunit.client.1.smithi136.stdout:4/239: dread d1d/d4d/f5b [0,4194304] 0 2024-04-06T03:40:33.050 INFO:tasks.workunit.client.1.smithi136.stdout:4/240: creat d1d/d31/d3e/f5d x:0 0 0 2024-04-06T03:40:33.052 INFO:tasks.workunit.client.1.smithi136.stdout:8/322: dwrite d3/f4d [0,4194304] 0 2024-04-06T03:40:33.055 INFO:tasks.workunit.client.0.smithi094.stdout:6/261: symlink dc/d11/d1b/l63 0 2024-04-06T03:40:33.057 INFO:tasks.workunit.client.1.smithi136.stdout:2/408: mkdir d3/d9/d10/d8a 0 2024-04-06T03:40:33.059 INFO:tasks.workunit.client.0.smithi094.stdout:0/320: dwrite d1/d9/dd/d18/d32/d42/f4a [0,4194304] 0 2024-04-06T03:40:33.060 INFO:tasks.workunit.client.1.smithi136.stdout:4/241: dread fc [0,4194304] 0 2024-04-06T03:40:33.060 INFO:tasks.workunit.client.1.smithi136.stdout:4/242: chown f8 665 1 2024-04-06T03:40:33.060 INFO:tasks.workunit.client.1.smithi136.stdout:4/243: fsync d1d/f4f 0 2024-04-06T03:40:33.060 INFO:tasks.workunit.client.1.smithi136.stdout:1/300: link d12/d1b/d28/l2d d12/d3b/d44/d5f/l65 0 2024-04-06T03:40:33.060 INFO:tasks.workunit.client.1.smithi136.stdout:1/301: chown d12/d1b/d28/d48/f50 82860 1 2024-04-06T03:40:33.061 INFO:tasks.workunit.client.1.smithi136.stdout:1/302: dread - d12/d53/f61 zero size 2024-04-06T03:40:33.061 INFO:tasks.workunit.client.0.smithi094.stdout:9/277: creat d2/dc/d46/f60 x:0 0 0 2024-04-06T03:40:33.067 INFO:tasks.workunit.client.0.smithi094.stdout:8/337: creat d18/d2c/d62/f72 x:0 0 0 2024-04-06T03:40:33.069 INFO:tasks.workunit.client.0.smithi094.stdout:0/321: dread d1/d9/dd/f1c [0,4194304] 0 2024-04-06T03:40:33.072 INFO:tasks.workunit.client.1.smithi136.stdout:7/302: symlink d5/df/d12/d1b/l6e 0 2024-04-06T03:40:33.072 INFO:tasks.workunit.client.1.smithi136.stdout:7/303: truncate d5/df/d11/d22/d2a/f48 258709 0 2024-04-06T03:40:33.074 INFO:tasks.workunit.client.0.smithi094.stdout:9/278: dwrite d2/dc/d21/d23/d2f/f5f [0,4194304] 0 2024-04-06T03:40:33.074 INFO:tasks.workunit.client.0.smithi094.stdout:7/319: rename d2/d6/c33 to d2/d6/dc/d12/d3a/d56/c5c 0 2024-04-06T03:40:33.074 INFO:tasks.workunit.client.1.smithi136.stdout:2/409: dwrite d3/d15/f30 [4194304,4194304] 0 2024-04-06T03:40:33.074 INFO:tasks.workunit.client.1.smithi136.stdout:2/410: write d3/d15/f6c [629170,26794] 0 2024-04-06T03:40:33.075 INFO:tasks.workunit.client.1.smithi136.stdout:9/292: symlink d17/d20/d25/d3d/l5e 0 2024-04-06T03:40:33.075 INFO:tasks.workunit.client.1.smithi136.stdout:9/293: creat d17/d21/d29/f5f x:0 0 0 2024-04-06T03:40:33.079 INFO:tasks.workunit.client.1.smithi136.stdout:8/323: mkdir d3/d2d/d5f 0 2024-04-06T03:40:33.080 INFO:tasks.workunit.client.0.smithi094.stdout:4/413: creat d0/df/d13/d4a/f8b x:0 0 0 2024-04-06T03:40:33.081 INFO:tasks.workunit.client.0.smithi094.stdout:4/414: dread - d0/df/d1e/d34/f88 zero size 2024-04-06T03:40:33.081 INFO:tasks.workunit.client.0.smithi094.stdout:7/320: dread d2/f1f [0,4194304] 0 2024-04-06T03:40:33.082 INFO:tasks.workunit.client.1.smithi136.stdout:4/244: link f1c d1d/d31/d3e/d56/f5e 0 2024-04-06T03:40:33.083 INFO:tasks.workunit.client.0.smithi094.stdout:3/367: symlink d9/d1e/d26/d55/l82 0 2024-04-06T03:40:33.084 INFO:tasks.workunit.client.1.smithi136.stdout:1/303: creat d12/d53/f66 x:0 0 0 2024-04-06T03:40:33.085 INFO:tasks.workunit.client.0.smithi094.stdout:6/262: mknod dc/d11/d1b/d24/d46/c64 0 2024-04-06T03:40:33.086 INFO:tasks.workunit.client.1.smithi136.stdout:5/329: getdents d1/dc/d10 0 2024-04-06T03:40:33.087 INFO:tasks.workunit.client.1.smithi136.stdout:5/330: dread - d1/dc/f5b zero size 2024-04-06T03:40:33.088 INFO:tasks.workunit.client.0.smithi094.stdout:5/241: symlink db/l5a 0 2024-04-06T03:40:33.089 INFO:tasks.workunit.client.1.smithi136.stdout:2/411: dwrite d3/d15/f6c [0,4194304] 0 2024-04-06T03:40:33.091 INFO:tasks.workunit.client.1.smithi136.stdout:0/272: rename d1/d2e to d1/d6/d2c/d57 0 2024-04-06T03:40:33.091 INFO:tasks.workunit.client.1.smithi136.stdout:0/273: read - d1/d6/d2c/d57/d43/f50 zero size 2024-04-06T03:40:33.092 INFO:tasks.workunit.client.0.smithi094.stdout:5/242: read db/d40/f55 [269280,74559] 0 2024-04-06T03:40:33.092 INFO:tasks.workunit.client.0.smithi094.stdout:5/243: write db/d19/d21/f26 [3548729,85884] 0 2024-04-06T03:40:33.092 INFO:tasks.workunit.client.0.smithi094.stdout:5/244: truncate f6 1011660 0 2024-04-06T03:40:33.093 INFO:tasks.workunit.client.1.smithi136.stdout:4/245: dread f1c [0,4194304] 0 2024-04-06T03:40:33.094 INFO:tasks.workunit.client.1.smithi136.stdout:1/304: write d12/d1b/d1e/d27/d37/f46 [4071078,100306] 0 2024-04-06T03:40:33.094 INFO:tasks.workunit.client.0.smithi094.stdout:8/338: creat d18/d2c/d37/f73 x:0 0 0 2024-04-06T03:40:33.096 INFO:tasks.workunit.client.1.smithi136.stdout:9/294: mkdir d17/d21/d32/d43/d56/d60 0 2024-04-06T03:40:33.097 INFO:tasks.workunit.client.0.smithi094.stdout:8/339: dread d18/d2c/f41 [0,4194304] 0 2024-04-06T03:40:33.098 INFO:tasks.workunit.client.1.smithi136.stdout:8/324: truncate d3/de/f56 72196 0 2024-04-06T03:40:33.101 INFO:tasks.workunit.client.0.smithi094.stdout:0/322: mkdir d1/d9/dd/d18/d2d/d67 0 2024-04-06T03:40:33.115 INFO:tasks.workunit.client.1.smithi136.stdout:3/253: sync 2024-04-06T03:40:33.115 INFO:tasks.workunit.client.1.smithi136.stdout:3/254: read - d0/d4/d12/d19/f50 zero size 2024-04-06T03:40:33.116 INFO:tasks.workunit.client.1.smithi136.stdout:3/255: fsync d0/f32 0 2024-04-06T03:40:33.116 INFO:tasks.workunit.client.1.smithi136.stdout:0/274: creat d1/d6/f58 x:0 0 0 2024-04-06T03:40:33.120 INFO:tasks.workunit.client.1.smithi136.stdout:5/331: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:33.120 INFO:tasks.workunit.client.1.smithi136.stdout:1/305: link d12/d3b/f4b d12/d3b/d44/f67 0 2024-04-06T03:40:33.121 INFO:tasks.workunit.client.1.smithi136.stdout:5/332: chown d1/dc/d10/f28 375627520 1 2024-04-06T03:40:33.121 INFO:tasks.workunit.client.1.smithi136.stdout:1/306: creat d12/d1b/d1e/d4d/f68 x:0 0 0 2024-04-06T03:40:33.124 INFO:tasks.workunit.client.0.smithi094.stdout:9/279: creat d2/dc/f61 x:0 0 0 2024-04-06T03:40:33.126 INFO:tasks.workunit.client.1.smithi136.stdout:8/325: dwrite d3/da/d26/d35/f1d [4194304,4194304] 0 2024-04-06T03:40:33.126 INFO:tasks.workunit.client.0.smithi094.stdout:1/302: rename d0/l68 to d0/d1/d16/d40/d50/l6a 0 2024-04-06T03:40:33.127 INFO:tasks.workunit.client.0.smithi094.stdout:5/245: dwrite db/d19/d2c/d57/f58 [0,4194304] 0 2024-04-06T03:40:33.128 INFO:tasks.workunit.client.0.smithi094.stdout:7/321: unlink d2/d6/l18 0 2024-04-06T03:40:33.132 INFO:tasks.workunit.client.1.smithi136.stdout:9/295: dwrite f11 [0,4194304] 0 2024-04-06T03:40:33.133 INFO:tasks.workunit.client.1.smithi136.stdout:9/296: creat d17/d19/f61 x:0 0 0 2024-04-06T03:40:33.133 INFO:tasks.workunit.client.0.smithi094.stdout:3/368: creat d9/d20/d76/f83 x:0 0 0 2024-04-06T03:40:33.134 INFO:tasks.workunit.client.1.smithi136.stdout:2/412: dwrite d3/d9/d10/d18/f74 [0,4194304] 0 2024-04-06T03:40:33.135 INFO:tasks.workunit.client.1.smithi136.stdout:2/413: stat d3/d9/fe 0 2024-04-06T03:40:33.135 INFO:tasks.workunit.client.1.smithi136.stdout:7/304: rename d5/df/d11/d22 to d5/df/d1d/d6f 0 2024-04-06T03:40:33.137 INFO:tasks.workunit.client.0.smithi094.stdout:1/303: read d0/d1/d16/d24/d30/f33 [1672711,95407] 0 2024-04-06T03:40:33.138 INFO:tasks.workunit.client.0.smithi094.stdout:6/263: truncate f7 1053083 0 2024-04-06T03:40:33.138 INFO:tasks.workunit.client.0.smithi094.stdout:6/264: write dc/d11/d1b/d24/f25 [1294325,84562] 0 2024-04-06T03:40:33.139 INFO:tasks.workunit.client.1.smithi136.stdout:0/275: dwrite d1/d6/f1f [0,4194304] 0 2024-04-06T03:40:33.139 INFO:tasks.workunit.client.1.smithi136.stdout:0/276: chown d1/d4e/c51 2123 1 2024-04-06T03:40:33.140 INFO:tasks.workunit.client.0.smithi094.stdout:5/246: dread db/f23 [0,4194304] 0 2024-04-06T03:40:33.140 INFO:tasks.workunit.client.0.smithi094.stdout:0/323: dwrite d1/d9/dd/d18/d1a/f4d [4194304,4194304] 0 2024-04-06T03:40:33.140 INFO:tasks.workunit.client.1.smithi136.stdout:8/326: mkdir d3/de/d60 0 2024-04-06T03:40:33.145 INFO:tasks.workunit.client.1.smithi136.stdout:0/277: dread d1/d6/fb [0,4194304] 0 2024-04-06T03:40:33.145 INFO:tasks.workunit.client.0.smithi094.stdout:9/280: creat d2/dc/d21/d26/f62 x:0 0 0 2024-04-06T03:40:33.146 INFO:tasks.workunit.client.1.smithi136.stdout:1/307: dwrite d12/d1b/d1e/d27/d37/f4e [0,4194304] 0 2024-04-06T03:40:33.147 INFO:tasks.workunit.client.1.smithi136.stdout:1/308: readlink d12/l18 0 2024-04-06T03:40:33.147 INFO:tasks.workunit.client.1.smithi136.stdout:5/333: dwrite d1/f61 [0,4194304] 0 2024-04-06T03:40:33.151 INFO:tasks.workunit.client.0.smithi094.stdout:7/322: mknod d2/d6/dc/d12/d29/c5d 0 2024-04-06T03:40:33.151 INFO:tasks.workunit.client.0.smithi094.stdout:7/323: truncate d2/d6/dc/d12/d29/d35/f3d 2495905 0 2024-04-06T03:40:33.151 INFO:tasks.workunit.client.0.smithi094.stdout:7/324: fdatasync d2/d6/dc/dd/f31 0 2024-04-06T03:40:33.151 INFO:tasks.workunit.client.0.smithi094.stdout:7/325: truncate d2/fb 1837800 0 2024-04-06T03:40:33.153 INFO:tasks.workunit.client.0.smithi094.stdout:3/369: symlink d9/d1d/d7d/l84 0 2024-04-06T03:40:33.153 INFO:tasks.workunit.client.0.smithi094.stdout:3/370: write d9/d10/f22 [5205429,105510] 0 2024-04-06T03:40:33.153 INFO:tasks.workunit.client.0.smithi094.stdout:3/371: write d9/d1d/f6a [417479,105345] 0 2024-04-06T03:40:33.157 INFO:tasks.workunit.client.0.smithi094.stdout:6/265: mknod dc/d11/d1b/d4e/c65 0 2024-04-06T03:40:33.157 INFO:tasks.workunit.client.0.smithi094.stdout:6/266: fdatasync dc/d11/f4b 0 2024-04-06T03:40:33.157 INFO:tasks.workunit.client.0.smithi094.stdout:6/267: stat dc/d11/d22/d50/d61 0 2024-04-06T03:40:33.159 INFO:tasks.workunit.client.0.smithi094.stdout:0/324: creat d1/d9/dd/d18/d32/d42/f68 x:0 0 0 2024-04-06T03:40:33.160 INFO:tasks.workunit.client.1.smithi136.stdout:8/327: dwrite d3/da/d26/d35/d3d/d48/f58 [0,4194304] 0 2024-04-06T03:40:33.163 INFO:tasks.workunit.client.0.smithi094.stdout:8/340: rename d18/f1a to d18/f74 0 2024-04-06T03:40:33.164 INFO:tasks.workunit.client.0.smithi094.stdout:3/372: dread d9/d1e/d26/f2c [0,4194304] 0 2024-04-06T03:40:33.165 INFO:tasks.workunit.client.1.smithi136.stdout:2/414: symlink d3/d45/d5f/l8b 0 2024-04-06T03:40:33.166 INFO:tasks.workunit.client.0.smithi094.stdout:9/281: mknod d2/dc/d21/d23/d47/d51/c63 0 2024-04-06T03:40:33.168 INFO:tasks.workunit.client.0.smithi094.stdout:7/326: dwrite d2/d6/dc/dd/f48 [0,4194304] 0 2024-04-06T03:40:33.168 INFO:tasks.workunit.client.0.smithi094.stdout:8/341: dread d18/f25 [0,4194304] 0 2024-04-06T03:40:33.169 INFO:tasks.workunit.client.1.smithi136.stdout:4/246: rename d1d/d2f/f51 to d1d/d31/d3e/f5f 0 2024-04-06T03:40:33.169 INFO:tasks.workunit.client.1.smithi136.stdout:4/247: truncate f8 2885641 0 2024-04-06T03:40:33.170 INFO:tasks.workunit.client.1.smithi136.stdout:8/328: dread d3/f9 [0,4194304] 0 2024-04-06T03:40:33.182 INFO:tasks.workunit.client.0.smithi094.stdout:0/325: dwrite d1/d9/dd/d18/d1a/f46 [0,4194304] 0 2024-04-06T03:40:33.182 INFO:tasks.workunit.client.0.smithi094.stdout:0/326: readlink d1/d9/l19 0 2024-04-06T03:40:33.183 INFO:tasks.workunit.client.0.smithi094.stdout:6/268: unlink dc/le 0 2024-04-06T03:40:33.184 INFO:tasks.workunit.client.1.smithi136.stdout:1/309: getdents d12/d1b/d1e/d27 0 2024-04-06T03:40:33.185 INFO:tasks.workunit.client.0.smithi094.stdout:8/342: dwrite f13 [0,4194304] 0 2024-04-06T03:40:33.185 INFO:tasks.workunit.client.1.smithi136.stdout:6/245: sync 2024-04-06T03:40:33.186 INFO:tasks.workunit.client.0.smithi094.stdout:3/373: creat d9/d1d/d7d/f85 x:0 0 0 2024-04-06T03:40:33.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/282: mkdir d2/d2c/d33/d64 0 2024-04-06T03:40:33.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/283: write d2/f7 [1568119,104091] 0 2024-04-06T03:40:33.188 INFO:tasks.workunit.client.0.smithi094.stdout:9/284: stat d2/f7 0 2024-04-06T03:40:33.189 INFO:tasks.workunit.client.0.smithi094.stdout:9/285: creat d2/dc/d46/f65 x:0 0 0 2024-04-06T03:40:33.190 INFO:tasks.workunit.client.1.smithi136.stdout:5/334: mknod d1/dc/d1c/d32/c62 0 2024-04-06T03:40:33.192 INFO:tasks.workunit.client.1.smithi136.stdout:8/329: dwrite d3/f9 [0,4194304] 0 2024-04-06T03:40:33.193 INFO:tasks.workunit.client.1.smithi136.stdout:6/246: dread d2/d25/f38 [0,4194304] 0 2024-04-06T03:40:33.194 INFO:tasks.workunit.client.1.smithi136.stdout:4/248: dwrite d1d/d31/f3f [0,4194304] 0 2024-04-06T03:40:33.195 INFO:tasks.workunit.client.1.smithi136.stdout:4/249: creat d1d/d2f/f60 x:0 0 0 2024-04-06T03:40:33.195 INFO:tasks.workunit.client.1.smithi136.stdout:2/415: link d3/d15/f36 d3/d15/d19/d43/d54/f8c 0 2024-04-06T03:40:33.195 INFO:tasks.workunit.client.1.smithi136.stdout:2/416: fdatasync d3/d15/f73 0 2024-04-06T03:40:33.195 INFO:tasks.workunit.client.1.smithi136.stdout:5/335: write d1/dc/d10/f28 [1221487,64541] 0 2024-04-06T03:40:33.195 INFO:tasks.workunit.client.1.smithi136.stdout:5/336: fsync d1/dc/d1c/f56 0 2024-04-06T03:40:33.195 INFO:tasks.workunit.client.1.smithi136.stdout:5/337: write d1/dc/d1c/d32/f38 [912621,4975] 0 2024-04-06T03:40:33.197 INFO:tasks.workunit.client.1.smithi136.stdout:3/256: rename d0/l2b to d0/d4/d12/d35/l52 0 2024-04-06T03:40:33.198 INFO:tasks.workunit.client.1.smithi136.stdout:9/297: rename d17/d20 to d17/d20/d25/d3c/d62 22 2024-04-06T03:40:33.199 INFO:tasks.workunit.client.1.smithi136.stdout:8/330: unlink d3/de/d18/l31 0 2024-04-06T03:40:33.203 INFO:tasks.workunit.client.1.smithi136.stdout:6/247: creat d2/d7/d26/d30/d4f/f57 x:0 0 0 2024-04-06T03:40:33.203 INFO:tasks.workunit.client.1.smithi136.stdout:6/248: dread - d2/d7/f31 zero size 2024-04-06T03:40:33.205 INFO:tasks.workunit.client.1.smithi136.stdout:9/298: dread d17/d21/d29/f2a [0,4194304] 0 2024-04-06T03:40:33.207 INFO:tasks.workunit.client.0.smithi094.stdout:8/343: mkdir d18/d5c/d75 0 2024-04-06T03:40:33.207 INFO:tasks.workunit.client.0.smithi094.stdout:8/344: write d18/d34/f3d [561831,128199] 0 2024-04-06T03:40:33.208 INFO:tasks.workunit.client.1.smithi136.stdout:8/331: write d3/de/ff [3412438,68098] 0 2024-04-06T03:40:33.208 INFO:tasks.workunit.client.1.smithi136.stdout:8/332: creat d3/de/f61 x:0 0 0 2024-04-06T03:40:33.208 INFO:tasks.workunit.client.0.smithi094.stdout:3/374: creat d9/d10/d7e/f86 x:0 0 0 2024-04-06T03:40:33.212 INFO:tasks.workunit.client.1.smithi136.stdout:2/417: creat d3/d4a/f8d x:0 0 0 2024-04-06T03:40:33.213 INFO:tasks.workunit.client.0.smithi094.stdout:9/286: truncate d2/dc/d21/f43 8302448 0 2024-04-06T03:40:33.213 INFO:tasks.workunit.client.0.smithi094.stdout:9/287: chown d2/dc/d46 494 1 2024-04-06T03:40:33.216 INFO:tasks.workunit.client.0.smithi094.stdout:7/327: creat d2/d6/dc/d12/d29/d3f/f5e x:0 0 0 2024-04-06T03:40:33.218 INFO:tasks.workunit.client.1.smithi136.stdout:5/338: truncate d1/dc/d1c/f36 3156483 0 2024-04-06T03:40:33.218 INFO:tasks.workunit.client.1.smithi136.stdout:5/339: fdatasync d1/f40 0 2024-04-06T03:40:33.219 INFO:tasks.workunit.client.0.smithi094.stdout:0/327: getdents d1/d9/dd/d18/d5d 0 2024-04-06T03:40:33.219 INFO:tasks.workunit.client.0.smithi094.stdout:0/328: write d1/d9/dd/d18/d32/d42/f45 [934380,84987] 0 2024-04-06T03:40:33.222 INFO:tasks.workunit.client.1.smithi136.stdout:7/305: rename d5/df/d1d/d6f/d6b to d5/df/d12/d1b/d3a/d4a/d70 0 2024-04-06T03:40:33.222 INFO:tasks.workunit.client.1.smithi136.stdout:5/340: dread d1/dc/d1c/f2c [0,4194304] 0 2024-04-06T03:40:33.222 INFO:tasks.workunit.client.1.smithi136.stdout:5/341: chown d1/d16/d17 31 1 2024-04-06T03:40:33.222 INFO:tasks.workunit.client.0.smithi094.stdout:6/269: symlink dc/d11/l66 0 2024-04-06T03:40:33.224 INFO:tasks.workunit.client.1.smithi136.stdout:3/257: mkdir d0/d4/d12/d35/d3c/d53 0 2024-04-06T03:40:33.226 INFO:tasks.workunit.client.1.smithi136.stdout:6/249: mkdir d2/de/d58 0 2024-04-06T03:40:33.226 INFO:tasks.workunit.client.1.smithi136.stdout:6/250: chown d2/de/f35 19 1 2024-04-06T03:40:33.227 INFO:tasks.workunit.client.1.smithi136.stdout:6/251: dread - d2/d7/d26/f54 zero size 2024-04-06T03:40:33.227 INFO:tasks.workunit.client.1.smithi136.stdout:6/252: fsync d2/f9 0 2024-04-06T03:40:33.230 INFO:tasks.workunit.client.0.smithi094.stdout:3/375: symlink d9/d1e/d26/d6c/l87 0 2024-04-06T03:40:33.230 INFO:tasks.workunit.client.1.smithi136.stdout:2/418: dwrite d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:33.231 INFO:tasks.workunit.client.1.smithi136.stdout:8/333: symlink d3/d2d/d5f/l62 0 2024-04-06T03:40:33.231 INFO:tasks.workunit.client.1.smithi136.stdout:8/334: chown d3/da/d26/d35/d53/f5b 1775 1 2024-04-06T03:40:33.231 INFO:tasks.workunit.client.1.smithi136.stdout:7/306: dread d5/df/f41 [0,4194304] 0 2024-04-06T03:40:33.232 INFO:tasks.workunit.client.0.smithi094.stdout:9/288: dwrite d2/dc/d21/d23/d47/d51/f52 [4194304,4194304] 0 2024-04-06T03:40:33.233 INFO:tasks.workunit.client.1.smithi136.stdout:2/419: dread d3/f64 [0,4194304] 0 2024-04-06T03:40:33.234 INFO:tasks.workunit.client.1.smithi136.stdout:4/250: getdents d1d/d31 0 2024-04-06T03:40:33.234 INFO:tasks.workunit.client.1.smithi136.stdout:4/251: chown d1d/f27 1977 1 2024-04-06T03:40:33.234 INFO:tasks.workunit.client.1.smithi136.stdout:4/252: dread - d1d/f49 zero size 2024-04-06T03:40:33.236 INFO:tasks.workunit.client.1.smithi136.stdout:7/307: dread d5/df/d12/f37 [0,4194304] 0 2024-04-06T03:40:33.237 INFO:tasks.workunit.client.1.smithi136.stdout:5/342: dread d1/dc/d1c/d32/f38 [0,4194304] 0 2024-04-06T03:40:33.240 INFO:tasks.workunit.client.1.smithi136.stdout:6/253: dread d2/d7/d26/d30/f50 [0,4194304] 0 2024-04-06T03:40:33.240 INFO:tasks.workunit.client.1.smithi136.stdout:6/254: dread - d2/d7/d26/d30/f3a zero size 2024-04-06T03:40:33.242 INFO:tasks.workunit.client.1.smithi136.stdout:0/278: rename d1/d6/d2c/d1a/d1c to d1/d6/d2c/d1a/d36/d59 0 2024-04-06T03:40:33.243 INFO:tasks.workunit.client.1.smithi136.stdout:3/258: symlink d0/d4/d12/d35/d3c/d53/l54 0 2024-04-06T03:40:33.245 INFO:tasks.workunit.client.1.smithi136.stdout:2/420: truncate d3/d9/d10/d18/f74 2078820 0 2024-04-06T03:40:33.245 INFO:tasks.workunit.client.1.smithi136.stdout:8/335: dread d3/da/d26/d35/f57 [0,4194304] 0 2024-04-06T03:40:33.246 INFO:tasks.workunit.client.0.smithi094.stdout:7/328: creat d2/d6/dc/d12/f5f x:0 0 0 2024-04-06T03:40:33.248 INFO:tasks.workunit.client.0.smithi094.stdout:0/329: mknod d1/d9/dd/d18/d2d/c69 0 2024-04-06T03:40:33.248 INFO:tasks.workunit.client.0.smithi094.stdout:0/330: dread - d1/d9/dd/d18/d1e/d22/d35/f58 zero size 2024-04-06T03:40:33.248 INFO:tasks.workunit.client.0.smithi094.stdout:7/329: read d2/d6/dc/dd/f31 [396877,79734] 0 2024-04-06T03:40:33.248 INFO:tasks.workunit.client.0.smithi094.stdout:7/330: fdatasync d2/d6/dc/d12/d29/d35/f59 0 2024-04-06T03:40:33.248 INFO:tasks.workunit.client.0.smithi094.stdout:7/331: creat d2/d6/dc/dd/df/d36/f60 x:0 0 0 2024-04-06T03:40:33.248 INFO:tasks.workunit.client.1.smithi136.stdout:4/253: creat d1d/d31/d3e/d56/f61 x:0 0 0 2024-04-06T03:40:33.249 INFO:tasks.workunit.client.1.smithi136.stdout:4/254: creat d1d/d31/d3e/f62 x:0 0 0 2024-04-06T03:40:33.249 INFO:tasks.workunit.client.0.smithi094.stdout:9/289: dwrite d2/dc/d46/f4e [0,4194304] 0 2024-04-06T03:40:33.251 INFO:tasks.workunit.client.1.smithi136.stdout:5/343: creat d1/d16/f63 x:0 0 0 2024-04-06T03:40:33.253 INFO:tasks.workunit.client.0.smithi094.stdout:3/376: mkdir d9/d20/d30/d88 0 2024-04-06T03:40:33.256 INFO:tasks.workunit.client.1.smithi136.stdout:6/255: creat d2/d25/f59 x:0 0 0 2024-04-06T03:40:33.256 INFO:tasks.workunit.client.1.smithi136.stdout:6/256: fdatasync d2/d25/f56 0 2024-04-06T03:40:33.261 INFO:tasks.workunit.client.1.smithi136.stdout:1/310: rename d12/d3b/d44/d5f/c41 to d12/c69 0 2024-04-06T03:40:33.262 INFO:tasks.workunit.client.1.smithi136.stdout:1/311: write d12/d3b/d44/f57 [40594,88229] 0 2024-04-06T03:40:33.262 INFO:tasks.workunit.client.1.smithi136.stdout:1/312: fsync d12/d1b/f29 0 2024-04-06T03:40:33.262 INFO:tasks.workunit.client.1.smithi136.stdout:0/279: creat d1/d6/d2c/f5a x:0 0 0 2024-04-06T03:40:33.263 INFO:tasks.workunit.client.0.smithi094.stdout:9/290: write d2/dc/d21/d23/d2f/f34 [1724512,108627] 0 2024-04-06T03:40:33.265 INFO:tasks.workunit.client.1.smithi136.stdout:2/421: mknod d3/d15/d19/c8e 0 2024-04-06T03:40:33.266 INFO:tasks.workunit.client.0.smithi094.stdout:3/377: dread d9/fd [0,4194304] 0 2024-04-06T03:40:33.267 INFO:tasks.workunit.client.0.smithi094.stdout:3/378: truncate d9/d1d/f36 1673641 0 2024-04-06T03:40:33.267 INFO:tasks.workunit.client.0.smithi094.stdout:8/345: rename d18/d2c/d37/d48 to d18/d3c/d76 0 2024-04-06T03:40:33.267 INFO:tasks.workunit.client.0.smithi094.stdout:8/346: dread - d18/d34/d52/f65 zero size 2024-04-06T03:40:33.269 INFO:tasks.workunit.client.0.smithi094.stdout:7/332: mkdir d2/d6/dc/d12/d61 0 2024-04-06T03:40:33.270 INFO:tasks.workunit.client.0.smithi094.stdout:3/379: symlink d9/d1e/d26/d55/d62/l89 0 2024-04-06T03:40:33.271 INFO:tasks.workunit.client.0.smithi094.stdout:0/331: dwrite d1/d9/dd/d18/d1e/d22/d35/f5c [0,4194304] 0 2024-04-06T03:40:33.273 INFO:tasks.workunit.client.0.smithi094.stdout:6/270: rename dc/d11/l1c to dc/d3a/l67 0 2024-04-06T03:40:33.273 INFO:tasks.workunit.client.0.smithi094.stdout:6/271: write dc/d11/d1b/f1f [690717,1573] 0 2024-04-06T03:40:33.273 INFO:tasks.workunit.client.1.smithi136.stdout:4/255: dwrite d1d/f22 [0,4194304] 0 2024-04-06T03:40:33.273 INFO:tasks.workunit.client.1.smithi136.stdout:8/336: dwrite d3/f5e [0,4194304] 0 2024-04-06T03:40:33.276 INFO:tasks.workunit.client.1.smithi136.stdout:5/344: truncate d1/d16/d4b/f57 3618265 0 2024-04-06T03:40:33.278 INFO:tasks.workunit.client.1.smithi136.stdout:6/257: creat d2/d7/f5a x:0 0 0 2024-04-06T03:40:33.281 INFO:tasks.workunit.client.1.smithi136.stdout:9/299: rename d17/d20/d25/d3d/d4b/f52 to d17/d21/d32/d3f/f63 0 2024-04-06T03:40:33.282 INFO:tasks.workunit.client.1.smithi136.stdout:1/313: symlink d12/d1b/d1e/d27/d37/l6a 0 2024-04-06T03:40:33.285 INFO:tasks.workunit.client.1.smithi136.stdout:2/422: getdents d3/d15/d19/d4e 0 2024-04-06T03:40:33.288 INFO:tasks.workunit.client.0.smithi094.stdout:8/347: dwrite d18/d2c/f2e [0,4194304] 0 2024-04-06T03:40:33.291 INFO:tasks.workunit.client.0.smithi094.stdout:0/332: creat d1/d9/dd/d18/d1e/d22/f6a x:0 0 0 2024-04-06T03:40:33.293 INFO:tasks.workunit.client.0.smithi094.stdout:6/272: creat dc/d11/d22/d50/d61/f68 x:0 0 0 2024-04-06T03:40:33.294 INFO:tasks.workunit.client.0.smithi094.stdout:6/273: dread dc/f40 [0,4194304] 0 2024-04-06T03:40:33.296 INFO:tasks.workunit.client.0.smithi094.stdout:7/333: getdents d2/d6/dc/d12 0 2024-04-06T03:40:33.296 INFO:tasks.workunit.client.0.smithi094.stdout:7/334: getdents d2/d6/dc/dd/d5a 0 2024-04-06T03:40:33.297 INFO:tasks.workunit.client.1.smithi136.stdout:4/256: symlink d1d/d31/d3e/l63 0 2024-04-06T03:40:33.300 INFO:tasks.workunit.client.0.smithi094.stdout:3/380: rmdir d9/d10/d27 39 2024-04-06T03:40:33.300 INFO:tasks.workunit.client.0.smithi094.stdout:3/381: readlink d9/d20/l51 0 2024-04-06T03:40:33.301 INFO:tasks.workunit.client.1.smithi136.stdout:2/423: dwrite d3/d15/d19/f1b [0,4194304] 0 2024-04-06T03:40:33.301 INFO:tasks.workunit.client.1.smithi136.stdout:2/424: chown d3/d9/d10/d18/d1d/f35 280529 1 2024-04-06T03:40:33.301 INFO:tasks.workunit.client.1.smithi136.stdout:2/425: fsync d3/d15/f73 0 2024-04-06T03:40:33.303 INFO:tasks.workunit.client.1.smithi136.stdout:9/300: link fa d17/d20/d25/d3c/d55/f64 0 2024-04-06T03:40:33.303 INFO:tasks.workunit.client.1.smithi136.stdout:9/301: write fb [1868769,89920] 0 2024-04-06T03:40:33.304 INFO:tasks.workunit.client.1.smithi136.stdout:9/302: write d17/d21/d29/f5f [345494,20442] 0 2024-04-06T03:40:33.304 INFO:tasks.workunit.client.0.smithi094.stdout:8/348: creat d18/d30/f77 x:0 0 0 2024-04-06T03:40:33.305 INFO:tasks.workunit.client.0.smithi094.stdout:2/371: sync 2024-04-06T03:40:33.306 INFO:tasks.workunit.client.1.smithi136.stdout:1/314: mkdir d12/d3b/d6b 0 2024-04-06T03:40:33.307 INFO:tasks.workunit.client.0.smithi094.stdout:6/274: creat dc/d11/d22/d50/f69 x:0 0 0 2024-04-06T03:40:33.308 INFO:tasks.workunit.client.1.smithi136.stdout:0/280: mknod d1/c5b 0 2024-04-06T03:40:33.310 INFO:tasks.workunit.client.1.smithi136.stdout:4/257: dwrite f14 [4194304,4194304] 0 2024-04-06T03:40:33.310 INFO:tasks.workunit.client.0.smithi094.stdout:7/335: mknod d2/d6/dc/dd/df/d36/c62 0 2024-04-06T03:40:33.310 INFO:tasks.workunit.client.0.smithi094.stdout:7/336: creat d2/f63 x:0 0 0 2024-04-06T03:40:33.310 INFO:tasks.workunit.client.0.smithi094.stdout:7/337: fdatasync d2/d6/dc/d12/d29/d35/f3d 0 2024-04-06T03:40:33.311 INFO:tasks.workunit.client.1.smithi136.stdout:6/258: getdents d2/de/d3d 0 2024-04-06T03:40:33.312 INFO:tasks.workunit.client.1.smithi136.stdout:9/303: dread d17/d19/f27 [0,4194304] 0 2024-04-06T03:40:33.312 INFO:tasks.workunit.client.1.smithi136.stdout:9/304: chown d17/d20 11123 1 2024-04-06T03:40:33.314 INFO:tasks.workunit.client.0.smithi094.stdout:8/349: symlink d18/d2c/l78 0 2024-04-06T03:40:33.315 INFO:tasks.workunit.client.1.smithi136.stdout:3/259: rename d0/d4/d12/d35 to d0/d4/d12/d19/d55 0 2024-04-06T03:40:33.315 INFO:tasks.workunit.client.1.smithi136.stdout:8/337: rename d3/da/d26/d35/d53 to d3/da/d26/d35/d53/d63 22 2024-04-06T03:40:33.316 INFO:tasks.workunit.client.1.smithi136.stdout:8/338: write d3/da/d26/d35/d3d/d48/f58 [4739860,21959] 0 2024-04-06T03:40:33.316 INFO:tasks.workunit.client.0.smithi094.stdout:2/372: rename d0/d8/d71 to d0/d8/d27/d2b/d76 0 2024-04-06T03:40:33.318 INFO:tasks.workunit.client.0.smithi094.stdout:6/275: mknod dc/d11/d22/d50/d61/c6a 0 2024-04-06T03:40:33.318 INFO:tasks.workunit.client.0.smithi094.stdout:6/276: dread - dc/d11/d22/d50/d61/f68 zero size 2024-04-06T03:40:33.321 INFO:tasks.workunit.client.1.smithi136.stdout:1/315: mknod d12/d3b/d44/c6c 0 2024-04-06T03:40:33.321 INFO:tasks.workunit.client.1.smithi136.stdout:1/316: fdatasync d12/d3b/d44/d5f/f30 0 2024-04-06T03:40:33.322 INFO:tasks.workunit.client.0.smithi094.stdout:3/382: getdents d9/d10 0 2024-04-06T03:40:33.322 INFO:tasks.workunit.client.0.smithi094.stdout:3/383: fdatasync d9/d1d/f42 0 2024-04-06T03:40:33.324 INFO:tasks.workunit.client.0.smithi094.stdout:8/350: unlink d18/f19 0 2024-04-06T03:40:33.325 INFO:tasks.workunit.client.1.smithi136.stdout:0/281: mkdir d1/d6/d2c/d57/d43/d4a/d5c 0 2024-04-06T03:40:33.327 INFO:tasks.workunit.client.0.smithi094.stdout:6/277: rename dc/f12 to dc/d3a/d4a/f6b 0 2024-04-06T03:40:33.328 INFO:tasks.workunit.client.1.smithi136.stdout:6/259: fsync d2/fa 0 2024-04-06T03:40:33.328 INFO:tasks.workunit.client.1.smithi136.stdout:6/260: chown d2/d7/d26/d30 3946384 1 2024-04-06T03:40:33.328 INFO:tasks.workunit.client.1.smithi136.stdout:6/261: stat d2/d7/f31 0 2024-04-06T03:40:33.329 INFO:tasks.workunit.client.1.smithi136.stdout:3/260: dwrite d0/f2 [0,4194304] 0 2024-04-06T03:40:33.330 INFO:tasks.workunit.client.1.smithi136.stdout:9/305: creat d17/d21/d32/d43/d56/f65 x:0 0 0 2024-04-06T03:40:33.332 INFO:tasks.workunit.client.1.smithi136.stdout:5/345: rename d1/dc/d1c/l5f to d1/dc/d10/l64 0 2024-04-06T03:40:33.332 INFO:tasks.workunit.client.1.smithi136.stdout:5/346: write d1/dc/f5b [662901,38631] 0 2024-04-06T03:40:33.332 INFO:tasks.workunit.client.0.smithi094.stdout:2/373: write d0/dd/d19/d31/f42 [382162,18739] 0 2024-04-06T03:40:33.332 INFO:tasks.workunit.client.0.smithi094.stdout:8/351: dread d18/d34/f3d [0,4194304] 0 2024-04-06T03:40:33.332 INFO:tasks.workunit.client.0.smithi094.stdout:2/374: truncate d0/d8/d27/f45 155694 0 2024-04-06T03:40:33.334 INFO:tasks.workunit.client.1.smithi136.stdout:0/282: creat d1/d6/d2c/d1a/d36/f5d x:0 0 0 2024-04-06T03:40:33.334 INFO:tasks.workunit.client.1.smithi136.stdout:6/262: rmdir d2/d7/d26/d30/d4f 39 2024-04-06T03:40:33.336 INFO:tasks.workunit.client.0.smithi094.stdout:2/375: mkdir d0/d8/d1d/d77 0 2024-04-06T03:40:33.338 INFO:tasks.workunit.client.1.smithi136.stdout:1/317: dwrite d12/d1b/d22/f43 [0,4194304] 0 2024-04-06T03:40:33.338 INFO:tasks.workunit.client.0.smithi094.stdout:2/376: write d0/d8/d27/d2b/f2c [101712,24758] 0 2024-04-06T03:40:33.338 INFO:tasks.workunit.client.0.smithi094.stdout:2/377: truncate d0/dd/d19/d31/d67/f6c 853844 0 2024-04-06T03:40:33.340 INFO:tasks.workunit.client.1.smithi136.stdout:4/258: rename d1d/f29 to d1d/d4d/f64 0 2024-04-06T03:40:33.341 INFO:tasks.workunit.client.0.smithi094.stdout:8/352: dread f13 [0,4194304] 0 2024-04-06T03:40:33.341 INFO:tasks.workunit.client.0.smithi094.stdout:8/353: dread - d18/d2c/d37/f73 zero size 2024-04-06T03:40:33.341 INFO:tasks.workunit.client.0.smithi094.stdout:8/354: creat d18/d34/d52/f79 x:0 0 0 2024-04-06T03:40:33.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/378: creat d0/d8/d27/f78 x:0 0 0 2024-04-06T03:40:33.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/379: write d0/d8/d27/d2b/f52 [967239,91436] 0 2024-04-06T03:40:33.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/380: write d0/d8/d4f/f5b [740045,40698] 0 2024-04-06T03:40:33.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/381: chown d0/dd/f62 1 1 2024-04-06T03:40:33.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/382: write d0/d8/d27/d2b/f2c [746435,57008] 0 2024-04-06T03:40:33.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/383: creat d0/d8/d1d/f79 x:0 0 0 2024-04-06T03:40:33.346 INFO:tasks.workunit.client.0.smithi094.stdout:3/384: write d9/d10/f81 [1362976,92824] 0 2024-04-06T03:40:33.346 INFO:tasks.workunit.client.0.smithi094.stdout:3/385: write d9/d10/d27/f38 [2922066,126948] 0 2024-04-06T03:40:33.347 INFO:tasks.workunit.client.1.smithi136.stdout:5/347: dread d1/f61 [0,4194304] 0 2024-04-06T03:40:33.348 INFO:tasks.workunit.client.1.smithi136.stdout:0/283: write d1/d6/d2c/d57/f42 [1805679,123929] 0 2024-04-06T03:40:33.349 INFO:tasks.workunit.client.0.smithi094.stdout:8/355: mknod d18/d5c/d75/c7a 0 2024-04-06T03:40:33.349 INFO:tasks.workunit.client.0.smithi094.stdout:8/356: write d18/d34/d52/f79 [591434,1016] 0 2024-04-06T03:40:33.352 INFO:tasks.workunit.client.1.smithi136.stdout:5/348: write d1/dc/d1c/f2c [1180066,74734] 0 2024-04-06T03:40:33.358 INFO:tasks.workunit.client.0.smithi094.stdout:2/384: symlink d0/dd/d1b/l7a 0 2024-04-06T03:40:33.360 INFO:tasks.workunit.client.0.smithi094.stdout:6/278: dwrite f5 [0,4194304] 0 2024-04-06T03:40:33.364 INFO:tasks.workunit.client.1.smithi136.stdout:5/349: dread d1/d16/d4b/f4f [0,4194304] 0 2024-04-06T03:40:33.364 INFO:tasks.workunit.client.1.smithi136.stdout:3/261: dwrite d0/d4/d12/d19/d2c/f40 [0,4194304] 0 2024-04-06T03:40:33.364 INFO:tasks.workunit.client.1.smithi136.stdout:9/306: dwrite d17/d19/f1f [0,4194304] 0 2024-04-06T03:40:33.365 INFO:tasks.workunit.client.0.smithi094.stdout:3/386: mknod d9/d20/c8a 0 2024-04-06T03:40:33.365 INFO:tasks.workunit.client.1.smithi136.stdout:8/339: rmdir d3 39 2024-04-06T03:40:33.367 INFO:tasks.workunit.client.1.smithi136.stdout:0/284: dwrite d1/d6/f1f [0,4194304] 0 2024-04-06T03:40:33.369 INFO:tasks.workunit.client.1.smithi136.stdout:6/263: mknod d2/de/d3d/d47/c5b 0 2024-04-06T03:40:33.370 INFO:tasks.workunit.client.0.smithi094.stdout:2/385: symlink d0/dd/d1b/d72/l7b 0 2024-04-06T03:40:33.370 INFO:tasks.workunit.client.1.smithi136.stdout:4/259: dwrite d1d/f20 [0,4194304] 0 2024-04-06T03:40:33.370 INFO:tasks.workunit.client.1.smithi136.stdout:4/260: dread - d1d/d2f/f57 zero size 2024-04-06T03:40:33.370 INFO:tasks.workunit.client.1.smithi136.stdout:4/261: truncate f1b 1046231 0 2024-04-06T03:40:33.370 INFO:tasks.workunit.client.1.smithi136.stdout:4/262: fsync d1d/d31/f44 0 2024-04-06T03:40:33.371 INFO:tasks.workunit.client.1.smithi136.stdout:1/318: mkdir d12/d3b/d60/d6d 0 2024-04-06T03:40:33.371 INFO:tasks.workunit.client.1.smithi136.stdout:1/319: fsync f3 0 2024-04-06T03:40:33.372 INFO:tasks.workunit.client.0.smithi094.stdout:8/357: dwrite d18/f67 [0,4194304] 0 2024-04-06T03:40:33.374 INFO:tasks.workunit.client.0.smithi094.stdout:3/387: truncate d9/d1e/d26/f28 973548 0 2024-04-06T03:40:33.375 INFO:tasks.workunit.client.1.smithi136.stdout:3/262: link d0/d4/d12/d19/d2e/d3b/d49/l4c d0/d4/d12/d19/d55/d3c/d53/l56 0 2024-04-06T03:40:33.375 INFO:tasks.workunit.client.1.smithi136.stdout:3/263: stat d0/f23 0 2024-04-06T03:40:33.377 INFO:tasks.workunit.client.1.smithi136.stdout:5/350: dwrite d1/d16/d17/f44 [4194304,4194304] 0 2024-04-06T03:40:33.377 INFO:tasks.workunit.client.1.smithi136.stdout:5/351: readlink l0 0 2024-04-06T03:40:33.378 INFO:tasks.workunit.client.1.smithi136.stdout:9/307: mkdir d17/d21/d29/d34/d66 0 2024-04-06T03:40:33.380 INFO:tasks.workunit.client.1.smithi136.stdout:0/285: rmdir d1/d4e 39 2024-04-06T03:40:33.383 INFO:tasks.workunit.client.1.smithi136.stdout:6/264: getdents d2/d7/d26/d30/d4f 0 2024-04-06T03:40:33.383 INFO:tasks.workunit.client.1.smithi136.stdout:6/265: chown d2/d7/d13/l1a 0 1 2024-04-06T03:40:33.389 INFO:tasks.workunit.client.0.smithi094.stdout:2/386: mknod d0/d8/d1d/c7c 0 2024-04-06T03:40:33.394 INFO:tasks.workunit.client.0.smithi094.stdout:3/388: mknod d9/d1e/d6e/d75/c8b 0 2024-04-06T03:40:33.394 INFO:tasks.workunit.client.0.smithi094.stdout:3/389: chown d9/d20/d76/l66 6671201 1 2024-04-06T03:40:33.394 INFO:tasks.workunit.client.0.smithi094.stdout:3/390: creat d9/f8c x:0 0 0 2024-04-06T03:40:33.395 INFO:tasks.workunit.client.0.smithi094.stdout:3/391: creat d9/f8d x:0 0 0 2024-04-06T03:40:33.395 INFO:tasks.workunit.client.1.smithi136.stdout:9/308: dwrite d17/d19/f2f [0,4194304] 0 2024-04-06T03:40:33.395 INFO:tasks.workunit.client.1.smithi136.stdout:9/309: write d17/d19/f61 [69605,104372] 0 2024-04-06T03:40:33.397 INFO:tasks.workunit.client.0.smithi094.stdout:2/387: rename d0/d8/fe to d0/dd/d19/d31/f7d 0 2024-04-06T03:40:33.398 INFO:tasks.workunit.client.0.smithi094.stdout:2/388: read d0/d8/d27/d2b/f52 [8119,100888] 0 2024-04-06T03:40:33.398 INFO:tasks.workunit.client.0.smithi094.stdout:2/389: write d0/f5 [13455583,75148] 0 2024-04-06T03:40:33.399 INFO:tasks.workunit.client.0.smithi094.stdout:8/358: creat d18/d3c/d49/f7b x:0 0 0 2024-04-06T03:40:33.404 INFO:tasks.workunit.client.1.smithi136.stdout:4/263: rename d1d/f38 to d1d/d31/d3e/d56/f65 0 2024-04-06T03:40:33.404 INFO:tasks.workunit.client.0.smithi094.stdout:2/390: dread d0/d8/d1d/f38 [0,4194304] 0 2024-04-06T03:40:33.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:33 smithi094 ceph-mon[29713]: pgmap v16: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 143 MiB/s rd, 195 MiB/s wr, 812 op/s 2024-04-06T03:40:33.494 INFO:tasks.workunit.client.0.smithi094.stdout:3/392: mkdir d9/d1e/d26/d6c/d8e 0 2024-04-06T03:40:33.497 INFO:tasks.workunit.client.1.smithi136.stdout:3/264: mkdir d0/d57 0 2024-04-06T03:40:33.498 INFO:tasks.workunit.client.1.smithi136.stdout:3/265: write d0/f11 [547671,128813] 0 2024-04-06T03:40:33.500 INFO:tasks.workunit.client.0.smithi094.stdout:2/391: rename d0/d18/c23 to d0/d8/d1d/d77/c7e 0 2024-04-06T03:40:33.506 INFO:tasks.workunit.client.1.smithi136.stdout:7/308: sync 2024-04-06T03:40:33.506 INFO:tasks.workunit.client.1.smithi136.stdout:5/352: link d1/dc/d1c/d32/l45 d1/dc/d15/l65 0 2024-04-06T03:40:33.506 INFO:tasks.workunit.client.1.smithi136.stdout:5/353: chown d1/d16/d4b/l5e 2 1 2024-04-06T03:40:33.508 INFO:tasks.workunit.client.1.smithi136.stdout:0/286: creat d1/f5e x:0 0 0 2024-04-06T03:40:33.509 INFO:tasks.workunit.client.1.smithi136.stdout:6/266: mknod d2/de/d58/c5c 0 2024-04-06T03:40:33.510 INFO:tasks.workunit.client.0.smithi094.stdout:3/393: dwrite d9/d10/d27/d3a/f40 [0,4194304] 0 2024-04-06T03:40:33.513 INFO:tasks.workunit.client.1.smithi136.stdout:8/340: rename d3/f4 to d3/da/d26/d35/d3d/f64 0 2024-04-06T03:40:33.514 INFO:tasks.workunit.client.1.smithi136.stdout:5/354: dread d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:33.516 INFO:tasks.workunit.client.1.smithi136.stdout:4/264: mknod d1d/d31/d3e/c66 0 2024-04-06T03:40:33.517 INFO:tasks.workunit.client.0.smithi094.stdout:2/392: symlink d0/dd/d3d/d49/l7f 0 2024-04-06T03:40:33.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/320: rmdir d12/d53 39 2024-04-06T03:40:33.517 INFO:tasks.workunit.client.1.smithi136.stdout:2/426: sync 2024-04-06T03:40:33.520 INFO:tasks.workunit.client.1.smithi136.stdout:4/265: dread d1d/d31/f48 [0,4194304] 0 2024-04-06T03:40:33.521 INFO:tasks.workunit.client.1.smithi136.stdout:3/266: creat d0/d4/d12/d19/d55/d3c/f58 x:0 0 0 2024-04-06T03:40:33.523 INFO:tasks.workunit.client.1.smithi136.stdout:0/287: creat d1/d6/d2c/d57/f5f x:0 0 0 2024-04-06T03:40:33.523 INFO:tasks.workunit.client.1.smithi136.stdout:0/288: chown d1/d6/d2c/d1a/f4b 2 1 2024-04-06T03:40:33.525 INFO:tasks.workunit.client.0.smithi094.stdout:2/393: read d0/dd/d3d/d49/f53 [150339,127674] 0 2024-04-06T03:40:33.525 INFO:tasks.workunit.client.0.smithi094.stdout:8/359: link d18/f27 d18/d2c/d37/d60/d70/f7c 0 2024-04-06T03:40:33.526 INFO:tasks.workunit.client.1.smithi136.stdout:6/267: dwrite d2/f9 [0,4194304] 0 2024-04-06T03:40:33.527 INFO:tasks.workunit.client.1.smithi136.stdout:7/309: dwrite d5/df/d1d/f1f [0,4194304] 0 2024-04-06T03:40:33.527 INFO:tasks.workunit.client.1.smithi136.stdout:7/310: readlink d5/df/d12/d1b/l6e 0 2024-04-06T03:40:33.528 INFO:tasks.workunit.client.1.smithi136.stdout:9/310: rename d17/d20/d25/d3d/f5d to d17/d21/d32/d43/d56/f67 0 2024-04-06T03:40:33.529 INFO:tasks.workunit.client.1.smithi136.stdout:9/311: read fb [1307074,39298] 0 2024-04-06T03:40:33.529 INFO:tasks.workunit.client.1.smithi136.stdout:9/312: fsync d17/d21/d32/d3f/f44 0 2024-04-06T03:40:33.541 INFO:tasks.workunit.client.1.smithi136.stdout:6/268: dwrite d2/d7/f12 [0,4194304] 0 2024-04-06T03:40:33.544 INFO:tasks.workunit.client.0.smithi094.stdout:1/304: sync 2024-04-06T03:40:33.544 INFO:tasks.workunit.client.0.smithi094.stdout:4/415: sync 2024-04-06T03:40:33.545 INFO:tasks.workunit.client.0.smithi094.stdout:1/305: write d0/d1/d16/d24/d30/f33 [4541359,48268] 0 2024-04-06T03:40:33.545 INFO:tasks.workunit.client.0.smithi094.stdout:1/306: chown d0/d1/d16/d40/d50/l51 92797 1 2024-04-06T03:40:33.545 INFO:tasks.workunit.client.0.smithi094.stdout:1/307: fsync d0/d4/d1b/d2f/d3c/d48/f4f 0 2024-04-06T03:40:33.545 INFO:tasks.workunit.client.0.smithi094.stdout:1/308: chown d0/d4/d1b/d2f/d3c/d48/f4f 52843981 1 2024-04-06T03:40:33.545 INFO:tasks.workunit.client.0.smithi094.stdout:1/309: chown d0/d7/d2d/l63 2004594543 1 2024-04-06T03:40:33.564 INFO:tasks.workunit.client.0.smithi094.stdout:1/310: dwrite d0/d1/f10 [0,4194304] 0 2024-04-06T03:40:33.564 INFO:tasks.workunit.client.1.smithi136.stdout:6/269: dwrite d2/d7/f12 [0,4194304] 0 2024-04-06T03:40:33.564 INFO:tasks.workunit.client.1.smithi136.stdout:6/270: write d2/f5 [1163329,88796] 0 2024-04-06T03:40:33.577 INFO:tasks.workunit.client.1.smithi136.stdout:8/341: creat d3/da/d26/d35/d39/f65 x:0 0 0 2024-04-06T03:40:33.577 INFO:tasks.workunit.client.1.smithi136.stdout:8/342: truncate d3/de/f56 318946 0 2024-04-06T03:40:33.578 INFO:tasks.workunit.client.0.smithi094.stdout:2/394: getdents d0/dd/d1b/d72 0 2024-04-06T03:40:33.580 INFO:tasks.workunit.client.1.smithi136.stdout:5/355: getdents d1/dc/d10 0 2024-04-06T03:40:33.580 INFO:tasks.workunit.client.1.smithi136.stdout:5/356: getdents d1/dc/d10 0 2024-04-06T03:40:33.584 INFO:tasks.workunit.client.1.smithi136.stdout:2/427: mknod d3/d9/d10/d18/d1d/d3c/c8f 0 2024-04-06T03:40:33.585 INFO:tasks.workunit.client.1.smithi136.stdout:5/357: write d1/f61 [1018674,122439] 0 2024-04-06T03:40:33.588 INFO:tasks.workunit.client.1.smithi136.stdout:4/266: truncate d1d/d31/d3e/f5f 3353258 0 2024-04-06T03:40:33.592 INFO:tasks.workunit.client.1.smithi136.stdout:3/267: unlink d0/d4/fd 0 2024-04-06T03:40:33.595 INFO:tasks.workunit.client.1.smithi136.stdout:7/311: mkdir d5/df/d12/d59/d71 0 2024-04-06T03:40:33.595 INFO:tasks.workunit.client.1.smithi136.stdout:7/312: dread - d5/df/d12/d1b/f47 zero size 2024-04-06T03:40:33.595 INFO:tasks.workunit.client.1.smithi136.stdout:7/313: write d5/df/d53/f39 [4154398,164] 0 2024-04-06T03:40:33.595 INFO:tasks.workunit.client.1.smithi136.stdout:7/314: write d5/df/d1d/f55 [331177,116855] 0 2024-04-06T03:40:33.596 INFO:tasks.workunit.client.0.smithi094.stdout:8/360: dwrite d18/f74 [4194304,4194304] 0 2024-04-06T03:40:33.596 INFO:tasks.workunit.client.0.smithi094.stdout:2/395: dwrite d0/d18/f55 [0,4194304] 0 2024-04-06T03:40:33.596 INFO:tasks.workunit.client.0.smithi094.stdout:2/396: chown d0/dd/d19 30692 1 2024-04-06T03:40:33.596 INFO:tasks.workunit.client.0.smithi094.stdout:2/397: write d0/d8/d1d/f38 [3045380,63245] 0 2024-04-06T03:40:33.596 INFO:tasks.workunit.client.0.smithi094.stdout:2/398: fdatasync d0/f64 0 2024-04-06T03:40:33.599 INFO:tasks.workunit.client.1.smithi136.stdout:9/313: link d17/l4a d17/d20/d25/d3d/d4b/l68 0 2024-04-06T03:40:33.599 INFO:tasks.workunit.client.1.smithi136.stdout:9/314: chown d17/d21/d29/d34/d46 29 1 2024-04-06T03:40:33.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:33 smithi136 ceph-mon[36436]: pgmap v16: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 143 MiB/s rd, 195 MiB/s wr, 812 op/s 2024-04-06T03:40:33.609 INFO:tasks.workunit.client.1.smithi136.stdout:3/268: dwrite d0/d4/d12/f15 [0,4194304] 0 2024-04-06T03:40:33.614 INFO:tasks.workunit.client.0.smithi094.stdout:4/416: creat d0/df/d13/f8c x:0 0 0 2024-04-06T03:40:33.632 INFO:tasks.workunit.client.1.smithi136.stdout:6/271: mkdir d2/de/d58/d5d 0 2024-04-06T03:40:33.648 INFO:tasks.workunit.client.0.smithi094.stdout:1/311: symlink d0/d4/d1b/d2f/d3c/d48/l6b 0 2024-04-06T03:40:33.648 INFO:tasks.workunit.client.0.smithi094.stdout:1/312: chown d0/d4/fc 0 1 2024-04-06T03:40:33.648 INFO:tasks.workunit.client.0.smithi094.stdout:1/313: fdatasync d0/d1/d16/f53 0 2024-04-06T03:40:33.648 INFO:tasks.workunit.client.0.smithi094.stdout:1/314: fdatasync d0/d7/d2d/f61 0 2024-04-06T03:40:33.649 INFO:tasks.workunit.client.1.smithi136.stdout:6/272: dwrite d2/de/f35 [0,4194304] 0 2024-04-06T03:40:33.649 INFO:tasks.workunit.client.1.smithi136.stdout:6/273: creat d2/de/f5e x:0 0 0 2024-04-06T03:40:33.649 INFO:tasks.workunit.client.1.smithi136.stdout:6/274: fdatasync d2/f9 0 2024-04-06T03:40:33.649 INFO:tasks.workunit.client.1.smithi136.stdout:6/275: write d2/f5 [1729655,50686] 0 2024-04-06T03:40:33.650 INFO:tasks.workunit.client.1.smithi136.stdout:6/276: fdatasync d2/d25/f59 0 2024-04-06T03:40:33.654 INFO:tasks.workunit.client.0.smithi094.stdout:3/394: creat d9/d1e/f8f x:0 0 0 2024-04-06T03:40:33.656 INFO:tasks.workunit.client.0.smithi094.stdout:1/315: dread d0/d4/f9 [4194304,4194304] 0 2024-04-06T03:40:33.669 INFO:tasks.workunit.client.0.smithi094.stdout:8/361: rename d18/d2c/c71 to d18/d2c/d37/d60/c7d 0 2024-04-06T03:40:33.684 INFO:tasks.workunit.client.0.smithi094.stdout:4/417: mknod d0/d15/c8d 0 2024-04-06T03:40:33.686 INFO:tasks.workunit.client.0.smithi094.stdout:3/395: mkdir d9/d1e/d6e/d75/d90 0 2024-04-06T03:40:33.687 INFO:tasks.workunit.client.1.smithi136.stdout:1/321: getdents d12/d1b/d22 0 2024-04-06T03:40:33.688 INFO:tasks.workunit.client.0.smithi094.stdout:1/316: creat d0/d4/d1b/d2f/d3c/f6c x:0 0 0 2024-04-06T03:40:33.690 INFO:tasks.workunit.client.1.smithi136.stdout:2/428: truncate d3/d15/d19/f39 2494927 0 2024-04-06T03:40:33.690 INFO:tasks.workunit.client.1.smithi136.stdout:2/429: chown d3/d45/d5f/l71 126567 1 2024-04-06T03:40:33.692 INFO:tasks.workunit.client.1.smithi136.stdout:5/358: mkdir d1/dc/d15/d20/d66 0 2024-04-06T03:40:33.694 INFO:tasks.workunit.client.0.smithi094.stdout:4/418: creat d0/df/d13/f8e x:0 0 0 2024-04-06T03:40:33.695 INFO:tasks.workunit.client.1.smithi136.stdout:1/322: dread f3 [0,4194304] 0 2024-04-06T03:40:33.695 INFO:tasks.workunit.client.1.smithi136.stdout:1/323: fsync d12/d53/f61 0 2024-04-06T03:40:33.695 INFO:tasks.workunit.client.1.smithi136.stdout:1/324: getdents d12/d3b/d6b 0 2024-04-06T03:40:33.696 INFO:tasks.workunit.client.1.smithi136.stdout:4/267: creat d1d/d31/f67 x:0 0 0 2024-04-06T03:40:33.696 INFO:tasks.workunit.client.1.smithi136.stdout:4/268: readlink d1d/d31/l47 0 2024-04-06T03:40:33.696 INFO:tasks.workunit.client.1.smithi136.stdout:4/269: write d1d/d31/d3e/d56/f61 [838572,76539] 0 2024-04-06T03:40:33.698 INFO:tasks.workunit.client.1.smithi136.stdout:9/315: unlink d17/d21/d29/d34/d46/l5a 0 2024-04-06T03:40:33.698 INFO:tasks.workunit.client.1.smithi136.stdout:9/316: creat d17/d19/f69 x:0 0 0 2024-04-06T03:40:33.699 INFO:tasks.workunit.client.0.smithi094.stdout:4/419: creat d0/d15/f8f x:0 0 0 2024-04-06T03:40:33.702 INFO:tasks.workunit.client.0.smithi094.stdout:4/420: truncate d0/d4/fd 2963657 0 2024-04-06T03:40:33.705 INFO:tasks.workunit.client.0.smithi094.stdout:4/421: symlink d0/d15/d3a/d3e/l90 0 2024-04-06T03:40:33.711 INFO:tasks.workunit.client.1.smithi136.stdout:1/325: dwrite d12/d53/f66 [0,4194304] 0 2024-04-06T03:40:33.713 INFO:tasks.workunit.client.1.smithi136.stdout:3/269: mknod d0/d4/d12/d3a/c59 0 2024-04-06T03:40:33.713 INFO:tasks.workunit.client.1.smithi136.stdout:3/270: write d0/d4/d12/d19/f50 [792693,16120] 0 2024-04-06T03:40:33.713 INFO:tasks.workunit.client.1.smithi136.stdout:3/271: write d0/d4/fa [278315,54853] 0 2024-04-06T03:40:33.713 INFO:tasks.workunit.client.1.smithi136.stdout:3/272: fdatasync d0/d4/d12/d3a/f3e 0 2024-04-06T03:40:33.714 INFO:tasks.workunit.client.0.smithi094.stdout:4/422: dread d0/d4/d4b/f6c [0,4194304] 0 2024-04-06T03:40:33.718 INFO:tasks.workunit.client.0.smithi094.stdout:4/423: unlink d0/d4/c53 0 2024-04-06T03:40:33.725 INFO:tasks.workunit.client.1.smithi136.stdout:2/430: truncate d3/d15/f2b 2867592 0 2024-04-06T03:40:33.726 INFO:tasks.workunit.client.1.smithi136.stdout:6/277: write d2/fd [4039631,7601] 0 2024-04-06T03:40:33.726 INFO:tasks.workunit.client.1.smithi136.stdout:6/278: getdents d2/de/d3d 0 2024-04-06T03:40:33.726 INFO:tasks.workunit.client.1.smithi136.stdout:4/270: creat d1d/d4d/f68 x:0 0 0 2024-04-06T03:40:33.727 INFO:tasks.workunit.client.1.smithi136.stdout:4/271: truncate d1d/d31/d3e/d56/f5e 5025837 0 2024-04-06T03:40:33.731 INFO:tasks.workunit.client.1.smithi136.stdout:9/317: symlink d17/d20/d25/d3d/d4b/l6a 0 2024-04-06T03:40:33.733 INFO:tasks.workunit.client.1.smithi136.stdout:2/431: write d3/d9/d10/d18/d1d/f35 [5325502,88429] 0 2024-04-06T03:40:33.733 INFO:tasks.workunit.client.0.smithi094.stdout:4/424: mkdir d0/df/d91 0 2024-04-06T03:40:33.735 INFO:tasks.workunit.client.1.smithi136.stdout:3/273: mkdir d0/d4/d12/d19/d55/d3c/d5a 0 2024-04-06T03:40:33.740 INFO:tasks.workunit.client.1.smithi136.stdout:4/272: mknod d1d/d2f/c69 0 2024-04-06T03:40:33.740 INFO:tasks.workunit.client.1.smithi136.stdout:4/273: stat d1d/d31/d3e/d55/f5c 0 2024-04-06T03:40:33.740 INFO:tasks.workunit.client.1.smithi136.stdout:4/274: truncate d1d/d31/f4c 840 0 2024-04-06T03:40:33.740 INFO:tasks.workunit.client.1.smithi136.stdout:4/275: readlink l6 0 2024-04-06T03:40:33.740 INFO:tasks.workunit.client.1.smithi136.stdout:4/276: truncate d1d/d31/f4c 159599 0 2024-04-06T03:40:33.740 INFO:tasks.workunit.client.1.smithi136.stdout:4/277: creat d1d/d4d/f6a x:0 0 0 2024-04-06T03:40:33.740 INFO:tasks.workunit.client.1.smithi136.stdout:4/278: chown d1d/f49 8 1 2024-04-06T03:40:33.740 INFO:tasks.workunit.client.1.smithi136.stdout:4/279: fsync d1d/f20 0 2024-04-06T03:40:33.741 INFO:tasks.workunit.client.1.smithi136.stdout:4/280: chown d1d/d31/f3b 5324848 1 2024-04-06T03:40:33.745 INFO:tasks.workunit.client.1.smithi136.stdout:3/274: mknod d0/d4/d12/d19/d2c/c5b 0 2024-04-06T03:40:33.745 INFO:tasks.workunit.client.1.smithi136.stdout:3/275: dread - d0/d4/d12/d2d/f4a zero size 2024-04-06T03:40:33.747 INFO:tasks.workunit.client.1.smithi136.stdout:9/318: rmdir d17/d21/d32/d3f 39 2024-04-06T03:40:33.748 INFO:tasks.workunit.client.1.smithi136.stdout:9/319: mknod d17/d21/d29/d34/d66/c6b 0 2024-04-06T03:40:33.748 INFO:tasks.workunit.client.1.smithi136.stdout:9/320: stat d17/d21/d29/f5f 0 2024-04-06T03:40:33.758 INFO:tasks.workunit.client.0.smithi094.stdout:4/425: mknod d0/df/d1e/c92 0 2024-04-06T03:40:33.759 INFO:tasks.workunit.client.1.smithi136.stdout:8/343: rename d3/f4d to d3/da/d26/d35/d3d/d48/f66 0 2024-04-06T03:40:33.761 INFO:tasks.workunit.client.0.smithi094.stdout:8/362: rmdir d18/d2c/d37 39 2024-04-06T03:40:33.761 INFO:tasks.workunit.client.0.smithi094.stdout:8/363: fdatasync d18/d3c/d76/f5b 0 2024-04-06T03:40:33.762 INFO:tasks.workunit.client.0.smithi094.stdout:8/364: symlink d18/d5c/d75/l7e 0 2024-04-06T03:40:33.763 INFO:tasks.workunit.client.1.smithi136.stdout:9/321: dwrite d17/d20/d25/f26 [0,4194304] 0 2024-04-06T03:40:33.784 INFO:tasks.workunit.client.1.smithi136.stdout:8/344: dwrite d3/f8 [0,4194304] 0 2024-04-06T03:40:33.784 INFO:tasks.workunit.client.1.smithi136.stdout:8/345: chown d3/d2d/d5f/l62 214 1 2024-04-06T03:40:33.788 INFO:tasks.workunit.client.0.smithi094.stdout:5/247: sync 2024-04-06T03:40:33.788 INFO:tasks.workunit.client.0.smithi094.stdout:4/426: write d0/d4/fd [293622,114388] 0 2024-04-06T03:40:33.789 INFO:tasks.workunit.client.1.smithi136.stdout:8/346: dread d3/de/f27 [0,4194304] 0 2024-04-06T03:40:33.789 INFO:tasks.workunit.client.0.smithi094.stdout:5/248: chown db/d19/f45 8111254 1 2024-04-06T03:40:33.790 INFO:tasks.workunit.client.0.smithi094.stdout:8/365: dwrite d18/d34/f3d [0,4194304] 0 2024-04-06T03:40:33.792 INFO:tasks.workunit.client.0.smithi094.stdout:5/249: mkdir db/d19/d2c/d5b 0 2024-04-06T03:40:33.794 INFO:tasks.workunit.client.0.smithi094.stdout:5/250: rmdir db/d19/d24 39 2024-04-06T03:40:33.797 INFO:tasks.workunit.client.0.smithi094.stdout:5/251: dread db/d19/d24/f34 [0,4194304] 0 2024-04-06T03:40:33.802 INFO:tasks.workunit.client.1.smithi136.stdout:8/347: dwrite d3/da/d26/f28 [4194304,4194304] 0 2024-04-06T03:40:33.803 INFO:tasks.workunit.client.0.smithi094.stdout:5/252: dread db/f11 [4194304,4194304] 0 2024-04-06T03:40:33.921 INFO:tasks.workunit.client.1.smithi136.stdout:8/348: creat d3/d2d/f67 x:0 0 0 2024-04-06T03:40:33.922 INFO:tasks.workunit.client.0.smithi094.stdout:3/396: rename d9/c57 to d9/d20/c91 0 2024-04-06T03:40:33.924 INFO:tasks.workunit.client.0.smithi094.stdout:5/253: unlink db/d19/f3d 0 2024-04-06T03:40:33.927 INFO:tasks.workunit.client.0.smithi094.stdout:1/317: rename d0/d4/d1b/d2f/c41 to d0/d1/d16/d3f/c6d 0 2024-04-06T03:40:33.927 INFO:tasks.workunit.client.0.smithi094.stdout:1/318: creat d0/d7/d2d/f6e x:0 0 0 2024-04-06T03:40:33.927 INFO:tasks.workunit.client.0.smithi094.stdout:1/319: read d0/d7/f26 [1500371,93973] 0 2024-04-06T03:40:33.927 INFO:tasks.workunit.client.0.smithi094.stdout:1/320: chown d0/d4/d37/d5f 1326 1 2024-04-06T03:40:33.927 INFO:tasks.workunit.client.0.smithi094.stdout:1/321: fsync d0/d4/fc 0 2024-04-06T03:40:33.928 INFO:tasks.workunit.client.1.smithi136.stdout:5/359: rename d1/c4c to d1/c67 0 2024-04-06T03:40:33.929 INFO:tasks.workunit.client.0.smithi094.stdout:3/397: unlink d9/d10/l58 0 2024-04-06T03:40:33.932 INFO:tasks.workunit.client.0.smithi094.stdout:5/254: dread db/f1e [0,4194304] 0 2024-04-06T03:40:33.933 INFO:tasks.workunit.client.0.smithi094.stdout:8/366: rename l15 to d18/d34/d52/l7f 0 2024-04-06T03:40:33.940 INFO:tasks.workunit.client.1.smithi136.stdout:8/349: link d3/da/d26/d35/d53/c5c d3/de/d60/c68 0 2024-04-06T03:40:33.941 INFO:tasks.workunit.client.0.smithi094.stdout:3/398: unlink d9/d1e/d26/d6c/l87 0 2024-04-06T03:40:33.941 INFO:tasks.workunit.client.0.smithi094.stdout:3/399: chown d9/d20/d2d/c49 513064 1 2024-04-06T03:40:33.944 INFO:tasks.workunit.client.0.smithi094.stdout:5/255: link db/d19/d24/f34 db/d19/d2c/d57/f5c 0 2024-04-06T03:40:33.945 INFO:tasks.workunit.client.0.smithi094.stdout:3/400: creat d9/d10/d27/d7c/f92 x:0 0 0 2024-04-06T03:40:33.945 INFO:tasks.workunit.client.0.smithi094.stdout:5/256: read db/d19/d24/d28/f3f [268723,85219] 0 2024-04-06T03:40:33.945 INFO:tasks.workunit.client.0.smithi094.stdout:5/257: write db/d19/d21/f26 [4695631,56968] 0 2024-04-06T03:40:33.948 INFO:tasks.workunit.client.0.smithi094.stdout:5/258: creat db/d19/f5d x:0 0 0 2024-04-06T03:40:33.950 INFO:tasks.workunit.client.0.smithi094.stdout:5/259: creat db/d19/d21/f5e x:0 0 0 2024-04-06T03:40:33.952 INFO:tasks.workunit.client.1.smithi136.stdout:7/315: rename d5/f46 to d5/df/d1d/d6f/d2c/f72 0 2024-04-06T03:40:33.953 INFO:tasks.workunit.client.1.smithi136.stdout:7/316: write d5/df/d53/f3c [4893207,85543] 0 2024-04-06T03:40:33.954 INFO:tasks.workunit.client.1.smithi136.stdout:5/360: dwrite d1/d16/f1b [0,4194304] 0 2024-04-06T03:40:33.954 INFO:tasks.workunit.client.1.smithi136.stdout:5/361: fdatasync d1/dc/d1c/f36 0 2024-04-06T03:40:33.954 INFO:tasks.workunit.client.1.smithi136.stdout:2/432: rename d3/d9/d10/f6a to d3/d15/f90 0 2024-04-06T03:40:33.955 INFO:tasks.workunit.client.1.smithi136.stdout:1/326: mknod d12/c6e 0 2024-04-06T03:40:33.956 INFO:tasks.workunit.client.1.smithi136.stdout:5/362: dread d1/dc/d10/f49 [0,4194304] 0 2024-04-06T03:40:33.958 INFO:tasks.workunit.client.1.smithi136.stdout:7/317: getdents d5/df/d12/d1b 0 2024-04-06T03:40:33.960 INFO:tasks.workunit.client.0.smithi094.stdout:3/401: write d9/f19 [2849423,110865] 0 2024-04-06T03:40:33.961 INFO:tasks.workunit.client.1.smithi136.stdout:0/289: sync 2024-04-06T03:40:33.961 INFO:tasks.workunit.client.1.smithi136.stdout:0/290: chown d1/d6/d2c/d1a/l3c 1824872 1 2024-04-06T03:40:33.961 INFO:tasks.workunit.client.1.smithi136.stdout:9/322: rename d17/d21/d32/f37 to d17/d21/d29/d34/d66/f6c 0 2024-04-06T03:40:33.963 INFO:tasks.workunit.client.1.smithi136.stdout:8/350: dwrite d3/da/d26/f28 [4194304,4194304] 0 2024-04-06T03:40:33.963 INFO:tasks.workunit.client.1.smithi136.stdout:2/433: dread d3/d15/d19/f39 [0,4194304] 0 2024-04-06T03:40:33.963 INFO:tasks.workunit.client.0.smithi094.stdout:8/367: dwrite f17 [0,4194304] 0 2024-04-06T03:40:33.963 INFO:tasks.workunit.client.0.smithi094.stdout:3/402: dread d9/d1d/f45 [0,4194304] 0 2024-04-06T03:40:33.964 INFO:tasks.workunit.client.1.smithi136.stdout:5/363: mkdir d1/d16/d4b/d68 0 2024-04-06T03:40:33.965 INFO:tasks.workunit.client.1.smithi136.stdout:7/318: rmdir d5/df 39 2024-04-06T03:40:33.966 INFO:tasks.workunit.client.0.smithi094.stdout:8/368: getdents d18/d5c 0 2024-04-06T03:40:33.967 INFO:tasks.workunit.client.1.smithi136.stdout:9/323: rmdir d17/d21/d29/d34/d66 39 2024-04-06T03:40:33.968 INFO:tasks.workunit.client.0.smithi094.stdout:5/260: dread db/d19/f41 [0,4194304] 0 2024-04-06T03:40:33.968 INFO:tasks.workunit.client.0.smithi094.stdout:5/261: write db/d19/d21/f49 [138468,33827] 0 2024-04-06T03:40:33.968 INFO:tasks.workunit.client.0.smithi094.stdout:8/369: dread d18/d2c/d62/f6d [0,4194304] 0 2024-04-06T03:40:33.968 INFO:tasks.workunit.client.0.smithi094.stdout:8/370: chown d18/d2c/d37/c51 455859 1 2024-04-06T03:40:33.968 INFO:tasks.workunit.client.0.smithi094.stdout:3/403: creat d9/d1e/d26/d55/f93 x:0 0 0 2024-04-06T03:40:33.970 INFO:tasks.workunit.client.1.smithi136.stdout:1/327: getdents d12/d3b/d44 0 2024-04-06T03:40:33.970 INFO:tasks.workunit.client.1.smithi136.stdout:1/328: truncate d12/d3b/f4c 467951 0 2024-04-06T03:40:33.970 INFO:tasks.workunit.client.1.smithi136.stdout:0/291: dread d1/d6/d2c/d57/f46 [0,4194304] 0 2024-04-06T03:40:33.971 INFO:tasks.workunit.client.1.smithi136.stdout:0/292: write d1/d6/d2c/d1a/d36/f44 [996322,129102] 0 2024-04-06T03:40:33.971 INFO:tasks.workunit.client.1.smithi136.stdout:0/293: write d1/d6/d2c/d1a/f28 [1047633,91630] 0 2024-04-06T03:40:33.973 INFO:tasks.workunit.client.0.smithi094.stdout:5/262: write db/d19/f41 [268344,95618] 0 2024-04-06T03:40:33.973 INFO:tasks.workunit.client.0.smithi094.stdout:5/263: write db/fc [4699891,122844] 0 2024-04-06T03:40:33.973 INFO:tasks.workunit.client.0.smithi094.stdout:8/371: mkdir d18/d3c/d49/d80 0 2024-04-06T03:40:33.973 INFO:tasks.workunit.client.0.smithi094.stdout:8/372: readlink d18/d3c/d49/l61 0 2024-04-06T03:40:33.974 INFO:tasks.workunit.client.1.smithi136.stdout:8/351: getdents d3/da/d26/d35/d3d/d48 0 2024-04-06T03:40:33.976 INFO:tasks.workunit.client.1.smithi136.stdout:2/434: mkdir d3/d9/d10/d18/d1d/d91 0 2024-04-06T03:40:33.978 INFO:tasks.workunit.client.1.smithi136.stdout:5/364: creat d1/dc/d10/d1e/f69 x:0 0 0 2024-04-06T03:40:33.981 INFO:tasks.workunit.client.0.smithi094.stdout:0/333: sync 2024-04-06T03:40:33.981 INFO:tasks.workunit.client.0.smithi094.stdout:9/291: sync 2024-04-06T03:40:33.981 INFO:tasks.workunit.client.0.smithi094.stdout:7/338: sync 2024-04-06T03:40:33.981 INFO:tasks.workunit.client.0.smithi094.stdout:6/279: sync 2024-04-06T03:40:33.981 INFO:tasks.workunit.client.0.smithi094.stdout:7/339: truncate d2/d6/f24 275733 0 2024-04-06T03:40:33.981 INFO:tasks.workunit.client.0.smithi094.stdout:6/280: creat dc/d11/d22/d50/d61/f6c x:0 0 0 2024-04-06T03:40:33.981 INFO:tasks.workunit.client.0.smithi094.stdout:6/281: read dc/d11/d1b/d24/f32 [511140,3170] 0 2024-04-06T03:40:33.981 INFO:tasks.workunit.client.0.smithi094.stdout:6/282: truncate dc/fd 4259298 0 2024-04-06T03:40:33.982 INFO:tasks.workunit.client.0.smithi094.stdout:2/399: sync 2024-04-06T03:40:33.982 INFO:tasks.workunit.client.0.smithi094.stdout:2/400: fsync d0/d8/d27/f78 0 2024-04-06T03:40:33.982 INFO:tasks.workunit.client.0.smithi094.stdout:2/401: chown d0/f1 13980 1 2024-04-06T03:40:33.982 INFO:tasks.workunit.client.0.smithi094.stdout:5/264: creat db/d19/d2c/d5b/f5f x:0 0 0 2024-04-06T03:40:33.982 INFO:tasks.workunit.client.0.smithi094.stdout:3/404: dwrite d9/f15 [0,4194304] 0 2024-04-06T03:40:33.983 INFO:tasks.workunit.client.1.smithi136.stdout:9/324: creat d17/d21/d32/d43/d51/f6d x:0 0 0 2024-04-06T03:40:33.984 INFO:tasks.workunit.client.0.smithi094.stdout:8/373: mknod d18/d2c/c81 0 2024-04-06T03:40:33.985 INFO:tasks.workunit.client.1.smithi136.stdout:0/294: creat d1/d6/d2c/d57/d43/f60 x:0 0 0 2024-04-06T03:40:33.985 INFO:tasks.workunit.client.1.smithi136.stdout:0/295: readlink d1/d6/d2c/d1a/l32 0 2024-04-06T03:40:33.987 INFO:tasks.workunit.client.0.smithi094.stdout:5/265: dread db/d19/d30/f44 [0,4194304] 0 2024-04-06T03:40:33.989 INFO:tasks.workunit.client.1.smithi136.stdout:8/352: creat d3/f69 x:0 0 0 2024-04-06T03:40:33.991 INFO:tasks.workunit.client.0.smithi094.stdout:9/292: creat d2/dc/d10/f66 x:0 0 0 2024-04-06T03:40:33.991 INFO:tasks.workunit.client.0.smithi094.stdout:9/293: dread - d2/d2c/d33/f44 zero size 2024-04-06T03:40:33.991 INFO:tasks.workunit.client.0.smithi094.stdout:9/294: write d2/f1d [787704,122659] 0 2024-04-06T03:40:33.991 INFO:tasks.workunit.client.0.smithi094.stdout:9/295: dread - d2/dc/f61 zero size 2024-04-06T03:40:33.994 INFO:tasks.workunit.client.1.smithi136.stdout:2/435: symlink d3/d9/d3b/d70/l92 0 2024-04-06T03:40:33.995 INFO:tasks.workunit.client.0.smithi094.stdout:2/402: dread d0/dd/d19/d31/f7d [0,4194304] 0 2024-04-06T03:40:33.997 INFO:tasks.workunit.client.0.smithi094.stdout:7/340: mknod d2/d6/dc/d12/d3a/d56/c64 0 2024-04-06T03:40:34.001 INFO:tasks.workunit.client.0.smithi094.stdout:6/283: link dc/d11/d1b/d24/d46/l59 dc/d3a/l6d 0 2024-04-06T03:40:34.004 INFO:tasks.workunit.client.0.smithi094.stdout:3/405: mknod d9/d20/d2d/c94 0 2024-04-06T03:40:34.004 INFO:tasks.workunit.client.0.smithi094.stdout:7/341: dread d2/d6/f30 [0,4194304] 0 2024-04-06T03:40:34.004 INFO:tasks.workunit.client.0.smithi094.stdout:7/342: write d2/f11 [5026259,89167] 0 2024-04-06T03:40:34.006 INFO:tasks.workunit.client.1.smithi136.stdout:0/296: dwrite d1/d6/d2c/d57/f46 [0,4194304] 0 2024-04-06T03:40:34.006 INFO:tasks.workunit.client.1.smithi136.stdout:0/297: chown d1 0 1 2024-04-06T03:40:34.006 INFO:tasks.workunit.client.1.smithi136.stdout:0/298: fdatasync d1/d6/f45 0 2024-04-06T03:40:34.007 INFO:tasks.workunit.client.0.smithi094.stdout:8/374: creat d18/f82 x:0 0 0 2024-04-06T03:40:34.009 INFO:tasks.workunit.client.1.smithi136.stdout:5/365: creat d1/dc/d1c/d32/f6a x:0 0 0 2024-04-06T03:40:34.011 INFO:tasks.workunit.client.1.smithi136.stdout:7/319: rmdir d5/df/d53 39 2024-04-06T03:40:34.014 INFO:tasks.workunit.client.1.smithi136.stdout:9/325: mkdir d17/d21/d32/d3f/d6e 0 2024-04-06T03:40:34.016 INFO:tasks.workunit.client.0.smithi094.stdout:2/403: mknod d0/d8/d27/c80 0 2024-04-06T03:40:34.016 INFO:tasks.workunit.client.0.smithi094.stdout:9/296: dwrite d2/d2c/d33/f4c [4194304,4194304] 0 2024-04-06T03:40:34.016 INFO:tasks.workunit.client.0.smithi094.stdout:2/404: dread d0/dd/d19/d31/d67/f6c [0,4194304] 0 2024-04-06T03:40:34.017 INFO:tasks.workunit.client.1.smithi136.stdout:8/353: creat d3/de/d18/f6a x:0 0 0 2024-04-06T03:40:34.020 INFO:tasks.workunit.client.0.smithi094.stdout:3/406: mknod d9/d10/c95 0 2024-04-06T03:40:34.020 INFO:tasks.workunit.client.1.smithi136.stdout:2/436: dwrite d3/d9/d10/d18/f81 [0,4194304] 0 2024-04-06T03:40:34.021 INFO:tasks.workunit.client.1.smithi136.stdout:0/299: link d1/d6/d2c/l41 d1/d6/l61 0 2024-04-06T03:40:34.021 INFO:tasks.workunit.client.0.smithi094.stdout:6/284: dwrite dc/d11/f4b [0,4194304] 0 2024-04-06T03:40:34.023 INFO:tasks.workunit.client.1.smithi136.stdout:5/366: rename d1/dc/c3e to d1/dc/d15/d20/d66/c6b 0 2024-04-06T03:40:34.025 INFO:tasks.workunit.client.1.smithi136.stdout:8/354: creat d3/da/d26/d35/d3d/f6b x:0 0 0 2024-04-06T03:40:34.031 INFO:tasks.workunit.client.0.smithi094.stdout:2/405: dread d0/f5 [4194304,4194304] 0 2024-04-06T03:40:34.032 INFO:tasks.workunit.client.1.smithi136.stdout:0/300: mkdir d1/d6/d2c/d57/d62 0 2024-04-06T03:40:34.036 INFO:tasks.workunit.client.1.smithi136.stdout:3/276: sync 2024-04-06T03:40:34.036 INFO:tasks.workunit.client.1.smithi136.stdout:3/277: creat d0/d4/d12/f5c x:0 0 0 2024-04-06T03:40:34.036 INFO:tasks.workunit.client.1.smithi136.stdout:4/281: sync 2024-04-06T03:40:34.036 INFO:tasks.workunit.client.1.smithi136.stdout:4/282: chown d1d/d31/l47 4381662 1 2024-04-06T03:40:34.036 INFO:tasks.workunit.client.1.smithi136.stdout:6/279: sync 2024-04-06T03:40:34.036 INFO:tasks.workunit.client.1.smithi136.stdout:6/280: chown d2/de/d3d/l55 1 1 2024-04-06T03:40:34.036 INFO:tasks.workunit.client.1.smithi136.stdout:6/281: dread - d2/d7/d26/d30/d4f/f57 zero size 2024-04-06T03:40:34.036 INFO:tasks.workunit.client.1.smithi136.stdout:9/326: rename d17/d20/d25/d3d/d4b/l6a to d17/l6f 0 2024-04-06T03:40:34.037 INFO:tasks.workunit.client.1.smithi136.stdout:3/278: dread d0/f32 [0,4194304] 0 2024-04-06T03:40:34.038 INFO:tasks.workunit.client.0.smithi094.stdout:0/334: rename d1/d9/dd/d18/d2d to d1/d9/dd/d18/d5d/d60/d6b 0 2024-04-06T03:40:34.038 INFO:tasks.workunit.client.0.smithi094.stdout:0/335: write d1/d10/f4f [746350,109312] 0 2024-04-06T03:40:34.038 INFO:tasks.workunit.client.0.smithi094.stdout:0/336: read - d1/d9/f5e zero size 2024-04-06T03:40:34.040 INFO:tasks.workunit.client.1.smithi136.stdout:0/301: creat d1/f63 x:0 0 0 2024-04-06T03:40:34.041 INFO:tasks.workunit.client.0.smithi094.stdout:7/343: dread d2/f11 [0,4194304] 0 2024-04-06T03:40:34.041 INFO:tasks.workunit.client.0.smithi094.stdout:7/344: write d2/d6/dc/f15 [284430,10818] 0 2024-04-06T03:40:34.041 INFO:tasks.workunit.client.0.smithi094.stdout:7/345: stat d2/d6/dc/dd/df/d36/f4d 0 2024-04-06T03:40:34.041 INFO:tasks.workunit.client.0.smithi094.stdout:7/346: chown d2/d6/dc/d12/d61 0 1 2024-04-06T03:40:34.041 INFO:tasks.workunit.client.0.smithi094.stdout:7/347: write d2/d6/fa [1968975,41458] 0 2024-04-06T03:40:34.041 INFO:tasks.workunit.client.0.smithi094.stdout:7/348: chown d2/d6/dc/dd/df/c1d 5 1 2024-04-06T03:40:34.042 INFO:tasks.workunit.client.0.smithi094.stdout:5/266: mknod db/c60 0 2024-04-06T03:40:34.044 INFO:tasks.workunit.client.0.smithi094.stdout:3/407: mknod d9/d20/c96 0 2024-04-06T03:40:34.044 INFO:tasks.workunit.client.1.smithi136.stdout:9/327: dread fb [0,4194304] 0 2024-04-06T03:40:34.044 INFO:tasks.workunit.client.1.smithi136.stdout:9/328: truncate d17/d19/f61 1084131 0 2024-04-06T03:40:34.044 INFO:tasks.workunit.client.1.smithi136.stdout:9/329: chown l14 530179046 1 2024-04-06T03:40:34.047 INFO:tasks.workunit.client.1.smithi136.stdout:3/279: dread d0/f2 [0,4194304] 0 2024-04-06T03:40:34.050 INFO:tasks.workunit.client.1.smithi136.stdout:0/302: dread d1/d6/d2c/d1a/d36/d59/f23 [0,4194304] 0 2024-04-06T03:40:34.051 INFO:tasks.workunit.client.1.smithi136.stdout:5/367: dwrite d1/dc/d10/f11 [4194304,4194304] 0 2024-04-06T03:40:34.051 INFO:tasks.workunit.client.1.smithi136.stdout:5/368: stat d1/d16/d4b/l5a 0 2024-04-06T03:40:34.054 INFO:tasks.workunit.client.0.smithi094.stdout:6/285: unlink dc/c14 0 2024-04-06T03:40:34.054 INFO:tasks.workunit.client.0.smithi094.stdout:6/286: truncate dc/d11/d22/d50/f69 902113 0 2024-04-06T03:40:34.058 INFO:tasks.workunit.client.0.smithi094.stdout:8/375: symlink d18/d30/d68/l83 0 2024-04-06T03:40:34.060 INFO:tasks.workunit.client.1.smithi136.stdout:3/280: mkdir d0/d4/d12/d3a/d43/d5d 0 2024-04-06T03:40:34.060 INFO:tasks.workunit.client.1.smithi136.stdout:3/281: chown d0/fb 27825 1 2024-04-06T03:40:34.065 INFO:tasks.workunit.client.0.smithi094.stdout:6/287: write dc/d11/d22/f34 [3279880,41848] 0 2024-04-06T03:40:34.066 INFO:tasks.workunit.client.0.smithi094.stdout:9/297: rename d2/dc/c31 to d2/dc/d21/d23/d47/c67 0 2024-04-06T03:40:34.066 INFO:tasks.workunit.client.1.smithi136.stdout:8/355: dwrite d3/da/d26/d35/d3d/f6b [0,4194304] 0 2024-04-06T03:40:34.067 INFO:tasks.workunit.client.1.smithi136.stdout:5/369: symlink d1/dc/d10/l6c 0 2024-04-06T03:40:34.069 INFO:tasks.workunit.client.0.smithi094.stdout:0/337: truncate d1/d9/dd/d18/d1a/f54 213237 0 2024-04-06T03:40:34.069 INFO:tasks.workunit.client.0.smithi094.stdout:0/338: readlink d1/l6 0 2024-04-06T03:40:34.071 INFO:tasks.workunit.client.1.smithi136.stdout:2/437: rename d3/d45 to d3/d9/d10/d8a/d93 0 2024-04-06T03:40:34.075 INFO:tasks.workunit.client.0.smithi094.stdout:9/298: write d2/dc/d21/f43 [7828039,72862] 0 2024-04-06T03:40:34.075 INFO:tasks.workunit.client.0.smithi094.stdout:9/299: write d2/f30 [470451,113967] 0 2024-04-06T03:40:34.076 INFO:tasks.workunit.client.0.smithi094.stdout:7/349: dwrite d2/d6/dc/dd/df/d36/f60 [0,4194304] 0 2024-04-06T03:40:34.076 INFO:tasks.workunit.client.0.smithi094.stdout:3/408: dwrite d9/d1e/d26/f2c [0,4194304] 0 2024-04-06T03:40:34.077 INFO:tasks.workunit.client.1.smithi136.stdout:9/330: dwrite d17/d21/d32/d43/d56/f67 [0,4194304] 0 2024-04-06T03:40:34.078 INFO:tasks.workunit.client.1.smithi136.stdout:4/283: dwrite d1d/d2f/f53 [0,4194304] 0 2024-04-06T03:40:34.079 INFO:tasks.workunit.client.0.smithi094.stdout:5/267: mkdir db/d19/d21/d61 0 2024-04-06T03:40:34.082 INFO:tasks.workunit.client.0.smithi094.stdout:8/376: creat d18/d2c/d37/d60/d70/f84 x:0 0 0 2024-04-06T03:40:34.088 INFO:tasks.workunit.client.1.smithi136.stdout:8/356: unlink d3/da/d26/d35/d3d/c45 0 2024-04-06T03:40:34.089 INFO:tasks.workunit.client.0.smithi094.stdout:2/406: rename d0/f1 to d0/dd/d19/f81 0 2024-04-06T03:40:34.091 INFO:tasks.workunit.client.1.smithi136.stdout:5/370: creat d1/dc/d10/f6d x:0 0 0 2024-04-06T03:40:34.093 INFO:tasks.workunit.client.0.smithi094.stdout:0/339: unlink d1/c11 0 2024-04-06T03:40:34.095 INFO:tasks.workunit.client.0.smithi094.stdout:0/340: dread d1/d9/dd/d18/f31 [0,4194304] 0 2024-04-06T03:40:34.099 INFO:tasks.workunit.client.1.smithi136.stdout:0/303: rename d1/d6/d2c/d57/c33 to d1/d6/d2c/d57/d43/d4a/c64 0 2024-04-06T03:40:34.100 INFO:tasks.workunit.client.0.smithi094.stdout:9/300: creat d2/dc/f68 x:0 0 0 2024-04-06T03:40:34.101 INFO:tasks.workunit.client.1.smithi136.stdout:2/438: rmdir d3/d15/d19/d43 39 2024-04-06T03:40:34.104 INFO:tasks.workunit.client.1.smithi136.stdout:2/439: dread d3/d15/d19/f60 [0,4194304] 0 2024-04-06T03:40:34.106 INFO:tasks.workunit.client.0.smithi094.stdout:7/350: mknod d2/d6/dc/d12/d61/c65 0 2024-04-06T03:40:34.106 INFO:tasks.workunit.client.0.smithi094.stdout:7/351: creat d2/d6/dc/dd/df/f66 x:0 0 0 2024-04-06T03:40:34.107 INFO:tasks.workunit.client.0.smithi094.stdout:3/409: stat d9/d20/d76/d6b/l78 0 2024-04-06T03:40:34.107 INFO:tasks.workunit.client.0.smithi094.stdout:3/410: chown d9/d10/d27/d3a/f40 7 1 2024-04-06T03:40:34.107 INFO:tasks.workunit.client.0.smithi094.stdout:3/411: write d9/f8d [360767,74104] 0 2024-04-06T03:40:34.108 INFO:tasks.workunit.client.0.smithi094.stdout:5/268: truncate db/d19/d24/f37 29161 0 2024-04-06T03:40:34.109 INFO:tasks.workunit.client.0.smithi094.stdout:5/269: chown db/d19/d24 37 1 2024-04-06T03:40:34.110 INFO:tasks.workunit.client.0.smithi094.stdout:8/377: mknod d18/d2c/d62/c85 0 2024-04-06T03:40:34.112 INFO:tasks.workunit.client.1.smithi136.stdout:8/357: mkdir d3/da/d26/d35/d3d/d48/d6c 0 2024-04-06T03:40:34.112 INFO:tasks.workunit.client.0.smithi094.stdout:2/407: mknod d0/d8/d27/c82 0 2024-04-06T03:40:34.113 INFO:tasks.workunit.client.1.smithi136.stdout:5/371: creat d1/dc/f6e x:0 0 0 2024-04-06T03:40:34.114 INFO:tasks.workunit.client.0.smithi094.stdout:0/341: truncate d1/f2c 2446868 0 2024-04-06T03:40:34.114 INFO:tasks.workunit.client.0.smithi094.stdout:0/342: chown d1/d9/dd/d18/d32/d42/f4a 3 1 2024-04-06T03:40:34.115 INFO:tasks.workunit.client.1.smithi136.stdout:0/304: truncate d1/d6/d2c/f40 3604726 0 2024-04-06T03:40:34.116 INFO:tasks.workunit.client.0.smithi094.stdout:9/301: mkdir d2/d2c/d33/d5a/d69 0 2024-04-06T03:40:34.117 INFO:tasks.workunit.client.0.smithi094.stdout:6/288: getdents dc/d3a 0 2024-04-06T03:40:34.117 INFO:tasks.workunit.client.0.smithi094.stdout:6/289: dread - dc/d3a/d4a/f56 zero size 2024-04-06T03:40:34.119 INFO:tasks.workunit.client.1.smithi136.stdout:2/440: creat d3/d15/d19/d43/f94 x:0 0 0 2024-04-06T03:40:34.119 INFO:tasks.workunit.client.1.smithi136.stdout:2/441: creat d3/d9/d10/d18/d4b/f95 x:0 0 0 2024-04-06T03:40:34.122 INFO:tasks.workunit.client.0.smithi094.stdout:2/408: write d0/d18/f55 [715785,49298] 0 2024-04-06T03:40:34.123 INFO:tasks.workunit.client.1.smithi136.stdout:4/284: unlink c15 0 2024-04-06T03:40:34.123 INFO:tasks.workunit.client.1.smithi136.stdout:4/285: fsync d1d/d2f/f60 0 2024-04-06T03:40:34.124 INFO:tasks.workunit.client.0.smithi094.stdout:7/352: truncate d2/d6/dc/dd/f28 3333152 0 2024-04-06T03:40:34.124 INFO:tasks.workunit.client.0.smithi094.stdout:7/353: chown d2/d6/dc/c21 767624801 1 2024-04-06T03:40:34.124 INFO:tasks.workunit.client.0.smithi094.stdout:7/354: chown d2/d6/dc/d12/l17 2175 1 2024-04-06T03:40:34.126 INFO:tasks.workunit.client.1.smithi136.stdout:8/358: mkdir d3/de/d18/d6d 0 2024-04-06T03:40:34.126 INFO:tasks.workunit.client.1.smithi136.stdout:5/372: write d1/dc/d10/d1e/f2f [7728698,20975] 0 2024-04-06T03:40:34.127 INFO:tasks.workunit.client.0.smithi094.stdout:3/412: mkdir d9/d10/d27/d97 0 2024-04-06T03:40:34.127 INFO:tasks.workunit.client.0.smithi094.stdout:3/413: truncate d9/d1d/f45 1371718 0 2024-04-06T03:40:34.128 INFO:tasks.workunit.client.0.smithi094.stdout:6/290: dread dc/d11/d1b/d24/f2e [4194304,4194304] 0 2024-04-06T03:40:34.128 INFO:tasks.workunit.client.0.smithi094.stdout:6/291: write dc/d11/d22/d50/d61/f68 [613657,125146] 0 2024-04-06T03:40:34.129 INFO:tasks.workunit.client.1.smithi136.stdout:0/305: creat d1/d6/d2c/d57/d62/f65 x:0 0 0 2024-04-06T03:40:34.129 INFO:tasks.workunit.client.1.smithi136.stdout:0/306: write d1/d6/d2c/d57/d62/f65 [356920,25379] 0 2024-04-06T03:40:34.130 INFO:tasks.workunit.client.0.smithi094.stdout:6/292: dread dc/f40 [0,4194304] 0 2024-04-06T03:40:34.130 INFO:tasks.workunit.client.0.smithi094.stdout:6/293: readlink dc/l15 0 2024-04-06T03:40:34.133 INFO:tasks.workunit.client.0.smithi094.stdout:8/378: symlink d18/d5c/d75/l86 0 2024-04-06T03:40:34.134 INFO:tasks.workunit.client.1.smithi136.stdout:2/442: symlink d3/d9/d10/d18/l96 0 2024-04-06T03:40:34.135 INFO:tasks.workunit.client.0.smithi094.stdout:9/302: write d2/dc/d46/f4e [1875305,66867] 0 2024-04-06T03:40:34.135 INFO:tasks.workunit.client.0.smithi094.stdout:0/343: dwrite d1/d9/dd/f3b [0,4194304] 0 2024-04-06T03:40:34.135 INFO:tasks.workunit.client.0.smithi094.stdout:0/344: readlink d1/d9/dd/d18/d1d/l39 0 2024-04-06T03:40:34.136 INFO:tasks.workunit.client.1.smithi136.stdout:5/373: fsync d1/dc/d1c/d32/f6a 0 2024-04-06T03:40:34.137 INFO:tasks.workunit.client.0.smithi094.stdout:6/294: write fa [1043793,2389] 0 2024-04-06T03:40:34.201 INFO:tasks.workunit.client.1.smithi136.stdout:8/359: rename d3/da/l49 to d3/d2d/l6e 0 2024-04-06T03:40:34.203 INFO:tasks.workunit.client.1.smithi136.stdout:0/307: link d1/d6/d2c/f3d d1/d6/d2c/d57/f66 0 2024-04-06T03:40:34.205 INFO:tasks.workunit.client.1.smithi136.stdout:2/443: symlink d3/l97 0 2024-04-06T03:40:34.207 INFO:tasks.workunit.client.1.smithi136.stdout:4/286: link l19 d1d/d4d/l6b 0 2024-04-06T03:40:34.209 INFO:tasks.workunit.client.1.smithi136.stdout:5/374: unlink d1/dc/d10/f49 0 2024-04-06T03:40:34.212 INFO:tasks.workunit.client.0.smithi094.stdout:2/409: rename d0/dd/d3d/d49/c50 to d0/d8/d4f/d70/c83 0 2024-04-06T03:40:34.214 INFO:tasks.workunit.client.0.smithi094.stdout:7/355: mknod d2/d6/dc/c67 0 2024-04-06T03:40:34.214 INFO:tasks.workunit.client.0.smithi094.stdout:7/356: write d2/d6/dc/dd/f2f [2503231,98031] 0 2024-04-06T03:40:34.215 INFO:tasks.workunit.client.0.smithi094.stdout:7/357: creat d2/d6/dc/dd/df/d36/f68 x:0 0 0 2024-04-06T03:40:34.215 INFO:tasks.workunit.client.0.smithi094.stdout:7/358: creat d2/d6/dc/dd/df/d36/f69 x:0 0 0 2024-04-06T03:40:34.219 INFO:tasks.workunit.client.1.smithi136.stdout:8/360: symlink d3/de/d18/l6f 0 2024-04-06T03:40:34.219 INFO:tasks.workunit.client.1.smithi136.stdout:8/361: getdents d3/de/d18/d6d 0 2024-04-06T03:40:34.219 INFO:tasks.workunit.client.1.smithi136.stdout:5/375: dread d1/d16/f1b [0,4194304] 0 2024-04-06T03:40:34.219 INFO:tasks.workunit.client.0.smithi094.stdout:9/303: mkdir d2/dc/d10/d6a 0 2024-04-06T03:40:34.220 INFO:tasks.workunit.client.0.smithi094.stdout:9/304: write d2/f1d [122357,100868] 0 2024-04-06T03:40:34.220 INFO:tasks.workunit.client.1.smithi136.stdout:4/287: dwrite d1d/f49 [0,4194304] 0 2024-04-06T03:40:34.220 INFO:tasks.workunit.client.1.smithi136.stdout:0/308: truncate d1/d6/d2c/d1a/d36/f44 881898 0 2024-04-06T03:40:34.220 INFO:tasks.workunit.client.1.smithi136.stdout:0/309: write d1/d6/d2c/d1a/f28 [1786790,117549] 0 2024-04-06T03:40:34.222 INFO:tasks.workunit.client.0.smithi094.stdout:6/295: mknod dc/c6e 0 2024-04-06T03:40:34.223 INFO:tasks.workunit.client.1.smithi136.stdout:5/376: mknod d1/d16/d17/c6f 0 2024-04-06T03:40:34.223 INFO:tasks.workunit.client.1.smithi136.stdout:5/377: read - d1/dc/d10/d1e/f69 zero size 2024-04-06T03:40:34.223 INFO:tasks.workunit.client.0.smithi094.stdout:2/410: mknod d0/dd/c84 0 2024-04-06T03:40:34.227 INFO:tasks.workunit.client.1.smithi136.stdout:4/288: mkdir d1d/d2f/d6c 0 2024-04-06T03:40:34.228 INFO:tasks.workunit.client.1.smithi136.stdout:4/289: write d1d/d4d/f6a [699917,106056] 0 2024-04-06T03:40:34.229 INFO:tasks.workunit.client.1.smithi136.stdout:0/310: creat d1/d6/d2c/d57/d43/d4a/f67 x:0 0 0 2024-04-06T03:40:34.230 INFO:tasks.workunit.client.1.smithi136.stdout:0/311: chown d1/d6/c2b 2303 1 2024-04-06T03:40:34.230 INFO:tasks.workunit.client.0.smithi094.stdout:8/379: truncate d18/f20 796496 0 2024-04-06T03:40:34.231 INFO:tasks.workunit.client.0.smithi094.stdout:0/345: rmdir d1/d9/dd/d18/d5d/d60/d6b 39 2024-04-06T03:40:34.232 INFO:tasks.workunit.client.0.smithi094.stdout:0/346: creat d1/d9/dd/d18/d32/f6c x:0 0 0 2024-04-06T03:40:34.232 INFO:tasks.workunit.client.0.smithi094.stdout:9/305: dread d2/d2c/f41 [0,4194304] 0 2024-04-06T03:40:34.234 INFO:tasks.workunit.client.0.smithi094.stdout:6/296: mknod dc/c6f 0 2024-04-06T03:40:34.236 INFO:tasks.workunit.client.1.smithi136.stdout:0/312: link d1/d6/d2c/c17 d1/d6/d2c/c68 0 2024-04-06T03:40:34.236 INFO:tasks.workunit.client.1.smithi136.stdout:8/362: dwrite d3/da/fb [0,4194304] 0 2024-04-06T03:40:34.237 INFO:tasks.workunit.client.0.smithi094.stdout:2/411: symlink d0/d8/l85 0 2024-04-06T03:40:34.238 INFO:tasks.workunit.client.0.smithi094.stdout:5/270: getdents db 0 2024-04-06T03:40:34.238 INFO:tasks.workunit.client.0.smithi094.stdout:5/271: readlink db/d19/d30/l4c 0 2024-04-06T03:40:34.238 INFO:tasks.workunit.client.0.smithi094.stdout:5/272: fsync db/d19/d21/f49 0 2024-04-06T03:40:34.238 INFO:tasks.workunit.client.0.smithi094.stdout:5/273: dread - db/d19/d2c/d5b/f5f zero size 2024-04-06T03:40:34.242 INFO:tasks.workunit.client.1.smithi136.stdout:4/290: link d1d/d31/d3e/l4a d1d/l6d 0 2024-04-06T03:40:34.245 INFO:tasks.workunit.client.1.smithi136.stdout:0/313: symlink d1/d6/d2c/d57/d43/l69 0 2024-04-06T03:40:34.245 INFO:tasks.workunit.client.1.smithi136.stdout:0/314: readlink d1/d6/d2c/l3b 0 2024-04-06T03:40:34.247 INFO:tasks.workunit.client.1.smithi136.stdout:8/363: mkdir d3/da/d26/d35/d39/d70 0 2024-04-06T03:40:34.247 INFO:tasks.workunit.client.1.smithi136.stdout:8/364: dread - d3/f69 zero size 2024-04-06T03:40:34.247 INFO:tasks.workunit.client.1.smithi136.stdout:8/365: truncate d3/da/fd 992845 0 2024-04-06T03:40:34.249 INFO:tasks.workunit.client.1.smithi136.stdout:4/291: link d1d/d31/d3e/d56/f61 d1d/d31/d3e/d56/f6e 0 2024-04-06T03:40:34.250 INFO:tasks.workunit.client.0.smithi094.stdout:0/347: rmdir d1/d10/d4e 39 2024-04-06T03:40:34.252 INFO:tasks.workunit.client.1.smithi136.stdout:0/315: creat d1/d6/d2c/d57/d43/d4a/d5c/f6a x:0 0 0 2024-04-06T03:40:34.252 INFO:tasks.workunit.client.1.smithi136.stdout:5/378: dwrite d1/f61 [0,4194304] 0 2024-04-06T03:40:34.253 INFO:tasks.workunit.client.0.smithi094.stdout:2/412: dwrite d0/d8/d27/d2b/f52 [0,4194304] 0 2024-04-06T03:40:34.253 INFO:tasks.workunit.client.0.smithi094.stdout:6/297: dwrite dc/d11/f20 [4194304,4194304] 0 2024-04-06T03:40:34.253 INFO:tasks.workunit.client.0.smithi094.stdout:6/298: chown dc/fd 3173963 1 2024-04-06T03:40:34.253 INFO:tasks.workunit.client.0.smithi094.stdout:6/299: dread - dc/d11/d1b/f52 zero size 2024-04-06T03:40:34.253 INFO:tasks.workunit.client.0.smithi094.stdout:6/300: dread - dc/d11/d1b/d24/d46/f55 zero size 2024-04-06T03:40:34.254 INFO:tasks.workunit.client.0.smithi094.stdout:9/306: mkdir d2/dc/d10/d2a/d2d/d6b 0 2024-04-06T03:40:34.255 INFO:tasks.workunit.client.1.smithi136.stdout:8/366: mknod d3/da/d26/d35/d39/c71 0 2024-04-06T03:40:34.255 INFO:tasks.workunit.client.1.smithi136.stdout:8/367: dread - d3/da/d26/f50 zero size 2024-04-06T03:40:34.257 INFO:tasks.workunit.client.0.smithi094.stdout:7/359: rmdir d2/d6/dc/d12/d51 0 2024-04-06T03:40:34.258 INFO:tasks.workunit.client.1.smithi136.stdout:4/292: chown l16 29767 1 2024-04-06T03:40:34.258 INFO:tasks.workunit.client.1.smithi136.stdout:4/293: dread - d1d/d31/f3b zero size 2024-04-06T03:40:34.259 INFO:tasks.workunit.client.0.smithi094.stdout:8/380: rename d18/d2c/d37/f73 to d18/f87 0 2024-04-06T03:40:34.260 INFO:tasks.workunit.client.1.smithi136.stdout:2/444: rmdir d3/d9/d10/d8a/d93 39 2024-04-06T03:40:34.265 INFO:tasks.workunit.client.1.smithi136.stdout:5/379: mkdir d1/dc/d1c/d32/d70 0 2024-04-06T03:40:34.265 INFO:tasks.workunit.client.1.smithi136.stdout:0/316: dwrite d1/d6/d2c/d1a/f1d [0,4194304] 0 2024-04-06T03:40:34.265 INFO:tasks.workunit.client.1.smithi136.stdout:5/380: truncate d1/dc/d1c/f56 139215 0 2024-04-06T03:40:34.267 INFO:tasks.workunit.client.1.smithi136.stdout:8/368: creat d3/da/d26/d35/d3d/d48/f72 x:0 0 0 2024-04-06T03:40:34.273 INFO:tasks.workunit.client.0.smithi094.stdout:0/348: symlink d1/d9/dd/l6d 0 2024-04-06T03:40:34.274 INFO:tasks.workunit.client.1.smithi136.stdout:4/294: dwrite d1d/d4d/f64 [0,4194304] 0 2024-04-06T03:40:34.274 INFO:tasks.workunit.client.1.smithi136.stdout:4/295: creat d1d/d31/d3e/d56/f6f x:0 0 0 2024-04-06T03:40:34.276 INFO:tasks.workunit.client.1.smithi136.stdout:5/381: link d1/dc/d15/c2e d1/dc/d15/d20/c71 0 2024-04-06T03:40:34.279 INFO:tasks.workunit.client.1.smithi136.stdout:0/317: rename d1/d6/d2c/d56 to d1/d6/d2c/d57/d43/d6b 0 2024-04-06T03:40:34.280 INFO:tasks.workunit.client.1.smithi136.stdout:2/445: dwrite d3/f64 [0,4194304] 0 2024-04-06T03:40:34.281 INFO:tasks.workunit.client.1.smithi136.stdout:8/369: mknod d3/de/c73 0 2024-04-06T03:40:34.282 INFO:tasks.workunit.client.0.smithi094.stdout:2/413: creat d0/dd/d19/d31/f86 x:0 0 0 2024-04-06T03:40:34.283 INFO:tasks.workunit.client.0.smithi094.stdout:2/414: dread - d0/dd/d19/d31/f6e zero size 2024-04-06T03:40:34.285 INFO:tasks.workunit.client.1.smithi136.stdout:1/329: sync 2024-04-06T03:40:34.286 INFO:tasks.workunit.client.0.smithi094.stdout:6/301: mknod dc/d11/d22/c70 0 2024-04-06T03:40:34.288 INFO:tasks.workunit.client.0.smithi094.stdout:9/307: mknod d2/d2c/d33/d5a/d69/c6c 0 2024-04-06T03:40:34.290 INFO:tasks.workunit.client.1.smithi136.stdout:5/382: rename d1/l4d to d1/dc/l72 0 2024-04-06T03:40:34.290 INFO:tasks.workunit.client.0.smithi094.stdout:0/349: dwrite d1/d9/dd/d18/d5d/f65 [0,4194304] 0 2024-04-06T03:40:34.296 INFO:tasks.workunit.client.1.smithi136.stdout:0/318: link d1/f4c d1/d6/d2c/d1a/d36/d59/f6c 0 2024-04-06T03:40:34.296 INFO:tasks.workunit.client.1.smithi136.stdout:0/319: stat d1/d6/d2c/d1a/f39 0 2024-04-06T03:40:34.296 INFO:tasks.workunit.client.1.smithi136.stdout:0/320: dread - d1/d6/f58 zero size 2024-04-06T03:40:34.296 INFO:tasks.workunit.client.1.smithi136.stdout:8/370: dwrite d3/de/d18/f1e [0,4194304] 0 2024-04-06T03:40:34.297 INFO:tasks.workunit.client.0.smithi094.stdout:2/415: read d0/dd/d19/f81 [3729308,42672] 0 2024-04-06T03:40:34.298 INFO:tasks.workunit.client.0.smithi094.stdout:7/360: symlink d2/d6/dc/d12/d61/l6a 0 2024-04-06T03:40:34.298 INFO:tasks.workunit.client.1.smithi136.stdout:1/330: dread d12/d3b/d44/d5f/f30 [0,4194304] 0 2024-04-06T03:40:34.300 INFO:tasks.workunit.client.0.smithi094.stdout:0/350: read d1/d9/dd/f3b [980003,23488] 0 2024-04-06T03:40:34.304 INFO:tasks.workunit.client.0.smithi094.stdout:8/381: creat d18/d2c/d37/d60/f88 x:0 0 0 2024-04-06T03:40:34.304 INFO:tasks.workunit.client.0.smithi094.stdout:8/382: chown d18/d34/d52 23 1 2024-04-06T03:40:34.304 INFO:tasks.workunit.client.0.smithi094.stdout:8/383: fdatasync f1 0 2024-04-06T03:40:34.307 INFO:tasks.workunit.client.0.smithi094.stdout:6/302: mkdir dc/d3a/d4a/d54/d71 0 2024-04-06T03:40:34.307 INFO:tasks.workunit.client.0.smithi094.stdout:2/416: dread d0/f64 [0,4194304] 0 2024-04-06T03:40:34.312 INFO:tasks.workunit.client.1.smithi136.stdout:0/321: dwrite d1/d6/d2c/d1a/f4b [0,4194304] 0 2024-04-06T03:40:34.313 INFO:tasks.workunit.client.1.smithi136.stdout:0/322: chown d1/d6/d2c/d1a/f4b 110974127 1 2024-04-06T03:40:34.313 INFO:tasks.workunit.client.1.smithi136.stdout:0/323: dread - d1/d6/d2c/f5a zero size 2024-04-06T03:40:34.313 INFO:tasks.workunit.client.0.smithi094.stdout:7/361: dread d2/d6/dc/dd/df/f2d [0,4194304] 0 2024-04-06T03:40:34.314 INFO:tasks.workunit.client.1.smithi136.stdout:5/383: dwrite d1/dc/d15/d20/f41 [0,4194304] 0 2024-04-06T03:40:34.314 INFO:tasks.workunit.client.0.smithi094.stdout:8/384: write d18/d2c/f47 [2731312,55575] 0 2024-04-06T03:40:34.315 INFO:tasks.workunit.client.0.smithi094.stdout:8/385: creat d18/d2c/d62/f89 x:0 0 0 2024-04-06T03:40:34.315 INFO:tasks.workunit.client.1.smithi136.stdout:2/446: rename d3/c4 to d3/d15/d19/d43/c98 0 2024-04-06T03:40:34.315 INFO:tasks.workunit.client.1.smithi136.stdout:2/447: fdatasync d3/d9/d10/d18/f74 0 2024-04-06T03:40:34.322 INFO:tasks.workunit.client.0.smithi094.stdout:7/362: write d2/d6/dc/f2b [3961512,18901] 0 2024-04-06T03:40:34.322 INFO:tasks.workunit.client.0.smithi094.stdout:7/363: creat d2/d6/dc/d12/d29/d3f/f6b x:0 0 0 2024-04-06T03:40:34.323 INFO:tasks.workunit.client.0.smithi094.stdout:0/351: symlink d1/d9/dd/d18/d1d/l6e 0 2024-04-06T03:40:34.326 INFO:tasks.workunit.client.1.smithi136.stdout:8/371: unlink d3/d2d/l5a 0 2024-04-06T03:40:34.326 INFO:tasks.workunit.client.1.smithi136.stdout:8/372: chown d3/f21 36654 1 2024-04-06T03:40:34.326 INFO:tasks.workunit.client.1.smithi136.stdout:8/373: creat d3/de/d18/f74 x:0 0 0 2024-04-06T03:40:34.326 INFO:tasks.workunit.client.1.smithi136.stdout:8/374: write d3/de/f61 [43631,49068] 0 2024-04-06T03:40:34.327 INFO:tasks.workunit.client.1.smithi136.stdout:2/448: write d3/d15/f2b [454747,54947] 0 2024-04-06T03:40:34.329 INFO:tasks.workunit.client.1.smithi136.stdout:1/331: mkdir d12/d1b/d1e/d6f 0 2024-04-06T03:40:34.329 INFO:tasks.workunit.client.1.smithi136.stdout:1/332: stat d12/d1b/d1e 0 2024-04-06T03:40:34.329 INFO:tasks.workunit.client.1.smithi136.stdout:1/333: write d12/d3b/d44/f67 [421833,42532] 0 2024-04-06T03:40:34.330 INFO:tasks.workunit.client.1.smithi136.stdout:8/375: dread d3/da/fd [0,4194304] 0 2024-04-06T03:40:34.330 INFO:tasks.workunit.client.1.smithi136.stdout:8/376: read - d3/f69 zero size 2024-04-06T03:40:34.332 INFO:tasks.workunit.client.1.smithi136.stdout:2/449: dread d3/d15/d19/f39 [0,4194304] 0 2024-04-06T03:40:34.332 INFO:tasks.workunit.client.1.smithi136.stdout:2/450: readlink d3/d15/d19/d4e/l5c 0 2024-04-06T03:40:34.335 INFO:tasks.workunit.client.0.smithi094.stdout:8/386: creat d18/d2c/d37/d60/d70/d66/f8a x:0 0 0 2024-04-06T03:40:34.335 INFO:tasks.workunit.client.0.smithi094.stdout:8/387: write d18/f82 [413891,2262] 0 2024-04-06T03:40:34.336 INFO:tasks.workunit.client.1.smithi136.stdout:5/384: dwrite d1/f61 [0,4194304] 0 2024-04-06T03:40:34.336 INFO:tasks.workunit.client.1.smithi136.stdout:5/385: dread - d1/d16/f63 zero size 2024-04-06T03:40:34.336 INFO:tasks.workunit.client.1.smithi136.stdout:0/324: dwrite d1/d6/d2c/d1a/d36/d59/f21 [0,4194304] 0 2024-04-06T03:40:34.337 INFO:tasks.workunit.client.0.smithi094.stdout:2/417: dwrite d0/f64 [0,4194304] 0 2024-04-06T03:40:34.338 INFO:tasks.workunit.client.0.smithi094.stdout:2/418: dread d0/d8/d4f/f5b [0,4194304] 0 2024-04-06T03:40:34.338 INFO:tasks.workunit.client.0.smithi094.stdout:2/419: write d0/dd/f62 [119667,78871] 0 2024-04-06T03:40:34.340 INFO:tasks.workunit.client.1.smithi136.stdout:1/334: dread fb [0,4194304] 0 2024-04-06T03:40:34.341 INFO:tasks.workunit.client.1.smithi136.stdout:1/335: fsync d12/d53/f61 0 2024-04-06T03:40:34.342 INFO:tasks.workunit.client.0.smithi094.stdout:7/364: creat d2/f6c x:0 0 0 2024-04-06T03:40:34.343 INFO:tasks.workunit.client.1.smithi136.stdout:6/282: sync 2024-04-06T03:40:34.343 INFO:tasks.workunit.client.1.smithi136.stdout:7/320: sync 2024-04-06T03:40:34.343 INFO:tasks.workunit.client.1.smithi136.stdout:3/282: sync 2024-04-06T03:40:34.343 INFO:tasks.workunit.client.1.smithi136.stdout:3/283: readlink d0/l28 0 2024-04-06T03:40:34.343 INFO:tasks.workunit.client.1.smithi136.stdout:7/321: chown d5/df/d12/l6a 1293 1 2024-04-06T03:40:34.343 INFO:tasks.workunit.client.1.smithi136.stdout:9/331: sync 2024-04-06T03:40:34.343 INFO:tasks.workunit.client.1.smithi136.stdout:3/284: truncate d0/d4/d12/d2d/f33 1663275 0 2024-04-06T03:40:34.343 INFO:tasks.workunit.client.1.smithi136.stdout:8/377: dwrite d3/da/fb [0,4194304] 0 2024-04-06T03:40:34.343 INFO:tasks.workunit.client.1.smithi136.stdout:8/378: stat d3/f10 0 2024-04-06T03:40:34.344 INFO:tasks.workunit.client.0.smithi094.stdout:0/352: truncate d1/d9/f20 680088 0 2024-04-06T03:40:34.347 INFO:tasks.workunit.client.0.smithi094.stdout:6/303: mkdir dc/d11/d1b/d72 0 2024-04-06T03:40:34.347 INFO:tasks.workunit.client.0.smithi094.stdout:6/304: write dc/f28 [123296,64493] 0 2024-04-06T03:40:34.349 INFO:tasks.workunit.client.1.smithi136.stdout:5/386: rmdir d1/d16/d4b/d68 0 2024-04-06T03:40:34.349 INFO:tasks.workunit.client.1.smithi136.stdout:5/387: chown d1/dc/d1c/f2c 5561086 1 2024-04-06T03:40:34.353 INFO:tasks.workunit.client.1.smithi136.stdout:8/379: dread d3/fc [0,4194304] 0 2024-04-06T03:40:34.353 INFO:tasks.workunit.client.1.smithi136.stdout:8/380: fdatasync d3/da/d26/f28 0 2024-04-06T03:40:34.358 INFO:tasks.workunit.client.0.smithi094.stdout:2/420: unlink d0/c56 0 2024-04-06T03:40:34.358 INFO:tasks.workunit.client.0.smithi094.stdout:2/421: chown d0/dd/d19/d31/f7d 1 1 2024-04-06T03:40:34.358 INFO:tasks.workunit.client.0.smithi094.stdout:2/422: fsync d0/f57 0 2024-04-06T03:40:34.359 INFO:tasks.workunit.client.0.smithi094.stdout:2/423: dread - d0/d8/d1d/f60 zero size 2024-04-06T03:40:34.361 INFO:tasks.workunit.client.0.smithi094.stdout:7/365: unlink d2/d6/dc/d12/l17 0 2024-04-06T03:40:34.361 INFO:tasks.workunit.client.0.smithi094.stdout:7/366: write d2/d6/dc/dd/df/f1a [2412267,55636] 0 2024-04-06T03:40:34.363 INFO:tasks.workunit.client.1.smithi136.stdout:3/285: dwrite d0/d4/d12/d2d/f33 [0,4194304] 0 2024-04-06T03:40:34.364 INFO:tasks.workunit.client.1.smithi136.stdout:9/332: dwrite d17/d21/d29/f2a [0,4194304] 0 2024-04-06T03:40:34.365 INFO:tasks.workunit.client.1.smithi136.stdout:4/296: rename d1d/d4d to d1d/d70 0 2024-04-06T03:40:34.365 INFO:tasks.workunit.client.1.smithi136.stdout:4/297: creat d1d/d31/d3e/f71 x:0 0 0 2024-04-06T03:40:34.367 INFO:tasks.workunit.client.0.smithi094.stdout:6/305: creat dc/d11/d22/d50/d61/f73 x:0 0 0 2024-04-06T03:40:34.369 INFO:tasks.workunit.client.0.smithi094.stdout:8/388: rmdir d18/d3c/d49 39 2024-04-06T03:40:34.369 INFO:tasks.workunit.client.0.smithi094.stdout:8/389: write d18/f82 [79456,16120] 0 2024-04-06T03:40:34.370 INFO:tasks.workunit.client.0.smithi094.stdout:8/390: chown d18/d34/d52/f65 24520 1 2024-04-06T03:40:34.373 INFO:tasks.workunit.client.0.smithi094.stdout:2/424: mkdir d0/d8/d1d/d77/d87 0 2024-04-06T03:40:34.374 INFO:tasks.workunit.client.1.smithi136.stdout:9/333: dread d17/d21/d29/f2a [0,4194304] 0 2024-04-06T03:40:34.374 INFO:tasks.workunit.client.0.smithi094.stdout:7/367: rmdir d2/d6 39 2024-04-06T03:40:34.376 INFO:tasks.workunit.client.1.smithi136.stdout:3/286: dwrite d0/f2 [0,4194304] 0 2024-04-06T03:40:34.378 INFO:tasks.workunit.client.0.smithi094.stdout:6/306: fdatasync dc/d11/f20 0 2024-04-06T03:40:34.380 INFO:tasks.workunit.client.0.smithi094.stdout:6/307: dread dc/d11/d1b/d24/f32 [0,4194304] 0 2024-04-06T03:40:34.380 INFO:tasks.workunit.client.0.smithi094.stdout:6/308: chown dc/c6f 7560230 1 2024-04-06T03:40:34.383 INFO:tasks.workunit.client.1.smithi136.stdout:6/283: creat d2/d7/d26/d30/d4f/f5f x:0 0 0 2024-04-06T03:40:34.384 INFO:tasks.workunit.client.1.smithi136.stdout:3/287: dread d0/d4/d12/f20 [0,4194304] 0 2024-04-06T03:40:34.384 INFO:tasks.workunit.client.1.smithi136.stdout:3/288: fsync d0/d4/d12/d2d/f4a 0 2024-04-06T03:40:34.384 INFO:tasks.workunit.client.1.smithi136.stdout:3/289: write d0/d4/d12/d19/f50 [1057142,25597] 0 2024-04-06T03:40:34.384 INFO:tasks.workunit.client.0.smithi094.stdout:2/425: creat d0/dd/d3d/f88 x:0 0 0 2024-04-06T03:40:34.384 INFO:tasks.workunit.client.0.smithi094.stdout:2/426: write d0/dd/d19/f4c [1913771,98097] 0 2024-04-06T03:40:34.387 INFO:tasks.workunit.client.0.smithi094.stdout:1/322: sync 2024-04-06T03:40:34.387 INFO:tasks.workunit.client.0.smithi094.stdout:4/427: sync 2024-04-06T03:40:34.387 INFO:tasks.workunit.client.1.smithi136.stdout:7/322: unlink d5/df/d1d/d6f/f28 0 2024-04-06T03:40:34.388 INFO:tasks.workunit.client.1.smithi136.stdout:7/323: write d5/df/d53/f3c [3864035,58884] 0 2024-04-06T03:40:34.391 INFO:tasks.workunit.client.0.smithi094.stdout:7/368: unlink d2/f1f 0 2024-04-06T03:40:34.392 INFO:tasks.workunit.client.0.smithi094.stdout:7/369: dread - d2/d6/dc/d12/d29/d35/f54 zero size 2024-04-06T03:40:34.397 INFO:tasks.workunit.client.1.smithi136.stdout:5/388: chown d1/cb 2527 1 2024-04-06T03:40:34.397 INFO:tasks.workunit.client.0.smithi094.stdout:6/309: dwrite dc/d11/d1b/d24/f43 [0,4194304] 0 2024-04-06T03:40:34.399 INFO:tasks.workunit.client.0.smithi094.stdout:2/427: dwrite d0/d18/f22 [4194304,4194304] 0 2024-04-06T03:40:34.400 INFO:tasks.workunit.client.1.smithi136.stdout:8/381: creat d3/da/d26/f75 x:0 0 0 2024-04-06T03:40:34.400 INFO:tasks.workunit.client.1.smithi136.stdout:8/382: creat d3/de/d18/f76 x:0 0 0 2024-04-06T03:40:34.401 INFO:tasks.workunit.client.0.smithi094.stdout:1/323: rmdir d0/d4/d1b/d2f/d4b 39 2024-04-06T03:40:34.401 INFO:tasks.workunit.client.0.smithi094.stdout:1/324: write d0/f5d [199029,42580] 0 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:7/324: dwrite d5/df/d3d/f3f [0,4194304] 0 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:8/383: dread d3/da/d26/d35/f3c [0,4194304] 0 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:8/384: write d3/de/d18/f74 [1011103,78851] 0 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:8/385: write d3/da/d26/d35/d3d/d48/f66 [4811701,59508] 0 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:8/386: chown d3/f10 10775238 1 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:8/387: chown d3/c16 3802000 1 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:8/388: fsync d3/de/ff 0 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:8/389: write d3/da/d26/d35/d3d/d48/f72 [735122,65593] 0 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:8/390: chown d3/de/d18/f6a 2003466360 1 2024-04-06T03:40:34.405 INFO:tasks.workunit.client.1.smithi136.stdout:8/391: write d3/de/ff [3608715,88455] 0 2024-04-06T03:40:34.407 INFO:tasks.workunit.client.1.smithi136.stdout:9/334: rename d17/d19/f2f to d17/d21/d32/d43/d56/d60/f70 0 2024-04-06T03:40:34.407 INFO:tasks.workunit.client.1.smithi136.stdout:9/335: chown d17/d21/d29/d34 15 1 2024-04-06T03:40:34.409 INFO:tasks.workunit.client.1.smithi136.stdout:8/392: fdatasync d3/da/d26/d35/d3d/d48/f72 0 2024-04-06T03:40:34.409 INFO:tasks.workunit.client.1.smithi136.stdout:6/284: getdents d2 0 2024-04-06T03:40:34.410 INFO:tasks.workunit.client.1.smithi136.stdout:6/285: dread - d2/d7/d13/f44 zero size 2024-04-06T03:40:34.410 INFO:tasks.workunit.client.1.smithi136.stdout:6/286: creat d2/d7/d26/d30/f60 x:0 0 0 2024-04-06T03:40:34.410 INFO:tasks.workunit.client.1.smithi136.stdout:6/287: stat d2/d25/f59 0 2024-04-06T03:40:34.411 INFO:tasks.workunit.client.0.smithi094.stdout:6/310: creat dc/d11/d1b/f74 x:0 0 0 2024-04-06T03:40:34.411 INFO:tasks.workunit.client.0.smithi094.stdout:6/311: stat c9 0 2024-04-06T03:40:34.411 INFO:tasks.workunit.client.0.smithi094.stdout:6/312: truncate dc/d3a/d4a/f56 380617 0 2024-04-06T03:40:34.417 INFO:tasks.workunit.client.0.smithi094.stdout:1/325: dwrite d0/d4/d1b/d2f/d3c/d48/f69 [0,4194304] 0 2024-04-06T03:40:34.418 INFO:tasks.workunit.client.1.smithi136.stdout:5/389: creat d1/dc/d15/f73 x:0 0 0 2024-04-06T03:40:34.418 INFO:tasks.workunit.client.0.smithi094.stdout:2/428: dwrite d0/dd/d19/d31/f7d [4194304,4194304] 0 2024-04-06T03:40:34.418 INFO:tasks.workunit.client.0.smithi094.stdout:2/429: chown d0/dd/d3d/d49/f53 8234160 1 2024-04-06T03:40:34.418 INFO:tasks.workunit.client.0.smithi094.stdout:2/430: write d0/dd/f40 [2058738,77574] 0 2024-04-06T03:40:34.418 INFO:tasks.workunit.client.0.smithi094.stdout:2/431: read - d0/d8/d1d/f79 zero size 2024-04-06T03:40:34.422 INFO:tasks.workunit.client.0.smithi094.stdout:6/313: dwrite dc/d11/d22/f44 [4194304,4194304] 0 2024-04-06T03:40:34.425 INFO:tasks.workunit.client.0.smithi094.stdout:4/428: mknod d0/df/d13/c93 0 2024-04-06T03:40:34.426 INFO:tasks.workunit.client.1.smithi136.stdout:0/325: getdents d1/d6/d2c/d1a/d36/d59 0 2024-04-06T03:40:34.429 INFO:tasks.workunit.client.1.smithi136.stdout:0/326: dread d1/d6/f45 [0,4194304] 0 2024-04-06T03:40:34.429 INFO:tasks.workunit.client.1.smithi136.stdout:0/327: stat d1/d6/d2c/d57/d43/d4a/d5c/f6a 0 2024-04-06T03:40:34.429 INFO:tasks.workunit.client.1.smithi136.stdout:0/328: chown d1/d6/d2c/d1a/l35 6682 1 2024-04-06T03:40:34.432 INFO:tasks.workunit.client.0.smithi094.stdout:1/326: creat d0/d4/d1b/f6f x:0 0 0 2024-04-06T03:40:34.432 INFO:tasks.workunit.client.0.smithi094.stdout:1/327: write d0/d1/d16/f53 [307103,597] 0 2024-04-06T03:40:34.435 INFO:tasks.workunit.client.1.smithi136.stdout:4/298: mkdir d1d/d72 0 2024-04-06T03:40:34.436 INFO:tasks.workunit.client.0.smithi094.stdout:2/432: symlink d0/dd/d19/d31/l89 0 2024-04-06T03:40:34.440 INFO:tasks.workunit.client.1.smithi136.stdout:7/325: rename d5/df/d12/d1b/c31 to d5/df/d1d/d6f/d2a/c73 0 2024-04-06T03:40:34.442 INFO:tasks.workunit.client.1.smithi136.stdout:9/336: stat c12 0 2024-04-06T03:40:34.442 INFO:tasks.workunit.client.1.smithi136.stdout:9/337: truncate d17/d21/d29/f5f 1099442 0 2024-04-06T03:40:34.442 INFO:tasks.workunit.client.1.smithi136.stdout:9/338: chown d17/d21/d32/f4f 0 1 2024-04-06T03:40:34.444 INFO:tasks.workunit.client.0.smithi094.stdout:4/429: mkdir d0/d15/d3a/d94 0 2024-04-06T03:40:34.444 INFO:tasks.workunit.client.0.smithi094.stdout:4/430: creat d0/d15/d3a/d3e/d46/f95 x:0 0 0 2024-04-06T03:40:34.444 INFO:tasks.workunit.client.0.smithi094.stdout:4/431: write d0/d15/d3a/d3e/d49/f73 [843117,44562] 0 2024-04-06T03:40:34.446 INFO:tasks.workunit.client.1.smithi136.stdout:8/393: unlink d3/de/d18/l6f 0 2024-04-06T03:40:34.448 INFO:tasks.workunit.client.0.smithi094.stdout:1/328: mkdir d0/d4/d37/d5f/d70 0 2024-04-06T03:40:34.452 INFO:tasks.workunit.client.1.smithi136.stdout:6/288: symlink d2/l61 0 2024-04-06T03:40:34.455 INFO:tasks.workunit.client.0.smithi094.stdout:1/329: write d0/d1/d16/d24/d30/d38/f4d [296068,130931] 0 2024-04-06T03:40:34.463 INFO:tasks.workunit.client.0.smithi094.stdout:0/353: dread d1/d9/f20 [0,4194304] 0 2024-04-06T03:40:34.465 INFO:tasks.workunit.client.0.smithi094.stdout:0/354: dread d1/d9/dd/d18/d1a/f54 [0,4194304] 0 2024-04-06T03:40:34.465 INFO:tasks.workunit.client.0.smithi094.stdout:0/355: fdatasync d1/d9/dd/f3b 0 2024-04-06T03:40:34.466 INFO:tasks.workunit.client.0.smithi094.stdout:6/314: link dc/d11/l4c dc/d3a/l75 0 2024-04-06T03:40:34.470 INFO:tasks.workunit.client.1.smithi136.stdout:5/390: symlink d1/dc/d1c/d32/d70/l74 0 2024-04-06T03:40:34.471 INFO:tasks.workunit.client.1.smithi136.stdout:0/329: truncate d1/d6/fd 2704194 0 2024-04-06T03:40:34.471 INFO:tasks.workunit.client.1.smithi136.stdout:0/330: readlink d1/d4e/l52 0 2024-04-06T03:40:34.473 INFO:tasks.workunit.client.0.smithi094.stdout:8/391: rename d18/d2c/d37/d60/d70/d66 to d18/d3c/d8b 0 2024-04-06T03:40:34.473 INFO:tasks.workunit.client.0.smithi094.stdout:8/392: write d18/d3c/d8b/f8a [50229,20413] 0 2024-04-06T03:40:34.474 INFO:tasks.workunit.client.0.smithi094.stdout:1/330: mknod d0/d7/d2d/c71 0 2024-04-06T03:40:34.474 INFO:tasks.workunit.client.0.smithi094.stdout:1/331: fdatasync d0/d1/d16/f3a 0 2024-04-06T03:40:34.477 INFO:tasks.workunit.client.0.smithi094.stdout:0/356: link d1/d9/l19 d1/d9/dd/d18/d5d/l6f 0 2024-04-06T03:40:34.479 INFO:tasks.workunit.client.1.smithi136.stdout:7/326: mknod d5/df/d1d/d6f/d2c/c74 0 2024-04-06T03:40:34.481 INFO:tasks.workunit.client.1.smithi136.stdout:9/339: mkdir d17/d20/d25/d71 0 2024-04-06T03:40:34.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/432: mknod d0/df/d1e/c96 0 2024-04-06T03:40:34.487 INFO:tasks.workunit.client.0.smithi094.stdout:4/433: dread d0/d23/f32 [0,4194304] 0 2024-04-06T03:40:34.488 INFO:tasks.workunit.client.1.smithi136.stdout:6/289: rename d2/d7/d4b to d2/de/d58/d5d/d62 0 2024-04-06T03:40:34.489 INFO:tasks.workunit.client.1.smithi136.stdout:9/340: write fa [190662,3833] 0 2024-04-06T03:40:34.490 INFO:tasks.workunit.client.0.smithi094.stdout:2/433: rename d0/d8/d4f/d70/c83 to d0/d18/d3f/c8a 0 2024-04-06T03:40:34.491 INFO:tasks.workunit.client.1.smithi136.stdout:5/391: readlink d1/l13 0 2024-04-06T03:40:34.493 INFO:tasks.workunit.client.1.smithi136.stdout:9/341: fdatasync d17/d20/d25/d3c/d55/f64 0 2024-04-06T03:40:34.493 INFO:tasks.workunit.client.1.smithi136.stdout:9/342: fdatasync d17/d21/d32/d3f/f44 0 2024-04-06T03:40:34.493 INFO:tasks.workunit.client.0.smithi094.stdout:2/434: dread d0/d8/d27/d2b/f2c [0,4194304] 0 2024-04-06T03:40:34.493 INFO:tasks.workunit.client.0.smithi094.stdout:8/393: creat d18/d3c/d76/f8c x:0 0 0 2024-04-06T03:40:34.495 INFO:tasks.workunit.client.1.smithi136.stdout:0/331: mkdir d1/d6/d2c/d57/d43/d6d 0 2024-04-06T03:40:34.497 INFO:tasks.workunit.client.0.smithi094.stdout:1/332: creat d0/d7/d1a/d64/f72 x:0 0 0 2024-04-06T03:40:34.498 INFO:tasks.workunit.client.1.smithi136.stdout:6/290: dread d2/d7/f2c [0,4194304] 0 2024-04-06T03:40:34.498 INFO:tasks.workunit.client.1.smithi136.stdout:6/291: fsync d2/d7/d26/f54 0 2024-04-06T03:40:34.498 INFO:tasks.workunit.client.1.smithi136.stdout:4/299: getdents d1d/d31/d3e/d56 0 2024-04-06T03:40:34.499 INFO:tasks.workunit.client.0.smithi094.stdout:0/357: truncate d1/d9/dd/d18/d1a/f46 3559830 0 2024-04-06T03:40:34.500 INFO:tasks.workunit.client.0.smithi094.stdout:0/358: truncate d1/d9/dd/d18/d1d/f57 1303004 0 2024-04-06T03:40:34.500 INFO:tasks.workunit.client.0.smithi094.stdout:0/359: chown d1/d9/dd/f3b 30 1 2024-04-06T03:40:34.501 INFO:tasks.workunit.client.0.smithi094.stdout:4/434: mknod d0/d15/c97 0 2024-04-06T03:40:34.503 INFO:tasks.workunit.client.1.smithi136.stdout:5/392: dread d1/d16/d17/f1a [0,4194304] 0 2024-04-06T03:40:34.505 INFO:tasks.workunit.client.0.smithi094.stdout:2/435: link d0/dd/d3d/d49/f53 d0/dd/f8b 0 2024-04-06T03:40:34.508 INFO:tasks.workunit.client.0.smithi094.stdout:8/394: link f14 d18/d5c/f8d 0 2024-04-06T03:40:34.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/333: creat d0/d1/f73 x:0 0 0 2024-04-06T03:40:34.513 INFO:tasks.workunit.client.0.smithi094.stdout:1/334: truncate d0/d4/d1b/d2f/d3c/f6c 555692 0 2024-04-06T03:40:34.513 INFO:tasks.workunit.client.1.smithi136.stdout:4/300: dread d1d/d31/d3e/d56/f5e [0,4194304] 0 2024-04-06T03:40:34.513 INFO:tasks.workunit.client.0.smithi094.stdout:2/436: write d0/d8/f11 [1782800,34313] 0 2024-04-06T03:40:34.514 INFO:tasks.workunit.client.0.smithi094.stdout:0/360: symlink d1/d9/dd/d18/d5d/d60/d6b/l70 0 2024-04-06T03:40:34.516 INFO:tasks.workunit.client.1.smithi136.stdout:6/292: dwrite d2/f21 [0,4194304] 0 2024-04-06T03:40:34.517 INFO:tasks.workunit.client.0.smithi094.stdout:4/435: creat d0/df/d13/f98 x:0 0 0 2024-04-06T03:40:34.517 INFO:tasks.workunit.client.0.smithi094.stdout:4/436: truncate d0/df/d13/d4a/f8b 802985 0 2024-04-06T03:40:34.519 INFO:tasks.workunit.client.0.smithi094.stdout:8/395: read d18/f25 [957007,47931] 0 2024-04-06T03:40:34.521 INFO:tasks.workunit.client.0.smithi094.stdout:2/437: unlink d0/d8/d27/l59 0 2024-04-06T03:40:34.521 INFO:tasks.workunit.client.0.smithi094.stdout:2/438: creat d0/dd/d19/d31/f8c x:0 0 0 2024-04-06T03:40:34.522 INFO:tasks.workunit.client.0.smithi094.stdout:2/439: read - d0/dd/d19/d31/f6e zero size 2024-04-06T03:40:34.522 INFO:tasks.workunit.client.0.smithi094.stdout:2/440: fdatasync d0/d8/d4f/f5b 0 2024-04-06T03:40:34.522 INFO:tasks.workunit.client.0.smithi094.stdout:4/437: write d0/df/d1e/f25 [5123990,12073] 0 2024-04-06T03:40:34.522 INFO:tasks.workunit.client.0.smithi094.stdout:4/438: write d0/d15/f40 [3683680,101650] 0 2024-04-06T03:40:34.524 INFO:tasks.workunit.client.0.smithi094.stdout:0/361: link d1/d9/dd/d18/d1e/d22/f6a d1/d9/dd/d18/d1e/f71 0 2024-04-06T03:40:34.524 INFO:tasks.workunit.client.0.smithi094.stdout:0/362: write d1/d9/dd/d18/d1e/d22/f37 [1849283,20669] 0 2024-04-06T03:40:34.525 INFO:tasks.workunit.client.1.smithi136.stdout:3/290: truncate d0/d4/d12/d2d/f33 872054 0 2024-04-06T03:40:34.525 INFO:tasks.workunit.client.1.smithi136.stdout:3/291: chown d0/d4/d12/d19/d2e 861522763 1 2024-04-06T03:40:34.525 INFO:tasks.workunit.client.1.smithi136.stdout:3/292: fdatasync d0/f1e 0 2024-04-06T03:40:34.525 INFO:tasks.workunit.client.1.smithi136.stdout:3/293: write d0/f32 [508046,104710] 0 2024-04-06T03:40:34.527 INFO:tasks.workunit.client.0.smithi094.stdout:8/396: mknod d18/d30/d68/c8e 0 2024-04-06T03:40:34.529 INFO:tasks.workunit.client.0.smithi094.stdout:1/335: dwrite d0/d7/f36 [0,4194304] 0 2024-04-06T03:40:34.529 INFO:tasks.workunit.client.0.smithi094.stdout:1/336: chown d0/d1/d16/d40/d50/l6a 30 1 2024-04-06T03:40:34.531 INFO:tasks.workunit.client.0.smithi094.stdout:2/441: truncate d0/f5 10521131 0 2024-04-06T03:40:34.533 INFO:tasks.workunit.client.1.smithi136.stdout:9/343: symlink d17/d21/d32/d43/d51/l72 0 2024-04-06T03:40:34.533 INFO:tasks.workunit.client.1.smithi136.stdout:9/344: fsync d17/d21/d32/d43/d56/f65 0 2024-04-06T03:40:34.533 INFO:tasks.workunit.client.1.smithi136.stdout:9/345: fsync fb 0 2024-04-06T03:40:34.537 INFO:tasks.workunit.client.1.smithi136.stdout:7/327: rename d5/df/d1d/d6f/d2a/c73 to d5/df/d1d/d6f/d2a/c75 0 2024-04-06T03:40:34.538 INFO:tasks.workunit.client.1.smithi136.stdout:0/332: creat d1/d53/f6e x:0 0 0 2024-04-06T03:40:34.540 INFO:tasks.workunit.client.1.smithi136.stdout:8/394: rmdir d3 39 2024-04-06T03:40:34.541 INFO:tasks.workunit.client.0.smithi094.stdout:4/439: symlink d0/df/d13/d4a/l99 0 2024-04-06T03:40:34.542 INFO:tasks.workunit.client.0.smithi094.stdout:2/442: dwrite d0/dd/f62 [0,4194304] 0 2024-04-06T03:40:34.543 INFO:tasks.workunit.client.1.smithi136.stdout:4/301: mknod d1d/d31/d3e/d55/c73 0 2024-04-06T03:40:34.546 INFO:tasks.workunit.client.0.smithi094.stdout:0/363: symlink d1/d9/dd/d18/d1e/d22/d35/l72 0 2024-04-06T03:40:34.550 INFO:tasks.workunit.client.0.smithi094.stdout:8/397: unlink cb 0 2024-04-06T03:40:34.550 INFO:tasks.workunit.client.0.smithi094.stdout:8/398: read - d18/d2c/d37/d60/f88 zero size 2024-04-06T03:40:34.550 INFO:tasks.workunit.client.0.smithi094.stdout:8/399: write d18/d3c/d76/f8c [762655,51163] 0 2024-04-06T03:40:34.557 INFO:tasks.workunit.client.0.smithi094.stdout:2/443: dwrite d0/d8/d4f/f5b [0,4194304] 0 2024-04-06T03:40:34.559 INFO:tasks.workunit.client.1.smithi136.stdout:3/294: link d0/d4/d12/d19/d2c/f40 d0/d4/d12/d19/d2e/d3b/d49/f5e 0 2024-04-06T03:40:34.560 INFO:tasks.workunit.client.1.smithi136.stdout:4/302: dwrite d1d/f22 [0,4194304] 0 2024-04-06T03:40:34.560 INFO:tasks.workunit.client.1.smithi136.stdout:3/295: write d0/f1e [9177858,94856] 0 2024-04-06T03:40:34.562 INFO:tasks.workunit.client.1.smithi136.stdout:9/346: rmdir d17/d21/d32/d3f/d6e 0 2024-04-06T03:40:34.563 INFO:tasks.workunit.client.0.smithi094.stdout:4/440: dwrite d0/d15/d3a/d61/d81/f89 [0,4194304] 0 2024-04-06T03:40:34.564 INFO:tasks.workunit.client.1.smithi136.stdout:7/328: mknod d5/df/d12/d59/d71/c76 0 2024-04-06T03:40:34.565 INFO:tasks.workunit.client.1.smithi136.stdout:0/333: symlink d1/d6/d2c/l6f 0 2024-04-06T03:40:34.565 INFO:tasks.workunit.client.1.smithi136.stdout:0/334: chown d1/d4e/l52 28 1 2024-04-06T03:40:34.567 INFO:tasks.workunit.client.1.smithi136.stdout:5/393: mkdir d1/d75 0 2024-04-06T03:40:34.570 INFO:tasks.workunit.client.1.smithi136.stdout:8/395: rename d3/f69 to d3/f77 0 2024-04-06T03:40:34.573 INFO:tasks.workunit.client.1.smithi136.stdout:7/329: read d5/df/d11/f1c [3445125,75763] 0 2024-04-06T03:40:34.573 INFO:tasks.workunit.client.1.smithi136.stdout:7/330: stat d5/df/d11/f14 0 2024-04-06T03:40:34.573 INFO:tasks.workunit.client.1.smithi136.stdout:6/293: getdents d2/d7/d26/d30/d4f 0 2024-04-06T03:40:34.577 INFO:tasks.workunit.client.1.smithi136.stdout:4/303: creat d1d/d70/f74 x:0 0 0 2024-04-06T03:40:34.580 INFO:tasks.workunit.client.1.smithi136.stdout:3/296: unlink d0/d4/d12/d2d/f4a 0 2024-04-06T03:40:34.581 INFO:tasks.workunit.client.0.smithi094.stdout:0/364: truncate d1/d9/dd/d18/f3a 1293145 0 2024-04-06T03:40:34.582 INFO:tasks.workunit.client.1.smithi136.stdout:9/347: creat d17/d20/d25/d3c/f73 x:0 0 0 2024-04-06T03:40:34.585 INFO:tasks.workunit.client.1.smithi136.stdout:0/335: symlink d1/d6/d2c/d57/d43/d4a/d5c/l70 0 2024-04-06T03:40:34.585 INFO:tasks.workunit.client.1.smithi136.stdout:0/336: stat d1/d6/d2c/d57 0 2024-04-06T03:40:34.586 INFO:tasks.workunit.client.1.smithi136.stdout:0/337: fsync d1/d6/d2c/d57/d43/f50 0 2024-04-06T03:40:34.586 INFO:tasks.workunit.client.1.smithi136.stdout:0/338: creat d1/d6/d2c/d57/d43/d4a/d5c/f71 x:0 0 0 2024-04-06T03:40:34.587 INFO:tasks.workunit.client.1.smithi136.stdout:5/394: truncate d1/dc/d1c/f34 521570 0 2024-04-06T03:40:34.589 INFO:tasks.workunit.client.1.smithi136.stdout:8/396: symlink d3/da/d26/d35/d3d/d48/d6c/l78 0 2024-04-06T03:40:34.592 INFO:tasks.workunit.client.1.smithi136.stdout:7/331: rename d5/l51 to d5/df/d1d/d54/l77 0 2024-04-06T03:40:34.592 INFO:tasks.workunit.client.1.smithi136.stdout:7/332: write d5/df/d1d/d6f/d2c/f72 [12019,63984] 0 2024-04-06T03:40:34.593 INFO:tasks.workunit.client.1.smithi136.stdout:7/333: read d5/df/d11/f1c [1953553,55653] 0 2024-04-06T03:40:34.594 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:34 smithi094 ceph-mon[29713]: pgmap v17: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 143 MiB/s rd, 191 MiB/s wr, 258 op/s 2024-04-06T03:40:34.597 INFO:tasks.workunit.client.1.smithi136.stdout:4/304: creat d1d/d72/f75 x:0 0 0 2024-04-06T03:40:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:34 smithi136 ceph-mon[36436]: pgmap v17: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 143 MiB/s rd, 191 MiB/s wr, 258 op/s 2024-04-06T03:40:34.601 INFO:tasks.workunit.client.1.smithi136.stdout:4/305: dread f1b [0,4194304] 0 2024-04-06T03:40:34.601 INFO:tasks.workunit.client.1.smithi136.stdout:4/306: fsync d1d/d31/f41 0 2024-04-06T03:40:34.601 INFO:tasks.workunit.client.1.smithi136.stdout:3/297: link d0/d4/f18 d0/d4/d12/d19/d2e/d3b/f5f 0 2024-04-06T03:40:34.601 INFO:tasks.workunit.client.1.smithi136.stdout:3/298: chown d0/d4/d12/d2d/c38 37102 1 2024-04-06T03:40:34.601 INFO:tasks.workunit.client.1.smithi136.stdout:3/299: stat d0/d4/d12/d3a/f3e 0 2024-04-06T03:40:34.601 INFO:tasks.workunit.client.1.smithi136.stdout:1/336: sync 2024-04-06T03:40:34.601 INFO:tasks.workunit.client.1.smithi136.stdout:2/451: sync 2024-04-06T03:40:34.602 INFO:tasks.workunit.client.1.smithi136.stdout:2/452: write d3/d9/d10/d8a/d93/d5f/f82 [907429,17336] 0 2024-04-06T03:40:34.603 INFO:tasks.workunit.client.1.smithi136.stdout:9/348: truncate d17/d19/f27 2628892 0 2024-04-06T03:40:34.604 INFO:tasks.workunit.client.1.smithi136.stdout:9/349: truncate f4 4557300 0 2024-04-06T03:40:34.604 INFO:tasks.workunit.client.1.smithi136.stdout:9/350: chown d17/d21/d29/d34/d46 5288984 1 2024-04-06T03:40:34.604 INFO:tasks.workunit.client.1.smithi136.stdout:9/351: creat d17/d21/d32/f74 x:0 0 0 2024-04-06T03:40:34.605 INFO:tasks.workunit.client.0.smithi094.stdout:4/441: getdents d0/df 0 2024-04-06T03:40:34.607 INFO:tasks.workunit.client.0.smithi094.stdout:1/337: mkdir d0/d7/d1a/d74 0 2024-04-06T03:40:34.609 INFO:tasks.workunit.client.1.smithi136.stdout:0/339: mkdir d1/d6/d2c/d57/d43/d6d/d72 0 2024-04-06T03:40:34.609 INFO:tasks.workunit.client.1.smithi136.stdout:0/340: chown d1/d6/d2c/d57/d43/d4a/c64 360 1 2024-04-06T03:40:34.609 INFO:tasks.workunit.client.1.smithi136.stdout:1/337: dread d12/d1b/d1e/d23/f3d [0,4194304] 0 2024-04-06T03:40:34.609 INFO:tasks.workunit.client.1.smithi136.stdout:5/395: dwrite d1/dc/d10/d1e/f69 [0,4194304] 0 2024-04-06T03:40:34.610 INFO:tasks.workunit.client.1.smithi136.stdout:8/397: truncate d3/f2a 2675609 0 2024-04-06T03:40:34.612 INFO:tasks.workunit.client.0.smithi094.stdout:0/365: creat d1/d10/f73 x:0 0 0 2024-04-06T03:40:34.614 INFO:tasks.workunit.client.1.smithi136.stdout:7/334: mknod d5/df/d3d/c78 0 2024-04-06T03:40:34.614 INFO:tasks.workunit.client.1.smithi136.stdout:7/335: chown d5/df/d11/l1a 6388407 1 2024-04-06T03:40:34.614 INFO:tasks.workunit.client.1.smithi136.stdout:7/336: stat d5/df/d1d/d6f/l40 0 2024-04-06T03:40:34.614 INFO:tasks.workunit.client.1.smithi136.stdout:7/337: creat d5/df/d11/f79 x:0 0 0 2024-04-06T03:40:34.615 INFO:tasks.workunit.client.0.smithi094.stdout:8/400: symlink d18/l8f 0 2024-04-06T03:40:34.617 INFO:tasks.workunit.client.1.smithi136.stdout:6/294: getdents d2/d7/d26/d30 0 2024-04-06T03:40:34.617 INFO:tasks.workunit.client.1.smithi136.stdout:6/295: write d2/d7/d26/d30/d4f/f5f [554271,39687] 0 2024-04-06T03:40:34.619 INFO:tasks.workunit.client.0.smithi094.stdout:2/444: mknod d0/d18/c8d 0 2024-04-06T03:40:34.619 INFO:tasks.workunit.client.1.smithi136.stdout:7/338: dread d5/fd [4194304,4194304] 0 2024-04-06T03:40:34.620 INFO:tasks.workunit.client.1.smithi136.stdout:1/338: dread f3 [4194304,4194304] 0 2024-04-06T03:40:34.620 INFO:tasks.workunit.client.1.smithi136.stdout:4/307: creat d1d/d2f/d6c/f76 x:0 0 0 2024-04-06T03:40:34.620 INFO:tasks.workunit.client.1.smithi136.stdout:4/308: write d1d/d31/f44 [1205430,121418] 0 2024-04-06T03:40:34.621 INFO:tasks.workunit.client.1.smithi136.stdout:5/396: dread d1/dc/d1c/f36 [0,4194304] 0 2024-04-06T03:40:34.622 INFO:tasks.workunit.client.1.smithi136.stdout:3/300: symlink d0/d4/l60 0 2024-04-06T03:40:34.623 INFO:tasks.workunit.client.1.smithi136.stdout:3/301: chown d0/d4/d12/d19/f50 463314607 1 2024-04-06T03:40:34.624 INFO:tasks.workunit.client.0.smithi094.stdout:8/401: read d18/f74 [6220554,123249] 0 2024-04-06T03:40:34.624 INFO:tasks.workunit.client.0.smithi094.stdout:8/402: chown d18/d34/d52/l7f 5 1 2024-04-06T03:40:34.624 INFO:tasks.workunit.client.0.smithi094.stdout:8/403: truncate d18/d2c/d37/d60/d70/f84 193000 0 2024-04-06T03:40:34.624 INFO:tasks.workunit.client.0.smithi094.stdout:8/404: write d18/d34/f58 [5012278,19598] 0 2024-04-06T03:40:34.625 INFO:tasks.workunit.client.1.smithi136.stdout:2/453: mknod d3/d9/d10/d8a/d93/c99 0 2024-04-06T03:40:34.627 INFO:tasks.workunit.client.0.smithi094.stdout:1/338: mknod d0/d4/c75 0 2024-04-06T03:40:34.627 INFO:tasks.workunit.client.0.smithi094.stdout:1/339: write d0/d4/d1b/f6f [523960,78879] 0 2024-04-06T03:40:34.632 INFO:tasks.workunit.client.0.smithi094.stdout:0/366: symlink d1/d10/l74 0 2024-04-06T03:40:34.634 INFO:tasks.workunit.client.1.smithi136.stdout:9/352: unlink d17/c1d 0 2024-04-06T03:40:34.634 INFO:tasks.workunit.client.1.smithi136.stdout:4/309: dwrite d1d/d72/f75 [0,4194304] 0 2024-04-06T03:40:34.634 INFO:tasks.workunit.client.1.smithi136.stdout:4/310: stat d1d/d2f/d6c 0 2024-04-06T03:40:34.635 INFO:tasks.workunit.client.0.smithi094.stdout:2/445: mkdir d0/d8/d27/d2b/d8e 0 2024-04-06T03:40:34.639 INFO:tasks.workunit.client.0.smithi094.stdout:4/442: mknod d0/d4/c9a 0 2024-04-06T03:40:34.640 INFO:tasks.workunit.client.0.smithi094.stdout:1/340: dwrite d0/d1/d16/d24/d30/d38/f4d [0,4194304] 0 2024-04-06T03:40:34.646 INFO:tasks.workunit.client.0.smithi094.stdout:8/405: stat d18/d2c/l3f 0 2024-04-06T03:40:34.648 INFO:tasks.workunit.client.1.smithi136.stdout:8/398: mkdir d3/da/d26/d35/d3d/d48/d6c/d79 0 2024-04-06T03:40:34.648 INFO:tasks.workunit.client.1.smithi136.stdout:8/399: stat d3/da/d26/d35/f57 0 2024-04-06T03:40:34.650 INFO:tasks.workunit.client.0.smithi094.stdout:0/367: dwrite d1/d9/dd/d18/d1a/f4d [0,4194304] 0 2024-04-06T03:40:34.653 INFO:tasks.workunit.client.0.smithi094.stdout:4/443: getdents d0 0 2024-04-06T03:40:34.655 INFO:tasks.workunit.client.0.smithi094.stdout:8/406: dread f13 [0,4194304] 0 2024-04-06T03:40:34.655 INFO:tasks.workunit.client.1.smithi136.stdout:6/296: creat d2/de/d58/d5d/f63 x:0 0 0 2024-04-06T03:40:34.657 INFO:tasks.workunit.client.0.smithi094.stdout:4/444: dread d0/df/d13/f36 [0,4194304] 0 2024-04-06T03:40:34.657 INFO:tasks.workunit.client.0.smithi094.stdout:2/446: mkdir d0/dd/d1b/d72/d8f 0 2024-04-06T03:40:34.658 INFO:tasks.workunit.client.1.smithi136.stdout:4/311: dwrite fc [0,4194304] 0 2024-04-06T03:40:34.661 INFO:tasks.workunit.client.1.smithi136.stdout:7/339: rename d5/f32 to d5/df/d12/d1b/d58/f7a 0 2024-04-06T03:40:34.661 INFO:tasks.workunit.client.1.smithi136.stdout:7/340: chown d5/df/d53 59617439 1 2024-04-06T03:40:34.662 INFO:tasks.workunit.client.1.smithi136.stdout:7/341: write d5/df/d12/d1b/f2d [803840,33125] 0 2024-04-06T03:40:34.662 INFO:tasks.workunit.client.1.smithi136.stdout:7/342: chown d5/df/d11/f79 5 1 2024-04-06T03:40:34.663 INFO:tasks.workunit.client.0.smithi094.stdout:1/341: dwrite d0/d7/f26 [0,4194304] 0 2024-04-06T03:40:34.663 INFO:tasks.workunit.client.0.smithi094.stdout:1/342: write d0/d7/d1a/d64/f72 [2923,37595] 0 2024-04-06T03:40:34.663 INFO:tasks.workunit.client.0.smithi094.stdout:1/343: readlink d0/d4/d37/l49 0 2024-04-06T03:40:34.663 INFO:tasks.workunit.client.0.smithi094.stdout:0/368: rename d1/d10/l66 to d1/d9/dd/d18/d5d/d60/d6b/l75 0 2024-04-06T03:40:34.663 INFO:tasks.workunit.client.0.smithi094.stdout:0/369: fdatasync d1/d9/dd/d18/d1e/d22/d35/f3c 0 2024-04-06T03:40:34.665 INFO:tasks.workunit.client.1.smithi136.stdout:1/339: creat d12/d3b/d6b/f70 x:0 0 0 2024-04-06T03:40:34.665 INFO:tasks.workunit.client.1.smithi136.stdout:1/340: write d12/d1b/d28/f5e [716648,113705] 0 2024-04-06T03:40:34.666 INFO:tasks.workunit.client.0.smithi094.stdout:4/445: mknod d0/d4/d67/c9b 0 2024-04-06T03:40:34.668 INFO:tasks.workunit.client.0.smithi094.stdout:2/447: stat d0/dd/d1b/c58 0 2024-04-06T03:40:34.670 INFO:tasks.workunit.client.1.smithi136.stdout:2/454: truncate d3/d9/d10/d8a/d93/f67 3057427 0 2024-04-06T03:40:34.670 INFO:tasks.workunit.client.1.smithi136.stdout:2/455: write d3/d15/f90 [418543,27649] 0 2024-04-06T03:40:34.671 INFO:tasks.workunit.client.0.smithi094.stdout:8/407: dwrite fd [0,4194304] 0 2024-04-06T03:40:34.672 INFO:tasks.workunit.client.0.smithi094.stdout:8/408: chown f11 0 1 2024-04-06T03:40:34.673 INFO:tasks.workunit.client.0.smithi094.stdout:1/344: write d0/d7/f52 [1345852,120048] 0 2024-04-06T03:40:34.677 INFO:tasks.workunit.client.0.smithi094.stdout:3/414: sync 2024-04-06T03:40:34.679 INFO:tasks.workunit.client.1.smithi136.stdout:0/341: symlink d1/l73 0 2024-04-06T03:40:34.679 INFO:tasks.workunit.client.1.smithi136.stdout:0/342: fdatasync d1/d6/d2c/f27 0 2024-04-06T03:40:34.680 INFO:tasks.workunit.client.1.smithi136.stdout:2/456: dread d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:34.682 INFO:tasks.workunit.client.1.smithi136.stdout:6/297: creat d2/de/d58/d5d/d62/f64 x:0 0 0 2024-04-06T03:40:34.682 INFO:tasks.workunit.client.1.smithi136.stdout:6/298: dread - d2/d7/f5a zero size 2024-04-06T03:40:34.683 INFO:tasks.workunit.client.1.smithi136.stdout:6/299: fsync d2/de/d3d/f46 0 2024-04-06T03:40:34.683 INFO:tasks.workunit.client.1.smithi136.stdout:6/300: creat d2/de/d58/d5d/f65 x:0 0 0 2024-04-06T03:40:34.683 INFO:tasks.workunit.client.1.smithi136.stdout:6/301: truncate d2/de/d58/d5d/f63 896903 0 2024-04-06T03:40:34.683 INFO:tasks.workunit.client.1.smithi136.stdout:6/302: fdatasync d2/d7/d26/d30/f60 0 2024-04-06T03:40:34.684 INFO:tasks.workunit.client.0.smithi094.stdout:8/409: write d18/d30/f4b [1298485,11989] 0 2024-04-06T03:40:34.684 INFO:tasks.workunit.client.0.smithi094.stdout:0/370: dwrite d1/f2c [0,4194304] 0 2024-04-06T03:40:34.686 INFO:tasks.workunit.client.1.smithi136.stdout:4/312: truncate d1d/d70/f64 2491151 0 2024-04-06T03:40:34.690 INFO:tasks.workunit.client.0.smithi094.stdout:3/415: write d9/f5f [1827802,81602] 0 2024-04-06T03:40:34.691 INFO:tasks.workunit.client.1.smithi136.stdout:5/397: rename d1/dc/d1c/f36 to d1/d16/d4b/f76 0 2024-04-06T03:40:34.692 INFO:tasks.workunit.client.0.smithi094.stdout:2/448: symlink d0/dd/d3d/d49/l90 0 2024-04-06T03:40:34.693 INFO:tasks.workunit.client.1.smithi136.stdout:7/343: stat d5/df/d1d/d6f/d2a/c75 0 2024-04-06T03:40:34.701 INFO:tasks.workunit.client.1.smithi136.stdout:3/302: rmdir d0/d4/d12/d19/d55/d3c 39 2024-04-06T03:40:34.704 INFO:tasks.workunit.client.1.smithi136.stdout:7/344: dread d5/df/d1d/f57 [0,4194304] 0 2024-04-06T03:40:34.704 INFO:tasks.workunit.client.1.smithi136.stdout:0/343: mknod d1/d6/d2c/d57/d43/d6b/c74 0 2024-04-06T03:40:34.705 INFO:tasks.workunit.client.1.smithi136.stdout:2/457: rmdir d3/d9/d10/d8a/d93 39 2024-04-06T03:40:34.705 INFO:tasks.workunit.client.1.smithi136.stdout:3/303: read d0/f11 [144685,12599] 0 2024-04-06T03:40:34.706 INFO:tasks.workunit.client.0.smithi094.stdout:8/410: symlink d18/d34/d52/l90 0 2024-04-06T03:40:34.708 INFO:tasks.workunit.client.0.smithi094.stdout:0/371: mkdir d1/d9/dd/d18/d5d/d60/d6b/d67/d76 0 2024-04-06T03:40:34.708 INFO:tasks.workunit.client.0.smithi094.stdout:0/372: creat d1/d9/dd/d18/d32/d42/f77 x:0 0 0 2024-04-06T03:40:34.708 INFO:tasks.workunit.client.1.smithi136.stdout:4/313: rmdir d1d/d31/d3e/d55 39 2024-04-06T03:40:34.710 INFO:tasks.workunit.client.0.smithi094.stdout:4/446: getdents d0/d15 0 2024-04-06T03:40:34.710 INFO:tasks.workunit.client.0.smithi094.stdout:4/447: readlink d0/df/d13/d4a/l4c 0 2024-04-06T03:40:34.711 INFO:tasks.workunit.client.1.smithi136.stdout:8/400: dwrite d3/f2a [0,4194304] 0 2024-04-06T03:40:34.714 INFO:tasks.workunit.client.1.smithi136.stdout:5/398: symlink d1/d16/d4b/l77 0 2024-04-06T03:40:34.714 INFO:tasks.workunit.client.1.smithi136.stdout:5/399: chown d1/dc/d1c/d32/c62 1672 1 2024-04-06T03:40:34.714 INFO:tasks.workunit.client.1.smithi136.stdout:5/400: fsync d1/dc/f6e 0 2024-04-06T03:40:34.714 INFO:tasks.workunit.client.1.smithi136.stdout:5/401: write d1/dc/d1c/f56 [11281,65736] 0 2024-04-06T03:40:34.714 INFO:tasks.workunit.client.1.smithi136.stdout:5/402: chown d1/cb 3861070 1 2024-04-06T03:40:34.714 INFO:tasks.workunit.client.1.smithi136.stdout:5/403: creat d1/dc/d1c/f78 x:0 0 0 2024-04-06T03:40:34.714 INFO:tasks.workunit.client.1.smithi136.stdout:5/404: creat d1/dc/d1c/f79 x:0 0 0 2024-04-06T03:40:34.715 INFO:tasks.workunit.client.0.smithi094.stdout:3/416: creat d9/d10/d27/f98 x:0 0 0 2024-04-06T03:40:34.715 INFO:tasks.workunit.client.1.smithi136.stdout:0/344: dread d1/f7 [0,4194304] 0 2024-04-06T03:40:34.716 INFO:tasks.workunit.client.1.smithi136.stdout:1/341: rmdir d12/d3b/d44/d5f/d5a 39 2024-04-06T03:40:34.716 INFO:tasks.workunit.client.1.smithi136.stdout:1/342: stat d12/d1b/d1e/d23/l59 0 2024-04-06T03:40:34.718 INFO:tasks.workunit.client.0.smithi094.stdout:2/449: mknod d0/d18/d3f/c91 0 2024-04-06T03:40:34.720 INFO:tasks.workunit.client.1.smithi136.stdout:2/458: creat d3/d15/d19/f9a x:0 0 0 2024-04-06T03:40:34.722 INFO:tasks.workunit.client.0.smithi094.stdout:1/345: rename d0/d1/d16/d24 to d0/d4/d1b/d2f/d76 0 2024-04-06T03:40:34.723 INFO:tasks.workunit.client.1.smithi136.stdout:1/343: write d12/d1b/d22/f43 [2815408,62092] 0 2024-04-06T03:40:34.725 INFO:tasks.workunit.client.1.smithi136.stdout:3/304: mkdir d0/d4/d12/d3a/d43/d5d/d61 0 2024-04-06T03:40:34.725 INFO:tasks.workunit.client.0.smithi094.stdout:8/411: chown d18/l26 1264 1 2024-04-06T03:40:34.731 INFO:tasks.workunit.client.0.smithi094.stdout:0/373: mkdir d1/d9/dd/d18/d78 0 2024-04-06T03:40:34.731 INFO:tasks.workunit.client.1.smithi136.stdout:0/345: dwrite d1/f5e [0,4194304] 0 2024-04-06T03:40:34.733 INFO:tasks.workunit.client.1.smithi136.stdout:8/401: creat d3/da/d26/f7a x:0 0 0 2024-04-06T03:40:34.735 INFO:tasks.workunit.client.0.smithi094.stdout:4/448: unlink d0/df/d13/d4a/l99 0 2024-04-06T03:40:34.738 INFO:tasks.workunit.client.0.smithi094.stdout:3/417: unlink d9/d10/f70 0 2024-04-06T03:40:34.738 INFO:tasks.workunit.client.0.smithi094.stdout:3/418: dread - d9/d1e/d26/d55/f93 zero size 2024-04-06T03:40:34.740 INFO:tasks.workunit.client.0.smithi094.stdout:2/450: rmdir d0/dd/d19 39 2024-04-06T03:40:34.740 INFO:tasks.workunit.client.0.smithi094.stdout:9/308: sync 2024-04-06T03:40:34.740 INFO:tasks.workunit.client.0.smithi094.stdout:5/274: sync 2024-04-06T03:40:34.741 INFO:tasks.workunit.client.1.smithi136.stdout:5/405: creat d1/f7a x:0 0 0 2024-04-06T03:40:34.742 INFO:tasks.workunit.client.0.smithi094.stdout:1/346: creat d0/d4/d1b/d2f/f77 x:0 0 0 2024-04-06T03:40:34.742 INFO:tasks.workunit.client.0.smithi094.stdout:1/347: chown d0/l3e 0 1 2024-04-06T03:40:34.742 INFO:tasks.workunit.client.0.smithi094.stdout:1/348: readlink d0/d4/d1b/d2f/d3c/d48/l6b 0 2024-04-06T03:40:34.743 INFO:tasks.workunit.client.1.smithi136.stdout:1/344: symlink d12/d1b/d28/d48/l71 0 2024-04-06T03:40:34.747 INFO:tasks.workunit.client.1.smithi136.stdout:9/353: rename d17/d21/d32/d43/d56 to d17/d20/d28/d49/d75 0 2024-04-06T03:40:34.748 INFO:tasks.workunit.client.1.smithi136.stdout:9/354: chown d17/d20/d28/d49/c4d 2 1 2024-04-06T03:40:34.749 INFO:tasks.workunit.client.1.smithi136.stdout:3/305: dwrite d0/f1 [0,4194304] 0 2024-04-06T03:40:34.750 INFO:tasks.workunit.client.0.smithi094.stdout:0/374: rename d1/d9/dd/c3d to d1/d9/dd/d18/d1a/d44/c79 0 2024-04-06T03:40:34.750 INFO:tasks.workunit.client.0.smithi094.stdout:0/375: readlink d1/d9/dd/d18/d1e/l26 0 2024-04-06T03:40:34.750 INFO:tasks.workunit.client.0.smithi094.stdout:6/315: sync 2024-04-06T03:40:34.751 INFO:tasks.workunit.client.0.smithi094.stdout:7/370: sync 2024-04-06T03:40:34.752 INFO:tasks.workunit.client.0.smithi094.stdout:4/449: mknod d0/d15/d3a/c9c 0 2024-04-06T03:40:34.758 INFO:tasks.workunit.client.0.smithi094.stdout:3/419: unlink d9/d1d/f3c 0 2024-04-06T03:40:34.759 INFO:tasks.workunit.client.0.smithi094.stdout:9/309: dread d2/d2c/d33/f4a [4194304,4194304] 0 2024-04-06T03:40:34.760 INFO:tasks.workunit.client.1.smithi136.stdout:0/346: creat d1/d4e/f75 x:0 0 0 2024-04-06T03:40:34.761 INFO:tasks.workunit.client.1.smithi136.stdout:0/347: creat d1/d6/d2c/d1a/d36/f76 x:0 0 0 2024-04-06T03:40:34.762 INFO:tasks.workunit.client.0.smithi094.stdout:2/451: symlink d0/dd/d19/l92 0 2024-04-06T03:40:34.763 INFO:tasks.workunit.client.1.smithi136.stdout:8/402: getdents d3/da/d26/d35/d3d 0 2024-04-06T03:40:34.764 INFO:tasks.workunit.client.0.smithi094.stdout:5/275: chown db/d19/c27 4762 1 2024-04-06T03:40:34.764 INFO:tasks.workunit.client.0.smithi094.stdout:5/276: chown db/d19/d30/f33 0 1 2024-04-06T03:40:34.765 INFO:tasks.workunit.client.1.smithi136.stdout:1/345: dwrite f10 [0,4194304] 0 2024-04-06T03:40:34.765 INFO:tasks.workunit.client.1.smithi136.stdout:1/346: stat d12/d1b/d1e/d4d/f68 0 2024-04-06T03:40:34.766 INFO:tasks.workunit.client.1.smithi136.stdout:5/406: mkdir d1/d16/d4b/d7b 0 2024-04-06T03:40:34.767 INFO:tasks.workunit.client.0.smithi094.stdout:1/349: creat d0/d7/f78 x:0 0 0 2024-04-06T03:40:34.768 INFO:tasks.workunit.client.0.smithi094.stdout:8/412: dwrite d18/f22 [0,4194304] 0 2024-04-06T03:40:34.768 INFO:tasks.workunit.client.0.smithi094.stdout:8/413: chown d18/d2c 1 1 2024-04-06T03:40:34.770 INFO:tasks.workunit.client.1.smithi136.stdout:6/303: rename d2/d7/d13/c4a to d2/c66 0 2024-04-06T03:40:34.772 INFO:tasks.workunit.client.0.smithi094.stdout:6/316: dwrite dc/d3a/d4a/d54/f58 [0,4194304] 0 2024-04-06T03:40:34.772 INFO:tasks.workunit.client.0.smithi094.stdout:6/317: dread - dc/d11/d22/d50/d61/f73 zero size 2024-04-06T03:40:34.772 INFO:tasks.workunit.client.0.smithi094.stdout:6/318: dread - dc/d11/d1b/d24/d46/f48 zero size 2024-04-06T03:40:34.772 INFO:tasks.workunit.client.0.smithi094.stdout:6/319: truncate dc/d11/d22/f57 1653410 0 2024-04-06T03:40:34.773 INFO:tasks.workunit.client.1.smithi136.stdout:9/355: symlink d17/d20/d25/d3b/l76 0 2024-04-06T03:40:34.773 INFO:tasks.workunit.client.1.smithi136.stdout:9/356: write d17/d21/d32/d3f/f63 [1013892,85959] 0 2024-04-06T03:40:34.773 INFO:tasks.workunit.client.1.smithi136.stdout:9/357: dread - d17/d20/d25/d3c/f73 zero size 2024-04-06T03:40:34.773 INFO:tasks.workunit.client.1.smithi136.stdout:9/358: fdatasync d17/d20/d28/d49/f54 0 2024-04-06T03:40:34.775 INFO:tasks.workunit.client.1.smithi136.stdout:9/359: write d17/d19/f61 [400767,27725] 0 2024-04-06T03:40:34.775 INFO:tasks.workunit.client.1.smithi136.stdout:9/360: creat d17/d20/d25/d3c/f77 x:0 0 0 2024-04-06T03:40:34.775 INFO:tasks.workunit.client.1.smithi136.stdout:9/361: creat d17/d21/d29/f78 x:0 0 0 2024-04-06T03:40:34.775 INFO:tasks.workunit.client.1.smithi136.stdout:9/362: chown d17/d21/d29/d34/l45 791955277 1 2024-04-06T03:40:34.775 INFO:tasks.workunit.client.1.smithi136.stdout:5/407: fdatasync d1/dc/d1c/f56 0 2024-04-06T03:40:34.776 INFO:tasks.workunit.client.0.smithi094.stdout:4/450: dwrite d0/d4/f14 [4194304,4194304] 0 2024-04-06T03:40:34.776 INFO:tasks.workunit.client.0.smithi094.stdout:1/350: dread d0/d4/d1b/d2f/d76/d30/f33 [0,4194304] 0 2024-04-06T03:40:34.776 INFO:tasks.workunit.client.0.smithi094.stdout:4/451: truncate d0/d23/f32 1327382 0 2024-04-06T03:40:34.776 INFO:tasks.workunit.client.0.smithi094.stdout:7/371: mknod d2/d6/dc/d12/d29/d35/d57/c6d 0 2024-04-06T03:40:34.782 INFO:tasks.workunit.client.1.smithi136.stdout:3/306: creat d0/d4/d12/d2d/d37/f62 x:0 0 0 2024-04-06T03:40:34.782 INFO:tasks.workunit.client.1.smithi136.stdout:9/363: dread f16 [0,4194304] 0 2024-04-06T03:40:34.785 INFO:tasks.workunit.client.0.smithi094.stdout:2/452: dwrite d0/dd/d19/d31/f42 [0,4194304] 0 2024-04-06T03:40:34.785 INFO:tasks.workunit.client.0.smithi094.stdout:2/453: write d0/d8/d27/f45 [826192,60134] 0 2024-04-06T03:40:34.785 INFO:tasks.workunit.client.0.smithi094.stdout:9/310: rename d2/dc/d10/d2a/d2d to d2/d2c/d33/d5a/d69/d6d 0 2024-04-06T03:40:34.786 INFO:tasks.workunit.client.0.smithi094.stdout:0/376: getdents d1/d9/dd/d18 0 2024-04-06T03:40:34.788 INFO:tasks.workunit.client.1.smithi136.stdout:8/403: link d3/da/d26/d35/d3d/d48/f72 d3/de/d18/d6d/f7b 0 2024-04-06T03:40:34.788 INFO:tasks.workunit.client.1.smithi136.stdout:6/304: dwrite d2/d25/f38 [4194304,4194304] 0 2024-04-06T03:40:34.788 INFO:tasks.workunit.client.1.smithi136.stdout:6/305: readlink d2/d7/d26/l33 0 2024-04-06T03:40:34.789 INFO:tasks.workunit.client.0.smithi094.stdout:8/414: mkdir d18/d3c/d49/d91 0 2024-04-06T03:40:34.789 INFO:tasks.workunit.client.0.smithi094.stdout:8/415: chown d18/d2c/d37/d60/d70/c55 1205 1 2024-04-06T03:40:34.790 INFO:tasks.workunit.client.0.smithi094.stdout:8/416: fdatasync d18/d34/f3b 0 2024-04-06T03:40:34.790 INFO:tasks.workunit.client.0.smithi094.stdout:8/417: read d18/d3c/d8b/f8a [54697,118540] 0 2024-04-06T03:40:34.792 INFO:tasks.workunit.client.1.smithi136.stdout:1/347: mkdir d12/d3b/d44/d5f/d5a/d72 0 2024-04-06T03:40:34.793 INFO:tasks.workunit.client.1.smithi136.stdout:1/348: chown d12/d1b/d1e 520620012 1 2024-04-06T03:40:34.797 INFO:tasks.workunit.client.1.smithi136.stdout:7/345: rename d5/df/d1d/f55 to d5/df/d1d/d6f/d4d/d69/f7b 0 2024-04-06T03:40:34.797 INFO:tasks.workunit.client.1.smithi136.stdout:7/346: write d5/df/d12/d1b/f47 [1001328,28091] 0 2024-04-06T03:40:34.797 INFO:tasks.workunit.client.1.smithi136.stdout:7/347: write d5/df/d1d/d6f/d2a/f48 [824917,115506] 0 2024-04-06T03:40:34.797 INFO:tasks.workunit.client.1.smithi136.stdout:7/348: chown d5/df/d12/l6a 0 1 2024-04-06T03:40:34.800 INFO:tasks.workunit.client.0.smithi094.stdout:6/320: chown dc/d11/l5a 0 1 2024-04-06T03:40:34.800 INFO:tasks.workunit.client.0.smithi094.stdout:6/321: fdatasync dc/f17 0 2024-04-06T03:40:34.800 INFO:tasks.workunit.client.0.smithi094.stdout:6/322: creat dc/d3a/d4a/d54/f76 x:0 0 0 2024-04-06T03:40:34.801 INFO:tasks.workunit.client.0.smithi094.stdout:9/311: dwrite d2/dc/f68 [0,4194304] 0 2024-04-06T03:40:34.801 INFO:tasks.workunit.client.1.smithi136.stdout:9/364: dwrite d17/d21/d32/d3f/f63 [0,4194304] 0 2024-04-06T03:40:34.802 INFO:tasks.workunit.client.0.smithi094.stdout:1/351: mknod d0/d4/d1b/d2f/c79 0 2024-04-06T03:40:34.802 INFO:tasks.workunit.client.0.smithi094.stdout:1/352: readlink d0/d1/l8 0 2024-04-06T03:40:34.805 INFO:tasks.workunit.client.1.smithi136.stdout:5/408: mkdir d1/dc/d15/d20/d7c 0 2024-04-06T03:40:34.807 INFO:tasks.workunit.client.0.smithi094.stdout:4/452: getdents d0/d15/d3a/d61 0 2024-04-06T03:40:34.809 INFO:tasks.workunit.client.1.smithi136.stdout:3/307: symlink d0/d4/d12/d3a/d51/l63 0 2024-04-06T03:40:34.809 INFO:tasks.workunit.client.1.smithi136.stdout:3/308: fdatasync d0/f1 0 2024-04-06T03:40:34.811 INFO:tasks.workunit.client.0.smithi094.stdout:7/372: unlink d2/d6/dc/dd/f4e 0 2024-04-06T03:40:34.815 INFO:tasks.workunit.client.0.smithi094.stdout:2/454: creat d0/dd/d1b/d72/d8f/f93 x:0 0 0 2024-04-06T03:40:34.815 INFO:tasks.workunit.client.0.smithi094.stdout:2/455: dread - d0/dd/d19/d31/f8c zero size 2024-04-06T03:40:34.815 INFO:tasks.workunit.client.0.smithi094.stdout:4/453: dread d0/df/d13/f37 [0,4194304] 0 2024-04-06T03:40:34.816 INFO:tasks.workunit.client.1.smithi136.stdout:8/404: creat d3/da/d26/d35/d53/f7c x:0 0 0 2024-04-06T03:40:34.819 INFO:tasks.workunit.client.0.smithi094.stdout:5/277: rename db/d19/f5d to db/d19/d21/f62 0 2024-04-06T03:40:34.819 INFO:tasks.workunit.client.0.smithi094.stdout:5/278: write db/d19/d24/d2e/f48 [464707,65145] 0 2024-04-06T03:40:34.821 INFO:tasks.workunit.client.0.smithi094.stdout:0/377: creat d1/f7a x:0 0 0 2024-04-06T03:40:34.821 INFO:tasks.workunit.client.0.smithi094.stdout:0/378: creat d1/d9/dd/d18/d32/f7b x:0 0 0 2024-04-06T03:40:34.822 INFO:tasks.workunit.client.0.smithi094.stdout:4/454: dread d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:34.822 INFO:tasks.workunit.client.0.smithi094.stdout:4/455: creat d0/df/d13/d4a/f9d x:0 0 0 2024-04-06T03:40:34.822 INFO:tasks.workunit.client.1.smithi136.stdout:7/349: dwrite d5/df/d1d/d6f/d2a/f49 [0,4194304] 0 2024-04-06T03:40:34.823 INFO:tasks.workunit.client.1.smithi136.stdout:4/314: rename d1d/d31/d3e/d56/f6e to d1d/d72/f77 0 2024-04-06T03:40:34.824 INFO:tasks.workunit.client.1.smithi136.stdout:1/349: dwrite d12/d1b/d28/f5e [0,4194304] 0 2024-04-06T03:40:34.824 INFO:tasks.workunit.client.1.smithi136.stdout:1/350: read - d12/d3b/d44/d5f/f63 zero size 2024-04-06T03:40:34.824 INFO:tasks.workunit.client.1.smithi136.stdout:1/351: truncate d12/d1b/d28/d48/f50 1309516 0 2024-04-06T03:40:34.829 INFO:tasks.workunit.client.0.smithi094.stdout:2/456: write d0/d18/f22 [1863241,1424] 0 2024-04-06T03:40:34.830 INFO:tasks.workunit.client.0.smithi094.stdout:0/379: write d1/d9/dd/d18/d1e/d22/d35/f3c [3388412,122544] 0 2024-04-06T03:40:34.832 INFO:tasks.workunit.client.1.smithi136.stdout:5/409: symlink d1/d16/d4b/l7d 0 2024-04-06T03:40:34.834 INFO:tasks.workunit.client.0.smithi094.stdout:6/323: creat dc/d11/d1b/d72/f77 x:0 0 0 2024-04-06T03:40:34.835 INFO:tasks.workunit.client.1.smithi136.stdout:3/309: mknod d0/d4/d12/d19/d55/c64 0 2024-04-06T03:40:34.838 INFO:tasks.workunit.client.1.smithi136.stdout:3/310: dread d0/d4/d12/d2d/f33 [0,4194304] 0 2024-04-06T03:40:34.839 INFO:tasks.workunit.client.1.smithi136.stdout:8/405: dwrite d3/da/d26/d35/f1d [0,4194304] 0 2024-04-06T03:40:34.839 INFO:tasks.workunit.client.1.smithi136.stdout:8/406: write d3/da/d26/d35/d3d/f6b [4910219,75345] 0 2024-04-06T03:40:34.839 INFO:tasks.workunit.client.1.smithi136.stdout:8/407: read - d3/da/f41 zero size 2024-04-06T03:40:34.841 INFO:tasks.workunit.client.1.smithi136.stdout:7/350: mknod d5/df/d1d/d54/d5c/c7c 0 2024-04-06T03:40:34.842 INFO:tasks.workunit.client.0.smithi094.stdout:9/312: symlink d2/d2c/d33/d37/l6e 0 2024-04-06T03:40:34.846 INFO:tasks.workunit.client.0.smithi094.stdout:1/353: creat d0/d1/d16/d40/f7a x:0 0 0 2024-04-06T03:40:34.847 INFO:tasks.workunit.client.1.smithi136.stdout:4/315: mkdir d1d/d31/d78 0 2024-04-06T03:40:34.850 INFO:tasks.workunit.client.0.smithi094.stdout:7/373: unlink d2/d6/dc/d12/l26 0 2024-04-06T03:40:34.850 INFO:tasks.workunit.client.0.smithi094.stdout:7/374: chown d2/d6/dc/d12/d29/d35/f54 56 1 2024-04-06T03:40:34.850 INFO:tasks.workunit.client.0.smithi094.stdout:7/375: write d2/d6/dc/dd/f28 [4050225,85024] 0 2024-04-06T03:40:34.853 INFO:tasks.workunit.client.0.smithi094.stdout:2/457: dwrite d0/d8/d27/d2b/f5d [0,4194304] 0 2024-04-06T03:40:34.853 INFO:tasks.workunit.client.0.smithi094.stdout:2/458: chown d0/d8/d27/c80 1 1 2024-04-06T03:40:34.857 INFO:tasks.workunit.client.0.smithi094.stdout:3/420: rmdir d9/d20/d3e 0 2024-04-06T03:40:34.857 INFO:tasks.workunit.client.0.smithi094.stdout:3/421: chown d9/d20 15 1 2024-04-06T03:40:34.861 INFO:tasks.workunit.client.0.smithi094.stdout:5/279: creat db/d19/d30/f63 x:0 0 0 2024-04-06T03:40:34.863 INFO:tasks.workunit.client.1.smithi136.stdout:3/311: dwrite d0/fb [0,4194304] 0 2024-04-06T03:40:34.865 INFO:tasks.workunit.client.1.smithi136.stdout:9/365: rmdir d17/d20/d25/d3d 39 2024-04-06T03:40:34.866 INFO:tasks.workunit.client.0.smithi094.stdout:4/456: creat d0/d77/f9e x:0 0 0 2024-04-06T03:40:34.868 INFO:tasks.workunit.client.1.smithi136.stdout:5/410: stat d1/c27 0 2024-04-06T03:40:34.868 INFO:tasks.workunit.client.1.smithi136.stdout:5/411: write d1/dc/d1c/d32/f38 [1184054,125169] 0 2024-04-06T03:40:34.868 INFO:tasks.workunit.client.1.smithi136.stdout:5/412: fsync d1/dc/d1c/d32/f6a 0 2024-04-06T03:40:34.868 INFO:tasks.workunit.client.1.smithi136.stdout:5/413: creat d1/d16/d4b/f7e x:0 0 0 2024-04-06T03:40:34.868 INFO:tasks.workunit.client.1.smithi136.stdout:5/414: read d1/dc/d10/d1e/f2f [4537521,59025] 0 2024-04-06T03:40:34.868 INFO:tasks.workunit.client.1.smithi136.stdout:5/415: write d1/dc/d1c/f2c [52389,31851] 0 2024-04-06T03:40:34.868 INFO:tasks.workunit.client.1.smithi136.stdout:5/416: truncate d1/dc/f6e 652603 0 2024-04-06T03:40:34.868 INFO:tasks.workunit.client.1.smithi136.stdout:5/417: readlink d1/dc/d1c/l58 0 2024-04-06T03:40:34.870 INFO:tasks.workunit.client.1.smithi136.stdout:8/408: mkdir d3/de/d18/d7d 0 2024-04-06T03:40:34.875 INFO:tasks.workunit.client.0.smithi094.stdout:9/313: dwrite d2/d2c/d33/f5c [0,4194304] 0 2024-04-06T03:40:34.875 INFO:tasks.workunit.client.0.smithi094.stdout:7/376: dwrite d2/d6/dc/d12/d29/d35/f54 [0,4194304] 0 2024-04-06T03:40:34.875 INFO:tasks.workunit.client.0.smithi094.stdout:7/377: fdatasync d2/fb 0 2024-04-06T03:40:34.875 INFO:tasks.workunit.client.0.smithi094.stdout:1/354: unlink d0/d4/d1b/d2f/d76/d30/f33 0 2024-04-06T03:40:34.875 INFO:tasks.workunit.client.1.smithi136.stdout:7/351: dwrite d5/df/d1d/d6f/f61 [4194304,4194304] 0 2024-04-06T03:40:34.876 INFO:tasks.workunit.client.1.smithi136.stdout:7/352: fdatasync d5/df/d1d/f1f 0 2024-04-06T03:40:34.878 INFO:tasks.workunit.client.1.smithi136.stdout:8/409: dread d3/de/d18/d6d/f7b [0,4194304] 0 2024-04-06T03:40:34.882 INFO:tasks.workunit.client.1.smithi136.stdout:5/418: dwrite d1/dc/d1c/f78 [0,4194304] 0 2024-04-06T03:40:34.882 INFO:tasks.workunit.client.1.smithi136.stdout:5/419: creat d1/dc/d15/f7f x:0 0 0 2024-04-06T03:40:34.884 INFO:tasks.workunit.client.0.smithi094.stdout:2/459: symlink d0/d8/d4f/l94 0 2024-04-06T03:40:34.886 INFO:tasks.workunit.client.0.smithi094.stdout:3/422: symlink d9/d10/d27/d97/l99 0 2024-04-06T03:40:34.888 INFO:tasks.workunit.client.0.smithi094.stdout:5/280: mknod db/d19/d2c/d57/c64 0 2024-04-06T03:40:34.889 INFO:tasks.workunit.client.1.smithi136.stdout:5/420: dread d1/d16/d4b/f4f [0,4194304] 0 2024-04-06T03:40:34.889 INFO:tasks.workunit.client.0.smithi094.stdout:6/324: rename dc/d11/d1b/d24/c3b to dc/d11/c78 0 2024-04-06T03:40:34.890 INFO:tasks.workunit.client.0.smithi094.stdout:6/325: truncate dc/d11/d1b/f74 692026 0 2024-04-06T03:40:34.894 INFO:tasks.workunit.client.1.smithi136.stdout:7/353: dwrite d5/df/d12/d1b/f2d [0,4194304] 0 2024-04-06T03:40:34.894 INFO:tasks.workunit.client.1.smithi136.stdout:7/354: write d5/df/d53/f43 [894035,109073] 0 2024-04-06T03:40:34.895 INFO:tasks.workunit.client.1.smithi136.stdout:1/352: rmdir d12/d1b/d1e/d27/d37 39 2024-04-06T03:40:34.900 INFO:tasks.workunit.client.0.smithi094.stdout:9/314: symlink d2/d2c/d33/d5a/d69/d6d/l6f 0 2024-04-06T03:40:34.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/378: mknod d2/d6/dc/dd/df/c6e 0 2024-04-06T03:40:34.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/379: chown d2/d6/dc/dd/l1b 37625491 1 2024-04-06T03:40:34.904 INFO:tasks.workunit.client.0.smithi094.stdout:1/355: unlink d0/d4/d1b/d2f/d3c/f6c 0 2024-04-06T03:40:34.905 INFO:tasks.workunit.client.1.smithi136.stdout:3/312: mknod d0/d4/d12/d3a/d43/d5d/c65 0 2024-04-06T03:40:34.906 INFO:tasks.workunit.client.0.smithi094.stdout:2/460: creat d0/dd/f95 x:0 0 0 2024-04-06T03:40:34.908 INFO:tasks.workunit.client.1.smithi136.stdout:9/366: creat d17/d20/d25/d3c/d55/f79 x:0 0 0 2024-04-06T03:40:34.910 INFO:tasks.workunit.client.0.smithi094.stdout:3/423: mkdir d9/d1e/d6e/d75/d90/d9a 0 2024-04-06T03:40:34.910 INFO:tasks.workunit.client.0.smithi094.stdout:3/424: chown d9/f8c 616238 1 2024-04-06T03:40:34.910 INFO:tasks.workunit.client.0.smithi094.stdout:6/326: dwrite dc/fd [0,4194304] 0 2024-04-06T03:40:34.911 INFO:tasks.workunit.client.1.smithi136.stdout:5/421: dwrite d1/dc/d1c/d32/f6a [0,4194304] 0 2024-04-06T03:40:34.911 INFO:tasks.workunit.client.1.smithi136.stdout:5/422: creat d1/d16/d4b/f80 x:0 0 0 2024-04-06T03:40:34.911 INFO:tasks.workunit.client.1.smithi136.stdout:5/423: fdatasync d1/d16/f1b 0 2024-04-06T03:40:34.913 INFO:tasks.workunit.client.0.smithi094.stdout:5/281: mkdir db/d19/d24/d28/d65 0 2024-04-06T03:40:34.916 INFO:tasks.workunit.client.1.smithi136.stdout:2/459: rename d3/d4a/d77/d7e to d3/d9b 0 2024-04-06T03:40:34.916 INFO:tasks.workunit.client.1.smithi136.stdout:2/460: fdatasync d3/d9/fe 0 2024-04-06T03:40:34.916 INFO:tasks.workunit.client.1.smithi136.stdout:2/461: truncate d3/d15/d19/d43/d54/f84 731093 0 2024-04-06T03:40:34.916 INFO:tasks.workunit.client.1.smithi136.stdout:2/462: creat d3/d9/d10/d18/d4b/f9c x:0 0 0 2024-04-06T03:40:34.917 INFO:tasks.workunit.client.1.smithi136.stdout:2/463: write d3/d9/d10/d18/f2a [2049134,20640] 0 2024-04-06T03:40:34.919 INFO:tasks.workunit.client.0.smithi094.stdout:4/457: rename d0/d15/d3a/d94 to d0/d4/d67/d9f 0 2024-04-06T03:40:34.919 INFO:tasks.workunit.client.0.smithi094.stdout:4/458: write d0/d15/d3a/d3e/d46/f62 [736750,114057] 0 2024-04-06T03:40:34.922 INFO:tasks.workunit.client.0.smithi094.stdout:7/380: getdents d2/d6/dc/d12/d61 0 2024-04-06T03:40:34.922 INFO:tasks.workunit.client.0.smithi094.stdout:7/381: read - d2/f63 zero size 2024-04-06T03:40:34.923 INFO:tasks.workunit.client.0.smithi094.stdout:5/282: dread db/f23 [0,4194304] 0 2024-04-06T03:40:34.924 INFO:tasks.workunit.client.0.smithi094.stdout:9/315: dwrite d2/d2c/d33/d5a/d69/d6d/f3b [0,4194304] 0 2024-04-06T03:40:34.924 INFO:tasks.workunit.client.0.smithi094.stdout:1/356: mknod d0/d4/c7b 0 2024-04-06T03:40:34.927 INFO:tasks.workunit.client.0.smithi094.stdout:2/461: unlink d0/d8/d27/d2b/f2c 0 2024-04-06T03:40:34.928 INFO:tasks.workunit.client.1.smithi136.stdout:5/424: dwrite d1/d16/d17/f44 [0,4194304] 0 2024-04-06T03:40:34.929 INFO:tasks.workunit.client.0.smithi094.stdout:2/462: dread d0/d8/d27/f45 [0,4194304] 0 2024-04-06T03:40:34.929 INFO:tasks.workunit.client.0.smithi094.stdout:3/425: mkdir d9/d10/d9b 0 2024-04-06T03:40:34.932 INFO:tasks.workunit.client.1.smithi136.stdout:2/464: dwrite d3/d15/d19/d43/d54/f8c [0,4194304] 0 2024-04-06T03:40:34.932 INFO:tasks.workunit.client.1.smithi136.stdout:2/465: write d3/d9/d10/d8a/d93/d5f/f82 [1185957,71213] 0 2024-04-06T03:40:34.932 INFO:tasks.workunit.client.0.smithi094.stdout:4/459: getdents d0/d4/d4b 0 2024-04-06T03:40:34.932 INFO:tasks.workunit.client.0.smithi094.stdout:4/460: chown d0/d15/d3a/d61 31 1 2024-04-06T03:40:34.932 INFO:tasks.workunit.client.0.smithi094.stdout:4/461: stat d0/c6d 0 2024-04-06T03:40:34.932 INFO:tasks.workunit.client.0.smithi094.stdout:4/462: readlink d0/d15/d3a/d3e/d50/l6b 0 2024-04-06T03:40:34.932 INFO:tasks.workunit.client.0.smithi094.stdout:4/463: chown d0/f8 59655958 1 2024-04-06T03:40:34.933 INFO:tasks.workunit.client.0.smithi094.stdout:6/327: rename dc/d11/d1b/d4e/c5b to dc/d11/d22/d50/c79 0 2024-04-06T03:40:34.934 INFO:tasks.workunit.client.0.smithi094.stdout:6/328: write dc/d11/d22/f57 [1633639,63572] 0 2024-04-06T03:40:34.935 INFO:tasks.workunit.client.0.smithi094.stdout:7/382: symlink d2/d6/dc/dd/df/l6f 0 2024-04-06T03:40:34.937 INFO:tasks.workunit.client.0.smithi094.stdout:3/426: dread d9/d20/f33 [0,4194304] 0 2024-04-06T03:40:34.940 INFO:tasks.workunit.client.0.smithi094.stdout:5/283: unlink db/d19/d24/d2e/f3a 0 2024-04-06T03:40:34.946 INFO:tasks.workunit.client.0.smithi094.stdout:9/316: creat d2/d2c/d33/d37/f70 x:0 0 0 2024-04-06T03:40:34.949 INFO:tasks.workunit.client.0.smithi094.stdout:1/357: symlink d0/d4/d37/d5f/l7c 0 2024-04-06T03:40:34.953 INFO:tasks.workunit.client.0.smithi094.stdout:4/464: mkdir d0/d15/d3a/d61/d81/da0 0 2024-04-06T03:40:34.957 INFO:tasks.workunit.client.1.smithi136.stdout:8/410: unlink d3/da/d26/d35/d3d/d48/f58 0 2024-04-06T03:40:34.958 INFO:tasks.workunit.client.0.smithi094.stdout:5/284: creat db/d19/d24/d28/d65/f66 x:0 0 0 2024-04-06T03:40:34.959 INFO:tasks.workunit.client.0.smithi094.stdout:5/285: write db/d19/d30/f44 [359258,2071] 0 2024-04-06T03:40:34.959 INFO:tasks.workunit.client.0.smithi094.stdout:2/463: write d0/dd/d1b/f3e [572672,91625] 0 2024-04-06T03:40:34.959 INFO:tasks.workunit.client.0.smithi094.stdout:6/329: dwrite dc/d3a/d4a/d54/f58 [0,4194304] 0 2024-04-06T03:40:34.960 INFO:tasks.workunit.client.0.smithi094.stdout:7/383: dwrite d2/d6/dc/dd/f28 [4194304,4194304] 0 2024-04-06T03:40:34.960 INFO:tasks.workunit.client.0.smithi094.stdout:3/427: dwrite d9/d10/d27/d3a/f40 [0,4194304] 0 2024-04-06T03:40:34.960 INFO:tasks.workunit.client.0.smithi094.stdout:3/428: fdatasync d9/fd 0 2024-04-06T03:40:34.960 INFO:tasks.workunit.client.0.smithi094.stdout:9/317: mknod d2/d2c/d33/d64/c71 0 2024-04-06T03:40:34.963 INFO:tasks.workunit.client.0.smithi094.stdout:1/358: unlink d0/c4a 0 2024-04-06T03:40:34.963 INFO:tasks.workunit.client.0.smithi094.stdout:1/359: readlink d0/d1/d16/l31 0 2024-04-06T03:40:34.967 INFO:tasks.workunit.client.1.smithi136.stdout:7/355: symlink d5/df/d1d/d6f/d4d/d69/l7d 0 2024-04-06T03:40:34.970 INFO:tasks.workunit.client.1.smithi136.stdout:1/353: rmdir d12/d3b/d60 39 2024-04-06T03:40:34.971 INFO:tasks.workunit.client.0.smithi094.stdout:5/286: mknod db/d40/c67 0 2024-04-06T03:40:34.973 INFO:tasks.workunit.client.1.smithi136.stdout:3/313: symlink d0/d4/d12/d19/d55/d3c/d46/l66 0 2024-04-06T03:40:34.973 INFO:tasks.workunit.client.1.smithi136.stdout:3/314: chown d0/d4/d12/d2d/l3d 138858 1 2024-04-06T03:40:34.975 INFO:tasks.workunit.client.1.smithi136.stdout:9/367: unlink d17/d20/d28/d49/d75/f67 0 2024-04-06T03:40:34.976 INFO:tasks.workunit.client.0.smithi094.stdout:2/464: mknod d0/d8/d4f/d70/c96 0 2024-04-06T03:40:34.986 INFO:tasks.workunit.client.1.smithi136.stdout:0/348: rename d1/d6/f19 to d1/d4e/f77 0 2024-04-06T03:40:34.986 INFO:tasks.workunit.client.0.smithi094.stdout:6/330: dwrite dc/d11/d22/f39 [0,4194304] 0 2024-04-06T03:40:34.987 INFO:tasks.workunit.client.0.smithi094.stdout:9/318: rename d2/dc/d21/d26/f62 to d2/dc/d21/d26/f72 0 2024-04-06T03:40:34.987 INFO:tasks.workunit.client.0.smithi094.stdout:9/319: chown d2/d2c/d33/d37/c39 622 1 2024-04-06T03:40:34.987 INFO:tasks.workunit.client.0.smithi094.stdout:9/320: write d2/dc/d21/d23/d2f/f57 [258402,127534] 0 2024-04-06T03:40:34.988 INFO:tasks.workunit.client.1.smithi136.stdout:9/368: dread f11 [0,4194304] 0 2024-04-06T03:40:34.988 INFO:tasks.workunit.client.1.smithi136.stdout:9/369: dread - d17/d20/d25/d3c/f73 zero size 2024-04-06T03:40:34.989 INFO:tasks.workunit.client.0.smithi094.stdout:1/360: dwrite d0/f19 [4194304,4194304] 0 2024-04-06T03:40:34.989 INFO:tasks.workunit.client.0.smithi094.stdout:1/361: stat d0/d4/d37 0 2024-04-06T03:40:34.989 INFO:tasks.workunit.client.0.smithi094.stdout:1/362: truncate d0/d4/d1b/f55 676766 0 2024-04-06T03:40:34.990 INFO:tasks.workunit.client.0.smithi094.stdout:3/429: dwrite d9/d20/f34 [0,4194304] 0 2024-04-06T03:40:34.990 INFO:tasks.workunit.client.0.smithi094.stdout:2/465: dwrite d0/dd/d19/d31/f8c [0,4194304] 0 2024-04-06T03:40:34.993 INFO:tasks.workunit.client.0.smithi094.stdout:4/465: symlink d0/df/d13/la1 0 2024-04-06T03:40:34.997 INFO:tasks.workunit.client.0.smithi094.stdout:5/287: rmdir db/d19/d21/d61 0 2024-04-06T03:40:35.003 INFO:tasks.workunit.client.0.smithi094.stdout:9/321: creat d2/d2c/d33/d5a/d69/d6d/d6b/f73 x:0 0 0 2024-04-06T03:40:35.006 INFO:tasks.workunit.client.0.smithi094.stdout:1/363: dread d0/d4/d1b/d2f/d76/d30/d38/f4d [0,4194304] 0 2024-04-06T03:40:35.006 INFO:tasks.workunit.client.0.smithi094.stdout:6/331: dwrite dc/d11/d1b/f52 [0,4194304] 0 2024-04-06T03:40:35.009 INFO:tasks.workunit.client.0.smithi094.stdout:2/466: read d0/d18/f24 [4099495,117027] 0 2024-04-06T03:40:35.010 INFO:tasks.workunit.client.0.smithi094.stdout:4/466: truncate d0/d4/f6 3877555 0 2024-04-06T03:40:35.011 INFO:tasks.workunit.client.0.smithi094.stdout:5/288: truncate db/d19/d21/f26 3523319 0 2024-04-06T03:40:35.011 INFO:tasks.workunit.client.0.smithi094.stdout:3/430: dwrite d9/d20/d30/f4d [0,4194304] 0 2024-04-06T03:40:35.012 INFO:tasks.workunit.client.1.smithi136.stdout:8/411: mkdir d3/da/d26/d35/d39/d70/d7e 0 2024-04-06T03:40:35.016 INFO:tasks.workunit.client.0.smithi094.stdout:9/322: mknod d2/dc/d21/d26/c74 0 2024-04-06T03:40:35.018 INFO:tasks.workunit.client.0.smithi094.stdout:4/467: write d0/d4/fb [2644964,108067] 0 2024-04-06T03:40:35.018 INFO:tasks.workunit.client.0.smithi094.stdout:4/468: chown d0/f44 9130 1 2024-04-06T03:40:35.019 INFO:tasks.workunit.client.1.smithi136.stdout:8/412: dread d3/de/ff [0,4194304] 0 2024-04-06T03:40:35.019 INFO:tasks.workunit.client.1.smithi136.stdout:7/356: link d5/df/d1d/d54/d5c/c7c d5/df/d1d/d6f/c7e 0 2024-04-06T03:40:35.020 INFO:tasks.workunit.client.1.smithi136.stdout:7/357: truncate d5/df/d12/d59/f5e 192247 0 2024-04-06T03:40:35.020 INFO:tasks.workunit.client.1.smithi136.stdout:7/358: creat d5/df/d1d/f7f x:0 0 0 2024-04-06T03:40:35.021 INFO:tasks.workunit.client.0.smithi094.stdout:3/431: dread d9/fd [0,4194304] 0 2024-04-06T03:40:35.021 INFO:tasks.workunit.client.0.smithi094.stdout:3/432: dread - d9/d1d/d7d/f85 zero size 2024-04-06T03:40:35.021 INFO:tasks.workunit.client.1.smithi136.stdout:1/354: mknod d12/d1b/d1e/c73 0 2024-04-06T03:40:35.023 INFO:tasks.workunit.client.0.smithi094.stdout:1/364: mkdir d0/d4/d1b/d2f/d76/d7d 0 2024-04-06T03:40:35.025 INFO:tasks.workunit.client.1.smithi136.stdout:3/315: unlink d0/f1e 0 2024-04-06T03:40:35.026 INFO:tasks.workunit.client.0.smithi094.stdout:6/332: getdents dc/d11/d1b/d4e 0 2024-04-06T03:40:35.029 INFO:tasks.workunit.client.1.smithi136.stdout:6/306: rename d2/d7/d26/d30/f60 to d2/d7/d26/d30/d4f/f67 0 2024-04-06T03:40:35.030 INFO:tasks.workunit.client.0.smithi094.stdout:5/289: creat db/d19/d24/d28/f68 x:0 0 0 2024-04-06T03:40:35.032 INFO:tasks.workunit.client.0.smithi094.stdout:6/333: dread dc/d11/d1b/f31 [0,4194304] 0 2024-04-06T03:40:35.033 INFO:tasks.workunit.client.1.smithi136.stdout:9/370: unlink d17/d20/d25/d3b/l4c 0 2024-04-06T03:40:35.035 INFO:tasks.workunit.client.1.smithi136.stdout:8/413: mkdir d3/da/d26/d35/d3d/d48/d6c/d7f 0 2024-04-06T03:40:35.037 INFO:tasks.workunit.client.0.smithi094.stdout:5/290: dread db/f1e [0,4194304] 0 2024-04-06T03:40:35.038 INFO:tasks.workunit.client.1.smithi136.stdout:4/316: sync 2024-04-06T03:40:35.038 INFO:tasks.workunit.client.1.smithi136.stdout:4/317: stat d1d/d31/d3e/d56/f65 0 2024-04-06T03:40:35.038 INFO:tasks.workunit.client.1.smithi136.stdout:4/318: truncate f8 3886951 0 2024-04-06T03:40:35.041 INFO:tasks.workunit.client.1.smithi136.stdout:7/359: creat d5/f80 x:0 0 0 2024-04-06T03:40:35.041 INFO:tasks.workunit.client.1.smithi136.stdout:3/316: dwrite d0/d4/d12/f20 [0,4194304] 0 2024-04-06T03:40:35.041 INFO:tasks.workunit.client.1.smithi136.stdout:3/317: write d0/d4/d12/f1a [1975377,114235] 0 2024-04-06T03:40:35.041 INFO:tasks.workunit.client.0.smithi094.stdout:9/323: truncate d2/dc/f68 1647357 0 2024-04-06T03:40:35.042 INFO:tasks.workunit.client.1.smithi136.stdout:2/466: sync 2024-04-06T03:40:35.042 INFO:tasks.workunit.client.1.smithi136.stdout:5/425: sync 2024-04-06T03:40:35.043 INFO:tasks.workunit.client.1.smithi136.stdout:9/371: read d17/d21/d29/f2a [1349595,11403] 0 2024-04-06T03:40:35.043 INFO:tasks.workunit.client.1.smithi136.stdout:7/360: dread d5/df/d3d/f63 [0,4194304] 0 2024-04-06T03:40:35.043 INFO:tasks.workunit.client.1.smithi136.stdout:7/361: chown d5 5058 1 2024-04-06T03:40:35.043 INFO:tasks.workunit.client.1.smithi136.stdout:7/362: fdatasync d5/df/d12/d59/f5e 0 2024-04-06T03:40:35.044 INFO:tasks.workunit.client.1.smithi136.stdout:1/355: mkdir d12/d3b/d74 0 2024-04-06T03:40:35.044 INFO:tasks.workunit.client.1.smithi136.stdout:1/356: truncate d12/d3b/d6b/f70 912312 0 2024-04-06T03:40:35.044 INFO:tasks.workunit.client.1.smithi136.stdout:1/357: chown d12/d1b/d1e/l4f 330 1 2024-04-06T03:40:35.044 INFO:tasks.workunit.client.1.smithi136.stdout:1/358: chown d12/d1b/l1c 2082325 1 2024-04-06T03:40:35.044 INFO:tasks.workunit.client.1.smithi136.stdout:1/359: truncate d12/d1b/d1e/d27/f42 995499 0 2024-04-06T03:40:35.046 INFO:tasks.workunit.client.0.smithi094.stdout:4/469: creat d0/d15/fa2 x:0 0 0 2024-04-06T03:40:35.047 INFO:tasks.workunit.client.1.smithi136.stdout:6/307: mkdir d2/d7/d26/d68 0 2024-04-06T03:40:35.047 INFO:tasks.workunit.client.1.smithi136.stdout:6/308: creat d2/de/f69 x:0 0 0 2024-04-06T03:40:35.049 INFO:tasks.workunit.client.0.smithi094.stdout:3/433: rename d9/d1e/d26/d55/f74 to d9/d1d/d7d/f9c 0 2024-04-06T03:40:35.050 INFO:tasks.workunit.client.1.smithi136.stdout:8/414: symlink d3/da/l80 0 2024-04-06T03:40:35.052 INFO:tasks.workunit.client.0.smithi094.stdout:1/365: symlink d0/d7/d1a/d64/l7e 0 2024-04-06T03:40:35.053 INFO:tasks.workunit.client.1.smithi136.stdout:4/319: symlink d1d/d31/l79 0 2024-04-06T03:40:35.054 INFO:tasks.workunit.client.0.smithi094.stdout:6/334: mknod dc/d11/d22/d50/c7a 0 2024-04-06T03:40:35.055 INFO:tasks.workunit.client.1.smithi136.stdout:9/372: dwrite d17/d20/d25/f26 [4194304,4194304] 0 2024-04-06T03:40:35.057 INFO:tasks.workunit.client.0.smithi094.stdout:5/291: creat db/f69 x:0 0 0 2024-04-06T03:40:35.058 INFO:tasks.workunit.client.0.smithi094.stdout:5/292: write db/d19/d24/f37 [318118,101411] 0 2024-04-06T03:40:35.058 INFO:tasks.workunit.client.0.smithi094.stdout:5/293: dread - db/d19/d24/d28/d65/f66 zero size 2024-04-06T03:40:35.061 INFO:tasks.workunit.client.0.smithi094.stdout:9/324: mknod d2/dc/d21/c75 0 2024-04-06T03:40:35.061 INFO:tasks.workunit.client.0.smithi094.stdout:9/325: readlink d2/l40 0 2024-04-06T03:40:35.064 INFO:tasks.workunit.client.0.smithi094.stdout:4/470: getdents d0/d15/d3a/d3e/d49 0 2024-04-06T03:40:35.064 INFO:tasks.workunit.client.0.smithi094.stdout:4/471: creat d0/d15/d3a/d3e/d46/fa3 x:0 0 0 2024-04-06T03:40:35.064 INFO:tasks.workunit.client.0.smithi094.stdout:4/472: readlink d0/d15/d3a/l6f 0 2024-04-06T03:40:35.067 INFO:tasks.workunit.client.1.smithi136.stdout:7/363: creat d5/df/d1d/d6f/d2c/f81 x:0 0 0 2024-04-06T03:40:35.067 INFO:tasks.workunit.client.1.smithi136.stdout:7/364: creat d5/f82 x:0 0 0 2024-04-06T03:40:35.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/366: write d0/d4/d1b/d2f/f54 [8077663,113899] 0 2024-04-06T03:40:35.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/367: dread - d0/d1/d16/d3f/f5c zero size 2024-04-06T03:40:35.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/368: chown d0/d4/d1b/d2f/c79 503391642 1 2024-04-06T03:40:35.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/369: chown d0/d4/d1b/f6f 56 1 2024-04-06T03:40:35.071 INFO:tasks.workunit.client.1.smithi136.stdout:1/360: mknod d12/c75 0 2024-04-06T03:40:35.071 INFO:tasks.workunit.client.1.smithi136.stdout:1/361: chown d12/d3b/d44/d5f/c52 60 1 2024-04-06T03:40:35.072 INFO:tasks.workunit.client.0.smithi094.stdout:6/335: dread dc/d11/f20 [0,4194304] 0 2024-04-06T03:40:35.072 INFO:tasks.workunit.client.0.smithi094.stdout:6/336: fdatasync dc/d11/d1b/d24/f2e 0 2024-04-06T03:40:35.072 INFO:tasks.workunit.client.0.smithi094.stdout:6/337: truncate dc/d11/d22/d50/d61/f6c 354029 0 2024-04-06T03:40:35.072 INFO:tasks.workunit.client.0.smithi094.stdout:6/338: read dc/d11/d1b/d24/f25 [3425130,19556] 0 2024-04-06T03:40:35.075 INFO:tasks.workunit.client.1.smithi136.stdout:1/362: read d12/d3b/d6b/f70 [87825,3663] 0 2024-04-06T03:40:35.075 INFO:tasks.workunit.client.1.smithi136.stdout:1/363: truncate d12/d1b/f35 726572 0 2024-04-06T03:40:35.075 INFO:tasks.workunit.client.1.smithi136.stdout:1/364: fdatasync f3 0 2024-04-06T03:40:35.077 INFO:tasks.workunit.client.0.smithi094.stdout:9/326: dwrite d2/dc/d10/f66 [0,4194304] 0 2024-04-06T03:40:35.078 INFO:tasks.workunit.client.0.smithi094.stdout:5/294: rename db/d19/d24/d28/f68 to db/d19/d24/d28/d65/f6a 0 2024-04-06T03:40:35.082 INFO:tasks.workunit.client.0.smithi094.stdout:6/339: write dc/fd [3049326,11508] 0 2024-04-06T03:40:35.083 INFO:tasks.workunit.client.1.smithi136.stdout:6/309: truncate d2/d7/d26/d30/f50 2378203 0 2024-04-06T03:40:35.083 INFO:tasks.workunit.client.0.smithi094.stdout:4/473: dwrite d0/df/d13/d4a/f8b [0,4194304] 0 2024-04-06T03:40:35.083 INFO:tasks.workunit.client.0.smithi094.stdout:4/474: dread - d0/d15/d3a/d3e/d50/f65 zero size 2024-04-06T03:40:35.085 INFO:tasks.workunit.client.1.smithi136.stdout:8/415: creat d3/de/d18/d7d/f81 x:0 0 0 2024-04-06T03:40:35.085 INFO:tasks.workunit.client.1.smithi136.stdout:8/416: readlink d3/d2d/d5f/l62 0 2024-04-06T03:40:35.087 INFO:tasks.workunit.client.1.smithi136.stdout:4/320: truncate d1d/d72/f75 1967434 0 2024-04-06T03:40:35.087 INFO:tasks.workunit.client.0.smithi094.stdout:5/295: dread db/d19/d30/f33 [0,4194304] 0 2024-04-06T03:40:35.093 INFO:tasks.workunit.client.1.smithi136.stdout:9/373: link d17/d19/f1f d17/d20/d28/d49/d75/d60/f7a 0 2024-04-06T03:40:35.094 INFO:tasks.workunit.client.0.smithi094.stdout:5/296: dread db/f23 [0,4194304] 0 2024-04-06T03:40:35.095 INFO:tasks.workunit.client.0.smithi094.stdout:5/297: write db/d19/d24/d2e/f36 [846694,27931] 0 2024-04-06T03:40:35.095 INFO:tasks.workunit.client.0.smithi094.stdout:5/298: dread - db/d40/f43 zero size 2024-04-06T03:40:35.095 INFO:tasks.workunit.client.0.smithi094.stdout:5/299: chown db/c15 10618 1 2024-04-06T03:40:35.095 INFO:tasks.workunit.client.1.smithi136.stdout:3/318: getdents d0 0 2024-04-06T03:40:35.102 INFO:tasks.workunit.client.1.smithi136.stdout:8/417: dread f1 [0,4194304] 0 2024-04-06T03:40:35.105 INFO:tasks.workunit.client.1.smithi136.stdout:5/426: rmdir d1/dc/d1c/d32 39 2024-04-06T03:40:35.105 INFO:tasks.workunit.client.1.smithi136.stdout:5/427: write d1/d16/d4b/f80 [586931,22827] 0 2024-04-06T03:40:35.108 INFO:tasks.workunit.client.1.smithi136.stdout:7/365: dwrite d5/df/d3d/f63 [0,4194304] 0 2024-04-06T03:40:35.109 INFO:tasks.workunit.client.1.smithi136.stdout:6/310: dwrite d2/d7/d26/d30/d4f/f67 [0,4194304] 0 2024-04-06T03:40:35.109 INFO:tasks.workunit.client.1.smithi136.stdout:6/311: write d2/d7/f31 [630042,123907] 0 2024-04-06T03:40:35.109 INFO:tasks.workunit.client.1.smithi136.stdout:6/312: truncate d2/d7/f31 1408120 0 2024-04-06T03:40:35.109 INFO:tasks.workunit.client.0.smithi094.stdout:1/370: dwrite d0/f19 [4194304,4194304] 0 2024-04-06T03:40:35.116 INFO:tasks.workunit.client.1.smithi136.stdout:1/365: link d12/c6e d12/d1b/d1e/d27/c76 0 2024-04-06T03:40:35.116 INFO:tasks.workunit.client.1.smithi136.stdout:1/366: getdents d12/d3b/d60/d6d 0 2024-04-06T03:40:35.116 INFO:tasks.workunit.client.0.smithi094.stdout:9/327: symlink d2/d2c/d33/d5a/l76 0 2024-04-06T03:40:35.117 INFO:tasks.workunit.client.0.smithi094.stdout:6/340: mkdir dc/d11/d1b/d24/d46/d7b 0 2024-04-06T03:40:35.117 INFO:tasks.workunit.client.0.smithi094.stdout:6/341: creat dc/d11/d1b/d72/f7c x:0 0 0 2024-04-06T03:40:35.117 INFO:tasks.workunit.client.0.smithi094.stdout:6/342: chown c6 750189 1 2024-04-06T03:40:35.117 INFO:tasks.workunit.client.0.smithi094.stdout:6/343: chown dc/d11/d22/f57 3688 1 2024-04-06T03:40:35.118 INFO:tasks.workunit.client.1.smithi136.stdout:3/319: write d0/d4/d12/d19/d2e/f34 [1778161,67591] 0 2024-04-06T03:40:35.119 INFO:tasks.workunit.client.1.smithi136.stdout:4/321: dwrite d1d/d31/f41 [0,4194304] 0 2024-04-06T03:40:35.122 INFO:tasks.workunit.client.1.smithi136.stdout:8/418: dwrite d3/de/d18/f74 [0,4194304] 0 2024-04-06T03:40:35.126 INFO:tasks.workunit.client.1.smithi136.stdout:6/313: dwrite d2/d7/d13/f44 [0,4194304] 0 2024-04-06T03:40:35.130 INFO:tasks.workunit.client.0.smithi094.stdout:4/475: creat d0/d15/d3a/d3e/d50/fa4 x:0 0 0 2024-04-06T03:40:35.132 INFO:tasks.workunit.client.1.smithi136.stdout:5/428: symlink d1/dc/d15/d20/d66/l81 0 2024-04-06T03:40:35.133 INFO:tasks.workunit.client.1.smithi136.stdout:5/429: chown d1/dc/d15 5994 1 2024-04-06T03:40:35.133 INFO:tasks.workunit.client.1.smithi136.stdout:5/430: write d1/dc/d10/f11 [1064506,56444] 0 2024-04-06T03:40:35.133 INFO:tasks.workunit.client.0.smithi094.stdout:5/300: symlink db/d19/d30/l6b 0 2024-04-06T03:40:35.137 INFO:tasks.workunit.client.0.smithi094.stdout:1/371: unlink d0/d4/d1b/l34 0 2024-04-06T03:40:35.138 INFO:tasks.workunit.client.1.smithi136.stdout:7/366: mknod d5/df/d12/d59/d71/c83 0 2024-04-06T03:40:35.138 INFO:tasks.workunit.client.1.smithi136.stdout:7/367: fdatasync d5/df/d1d/f7f 0 2024-04-06T03:40:35.140 INFO:tasks.workunit.client.0.smithi094.stdout:9/328: rename d2/dc/f61 to d2/dc/d21/d23/d47/f77 0 2024-04-06T03:40:35.140 INFO:tasks.workunit.client.1.smithi136.stdout:6/314: dwrite d2/d25/f56 [0,4194304] 0 2024-04-06T03:40:35.145 INFO:tasks.workunit.client.1.smithi136.stdout:1/367: truncate f3 5949891 0 2024-04-06T03:40:35.146 INFO:tasks.workunit.client.0.smithi094.stdout:4/476: creat d0/d4/d67/d9f/fa5 x:0 0 0 2024-04-06T03:40:35.148 INFO:tasks.workunit.client.1.smithi136.stdout:3/320: link d0/d4/d12/d2d/d37/f62 d0/d4/d12/d19/d2c/f67 0 2024-04-06T03:40:35.148 INFO:tasks.workunit.client.1.smithi136.stdout:3/321: chown d0/d4/d12/d3a/d43 135359737 1 2024-04-06T03:40:35.150 INFO:tasks.workunit.client.0.smithi094.stdout:5/301: symlink db/d19/d2c/d5b/l6c 0 2024-04-06T03:40:35.151 INFO:tasks.workunit.client.0.smithi094.stdout:1/372: rename d0 to d0/d7/d1a/d7f 22 2024-04-06T03:40:35.153 INFO:tasks.workunit.client.0.smithi094.stdout:6/344: link c6 dc/d11/c7d 0 2024-04-06T03:40:35.154 INFO:tasks.workunit.client.1.smithi136.stdout:3/322: read d0/fb [1691533,92995] 0 2024-04-06T03:40:35.155 INFO:tasks.workunit.client.0.smithi094.stdout:5/302: write db/f1e [3866289,37920] 0 2024-04-06T03:40:35.155 INFO:tasks.workunit.client.0.smithi094.stdout:5/303: chown db/c10 1239978 1 2024-04-06T03:40:35.157 INFO:tasks.workunit.client.1.smithi136.stdout:4/322: creat d1d/d31/d78/f7a x:0 0 0 2024-04-06T03:40:35.159 INFO:tasks.workunit.client.0.smithi094.stdout:1/373: mknod d0/d1/d27/c80 0 2024-04-06T03:40:35.160 INFO:tasks.workunit.client.0.smithi094.stdout:1/374: chown d0/d1/d16/d3f/c6d 1 1 2024-04-06T03:40:35.163 INFO:tasks.workunit.client.0.smithi094.stdout:6/345: mknod dc/d11/d1b/c7e 0 2024-04-06T03:40:35.165 INFO:tasks.workunit.client.1.smithi136.stdout:8/419: rename d3/de/d18/d7d/f81 to d3/da/d26/d35/d3d/f82 0 2024-04-06T03:40:35.165 INFO:tasks.workunit.client.0.smithi094.stdout:6/346: fsync dc/d11/d1b/d72/f7c 0 2024-04-06T03:40:35.165 INFO:tasks.workunit.client.0.smithi094.stdout:6/347: chown dc/d11/d1b/d24/f32 29813 1 2024-04-06T03:40:35.172 INFO:tasks.workunit.client.0.smithi094.stdout:9/329: rename d2/d2c/d33/d5a to d2/dc/d21/d26/d78 0 2024-04-06T03:40:35.175 INFO:tasks.workunit.client.1.smithi136.stdout:8/420: dread d3/f5e [0,4194304] 0 2024-04-06T03:40:35.175 INFO:tasks.workunit.client.1.smithi136.stdout:8/421: write d3/de/f4b [988928,48234] 0 2024-04-06T03:40:35.176 INFO:tasks.workunit.client.0.smithi094.stdout:6/348: mkdir dc/d3a/d7f 0 2024-04-06T03:40:35.177 INFO:tasks.workunit.client.0.smithi094.stdout:1/375: dwrite d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:35.177 INFO:tasks.workunit.client.1.smithi136.stdout:0/349: sync 2024-04-06T03:40:35.179 INFO:tasks.workunit.client.1.smithi136.stdout:7/368: mknod d5/df/d12/c84 0 2024-04-06T03:40:35.179 INFO:tasks.workunit.client.0.smithi094.stdout:1/376: rmdir d0/d1/d16 39 2024-04-06T03:40:35.180 INFO:tasks.workunit.client.0.smithi094.stdout:1/377: read d0/d4/d1b/d2f/d3c/d48/f4f [249491,9437] 0 2024-04-06T03:40:35.180 INFO:tasks.workunit.client.0.smithi094.stdout:1/378: fsync d0/d7/d2d/f61 0 2024-04-06T03:40:35.180 INFO:tasks.workunit.client.0.smithi094.stdout:1/379: fdatasync d0/d7/d1a/d64/f72 0 2024-04-06T03:40:35.182 INFO:tasks.workunit.client.0.smithi094.stdout:5/304: rename db/d19/d2c/d5b to db/d19/d2c/d6d 0 2024-04-06T03:40:35.182 INFO:tasks.workunit.client.0.smithi094.stdout:5/305: fdatasync db/f23 0 2024-04-06T03:40:35.183 INFO:tasks.workunit.client.1.smithi136.stdout:6/315: creat d2/de/d3d/f6a x:0 0 0 2024-04-06T03:40:35.184 INFO:tasks.workunit.client.0.smithi094.stdout:6/349: read dc/d3a/d4a/d54/f58 [2396439,33349] 0 2024-04-06T03:40:35.185 INFO:tasks.workunit.client.0.smithi094.stdout:1/380: creat d0/d4/d1b/d2f/d76/d30/d38/f81 x:0 0 0 2024-04-06T03:40:35.185 INFO:tasks.workunit.client.0.smithi094.stdout:9/330: dwrite d2/d2c/d33/f4c [4194304,4194304] 0 2024-04-06T03:40:35.185 INFO:tasks.workunit.client.1.smithi136.stdout:6/316: fdatasync d2/d7/f31 0 2024-04-06T03:40:35.190 INFO:tasks.workunit.client.1.smithi136.stdout:2/467: sync 2024-04-06T03:40:35.191 INFO:tasks.workunit.client.0.smithi094.stdout:5/306: mknod db/d19/d24/d2e/c6e 0 2024-04-06T03:40:35.191 INFO:tasks.workunit.client.0.smithi094.stdout:5/307: chown db/d19/d21/f62 0 1 2024-04-06T03:40:35.191 INFO:tasks.workunit.client.0.smithi094.stdout:5/308: chown db/d19/d24/d2e/c1a 1850 1 2024-04-06T03:40:35.191 INFO:tasks.workunit.client.0.smithi094.stdout:5/309: truncate db/d19/d2c/d57/f58 5192458 0 2024-04-06T03:40:35.191 INFO:tasks.workunit.client.0.smithi094.stdout:5/310: creat db/f6f x:0 0 0 2024-04-06T03:40:35.191 INFO:tasks.workunit.client.0.smithi094.stdout:5/311: creat db/d19/d2c/d57/f70 x:0 0 0 2024-04-06T03:40:35.191 INFO:tasks.workunit.client.0.smithi094.stdout:1/381: rename d0/d4/d37/l49 to d0/d4/d1b/d2f/d3c/d48/d67/l82 0 2024-04-06T03:40:35.192 INFO:tasks.workunit.client.1.smithi136.stdout:1/368: mkdir d12/d3b/d44/d5f/d5a/d72/d77 0 2024-04-06T03:40:35.193 INFO:tasks.workunit.client.0.smithi094.stdout:9/331: symlink d2/dc/d21/d26/d78/d69/d6d/d6b/l79 0 2024-04-06T03:40:35.193 INFO:tasks.workunit.client.0.smithi094.stdout:9/332: creat d2/dc/d10/f7a x:0 0 0 2024-04-06T03:40:35.193 INFO:tasks.workunit.client.0.smithi094.stdout:9/333: fsync d2/d2c/d33/f44 0 2024-04-06T03:40:35.194 INFO:tasks.workunit.client.1.smithi136.stdout:3/323: symlink d0/d4/d12/d3a/l68 0 2024-04-06T03:40:35.197 INFO:tasks.workunit.client.1.smithi136.stdout:8/422: getdents d3/d2d/d5f 0 2024-04-06T03:40:35.198 INFO:tasks.workunit.client.0.smithi094.stdout:6/350: dwrite dc/f28 [0,4194304] 0 2024-04-06T03:40:35.198 INFO:tasks.workunit.client.0.smithi094.stdout:6/351: truncate dc/d11/d1b/f74 995135 0 2024-04-06T03:40:35.198 INFO:tasks.workunit.client.0.smithi094.stdout:6/352: write dc/d11/d22/d50/d61/f6c [1286546,72609] 0 2024-04-06T03:40:35.198 INFO:tasks.workunit.client.0.smithi094.stdout:6/353: creat dc/d11/d22/d50/f80 x:0 0 0 2024-04-06T03:40:35.199 INFO:tasks.workunit.client.0.smithi094.stdout:6/354: stat f5 0 2024-04-06T03:40:35.199 INFO:tasks.workunit.client.0.smithi094.stdout:6/355: write dc/d11/d1b/f74 [538996,126123] 0 2024-04-06T03:40:35.199 INFO:tasks.workunit.client.0.smithi094.stdout:6/356: creat dc/d11/d1b/d72/f81 x:0 0 0 2024-04-06T03:40:35.199 INFO:tasks.workunit.client.0.smithi094.stdout:6/357: write fa [1756844,130875] 0 2024-04-06T03:40:35.201 INFO:tasks.workunit.client.0.smithi094.stdout:5/312: unlink l5 0 2024-04-06T03:40:35.201 INFO:tasks.workunit.client.0.smithi094.stdout:5/313: write db/f69 [84562,72230] 0 2024-04-06T03:40:35.201 INFO:tasks.workunit.client.0.smithi094.stdout:5/314: write db/d19/d24/f53 [1461178,59068] 0 2024-04-06T03:40:35.201 INFO:tasks.workunit.client.0.smithi094.stdout:5/315: write db/d19/d2c/d6d/f5f [205750,66690] 0 2024-04-06T03:40:35.201 INFO:tasks.workunit.client.1.smithi136.stdout:5/431: mknod d1/c82 0 2024-04-06T03:40:35.201 INFO:tasks.workunit.client.1.smithi136.stdout:9/374: sync 2024-04-06T03:40:35.202 INFO:tasks.workunit.client.1.smithi136.stdout:0/350: mkdir d1/d6/d2c/d57/d43/d6d/d72/d78 0 2024-04-06T03:40:35.202 INFO:tasks.workunit.client.1.smithi136.stdout:0/351: chown d1/d4e/f55 8776170 1 2024-04-06T03:40:35.203 INFO:tasks.workunit.client.1.smithi136.stdout:0/352: stat d1/d6/d2c/d57 0 2024-04-06T03:40:35.203 INFO:tasks.workunit.client.1.smithi136.stdout:0/353: dread - d1/d6/d2c/d57/d43/f60 zero size 2024-04-06T03:40:35.204 INFO:tasks.workunit.client.0.smithi094.stdout:1/382: mknod d0/d4/d1b/d2f/c83 0 2024-04-06T03:40:35.204 INFO:tasks.workunit.client.0.smithi094.stdout:1/383: dread - d0/d7/f78 zero size 2024-04-06T03:40:35.206 INFO:tasks.workunit.client.1.smithi136.stdout:7/369: symlink d5/df/d1d/d6f/d2c/l85 0 2024-04-06T03:40:35.206 INFO:tasks.workunit.client.1.smithi136.stdout:7/370: write d5/df/d1d/d6f/d2c/f72 [358809,22986] 0 2024-04-06T03:40:35.207 INFO:tasks.workunit.client.1.smithi136.stdout:7/371: truncate d5/f82 909431 0 2024-04-06T03:40:35.208 INFO:tasks.workunit.client.1.smithi136.stdout:6/317: truncate d2/d7/d26/d30/d4f/f67 941690 0 2024-04-06T03:40:35.210 INFO:tasks.workunit.client.1.smithi136.stdout:1/369: dwrite f3 [4194304,4194304] 0 2024-04-06T03:40:35.210 INFO:tasks.workunit.client.0.smithi094.stdout:9/334: creat d2/f7b x:0 0 0 2024-04-06T03:40:35.214 INFO:tasks.workunit.client.1.smithi136.stdout:3/324: dread d0/d4/f45 [0,4194304] 0 2024-04-06T03:40:35.214 INFO:tasks.workunit.client.1.smithi136.stdout:3/325: chown d0/d4/d12/c4d 460 1 2024-04-06T03:40:35.215 INFO:tasks.workunit.client.1.smithi136.stdout:3/326: read d0/d4/d12/d19/f29 [1177018,92587] 0 2024-04-06T03:40:35.215 INFO:tasks.workunit.client.1.smithi136.stdout:3/327: write d0/d4/d12/f1a [1138975,17037] 0 2024-04-06T03:40:35.217 INFO:tasks.workunit.client.1.smithi136.stdout:0/354: symlink d1/d6/d2c/l79 0 2024-04-06T03:40:35.220 INFO:tasks.workunit.client.1.smithi136.stdout:9/375: dwrite d17/d20/d25/d3c/f77 [0,4194304] 0 2024-04-06T03:40:35.223 INFO:tasks.workunit.client.1.smithi136.stdout:5/432: dwrite d1/f61 [0,4194304] 0 2024-04-06T03:40:35.224 INFO:tasks.workunit.client.1.smithi136.stdout:6/318: link d2/d7/d26/d30/d4f/f57 d2/de/f6b 0 2024-04-06T03:40:35.229 INFO:tasks.workunit.client.1.smithi136.stdout:1/370: creat d12/d1b/d1e/d27/d37/f78 x:0 0 0 2024-04-06T03:40:35.231 INFO:tasks.workunit.client.0.smithi094.stdout:1/384: link d0/d7/d2d/l63 d0/d4/d1b/l84 0 2024-04-06T03:40:35.233 INFO:tasks.workunit.client.1.smithi136.stdout:4/323: rename d1d/d72 to d1d/d31/d3e/d56/d7b 0 2024-04-06T03:40:35.234 INFO:tasks.workunit.client.0.smithi094.stdout:9/335: creat d2/dc/d21/d23/f7c x:0 0 0 2024-04-06T03:40:35.237 INFO:tasks.workunit.client.1.smithi136.stdout:3/328: mkdir d0/d4/d12/d3a/d69 0 2024-04-06T03:40:35.237 INFO:tasks.workunit.client.1.smithi136.stdout:3/329: fsync d0/d4/d12/d19/d55/f24 0 2024-04-06T03:40:35.237 INFO:tasks.workunit.client.0.smithi094.stdout:6/358: mknod dc/d11/c82 0 2024-04-06T03:40:35.241 INFO:tasks.workunit.client.0.smithi094.stdout:5/316: rmdir db/d19/d21 39 2024-04-06T03:40:35.243 INFO:tasks.workunit.client.1.smithi136.stdout:9/376: dwrite d17/d20/d25/d3c/d55/f79 [0,4194304] 0 2024-04-06T03:40:35.243 INFO:tasks.workunit.client.1.smithi136.stdout:1/371: dwrite d12/d1b/d1e/d4d/f68 [0,4194304] 0 2024-04-06T03:40:35.243 INFO:tasks.workunit.client.1.smithi136.stdout:1/372: chown d12/c20 209497321 1 2024-04-06T03:40:35.243 INFO:tasks.workunit.client.0.smithi094.stdout:1/385: symlink d0/d4/d37/d5f/l85 0 2024-04-06T03:40:35.244 INFO:tasks.workunit.client.0.smithi094.stdout:1/386: creat d0/d7/f86 x:0 0 0 2024-04-06T03:40:35.246 INFO:tasks.workunit.client.0.smithi094.stdout:9/336: mknod d2/dc/d21/d23/d47/d51/c7d 0 2024-04-06T03:40:35.251 INFO:tasks.workunit.client.0.smithi094.stdout:5/317: unlink db/d19/d24/d2e/c1c 0 2024-04-06T03:40:35.254 INFO:tasks.workunit.client.0.smithi094.stdout:9/337: rename d2/dc/d10/f7a to d2/dc/d46/f7e 0 2024-04-06T03:40:35.254 INFO:tasks.workunit.client.0.smithi094.stdout:9/338: fdatasync d2/d2c/d33/f4c 0 2024-04-06T03:40:35.255 INFO:tasks.workunit.client.0.smithi094.stdout:5/318: unlink db/d19/d24/d2e/c1a 0 2024-04-06T03:40:35.258 INFO:tasks.workunit.client.0.smithi094.stdout:5/319: rename db/d19/d24/d28/c56 to db/d19/c71 0 2024-04-06T03:40:35.258 INFO:tasks.workunit.client.0.smithi094.stdout:5/320: dread - db/f29 zero size 2024-04-06T03:40:35.258 INFO:tasks.workunit.client.1.smithi136.stdout:4/324: dwrite d1d/d31/d3e/d55/f5c [0,4194304] 0 2024-04-06T03:40:35.259 INFO:tasks.workunit.client.0.smithi094.stdout:5/321: creat db/d19/d24/f72 x:0 0 0 2024-04-06T03:40:35.260 INFO:tasks.workunit.client.1.smithi136.stdout:7/372: rmdir d5 39 2024-04-06T03:40:35.260 INFO:tasks.workunit.client.1.smithi136.stdout:7/373: stat d5/df/d12/d1b/f2d 0 2024-04-06T03:40:35.260 INFO:tasks.workunit.client.1.smithi136.stdout:7/374: chown d5/df/d1d/d6f/d4d/d69/l7d 920606 1 2024-04-06T03:40:35.261 INFO:tasks.workunit.client.1.smithi136.stdout:5/433: getdents d1/d16/d17 0 2024-04-06T03:40:35.262 INFO:tasks.workunit.client.0.smithi094.stdout:5/322: creat db/d19/d2c/f73 x:0 0 0 2024-04-06T03:40:35.263 INFO:tasks.workunit.client.0.smithi094.stdout:1/387: dwrite d0/d7/f36 [0,4194304] 0 2024-04-06T03:40:35.265 INFO:tasks.workunit.client.0.smithi094.stdout:6/359: dwrite dc/d11/d1b/f52 [0,4194304] 0 2024-04-06T03:40:35.268 INFO:tasks.workunit.client.1.smithi136.stdout:3/330: mkdir d0/d4/d12/d3a/d43/d5d/d6a 0 2024-04-06T03:40:35.269 INFO:tasks.workunit.client.1.smithi136.stdout:4/325: write d1d/f20 [2665180,55539] 0 2024-04-06T03:40:35.270 INFO:tasks.workunit.client.0.smithi094.stdout:6/360: symlink dc/d3a/d4a/d54/d71/l83 0 2024-04-06T03:40:35.270 INFO:tasks.workunit.client.0.smithi094.stdout:6/361: fsync dc/d3a/f41 0 2024-04-06T03:40:35.270 INFO:tasks.workunit.client.0.smithi094.stdout:6/362: read dc/d11/d1b/f1f [422774,4033] 0 2024-04-06T03:40:35.271 INFO:tasks.workunit.client.0.smithi094.stdout:1/388: write d0/d4/d1b/d2f/d3c/d48/f69 [2231748,79533] 0 2024-04-06T03:40:35.271 INFO:tasks.workunit.client.0.smithi094.stdout:5/323: dread db/d19/f41 [0,4194304] 0 2024-04-06T03:40:35.272 INFO:tasks.workunit.client.0.smithi094.stdout:9/339: dwrite d2/dc/d10/f12 [0,4194304] 0 2024-04-06T03:40:35.273 INFO:tasks.workunit.client.0.smithi094.stdout:6/363: rename dc/d11/d1b/d24/d46/f55 to dc/d3a/d7f/f84 0 2024-04-06T03:40:35.276 INFO:tasks.workunit.client.1.smithi136.stdout:9/377: rename d17/d21/d32/d3f/l5c to d17/d21/d32/d3f/l7b 0 2024-04-06T03:40:35.278 INFO:tasks.workunit.client.1.smithi136.stdout:1/373: mkdir d12/d3b/d6b/d79 0 2024-04-06T03:40:35.278 INFO:tasks.workunit.client.1.smithi136.stdout:1/374: write d12/d1b/d28/d48/f58 [3602967,34009] 0 2024-04-06T03:40:35.281 INFO:tasks.workunit.client.1.smithi136.stdout:5/434: truncate d1/d16/d4b/f76 2403152 0 2024-04-06T03:40:35.285 INFO:tasks.workunit.client.0.smithi094.stdout:6/364: mknod dc/d11/d1b/d24/d46/d7b/c85 0 2024-04-06T03:40:35.285 INFO:tasks.workunit.client.0.smithi094.stdout:6/365: chown dc/l26 61901 1 2024-04-06T03:40:35.285 INFO:tasks.workunit.client.0.smithi094.stdout:6/366: truncate dc/d11/d1b/d72/f77 672856 0 2024-04-06T03:40:35.286 INFO:tasks.workunit.client.0.smithi094.stdout:5/324: getdents db/d19/d30 0 2024-04-06T03:40:35.288 INFO:tasks.workunit.client.1.smithi136.stdout:2/468: rename d3/d9/d10 to d3/d4a/d9d 0 2024-04-06T03:40:35.292 INFO:tasks.workunit.client.0.smithi094.stdout:1/389: rename d0/d4/d1b/l42 to d0/d7/d1a/l87 0 2024-04-06T03:40:35.292 INFO:tasks.workunit.client.0.smithi094.stdout:1/390: creat d0/d4/d1b/d2f/d3c/d48/f88 x:0 0 0 2024-04-06T03:40:35.292 INFO:tasks.workunit.client.0.smithi094.stdout:1/391: creat d0/d4/d1b/d2f/d3c/d48/f89 x:0 0 0 2024-04-06T03:40:35.295 INFO:tasks.workunit.client.1.smithi136.stdout:4/326: mknod d1d/d31/d3e/d56/d7b/c7c 0 2024-04-06T03:40:35.295 INFO:tasks.workunit.client.1.smithi136.stdout:4/327: creat d1d/d2f/f7d x:0 0 0 2024-04-06T03:40:35.296 INFO:tasks.workunit.client.1.smithi136.stdout:4/328: creat d1d/d2f/d6c/f7e x:0 0 0 2024-04-06T03:40:35.296 INFO:tasks.workunit.client.1.smithi136.stdout:4/329: fdatasync d1d/d70/f6a 0 2024-04-06T03:40:35.296 INFO:tasks.workunit.client.1.smithi136.stdout:4/330: write d1d/d31/f67 [1023043,120449] 0 2024-04-06T03:40:35.296 INFO:tasks.workunit.client.0.smithi094.stdout:9/340: dwrite d2/dc/d21/d23/d2f/f5f [4194304,4194304] 0 2024-04-06T03:40:35.299 INFO:tasks.workunit.client.1.smithi136.stdout:3/331: dread d0/d4/f22 [8388608,4194304] 0 2024-04-06T03:40:35.299 INFO:tasks.workunit.client.1.smithi136.stdout:9/378: unlink d17/d20/d25/d3d/d4b/l68 0 2024-04-06T03:40:35.300 INFO:tasks.workunit.client.1.smithi136.stdout:9/379: readlink d17/d20/d25/l2b 0 2024-04-06T03:40:35.301 INFO:tasks.workunit.client.1.smithi136.stdout:1/375: dwrite d12/f24 [0,4194304] 0 2024-04-06T03:40:35.301 INFO:tasks.workunit.client.0.smithi094.stdout:5/325: dwrite db/d19/d24/f37 [0,4194304] 0 2024-04-06T03:40:35.301 INFO:tasks.workunit.client.0.smithi094.stdout:5/326: fsync db/d19/d2c/d57/f58 0 2024-04-06T03:40:35.305 INFO:tasks.workunit.client.1.smithi136.stdout:5/435: creat d1/d16/d4b/d7b/f83 x:0 0 0 2024-04-06T03:40:35.306 INFO:tasks.workunit.client.1.smithi136.stdout:8/423: rename d3/da/d26/d35/d39/l52 to d3/da/d26/d35/d3d/d48/d6c/d79/l83 0 2024-04-06T03:40:35.311 INFO:tasks.workunit.client.1.smithi136.stdout:2/469: creat d3/d9/d3b/f9e x:0 0 0 2024-04-06T03:40:35.311 INFO:tasks.workunit.client.0.smithi094.stdout:5/327: mknod db/d19/d2c/d6d/c74 0 2024-04-06T03:40:35.314 INFO:tasks.workunit.client.1.smithi136.stdout:4/331: mkdir d1d/d31/d3e/d55/d7f 0 2024-04-06T03:40:35.314 INFO:tasks.workunit.client.0.smithi094.stdout:1/392: dwrite d0/d4/d1b/d2f/f54 [0,4194304] 0 2024-04-06T03:40:35.316 INFO:tasks.workunit.client.0.smithi094.stdout:5/328: creat db/d19/d24/d2e/f75 x:0 0 0 2024-04-06T03:40:35.316 INFO:tasks.workunit.client.0.smithi094.stdout:5/329: creat db/f76 x:0 0 0 2024-04-06T03:40:35.316 INFO:tasks.workunit.client.0.smithi094.stdout:5/330: stat db/d19/d24/d2e/f48 0 2024-04-06T03:40:35.318 INFO:tasks.workunit.client.0.smithi094.stdout:6/367: dread dc/d11/f4b [0,4194304] 0 2024-04-06T03:40:35.318 INFO:tasks.workunit.client.1.smithi136.stdout:3/332: unlink d0/d4/d12/d19/d2e/f34 0 2024-04-06T03:40:35.319 INFO:tasks.workunit.client.1.smithi136.stdout:4/332: dread d1d/d31/d3e/d56/f61 [0,4194304] 0 2024-04-06T03:40:35.319 INFO:tasks.workunit.client.1.smithi136.stdout:4/333: dread - d1d/d2f/f43 zero size 2024-04-06T03:40:35.319 INFO:tasks.workunit.client.1.smithi136.stdout:3/333: write d0/d4/d12/f1a [1846437,15834] 0 2024-04-06T03:40:35.320 INFO:tasks.workunit.client.1.smithi136.stdout:0/355: sync 2024-04-06T03:40:35.321 INFO:tasks.workunit.client.1.smithi136.stdout:1/376: dwrite d12/d1b/d1e/d27/d37/f46 [4194304,4194304] 0 2024-04-06T03:40:35.322 INFO:tasks.workunit.client.1.smithi136.stdout:1/377: readlink d12/l18 0 2024-04-06T03:40:35.322 INFO:tasks.workunit.client.1.smithi136.stdout:9/380: creat d17/d21/f7c x:0 0 0 2024-04-06T03:40:35.324 INFO:tasks.workunit.client.1.smithi136.stdout:1/378: write d12/d3b/d44/f67 [154598,32946] 0 2024-04-06T03:40:35.325 INFO:tasks.workunit.client.0.smithi094.stdout:1/393: creat d0/d4/d1b/d2f/d4b/f8a x:0 0 0 2024-04-06T03:40:35.327 INFO:tasks.workunit.client.0.smithi094.stdout:9/341: dwrite d2/d2c/d33/f44 [0,4194304] 0 2024-04-06T03:40:35.330 INFO:tasks.workunit.client.0.smithi094.stdout:6/368: symlink dc/l86 0 2024-04-06T03:40:35.332 INFO:tasks.workunit.client.1.smithi136.stdout:6/319: rename d2/cf to d2/de/c6c 0 2024-04-06T03:40:35.333 INFO:tasks.workunit.client.0.smithi094.stdout:1/394: creat d0/d4/f8b x:0 0 0 2024-04-06T03:40:35.334 INFO:tasks.workunit.client.0.smithi094.stdout:1/395: dread - d0/d7/f78 zero size 2024-04-06T03:40:35.334 INFO:tasks.workunit.client.1.smithi136.stdout:8/424: getdents d3/d2d 0 2024-04-06T03:40:35.335 INFO:tasks.workunit.client.1.smithi136.stdout:8/425: fsync d3/de/f4b 0 2024-04-06T03:40:35.337 INFO:tasks.workunit.client.1.smithi136.stdout:0/356: creat d1/d6/d2c/d57/d43/d6d/d72/d78/f7a x:0 0 0 2024-04-06T03:40:35.338 INFO:tasks.workunit.client.0.smithi094.stdout:5/331: dwrite db/d19/d2c/f2d [0,4194304] 0 2024-04-06T03:40:35.338 INFO:tasks.workunit.client.0.smithi094.stdout:5/332: truncate db/d19/d2c/d6d/f5f 410032 0 2024-04-06T03:40:35.339 INFO:tasks.workunit.client.0.smithi094.stdout:9/342: creat d2/d2c/d33/d37/f7f x:0 0 0 2024-04-06T03:40:35.339 INFO:tasks.workunit.client.1.smithi136.stdout:5/436: dwrite d1/f61 [0,4194304] 0 2024-04-06T03:40:35.339 INFO:tasks.workunit.client.1.smithi136.stdout:2/470: dwrite d3/d4a/d9d/d18/d4b/f85 [0,4194304] 0 2024-04-06T03:40:35.340 INFO:tasks.workunit.client.0.smithi094.stdout:6/369: mkdir dc/d11/d1b/d72/d87 0 2024-04-06T03:40:35.340 INFO:tasks.workunit.client.0.smithi094.stdout:6/370: write dc/d11/d1b/d24/d46/f48 [707911,70700] 0 2024-04-06T03:40:35.342 INFO:tasks.workunit.client.1.smithi136.stdout:9/381: link f13 d17/d20/d28/d49/d75/d60/f7d 0 2024-04-06T03:40:35.342 INFO:tasks.workunit.client.1.smithi136.stdout:9/382: stat d17/d20/d25/d3d/l5e 0 2024-04-06T03:40:35.342 INFO:tasks.workunit.client.0.smithi094.stdout:1/396: mkdir d0/d1/d16/d3f/d8c 0 2024-04-06T03:40:35.345 INFO:tasks.workunit.client.1.smithi136.stdout:3/334: rename d0/l2a to d0/d4/d12/d3a/l6b 0 2024-04-06T03:40:35.345 INFO:tasks.workunit.client.1.smithi136.stdout:3/335: chown d0/d4/d12/d19/d2c 3 1 2024-04-06T03:40:35.347 INFO:tasks.workunit.client.1.smithi136.stdout:6/320: truncate d2/d7/d13/f44 1318079 0 2024-04-06T03:40:35.347 INFO:tasks.workunit.client.1.smithi136.stdout:9/383: dread d17/d19/f27 [0,4194304] 0 2024-04-06T03:40:35.347 INFO:tasks.workunit.client.1.smithi136.stdout:9/384: write d17/d19/f61 [1840701,55564] 0 2024-04-06T03:40:35.348 INFO:tasks.workunit.client.0.smithi094.stdout:9/343: write d2/dc/d21/d26/d78/d69/d6d/f3b [1323736,1379] 0 2024-04-06T03:40:35.349 INFO:tasks.workunit.client.1.smithi136.stdout:4/334: rmdir d1d/d31/d78 39 2024-04-06T03:40:35.349 INFO:tasks.workunit.client.1.smithi136.stdout:4/335: write d1d/d31/d3e/d56/f65 [1213922,12429] 0 2024-04-06T03:40:35.349 INFO:tasks.workunit.client.1.smithi136.stdout:4/336: creat d1d/d2f/f80 x:0 0 0 2024-04-06T03:40:35.350 INFO:tasks.workunit.client.1.smithi136.stdout:0/357: mknod d1/d6/c7b 0 2024-04-06T03:40:35.356 INFO:tasks.workunit.client.1.smithi136.stdout:9/385: dread d17/d20/d28/d49/d75/d60/f7a [0,4194304] 0 2024-04-06T03:40:35.362 INFO:tasks.workunit.client.0.smithi094.stdout:6/371: mkdir dc/d11/d1b/d72/d88 0 2024-04-06T03:40:35.362 INFO:tasks.workunit.client.0.smithi094.stdout:6/372: fsync dc/d3a/d4a/f6b 0 2024-04-06T03:40:35.362 INFO:tasks.workunit.client.0.smithi094.stdout:6/373: readlink dc/l86 0 2024-04-06T03:40:35.362 INFO:tasks.workunit.client.0.smithi094.stdout:6/374: fdatasync f8 0 2024-04-06T03:40:35.363 INFO:tasks.workunit.client.0.smithi094.stdout:1/397: creat d0/d4/d1b/d2f/d76/d30/d38/f8d x:0 0 0 2024-04-06T03:40:35.363 INFO:tasks.workunit.client.0.smithi094.stdout:1/398: readlink d0/d4/d1b/d2f/d76/l28 0 2024-04-06T03:40:35.365 INFO:tasks.workunit.client.1.smithi136.stdout:2/471: symlink d3/d4a/d77/l9f 0 2024-04-06T03:40:35.366 INFO:tasks.workunit.client.0.smithi094.stdout:9/344: mkdir d2/dc/d80 0 2024-04-06T03:40:35.366 INFO:tasks.workunit.client.0.smithi094.stdout:9/345: truncate d2/dc/d46/f7e 489413 0 2024-04-06T03:40:35.368 INFO:tasks.workunit.client.0.smithi094.stdout:6/375: truncate dc/d11/d1b/d24/f25 3966558 0 2024-04-06T03:40:35.368 INFO:tasks.workunit.client.1.smithi136.stdout:3/336: fdatasync d0/d4/fa 0 2024-04-06T03:40:35.369 INFO:tasks.workunit.client.0.smithi094.stdout:5/333: rename db/d19/d24/d28/d65 to db/d40/d77 0 2024-04-06T03:40:35.369 INFO:tasks.workunit.client.0.smithi094.stdout:5/334: chown db/d19/d2c/d6d/c74 14523381 1 2024-04-06T03:40:35.369 INFO:tasks.workunit.client.0.smithi094.stdout:5/335: dread - db/d40/f43 zero size 2024-04-06T03:40:35.371 INFO:tasks.workunit.client.1.smithi136.stdout:6/321: creat d2/d7/d26/d30/d4f/f6d x:0 0 0 2024-04-06T03:40:35.371 INFO:tasks.workunit.client.1.smithi136.stdout:6/322: dread - d2/de/d3d/f6a zero size 2024-04-06T03:40:35.371 INFO:tasks.workunit.client.0.smithi094.stdout:6/376: truncate dc/d11/d22/d50/d61/f6c 436579 0 2024-04-06T03:40:35.373 INFO:tasks.workunit.client.1.smithi136.stdout:3/337: mknod d0/d4/d12/d3a/d51/c6c 0 2024-04-06T03:40:35.373 INFO:tasks.workunit.client.1.smithi136.stdout:3/338: chown d0/d4/d12/d2d/f39 0 1 2024-04-06T03:40:35.373 INFO:tasks.workunit.client.1.smithi136.stdout:9/386: dwrite d17/d20/d28/d49/f54 [0,4194304] 0 2024-04-06T03:40:35.373 INFO:tasks.workunit.client.1.smithi136.stdout:9/387: fsync d17/d21/d32/d3f/f44 0 2024-04-06T03:40:35.374 INFO:tasks.workunit.client.1.smithi136.stdout:1/379: rename d12/d1b/d28/d48 to d12/d3b/d74/d7a 0 2024-04-06T03:40:35.374 INFO:tasks.workunit.client.1.smithi136.stdout:1/380: fdatasync d12/d3b/d44/f54 0 2024-04-06T03:40:35.376 INFO:tasks.workunit.client.1.smithi136.stdout:3/339: truncate d0/d4/f22 5097061 0 2024-04-06T03:40:35.376 INFO:tasks.workunit.client.1.smithi136.stdout:3/340: fsync d0/d4/d12/f20 0 2024-04-06T03:40:35.377 INFO:tasks.workunit.client.1.smithi136.stdout:9/388: truncate d17/d20/d25/d3c/f77 2706602 0 2024-04-06T03:40:35.377 INFO:tasks.workunit.client.1.smithi136.stdout:9/389: chown d17/d20/d25/c2c 2050999 1 2024-04-06T03:40:35.377 INFO:tasks.workunit.client.1.smithi136.stdout:3/341: creat d0/d4/d12/d2d/d37/f6d x:0 0 0 2024-04-06T03:40:35.379 INFO:tasks.workunit.client.1.smithi136.stdout:8/426: rename d3/da/d26/d35/d3d/d48/d6c/l78 to d3/da/d26/d35/d39/l84 0 2024-04-06T03:40:35.379 INFO:tasks.workunit.client.1.smithi136.stdout:8/427: chown d3/de/d18/c30 583671 1 2024-04-06T03:40:35.379 INFO:tasks.workunit.client.1.smithi136.stdout:8/428: getdents d3/da/d26/d35/d3d/d48/d6c/d7f 0 2024-04-06T03:40:35.379 INFO:tasks.workunit.client.1.smithi136.stdout:8/429: readlink d3/da/d26/d35/l46 0 2024-04-06T03:40:35.379 INFO:tasks.workunit.client.1.smithi136.stdout:8/430: creat d3/da/d26/f85 x:0 0 0 2024-04-06T03:40:35.382 INFO:tasks.workunit.client.0.smithi094.stdout:9/346: dread d2/f14 [0,4194304] 0 2024-04-06T03:40:35.382 INFO:tasks.workunit.client.0.smithi094.stdout:9/347: dread - d2/dc/d46/f60 zero size 2024-04-06T03:40:35.383 INFO:tasks.workunit.client.1.smithi136.stdout:3/342: symlink d0/d4/d12/d3a/d43/d5d/d6a/l6e 0 2024-04-06T03:40:35.387 INFO:tasks.workunit.client.1.smithi136.stdout:4/337: write d1d/f52 [2299719,92505] 0 2024-04-06T03:40:35.390 INFO:tasks.workunit.client.1.smithi136.stdout:0/358: dwrite d1/d6/d2c/d1a/d36/d59/f23 [0,4194304] 0 2024-04-06T03:40:35.390 INFO:tasks.workunit.client.1.smithi136.stdout:9/390: write d17/d20/d28/d49/d75/d60/f70 [2177723,24968] 0 2024-04-06T03:40:35.393 INFO:tasks.workunit.client.0.smithi094.stdout:5/336: dwrite db/d19/d24/d2e/f36 [0,4194304] 0 2024-04-06T03:40:35.397 INFO:tasks.workunit.client.0.smithi094.stdout:1/399: dwrite d0/d4/d1b/d2f/d3c/d48/f4f [0,4194304] 0 2024-04-06T03:40:35.399 INFO:tasks.workunit.client.1.smithi136.stdout:2/472: dwrite d3/d4a/d9d/f6b [0,4194304] 0 2024-04-06T03:40:35.399 INFO:tasks.workunit.client.0.smithi094.stdout:6/377: dwrite f5 [0,4194304] 0 2024-04-06T03:40:35.404 INFO:tasks.workunit.client.1.smithi136.stdout:0/359: dread d1/d6/d2c/d1a/f39 [4194304,4194304] 0 2024-04-06T03:40:35.408 INFO:tasks.workunit.client.0.smithi094.stdout:6/378: dread f8 [0,4194304] 0 2024-04-06T03:40:35.409 INFO:tasks.workunit.client.0.smithi094.stdout:6/379: stat dc/d11/d1b/c2c 0 2024-04-06T03:40:35.409 INFO:tasks.workunit.client.0.smithi094.stdout:6/380: readlink dc/l3e 0 2024-04-06T03:40:35.457 INFO:tasks.workunit.client.1.smithi136.stdout:5/437: rename d1/f61 to d1/dc/d15/d20/f84 0 2024-04-06T03:40:35.457 INFO:tasks.workunit.client.1.smithi136.stdout:5/438: write d1/dc/d15/f73 [147934,79105] 0 2024-04-06T03:40:35.459 INFO:tasks.workunit.client.0.smithi094.stdout:9/348: symlink d2/dc/d10/d6a/l81 0 2024-04-06T03:40:35.460 INFO:tasks.workunit.client.1.smithi136.stdout:8/431: mkdir d3/da/d26/d35/d53/d86 0 2024-04-06T03:40:35.462 INFO:tasks.workunit.client.1.smithi136.stdout:3/343: fsync d0/d4/fe 0 2024-04-06T03:40:35.462 INFO:tasks.workunit.client.1.smithi136.stdout:3/344: chown d0/d4/d12/d2d/f39 2382074 1 2024-04-06T03:40:35.462 INFO:tasks.workunit.client.1.smithi136.stdout:3/345: truncate d0/d4/d12/d2d/f39 413250 0 2024-04-06T03:40:35.466 INFO:tasks.workunit.client.1.smithi136.stdout:4/338: creat d1d/d31/d3e/d56/f81 x:0 0 0 2024-04-06T03:40:35.473 INFO:tasks.workunit.client.1.smithi136.stdout:2/473: link d3/d15/f30 d3/d4a/d9d/d18/d1d/d3c/fa0 0 2024-04-06T03:40:35.473 INFO:tasks.workunit.client.1.smithi136.stdout:7/375: sync 2024-04-06T03:40:35.475 INFO:tasks.workunit.client.0.smithi094.stdout:5/337: creat db/d19/d2c/d6d/f78 x:0 0 0 2024-04-06T03:40:35.475 INFO:tasks.workunit.client.0.smithi094.stdout:5/338: creat db/d19/d2c/f79 x:0 0 0 2024-04-06T03:40:35.475 INFO:tasks.workunit.client.0.smithi094.stdout:5/339: read - db/f6f zero size 2024-04-06T03:40:35.475 INFO:tasks.workunit.client.0.smithi094.stdout:5/340: creat db/f7a x:0 0 0 2024-04-06T03:40:35.476 INFO:tasks.workunit.client.1.smithi136.stdout:0/360: mkdir d1/d6/d2c/d1a/d7c 0 2024-04-06T03:40:35.481 INFO:tasks.workunit.client.1.smithi136.stdout:5/439: creat d1/d16/f85 x:0 0 0 2024-04-06T03:40:35.482 INFO:tasks.workunit.client.1.smithi136.stdout:7/376: dread d5/df/f10 [0,4194304] 0 2024-04-06T03:40:35.483 INFO:tasks.workunit.client.1.smithi136.stdout:8/432: dwrite d3/f5e [0,4194304] 0 2024-04-06T03:40:35.483 INFO:tasks.workunit.client.0.smithi094.stdout:1/400: rename d0/d4/d1b/d2f/d3c/d48/f4f to d0/d1/d16/d3f/d8c/f8e 0 2024-04-06T03:40:35.485 INFO:tasks.workunit.client.1.smithi136.stdout:3/346: mkdir d0/d4/d12/d19/d55/d6f 0 2024-04-06T03:40:35.486 INFO:tasks.workunit.client.1.smithi136.stdout:7/377: dread d5/df/f10 [0,4194304] 0 2024-04-06T03:40:35.486 INFO:tasks.workunit.client.1.smithi136.stdout:7/378: fdatasync d5/df/f41 0 2024-04-06T03:40:35.486 INFO:tasks.workunit.client.1.smithi136.stdout:7/379: chown d5/l1e 2 1 2024-04-06T03:40:35.488 INFO:tasks.workunit.client.1.smithi136.stdout:9/391: symlink d17/d21/d29/d34/d46/l7e 0 2024-04-06T03:40:35.489 INFO:tasks.workunit.client.0.smithi094.stdout:8/418: sync 2024-04-06T03:40:35.489 INFO:tasks.workunit.client.1.smithi136.stdout:4/339: dwrite d1d/f52 [0,4194304] 0 2024-04-06T03:40:35.490 INFO:tasks.workunit.client.0.smithi094.stdout:6/381: creat dc/d3a/f89 x:0 0 0 2024-04-06T03:40:35.490 INFO:tasks.workunit.client.0.smithi094.stdout:6/382: chown dc/d11/d1b/l1d 7944 1 2024-04-06T03:40:35.491 INFO:tasks.workunit.client.1.smithi136.stdout:2/474: truncate d3/d4a/d9d/d18/d1d/d3c/f40 436409 0 2024-04-06T03:40:35.495 INFO:tasks.workunit.client.1.smithi136.stdout:0/361: symlink d1/d4e/l7d 0 2024-04-06T03:40:35.496 INFO:tasks.workunit.client.1.smithi136.stdout:0/362: chown d1/f18 112 1 2024-04-06T03:40:35.496 INFO:tasks.workunit.client.1.smithi136.stdout:0/363: write d1/f63 [159971,92313] 0 2024-04-06T03:40:35.496 INFO:tasks.workunit.client.1.smithi136.stdout:0/364: dread - d1/d6/d2c/d1a/d36/d59/f38 zero size 2024-04-06T03:40:35.496 INFO:tasks.workunit.client.1.smithi136.stdout:5/440: dread d1/f4 [0,4194304] 0 2024-04-06T03:40:35.498 INFO:tasks.workunit.client.0.smithi094.stdout:1/401: dwrite d0/d1/d16/d40/f7a [0,4194304] 0 2024-04-06T03:40:35.499 INFO:tasks.workunit.client.0.smithi094.stdout:5/341: unlink c7 0 2024-04-06T03:40:35.499 INFO:tasks.workunit.client.0.smithi094.stdout:5/342: truncate db/d19/d21/f62 903291 0 2024-04-06T03:40:35.502 INFO:tasks.workunit.client.1.smithi136.stdout:2/475: dread d3/d4a/d9d/d18/d1d/f20 [0,4194304] 0 2024-04-06T03:40:35.504 INFO:tasks.workunit.client.0.smithi094.stdout:9/349: rename d2/dc/d10/l15 to d2/dc/d21/d23/d2f/l82 0 2024-04-06T03:40:35.504 INFO:tasks.workunit.client.0.smithi094.stdout:9/350: stat d2/dc/f56 0 2024-04-06T03:40:35.504 INFO:tasks.workunit.client.0.smithi094.stdout:9/351: creat d2/dc/d21/d23/d2f/f83 x:0 0 0 2024-04-06T03:40:35.506 INFO:tasks.workunit.client.0.smithi094.stdout:6/383: creat dc/d11/d1b/d72/d87/f8a x:0 0 0 2024-04-06T03:40:35.506 INFO:tasks.workunit.client.0.smithi094.stdout:6/384: fsync f7 0 2024-04-06T03:40:35.510 INFO:tasks.workunit.client.1.smithi136.stdout:8/433: truncate d3/da/d26/f28 3663808 0 2024-04-06T03:40:35.510 INFO:tasks.workunit.client.1.smithi136.stdout:8/434: getdents d3/da/d26/d35/d53/d86 0 2024-04-06T03:40:35.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/402: mkdir d0/d1/d8f 0 2024-04-06T03:40:35.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/403: write d0/d1/f65 [1019354,67339] 0 2024-04-06T03:40:35.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/404: read - d0/d1/f73 zero size 2024-04-06T03:40:35.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/405: creat d0/d7/f90 x:0 0 0 2024-04-06T03:40:35.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/406: chown d0/d4/f39 26 1 2024-04-06T03:40:35.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/407: write d0/d4/d1b/d2f/d76/d30/d38/f8d [437588,28839] 0 2024-04-06T03:40:35.516 INFO:tasks.workunit.client.1.smithi136.stdout:2/476: dread d3/d4a/d9d/f6b [0,4194304] 0 2024-04-06T03:40:35.516 INFO:tasks.workunit.client.1.smithi136.stdout:2/477: truncate d3/f83 799365 0 2024-04-06T03:40:35.517 INFO:tasks.workunit.client.1.smithi136.stdout:2/478: stat d3/f68 0 2024-04-06T03:40:35.520 INFO:tasks.workunit.client.1.smithi136.stdout:7/380: dwrite d5/df/d1d/f1f [0,4194304] 0 2024-04-06T03:40:35.524 INFO:tasks.workunit.client.0.smithi094.stdout:9/352: dwrite d2/dc/d21/d26/f29 [0,4194304] 0 2024-04-06T03:40:35.524 INFO:tasks.workunit.client.1.smithi136.stdout:0/365: dwrite d1/d4e/f75 [0,4194304] 0 2024-04-06T03:40:35.525 INFO:tasks.workunit.client.1.smithi136.stdout:0/366: dread - d1/d6/d2c/d1a/d36/d59/f38 zero size 2024-04-06T03:40:35.525 INFO:tasks.workunit.client.1.smithi136.stdout:0/367: dread - d1/d4e/f55 zero size 2024-04-06T03:40:35.525 INFO:tasks.workunit.client.1.smithi136.stdout:3/347: dwrite d0/d4/f22 [0,4194304] 0 2024-04-06T03:40:35.525 INFO:tasks.workunit.client.1.smithi136.stdout:3/348: fdatasync d0/d4/d12/d19/d2e/d3b/d49/f5e 0 2024-04-06T03:40:35.532 INFO:tasks.workunit.client.0.smithi094.stdout:1/408: dwrite d0/d4/d1b/f6f [0,4194304] 0 2024-04-06T03:40:35.533 INFO:tasks.workunit.client.0.smithi094.stdout:1/409: creat d0/d4/d1b/d2f/d4b/f91 x:0 0 0 2024-04-06T03:40:35.535 INFO:tasks.workunit.client.1.smithi136.stdout:2/479: dwrite d3/d4a/d9d/f22 [0,4194304] 0 2024-04-06T03:40:35.573 INFO:tasks.workunit.client.1.smithi136.stdout:9/392: symlink d17/d19/l7f 0 2024-04-06T03:40:35.574 INFO:tasks.workunit.client.1.smithi136.stdout:9/393: write d17/d20/d25/d3c/d55/f79 [4508339,130049] 0 2024-04-06T03:40:35.574 INFO:tasks.workunit.client.0.smithi094.stdout:8/419: rename d18/f87 to d18/d30/d68/f92 0 2024-04-06T03:40:35.575 INFO:tasks.workunit.client.1.smithi136.stdout:4/340: symlink d1d/d31/l82 0 2024-04-06T03:40:35.575 INFO:tasks.workunit.client.1.smithi136.stdout:4/341: chown d1d/d2f/c69 12766 1 2024-04-06T03:40:35.576 INFO:tasks.workunit.client.1.smithi136.stdout:4/342: write d1d/d2f/f43 [820365,62740] 0 2024-04-06T03:40:35.579 INFO:tasks.workunit.client.1.smithi136.stdout:6/323: rename d2/de to d2/d7/d26/d30/d6e 0 2024-04-06T03:40:35.580 INFO:tasks.workunit.client.1.smithi136.stdout:8/435: creat d3/da/d26/d35/d53/d86/f87 x:0 0 0 2024-04-06T03:40:35.588 INFO:tasks.workunit.client.0.smithi094.stdout:1/410: mknod d0/d7/c92 0 2024-04-06T03:40:35.588 INFO:tasks.workunit.client.0.smithi094.stdout:1/411: chown d0/d7/d2d/l44 468248885 1 2024-04-06T03:40:35.589 INFO:tasks.workunit.client.1.smithi136.stdout:0/368: rename d1/d6/d2c/d57/d43/l69 to d1/d6/d2c/d57/d43/l7e 0 2024-04-06T03:40:35.591 INFO:tasks.workunit.client.1.smithi136.stdout:3/349: creat d0/d4/d12/d3a/d51/f70 x:0 0 0 2024-04-06T03:40:35.591 INFO:tasks.workunit.client.1.smithi136.stdout:2/480: write d3/d4a/d9d/d18/d1d/d3c/f40 [1258967,127500] 0 2024-04-06T03:40:35.593 INFO:tasks.workunit.client.1.smithi136.stdout:8/436: dwrite d3/de/d18/f74 [0,4194304] 0 2024-04-06T03:40:35.594 INFO:tasks.workunit.client.1.smithi136.stdout:3/350: dread d0/d4/d12/d2d/f33 [0,4194304] 0 2024-04-06T03:40:35.594 INFO:tasks.workunit.client.0.smithi094.stdout:5/343: rename c4 to db/d40/d77/c7b 0 2024-04-06T03:40:35.594 INFO:tasks.workunit.client.0.smithi094.stdout:5/344: chown db/c10 99 1 2024-04-06T03:40:35.595 INFO:tasks.workunit.client.0.smithi094.stdout:5/345: truncate db/fd 2273845 0 2024-04-06T03:40:35.597 INFO:tasks.workunit.client.0.smithi094.stdout:1/412: creat d0/f93 x:0 0 0 2024-04-06T03:40:35.597 INFO:tasks.workunit.client.0.smithi094.stdout:1/413: write d0/d7/f78 [686783,39261] 0 2024-04-06T03:40:35.597 INFO:tasks.workunit.client.0.smithi094.stdout:1/414: truncate d0/d1/d16/d3f/f5c 909417 0 2024-04-06T03:40:35.597 INFO:tasks.workunit.client.0.smithi094.stdout:1/415: stat d0/d4/c75 0 2024-04-06T03:40:35.597 INFO:tasks.workunit.client.0.smithi094.stdout:1/416: truncate d0/d4/d1b/d2f/f77 348267 0 2024-04-06T03:40:35.602 INFO:tasks.workunit.client.0.smithi094.stdout:8/420: rename f11 to d18/d5c/d75/f93 0 2024-04-06T03:40:35.602 INFO:tasks.workunit.client.0.smithi094.stdout:1/417: rename d0/d4 to d0/d4/d37/d5f/d70/d94 22 2024-04-06T03:40:35.602 INFO:tasks.workunit.client.0.smithi094.stdout:1/418: truncate d0/d7/d2d/f61 529415 0 2024-04-06T03:40:35.602 INFO:tasks.workunit.client.0.smithi094.stdout:1/419: dread d0/d1/d16/d3f/f5c [0,4194304] 0 2024-04-06T03:40:35.602 INFO:tasks.workunit.client.0.smithi094.stdout:5/346: mkdir db/d7c 0 2024-04-06T03:40:35.602 INFO:tasks.workunit.client.1.smithi136.stdout:8/437: dread d3/da/d26/d35/f57 [0,4194304] 0 2024-04-06T03:40:35.603 INFO:tasks.workunit.client.0.smithi094.stdout:8/421: mkdir d18/d3c/d49/d91/d94 0 2024-04-06T03:40:35.604 INFO:tasks.workunit.client.0.smithi094.stdout:1/420: mkdir d0/d4/d1b/d2f/d4b/d95 0 2024-04-06T03:40:35.604 INFO:tasks.workunit.client.0.smithi094.stdout:1/421: write d0/d4/d1b/d2f/d3c/d48/f69 [4837930,82551] 0 2024-04-06T03:40:35.605 INFO:tasks.workunit.client.0.smithi094.stdout:8/422: dread d18/f25 [0,4194304] 0 2024-04-06T03:40:35.606 INFO:tasks.workunit.client.0.smithi094.stdout:5/347: mknod db/d19/d24/c7d 0 2024-04-06T03:40:35.606 INFO:tasks.workunit.client.0.smithi094.stdout:5/348: readlink db/d19/d30/l6b 0 2024-04-06T03:40:35.606 INFO:tasks.workunit.client.0.smithi094.stdout:5/349: write db/d19/d24/f72 [1046603,25789] 0 2024-04-06T03:40:35.606 INFO:tasks.workunit.client.0.smithi094.stdout:5/350: getdents db/d19/d30 0 2024-04-06T03:40:35.608 INFO:tasks.workunit.client.0.smithi094.stdout:1/422: link d0/d4/d1b/d2f/f77 d0/d1/d8f/f96 0 2024-04-06T03:40:35.611 INFO:tasks.workunit.client.0.smithi094.stdout:8/423: mknod d18/d2c/d37/c95 0 2024-04-06T03:40:35.611 INFO:tasks.workunit.client.0.smithi094.stdout:8/424: write f17 [5167128,90619] 0 2024-04-06T03:40:35.611 INFO:tasks.workunit.client.0.smithi094.stdout:8/425: fdatasync fd 0 2024-04-06T03:40:35.613 INFO:tasks.workunit.client.0.smithi094.stdout:5/351: creat db/d7c/f7e x:0 0 0 2024-04-06T03:40:35.613 INFO:tasks.workunit.client.0.smithi094.stdout:5/352: fdatasync db/d7c/f7e 0 2024-04-06T03:40:35.616 INFO:tasks.workunit.client.1.smithi136.stdout:4/343: creat d1d/d31/d3e/d55/d7f/f83 x:0 0 0 2024-04-06T03:40:35.617 INFO:tasks.workunit.client.0.smithi094.stdout:8/426: mkdir d18/d34/d96 0 2024-04-06T03:40:35.621 INFO:tasks.workunit.client.1.smithi136.stdout:5/441: rmdir d1/dc/d10 39 2024-04-06T03:40:35.622 INFO:tasks.workunit.client.0.smithi094.stdout:5/353: dread db/d19/f41 [0,4194304] 0 2024-04-06T03:40:35.623 INFO:tasks.workunit.client.0.smithi094.stdout:8/427: unlink d18/l2f 0 2024-04-06T03:40:35.624 INFO:tasks.workunit.client.0.smithi094.stdout:1/423: dwrite d0/d7/f52 [4194304,4194304] 0 2024-04-06T03:40:35.626 INFO:tasks.workunit.client.1.smithi136.stdout:4/344: write d1d/f52 [2828893,100942] 0 2024-04-06T03:40:35.629 INFO:tasks.workunit.client.1.smithi136.stdout:1/381: rename d12/f16 to d12/d3b/f7b 0 2024-04-06T03:40:35.629 INFO:tasks.workunit.client.1.smithi136.stdout:1/382: chown d12/d1b/f29 31586982 1 2024-04-06T03:40:35.629 INFO:tasks.workunit.client.1.smithi136.stdout:1/383: truncate d12/d1b/f35 1350392 0 2024-04-06T03:40:35.630 INFO:tasks.workunit.client.0.smithi094.stdout:5/354: chown db/d19/c3b 11 1 2024-04-06T03:40:35.633 INFO:tasks.workunit.client.1.smithi136.stdout:6/324: mknod d2/d7/d26/d30/c6f 0 2024-04-06T03:40:35.633 INFO:tasks.workunit.client.1.smithi136.stdout:6/325: chown d2/d25/f56 2028785794 1 2024-04-06T03:40:35.635 INFO:tasks.workunit.client.1.smithi136.stdout:0/369: creat d1/d6/d2c/d57/d43/d6d/d72/f7f x:0 0 0 2024-04-06T03:40:35.638 INFO:tasks.workunit.client.1.smithi136.stdout:1/384: write f3 [5528143,36364] 0 2024-04-06T03:40:35.641 INFO:tasks.workunit.client.0.smithi094.stdout:8/428: dwrite d18/d34/f58 [0,4194304] 0 2024-04-06T03:40:35.643 INFO:tasks.workunit.client.1.smithi136.stdout:3/351: creat d0/d4/d12/d19/d55/d3c/d5a/f71 x:0 0 0 2024-04-06T03:40:35.643 INFO:tasks.workunit.client.1.smithi136.stdout:6/326: dread d2/d25/f3b [0,4194304] 0 2024-04-06T03:40:35.643 INFO:tasks.workunit.client.1.smithi136.stdout:6/327: write d2/d7/d26/d30/d6e/f6b [977735,49104] 0 2024-04-06T03:40:35.646 INFO:tasks.workunit.client.1.smithi136.stdout:8/438: mkdir d3/da/d26/d88 0 2024-04-06T03:40:35.648 INFO:tasks.workunit.client.1.smithi136.stdout:9/394: getdents d17/d20/d28/d49/d75 0 2024-04-06T03:40:35.648 INFO:tasks.workunit.client.1.smithi136.stdout:9/395: chown d17/l4a 5193768 1 2024-04-06T03:40:35.651 INFO:tasks.workunit.client.1.smithi136.stdout:5/442: symlink d1/dc/l86 0 2024-04-06T03:40:35.653 INFO:tasks.workunit.client.1.smithi136.stdout:4/345: symlink d1d/d70/l84 0 2024-04-06T03:40:35.653 INFO:tasks.workunit.client.1.smithi136.stdout:4/346: chown d1d/d31/d3e/c66 150 1 2024-04-06T03:40:35.655 INFO:tasks.workunit.client.0.smithi094.stdout:8/429: dwrite d18/d2c/f47 [0,4194304] 0 2024-04-06T03:40:35.655 INFO:tasks.workunit.client.0.smithi094.stdout:8/430: write d18/d30/f77 [599000,54161] 0 2024-04-06T03:40:35.656 INFO:tasks.workunit.client.1.smithi136.stdout:2/481: rename d3/d9/d3b/f9e to d3/d4a/d9d/d18/d1d/d3c/fa1 0 2024-04-06T03:40:35.657 INFO:tasks.workunit.client.1.smithi136.stdout:2/482: write d3/d15/f36 [4871958,65302] 0 2024-04-06T03:40:35.657 INFO:tasks.workunit.client.1.smithi136.stdout:2/483: write d3/d15/d19/f60 [43584,39330] 0 2024-04-06T03:40:35.660 INFO:tasks.workunit.client.1.smithi136.stdout:0/370: creat d1/d6/d2c/d57/d43/d6b/f80 x:0 0 0 2024-04-06T03:40:35.660 INFO:tasks.workunit.client.1.smithi136.stdout:0/371: creat d1/d6/d2c/d1a/f81 x:0 0 0 2024-04-06T03:40:35.661 INFO:tasks.workunit.client.0.smithi094.stdout:1/424: rmdir d0/d7/d2d 39 2024-04-06T03:40:35.665 INFO:tasks.workunit.client.1.smithi136.stdout:5/443: dwrite d1/dc/f5b [0,4194304] 0 2024-04-06T03:40:35.665 INFO:tasks.workunit.client.1.smithi136.stdout:4/347: write d1d/f22 [2018449,40278] 0 2024-04-06T03:40:35.665 INFO:tasks.workunit.client.1.smithi136.stdout:4/348: chown d1d/d31/d3e/d55/c59 43095 1 2024-04-06T03:40:35.666 INFO:tasks.workunit.client.0.smithi094.stdout:5/355: rename db/d19/l2b to db/d19/d24/d28/l7f 0 2024-04-06T03:40:35.666 INFO:tasks.workunit.client.0.smithi094.stdout:5/356: truncate db/d19/d2c/d57/f70 209136 0 2024-04-06T03:40:35.666 INFO:tasks.workunit.client.0.smithi094.stdout:5/357: dread - db/f76 zero size 2024-04-06T03:40:35.666 INFO:tasks.workunit.client.0.smithi094.stdout:5/358: readlink db/d19/d30/l4c 0 2024-04-06T03:40:35.667 INFO:tasks.workunit.client.1.smithi136.stdout:1/385: truncate d12/f39 3240735 0 2024-04-06T03:40:35.667 INFO:tasks.workunit.client.1.smithi136.stdout:1/386: write d12/f38 [4890494,124699] 0 2024-04-06T03:40:35.669 INFO:tasks.workunit.client.1.smithi136.stdout:3/352: link d0/d4/d12/d19/d55/d3c/f58 d0/d4/d12/d19/d55/d3c/d46/f72 0 2024-04-06T03:40:35.669 INFO:tasks.workunit.client.1.smithi136.stdout:3/353: read d0/d4/d12/d2d/f39 [97505,75978] 0 2024-04-06T03:40:35.670 INFO:tasks.workunit.client.1.smithi136.stdout:3/354: read d0/d4/fe [4447333,28144] 0 2024-04-06T03:40:35.676 INFO:tasks.workunit.client.0.smithi094.stdout:8/431: creat d18/d30/d68/f97 x:0 0 0 2024-04-06T03:40:35.678 INFO:tasks.workunit.client.0.smithi094.stdout:5/359: mkdir db/d80 0 2024-04-06T03:40:35.681 INFO:tasks.workunit.client.1.smithi136.stdout:6/328: rename d2/d7/d26/l4e to d2/d7/d26/d30/d6e/d58/d5d/l70 0 2024-04-06T03:40:35.681 INFO:tasks.workunit.client.1.smithi136.stdout:7/381: sync 2024-04-06T03:40:35.681 INFO:tasks.workunit.client.1.smithi136.stdout:7/382: write d5/df/d53/f39 [2032886,31561] 0 2024-04-06T03:40:35.682 INFO:tasks.workunit.client.0.smithi094.stdout:8/432: mknod d18/d2c/d37/d60/d70/c98 0 2024-04-06T03:40:35.682 INFO:tasks.workunit.client.0.smithi094.stdout:8/433: creat d18/d30/f99 x:0 0 0 2024-04-06T03:40:35.683 INFO:tasks.workunit.client.1.smithi136.stdout:7/383: write d5/df/d53/f43 [495317,10849] 0 2024-04-06T03:40:35.684 INFO:tasks.workunit.client.0.smithi094.stdout:5/360: creat db/d19/d30/f81 x:0 0 0 2024-04-06T03:40:35.691 INFO:tasks.workunit.client.1.smithi136.stdout:1/387: unlink d12/d3b/d44/f54 0 2024-04-06T03:40:35.691 INFO:tasks.workunit.client.1.smithi136.stdout:1/388: write d12/d3b/d44/f57 [1146781,109103] 0 2024-04-06T03:40:35.692 INFO:tasks.workunit.client.1.smithi136.stdout:3/355: creat d0/d4/d12/d3a/d43/d5d/f73 x:0 0 0 2024-04-06T03:40:35.692 INFO:tasks.workunit.client.1.smithi136.stdout:3/356: chown d0/d4/d12/d19/d55/d3c/d53 58835 1 2024-04-06T03:40:35.693 INFO:tasks.workunit.client.1.smithi136.stdout:3/357: dread - d0/d4/d12/d19/d55/d3c/f58 zero size 2024-04-06T03:40:35.695 INFO:tasks.workunit.client.1.smithi136.stdout:8/439: rename d3/da/d26/d35/d39/f4f to d3/da/d26/d35/f89 0 2024-04-06T03:40:35.695 INFO:tasks.workunit.client.1.smithi136.stdout:8/440: dread - d3/de/d18/f6a zero size 2024-04-06T03:40:35.695 INFO:tasks.workunit.client.1.smithi136.stdout:7/384: creat d5/df/d12/d1b/d3a/d4a/f86 x:0 0 0 2024-04-06T03:40:35.696 INFO:tasks.workunit.client.1.smithi136.stdout:5/444: getdents d1/dc/d1c 0 2024-04-06T03:40:35.698 INFO:tasks.workunit.client.1.smithi136.stdout:3/358: symlink d0/d4/d12/d3a/d69/l74 0 2024-04-06T03:40:35.698 INFO:tasks.workunit.client.1.smithi136.stdout:3/359: chown d0/d4/d12/d19 22187157 1 2024-04-06T03:40:35.698 INFO:tasks.workunit.client.1.smithi136.stdout:8/441: rmdir d3/d2d 39 2024-04-06T03:40:35.700 INFO:tasks.workunit.client.1.smithi136.stdout:4/349: rename d1d/c28 to d1d/d31/d3e/c85 0 2024-04-06T03:40:35.701 INFO:tasks.workunit.client.1.smithi136.stdout:4/350: readlink d1d/d70/l54 0 2024-04-06T03:40:35.702 INFO:tasks.workunit.client.1.smithi136.stdout:3/360: creat d0/d4/d12/d19/d2e/f75 x:0 0 0 2024-04-06T03:40:35.702 INFO:tasks.workunit.client.1.smithi136.stdout:7/385: dread d5/df/d11/f14 [0,4194304] 0 2024-04-06T03:40:35.705 INFO:tasks.workunit.client.1.smithi136.stdout:7/386: symlink d5/df/d11/l87 0 2024-04-06T03:40:35.722 INFO:tasks.workunit.client.1.smithi136.stdout:4/351: dwrite d1d/f52 [0,4194304] 0 2024-04-06T03:40:35.724 INFO:tasks.workunit.client.1.smithi136.stdout:4/352: getdents d1d/d31 0 2024-04-06T03:40:35.726 INFO:tasks.workunit.client.1.smithi136.stdout:4/353: mknod d1d/d70/c86 0 2024-04-06T03:40:35.728 INFO:tasks.workunit.client.1.smithi136.stdout:4/354: rmdir d1d/d2f/d6c 39 2024-04-06T03:40:35.730 INFO:tasks.workunit.client.1.smithi136.stdout:4/355: mknod d1d/d70/c87 0 2024-04-06T03:40:35.736 INFO:tasks.workunit.client.1.smithi136.stdout:8/442: write d3/da/d26/d35/d3d/d48/f66 [1881446,50117] 0 2024-04-06T03:40:35.736 INFO:tasks.workunit.client.1.smithi136.stdout:0/372: dwrite d1/d6/d2c/d1a/d36/d59/f6c [0,4194304] 0 2024-04-06T03:40:35.736 INFO:tasks.workunit.client.1.smithi136.stdout:8/443: write d3/de/f56 [1314318,117673] 0 2024-04-06T03:40:35.736 INFO:tasks.workunit.client.1.smithi136.stdout:5/445: rmdir d1 39 2024-04-06T03:40:35.736 INFO:tasks.workunit.client.1.smithi136.stdout:3/361: dwrite d0/f23 [0,4194304] 0 2024-04-06T03:40:35.737 INFO:tasks.workunit.client.0.smithi094.stdout:5/361: dwrite db/d40/d77/f6a [0,4194304] 0 2024-04-06T03:40:35.737 INFO:tasks.workunit.client.0.smithi094.stdout:1/425: dwrite d0/d7/f46 [0,4194304] 0 2024-04-06T03:40:35.737 INFO:tasks.workunit.client.1.smithi136.stdout:1/389: dwrite d12/d3b/d44/d5f/f63 [0,4194304] 0 2024-04-06T03:40:35.738 INFO:tasks.workunit.client.1.smithi136.stdout:1/390: readlink d12/d3b/d74/d7a/l71 0 2024-04-06T03:40:35.738 INFO:tasks.workunit.client.1.smithi136.stdout:6/329: dwrite d2/d25/f3b [0,4194304] 0 2024-04-06T03:40:35.738 INFO:tasks.workunit.client.1.smithi136.stdout:6/330: write d2/d7/d26/d30/d4f/f5f [1574384,82771] 0 2024-04-06T03:40:35.739 INFO:tasks.workunit.client.1.smithi136.stdout:0/373: mkdir d1/d6/d2c/d1a/d82 0 2024-04-06T03:40:35.742 INFO:tasks.workunit.client.1.smithi136.stdout:6/331: dread d2/d7/f31 [0,4194304] 0 2024-04-06T03:40:35.742 INFO:tasks.workunit.client.1.smithi136.stdout:6/332: write d2/d7/f5a [1024132,15604] 0 2024-04-06T03:40:35.745 INFO:tasks.workunit.client.1.smithi136.stdout:8/444: mknod d3/da/d26/d35/d39/d70/c8a 0 2024-04-06T03:40:35.745 INFO:tasks.workunit.client.1.smithi136.stdout:3/362: read d0/f1 [2930609,47995] 0 2024-04-06T03:40:35.746 INFO:tasks.workunit.client.0.smithi094.stdout:5/362: creat db/d40/d77/f82 x:0 0 0 2024-04-06T03:40:35.749 INFO:tasks.workunit.client.1.smithi136.stdout:6/333: dread d2/d7/d26/d30/f50 [0,4194304] 0 2024-04-06T03:40:35.752 INFO:tasks.workunit.client.1.smithi136.stdout:1/391: symlink d12/d3b/d44/d5f/d5a/d72/d77/l7c 0 2024-04-06T03:40:35.755 INFO:tasks.workunit.client.1.smithi136.stdout:4/356: dwrite d1d/d31/f58 [0,4194304] 0 2024-04-06T03:40:35.756 INFO:tasks.workunit.client.1.smithi136.stdout:4/357: chown d1d/d70/f6a 48 1 2024-04-06T03:40:35.759 INFO:tasks.workunit.client.1.smithi136.stdout:3/363: dread d0/f2 [0,4194304] 0 2024-04-06T03:40:35.759 INFO:tasks.workunit.client.1.smithi136.stdout:3/364: fsync d0/f1 0 2024-04-06T03:40:35.760 INFO:tasks.workunit.client.1.smithi136.stdout:0/374: mknod d1/d6/d2c/d57/d43/d6b/c83 0 2024-04-06T03:40:35.760 INFO:tasks.workunit.client.1.smithi136.stdout:0/375: chown d1/d6/d2c/d1a/l32 12327 1 2024-04-06T03:40:35.760 INFO:tasks.workunit.client.0.smithi094.stdout:8/434: getdents d18/d2c/d37/d60/d70 0 2024-04-06T03:40:35.760 INFO:tasks.workunit.client.1.smithi136.stdout:8/445: dwrite d3/da/d26/d35/f89 [0,4194304] 0 2024-04-06T03:40:35.762 INFO:tasks.workunit.client.0.smithi094.stdout:2/467: sync 2024-04-06T03:40:35.762 INFO:tasks.workunit.client.0.smithi094.stdout:0/380: sync 2024-04-06T03:40:35.762 INFO:tasks.workunit.client.0.smithi094.stdout:7/384: sync 2024-04-06T03:40:35.762 INFO:tasks.workunit.client.1.smithi136.stdout:3/365: dread d0/f32 [0,4194304] 0 2024-04-06T03:40:35.768 INFO:tasks.workunit.client.0.smithi094.stdout:1/426: rename d0/d1/d16/l31 to d0/d4/d37/d5f/d70/l97 0 2024-04-06T03:40:35.770 INFO:tasks.workunit.client.1.smithi136.stdout:6/334: dwrite d2/d7/f2c [0,4194304] 0 2024-04-06T03:40:35.770 INFO:tasks.workunit.client.0.smithi094.stdout:5/363: mkdir db/d19/d30/d83 0 2024-04-06T03:40:35.775 INFO:tasks.workunit.client.0.smithi094.stdout:8/435: unlink d18/d2c/d37/c95 0 2024-04-06T03:40:35.776 INFO:tasks.workunit.client.0.smithi094.stdout:8/436: chown d18/f20 193 1 2024-04-06T03:40:35.776 INFO:tasks.workunit.client.0.smithi094.stdout:0/381: read d1/d9/dd/d18/d1a/f46 [2349787,47070] 0 2024-04-06T03:40:35.776 INFO:tasks.workunit.client.0.smithi094.stdout:0/382: chown d1/d9/dd/d18/d1d/c36 4 1 2024-04-06T03:40:35.781 INFO:tasks.workunit.client.0.smithi094.stdout:7/385: creat d2/d6/dc/dd/d5a/f70 x:0 0 0 2024-04-06T03:40:35.783 INFO:tasks.workunit.client.1.smithi136.stdout:1/392: dwrite d12/d3b/d74/d7a/f50 [0,4194304] 0 2024-04-06T03:40:35.783 INFO:tasks.workunit.client.1.smithi136.stdout:4/358: symlink d1d/d31/l88 0 2024-04-06T03:40:35.783 INFO:tasks.workunit.client.1.smithi136.stdout:1/393: chown d12/d1b/d1e/c33 2537800 1 2024-04-06T03:40:35.784 INFO:tasks.workunit.client.1.smithi136.stdout:0/376: rmdir d1/d6/d2c/d1a/d36/d59 39 2024-04-06T03:40:35.785 INFO:tasks.workunit.client.0.smithi094.stdout:5/364: symlink db/d19/d24/d28/l84 0 2024-04-06T03:40:35.786 INFO:tasks.workunit.client.1.smithi136.stdout:3/366: creat d0/d4/f76 x:0 0 0 2024-04-06T03:40:35.786 INFO:tasks.workunit.client.0.smithi094.stdout:8/437: symlink d18/d2c/d37/d60/l9a 0 2024-04-06T03:40:35.788 INFO:tasks.workunit.client.0.smithi094.stdout:8/438: write d18/d2c/d37/d60/d70/f84 [12465,62597] 0 2024-04-06T03:40:35.788 INFO:tasks.workunit.client.0.smithi094.stdout:8/439: readlink lc 0 2024-04-06T03:40:35.789 INFO:tasks.workunit.client.0.smithi094.stdout:0/383: mknod d1/d9/dd/d18/d32/c7c 0 2024-04-06T03:40:35.789 INFO:tasks.workunit.client.0.smithi094.stdout:0/384: chown d1/d9/dd/d18/d5d/d60/d6b/l75 3 1 2024-04-06T03:40:35.789 INFO:tasks.workunit.client.0.smithi094.stdout:0/385: creat d1/d9/dd/d18/d32/f7d x:0 0 0 2024-04-06T03:40:35.789 INFO:tasks.workunit.client.0.smithi094.stdout:0/386: stat d1/d9/dd/d18/d5d/f65 0 2024-04-06T03:40:35.791 INFO:tasks.workunit.client.1.smithi136.stdout:8/446: dwrite d3/da/d26/d35/d3d/f82 [0,4194304] 0 2024-04-06T03:40:35.792 INFO:tasks.workunit.client.0.smithi094.stdout:2/468: symlink d0/d18/l97 0 2024-04-06T03:40:35.792 INFO:tasks.workunit.client.0.smithi094.stdout:2/469: read d0/dd/d19/f81 [767439,99609] 0 2024-04-06T03:40:35.794 INFO:tasks.workunit.client.0.smithi094.stdout:7/386: creat d2/d6/dc/d12/d3a/d56/f71 x:0 0 0 2024-04-06T03:40:35.797 INFO:tasks.workunit.client.1.smithi136.stdout:4/359: mkdir d1d/d31/d3e/d55/d89 0 2024-04-06T03:40:35.797 INFO:tasks.workunit.client.0.smithi094.stdout:8/440: dread f1 [0,4194304] 0 2024-04-06T03:40:35.797 INFO:tasks.workunit.client.1.smithi136.stdout:1/394: write d12/d1b/d28/f5e [3258134,114708] 0 2024-04-06T03:40:35.799 INFO:tasks.workunit.client.0.smithi094.stdout:5/365: mkdir db/d19/d24/d28/d85 0 2024-04-06T03:40:35.799 INFO:tasks.workunit.client.0.smithi094.stdout:5/366: chown db/d19/c3b 420763 1 2024-04-06T03:40:35.801 INFO:tasks.workunit.client.1.smithi136.stdout:0/377: mkdir d1/d6/d2c/d84 0 2024-04-06T03:40:35.808 INFO:tasks.workunit.client.1.smithi136.stdout:3/367: rmdir d0/d4/d12/d19/d2e/d3b/d49 39 2024-04-06T03:40:35.808 INFO:tasks.workunit.client.0.smithi094.stdout:1/427: dwrite d0/d4/d1b/f6f [0,4194304] 0 2024-04-06T03:40:35.810 INFO:tasks.workunit.client.0.smithi094.stdout:0/387: creat d1/d9/dd/d18/d1a/f7e x:0 0 0 2024-04-06T03:40:35.810 INFO:tasks.workunit.client.0.smithi094.stdout:0/388: fdatasync d1/d9/dd/d18/d1e/d22/d35/f3c 0 2024-04-06T03:40:35.812 INFO:tasks.workunit.client.0.smithi094.stdout:2/470: dread d0/d8/d4f/f5b [0,4194304] 0 2024-04-06T03:40:35.812 INFO:tasks.workunit.client.0.smithi094.stdout:6/385: sync 2024-04-06T03:40:35.813 INFO:tasks.workunit.client.0.smithi094.stdout:3/434: sync 2024-04-06T03:40:35.813 INFO:tasks.workunit.client.0.smithi094.stdout:4/477: sync 2024-04-06T03:40:35.813 INFO:tasks.workunit.client.0.smithi094.stdout:3/435: stat d9/d10/l17 0 2024-04-06T03:40:35.814 INFO:tasks.workunit.client.1.smithi136.stdout:6/335: dwrite d2/d7/d26/d30/f50 [0,4194304] 0 2024-04-06T03:40:35.814 INFO:tasks.workunit.client.1.smithi136.stdout:6/336: chown d2/d25/f59 800233138 1 2024-04-06T03:40:35.818 INFO:tasks.workunit.client.0.smithi094.stdout:2/471: unlink d0/dd/c28 0 2024-04-06T03:40:35.822 INFO:tasks.workunit.client.0.smithi094.stdout:6/386: getdents dc/d11 0 2024-04-06T03:40:35.823 INFO:tasks.workunit.client.0.smithi094.stdout:6/387: truncate dc/d11/d22/d50/f80 248793 0 2024-04-06T03:40:35.823 INFO:tasks.workunit.client.0.smithi094.stdout:9/353: sync 2024-04-06T03:40:35.825 INFO:tasks.workunit.client.1.smithi136.stdout:1/395: creat d12/d1b/d1e/d27/f7d x:0 0 0 2024-04-06T03:40:35.826 INFO:tasks.workunit.client.1.smithi136.stdout:8/447: dwrite d3/da/d26/d35/d53/d86/f87 [0,4194304] 0 2024-04-06T03:40:35.827 INFO:tasks.workunit.client.0.smithi094.stdout:3/436: truncate d9/f5f 1885054 0 2024-04-06T03:40:35.827 INFO:tasks.workunit.client.0.smithi094.stdout:3/437: chown d9 93031 1 2024-04-06T03:40:35.828 INFO:tasks.workunit.client.0.smithi094.stdout:4/478: read d0/d15/f1a [888607,101787] 0 2024-04-06T03:40:35.828 INFO:tasks.workunit.client.0.smithi094.stdout:4/479: chown d0/df/d13/d4a/l60 33774484 1 2024-04-06T03:40:35.829 INFO:tasks.workunit.client.1.smithi136.stdout:0/378: rmdir d1/d6/d2c/d57/d43/d6b 39 2024-04-06T03:40:35.832 INFO:tasks.workunit.client.0.smithi094.stdout:2/472: rename d0/d8/d4f/d70 to d0/dd/d19/d31/d67/d98 0 2024-04-06T03:40:35.833 INFO:tasks.workunit.client.1.smithi136.stdout:9/396: sync 2024-04-06T03:40:35.834 INFO:tasks.workunit.client.0.smithi094.stdout:6/388: symlink dc/d11/d1b/d72/l8b 0 2024-04-06T03:40:35.837 INFO:tasks.workunit.client.0.smithi094.stdout:9/354: symlink d2/dc/d10/d2a/l84 0 2024-04-06T03:40:35.839 INFO:tasks.workunit.client.0.smithi094.stdout:5/367: dwrite db/d19/d24/f53 [0,4194304] 0 2024-04-06T03:40:35.839 INFO:tasks.workunit.client.0.smithi094.stdout:4/480: creat d0/d15/d3a/fa6 x:0 0 0 2024-04-06T03:40:35.845 INFO:tasks.workunit.client.0.smithi094.stdout:8/441: dwrite d18/d3c/d76/f5b [4194304,4194304] 0 2024-04-06T03:40:35.846 INFO:tasks.workunit.client.0.smithi094.stdout:8/442: dread - d18/d34/f3b zero size 2024-04-06T03:40:35.846 INFO:tasks.workunit.client.0.smithi094.stdout:8/443: creat d18/d2c/f9b x:0 0 0 2024-04-06T03:40:35.846 INFO:tasks.workunit.client.0.smithi094.stdout:1/428: dwrite d0/d1/d16/f1c [0,4194304] 0 2024-04-06T03:40:35.846 INFO:tasks.workunit.client.1.smithi136.stdout:4/360: dwrite d1d/d31/d3e/d56/f5e [0,4194304] 0 2024-04-06T03:40:35.847 INFO:tasks.workunit.client.1.smithi136.stdout:8/448: dwrite d3/da/d26/f7a [0,4194304] 0 2024-04-06T03:40:35.848 INFO:tasks.workunit.client.0.smithi094.stdout:3/438: dread d9/d10/f22 [0,4194304] 0 2024-04-06T03:40:35.848 INFO:tasks.workunit.client.0.smithi094.stdout:0/389: dwrite d1/d9/f20 [0,4194304] 0 2024-04-06T03:40:35.848 INFO:tasks.workunit.client.0.smithi094.stdout:3/439: write d9/d1e/f8f [14196,59902] 0 2024-04-06T03:40:35.849 INFO:tasks.workunit.client.0.smithi094.stdout:3/440: fsync d9/d10/d7e/f86 0 2024-04-06T03:40:35.849 INFO:tasks.workunit.client.0.smithi094.stdout:3/441: write d9/d10/f47 [461357,3715] 0 2024-04-06T03:40:35.849 INFO:tasks.workunit.client.0.smithi094.stdout:5/368: rename db/d19/d24/d2e/c6e to db/d19/d2c/d6d/c86 0 2024-04-06T03:40:35.849 INFO:tasks.workunit.client.0.smithi094.stdout:6/389: write dc/d11/d22/f39 [2608400,56382] 0 2024-04-06T03:40:35.849 INFO:tasks.workunit.client.0.smithi094.stdout:7/387: dwrite d2/d6/dc/dd/df/d36/f68 [0,4194304] 0 2024-04-06T03:40:35.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/481: getdents d0/d4/d67/d9f 0 2024-04-06T03:40:35.857 INFO:tasks.workunit.client.1.smithi136.stdout:9/397: creat d17/d20/d25/d3d/d4b/f80 x:0 0 0 2024-04-06T03:40:35.857 INFO:tasks.workunit.client.1.smithi136.stdout:9/398: readlink d17/d20/d28/l2d 0 2024-04-06T03:40:35.857 INFO:tasks.workunit.client.1.smithi136.stdout:9/399: read d17/d19/f35 [1684690,10322] 0 2024-04-06T03:40:35.862 INFO:tasks.workunit.client.0.smithi094.stdout:9/355: dwrite d2/d2c/d33/d37/f7f [0,4194304] 0 2024-04-06T03:40:35.865 INFO:tasks.workunit.client.0.smithi094.stdout:7/388: dread d2/d6/dc/dd/f2f [0,4194304] 0 2024-04-06T03:40:35.865 INFO:tasks.workunit.client.0.smithi094.stdout:7/389: truncate d2/d6/dc/dd/df/d36/f4d 737675 0 2024-04-06T03:40:35.867 INFO:tasks.workunit.client.1.smithi136.stdout:6/337: truncate d2/d7/f2c 3941913 0 2024-04-06T03:40:35.868 INFO:tasks.workunit.client.0.smithi094.stdout:8/444: mknod d18/d2c/c9c 0 2024-04-06T03:40:35.868 INFO:tasks.workunit.client.0.smithi094.stdout:8/445: write d18/d2c/f9b [197181,23977] 0 2024-04-06T03:40:35.869 INFO:tasks.workunit.client.1.smithi136.stdout:2/484: sync 2024-04-06T03:40:35.869 INFO:tasks.workunit.client.1.smithi136.stdout:2/485: write d3/d9/f89 [199755,22017] 0 2024-04-06T03:40:35.873 INFO:tasks.workunit.client.0.smithi094.stdout:1/429: mknod d0/d7/c98 0 2024-04-06T03:40:35.873 INFO:tasks.workunit.client.1.smithi136.stdout:7/387: sync 2024-04-06T03:40:35.873 INFO:tasks.workunit.client.1.smithi136.stdout:5/446: sync 2024-04-06T03:40:35.874 INFO:tasks.workunit.client.1.smithi136.stdout:7/388: read - d5/df/d11/f79 zero size 2024-04-06T03:40:35.875 INFO:tasks.workunit.client.0.smithi094.stdout:6/390: dwrite dc/d3a/d4a/d54/f76 [0,4194304] 0 2024-04-06T03:40:35.875 INFO:tasks.workunit.client.0.smithi094.stdout:6/391: creat dc/d11/d1b/d72/f8c x:0 0 0 2024-04-06T03:40:35.876 INFO:tasks.workunit.client.1.smithi136.stdout:8/449: dwrite d3/de/f3a [4194304,4194304] 0 2024-04-06T03:40:35.876 INFO:tasks.workunit.client.1.smithi136.stdout:8/450: fdatasync d3/de/f4b 0 2024-04-06T03:40:35.876 INFO:tasks.workunit.client.1.smithi136.stdout:8/451: dread - d3/da/d26/f50 zero size 2024-04-06T03:40:35.876 INFO:tasks.workunit.client.1.smithi136.stdout:8/452: stat d3/da/l25 0 2024-04-06T03:40:35.877 INFO:tasks.workunit.client.1.smithi136.stdout:4/361: dwrite d1d/d2f/d6c/f76 [0,4194304] 0 2024-04-06T03:40:35.877 INFO:tasks.workunit.client.1.smithi136.stdout:4/362: write d1d/d70/f68 [112415,38292] 0 2024-04-06T03:40:35.877 INFO:tasks.workunit.client.0.smithi094.stdout:3/442: dwrite d9/d1d/f36 [0,4194304] 0 2024-04-06T03:40:35.877 INFO:tasks.workunit.client.0.smithi094.stdout:3/443: write d9/f19 [1111376,57330] 0 2024-04-06T03:40:35.877 INFO:tasks.workunit.client.0.smithi094.stdout:0/390: symlink d1/d9/dd/d18/d78/l7f 0 2024-04-06T03:40:35.881 INFO:tasks.workunit.client.1.smithi136.stdout:9/400: mkdir d17/d20/d28/d81 0 2024-04-06T03:40:35.882 INFO:tasks.workunit.client.0.smithi094.stdout:5/369: creat db/d19/d30/d83/f87 x:0 0 0 2024-04-06T03:40:35.885 INFO:tasks.workunit.client.1.smithi136.stdout:2/486: getdents d3/d4a/d9d 0 2024-04-06T03:40:35.885 INFO:tasks.workunit.client.1.smithi136.stdout:7/389: dread d5/df/d12/d1b/f30 [0,4194304] 0 2024-04-06T03:40:35.886 INFO:tasks.workunit.client.1.smithi136.stdout:5/447: truncate d1/dc/d10/f11 2271843 0 2024-04-06T03:40:35.886 INFO:tasks.workunit.client.0.smithi094.stdout:1/430: write d0/d7/f52 [7987516,67098] 0 2024-04-06T03:40:35.887 INFO:tasks.workunit.client.0.smithi094.stdout:1/431: chown d0/d1/lb 24 1 2024-04-06T03:40:35.887 INFO:tasks.workunit.client.0.smithi094.stdout:1/432: chown d0/d4/d1b/d2f/d76/d30/d38/f81 11883211 1 2024-04-06T03:40:35.889 INFO:tasks.workunit.client.1.smithi136.stdout:1/396: rename d12/d1b/d1e/d27 to d12/d3b/d60/d7e 0 2024-04-06T03:40:35.889 INFO:tasks.workunit.client.1.smithi136.stdout:1/397: creat d12/d3b/d44/d5f/f7f x:0 0 0 2024-04-06T03:40:35.891 INFO:tasks.workunit.client.1.smithi136.stdout:9/401: write d17/d21/d32/d3f/f63 [4170656,123033] 0 2024-04-06T03:40:35.892 INFO:tasks.workunit.client.1.smithi136.stdout:9/402: write d17/d20/d28/d49/d75/f65 [172163,61395] 0 2024-04-06T03:40:35.892 INFO:tasks.workunit.client.1.smithi136.stdout:8/453: mknod d3/de/d18/d6d/c8b 0 2024-04-06T03:40:35.894 INFO:tasks.workunit.client.0.smithi094.stdout:0/391: dwrite d1/d9/dd/f3b [0,4194304] 0 2024-04-06T03:40:35.894 INFO:tasks.workunit.client.0.smithi094.stdout:5/370: read db/d19/d24/f37 [4034806,128540] 0 2024-04-06T03:40:35.895 INFO:tasks.workunit.client.0.smithi094.stdout:0/392: dread d1/d9/dd/d18/d1e/d22/d35/f52 [0,4194304] 0 2024-04-06T03:40:35.899 INFO:tasks.workunit.client.1.smithi136.stdout:5/448: dwrite d1/dc/d10/f6d [0,4194304] 0 2024-04-06T03:40:35.899 INFO:tasks.workunit.client.1.smithi136.stdout:4/363: creat d1d/d2f/d6c/f8a x:0 0 0 2024-04-06T03:40:35.901 INFO:tasks.workunit.client.1.smithi136.stdout:0/379: rename d1/d6/d2c/d57/d43/d6d/d72 to d1/d6/d2c/d57/d43/d6d/d85 0 2024-04-06T03:40:35.901 INFO:tasks.workunit.client.1.smithi136.stdout:0/380: chown d1/d6/d2c/d57/l2f 241327872 1 2024-04-06T03:40:35.901 INFO:tasks.workunit.client.1.smithi136.stdout:9/403: dread d17/d21/d29/f3a [0,4194304] 0 2024-04-06T03:40:35.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/390: unlink d2/d6/dc/d12/d3a/d56/c64 0 2024-04-06T03:40:35.904 INFO:tasks.workunit.client.0.smithi094.stdout:8/446: fdatasync d18/f64 0 2024-04-06T03:40:35.905 INFO:tasks.workunit.client.0.smithi094.stdout:8/447: write d18/d30/f4b [3775056,11322] 0 2024-04-06T03:40:35.912 INFO:tasks.workunit.client.1.smithi136.stdout:0/381: dread d1/d6/d2c/d1a/d36/d59/f23 [0,4194304] 0 2024-04-06T03:40:35.913 INFO:tasks.workunit.client.1.smithi136.stdout:2/487: creat d3/d4a/d9d/d18/d1d/d3c/d58/fa2 x:0 0 0 2024-04-06T03:40:35.913 INFO:tasks.workunit.client.0.smithi094.stdout:6/392: truncate dc/f17 3286603 0 2024-04-06T03:40:35.915 INFO:tasks.workunit.client.1.smithi136.stdout:7/390: rename d5/df/d12/d59/d71/c76 to d5/df/d12/d1b/d3a/d4a/c88 0 2024-04-06T03:40:35.915 INFO:tasks.workunit.client.1.smithi136.stdout:9/404: dwrite d17/d21/d32/d3f/f41 [0,4194304] 0 2024-04-06T03:40:35.915 INFO:tasks.workunit.client.1.smithi136.stdout:9/405: fdatasync d17/d20/d25/d3c/f77 0 2024-04-06T03:40:35.918 INFO:tasks.workunit.client.1.smithi136.stdout:1/398: mknod d12/d3b/d44/d5f/d5a/d72/c80 0 2024-04-06T03:40:35.919 INFO:tasks.workunit.client.1.smithi136.stdout:8/454: rmdir d3/da 39 2024-04-06T03:40:35.924 INFO:tasks.workunit.client.0.smithi094.stdout:1/433: symlink d0/d4/d1b/d2f/d76/l99 0 2024-04-06T03:40:35.924 INFO:tasks.workunit.client.0.smithi094.stdout:1/434: fsync d0/d4/fc 0 2024-04-06T03:40:35.926 INFO:tasks.workunit.client.0.smithi094.stdout:5/371: symlink db/d19/d30/l88 0 2024-04-06T03:40:35.926 INFO:tasks.workunit.client.0.smithi094.stdout:5/372: readlink db/l5a 0 2024-04-06T03:40:35.928 INFO:tasks.workunit.client.1.smithi136.stdout:4/364: mkdir d1d/d70/d8b 0 2024-04-06T03:40:35.928 INFO:tasks.workunit.client.1.smithi136.stdout:4/365: fdatasync d1d/d70/f68 0 2024-04-06T03:40:35.928 INFO:tasks.workunit.client.0.smithi094.stdout:5/373: dread db/d40/f55 [0,4194304] 0 2024-04-06T03:40:35.928 INFO:tasks.workunit.client.0.smithi094.stdout:5/374: dread - db/f7a zero size 2024-04-06T03:40:35.928 INFO:tasks.workunit.client.0.smithi094.stdout:5/375: readlink db/d19/d21/l4b 0 2024-04-06T03:40:35.929 INFO:tasks.workunit.client.0.smithi094.stdout:5/376: fdatasync db/d19/d24/d2e/f75 0 2024-04-06T03:40:35.929 INFO:tasks.workunit.client.0.smithi094.stdout:5/377: creat db/d19/d24/f89 x:0 0 0 2024-04-06T03:40:35.929 INFO:tasks.workunit.client.0.smithi094.stdout:5/378: chown db/d19/d30/c46 32 1 2024-04-06T03:40:35.929 INFO:tasks.workunit.client.0.smithi094.stdout:0/393: rename d1/d9/dd/d18/d32/f7b to d1/d9/dd/d18/d1a/d44/f80 0 2024-04-06T03:40:35.931 INFO:tasks.workunit.client.1.smithi136.stdout:8/455: write d3/da/d26/d35/d3d/f82 [3559129,93336] 0 2024-04-06T03:40:35.931 INFO:tasks.workunit.client.1.smithi136.stdout:0/382: mkdir d1/d6/d2c/d57/d43/d6d/d85/d86 0 2024-04-06T03:40:35.931 INFO:tasks.workunit.client.1.smithi136.stdout:0/383: write d1/d6/d2c/d57/d43/d6d/d85/f7f [39573,1970] 0 2024-04-06T03:40:35.932 INFO:tasks.workunit.client.0.smithi094.stdout:9/356: truncate d2/d2c/d33/d37/f7f 2634986 0 2024-04-06T03:40:35.932 INFO:tasks.workunit.client.0.smithi094.stdout:9/357: write d2/f7 [1276025,59917] 0 2024-04-06T03:40:35.934 INFO:tasks.workunit.client.1.smithi136.stdout:2/488: mknod d3/d4a/d9d/ca3 0 2024-04-06T03:40:35.934 INFO:tasks.workunit.client.1.smithi136.stdout:2/489: chown d3/d4a/d9d/d18/d1d/f20 37880 1 2024-04-06T03:40:35.934 INFO:tasks.workunit.client.1.smithi136.stdout:2/490: readlink d3/d15/d19/d4e/l5c 0 2024-04-06T03:40:35.936 INFO:tasks.workunit.client.1.smithi136.stdout:4/366: dread d1d/f49 [0,4194304] 0 2024-04-06T03:40:35.936 INFO:tasks.workunit.client.1.smithi136.stdout:4/367: write d1d/d70/f64 [2677226,11188] 0 2024-04-06T03:40:35.937 INFO:tasks.workunit.client.0.smithi094.stdout:7/391: symlink d2/d6/dc/d12/d29/d35/l72 0 2024-04-06T03:40:35.939 INFO:tasks.workunit.client.0.smithi094.stdout:1/435: dwrite d0/d1/d8f/f96 [0,4194304] 0 2024-04-06T03:40:35.939 INFO:tasks.workunit.client.1.smithi136.stdout:5/449: rename d1/dc/d15/l24 to d1/dc/d1c/l87 0 2024-04-06T03:40:35.940 INFO:tasks.workunit.client.1.smithi136.stdout:5/450: readlink d1/dc/d10/l64 0 2024-04-06T03:40:35.940 INFO:tasks.workunit.client.0.smithi094.stdout:8/448: fdatasync d18/f6b 0 2024-04-06T03:40:35.941 INFO:tasks.workunit.client.1.smithi136.stdout:7/391: creat d5/df/d11/f89 x:0 0 0 2024-04-06T03:40:35.941 INFO:tasks.workunit.client.1.smithi136.stdout:7/392: fdatasync d5/df/d1d/d6f/d2a/f5d 0 2024-04-06T03:40:35.944 INFO:tasks.workunit.client.0.smithi094.stdout:0/394: dwrite d1/d9/fc [0,4194304] 0 2024-04-06T03:40:35.952 INFO:tasks.workunit.client.0.smithi094.stdout:7/392: dread d2/d6/dc/d12/d29/d35/f54 [0,4194304] 0 2024-04-06T03:40:35.953 INFO:tasks.workunit.client.1.smithi136.stdout:4/368: dread d1d/f20 [0,4194304] 0 2024-04-06T03:40:35.959 INFO:tasks.workunit.client.1.smithi136.stdout:1/399: rmdir d12/d3b/d6b/d79 0 2024-04-06T03:40:35.961 INFO:tasks.workunit.client.0.smithi094.stdout:7/393: dread d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:35.962 INFO:tasks.workunit.client.0.smithi094.stdout:0/395: dwrite d1/f55 [0,4194304] 0 2024-04-06T03:40:35.963 INFO:tasks.workunit.client.0.smithi094.stdout:6/393: mknod dc/d3a/c8d 0 2024-04-06T03:40:35.965 INFO:tasks.workunit.client.1.smithi136.stdout:2/491: creat d3/d4a/d9d/d18/fa4 x:0 0 0 2024-04-06T03:40:35.967 INFO:tasks.workunit.client.1.smithi136.stdout:7/393: dwrite d5/df/d11/f1c [0,4194304] 0 2024-04-06T03:40:35.968 INFO:tasks.workunit.client.1.smithi136.stdout:0/384: rename d1/d6/d2c/d1a/d36/d59/f6c to d1/d6/d2c/f87 0 2024-04-06T03:40:35.968 INFO:tasks.workunit.client.1.smithi136.stdout:0/385: chown d1/d6/d2c/d1a/l1b 154 1 2024-04-06T03:40:35.968 INFO:tasks.workunit.client.1.smithi136.stdout:0/386: creat d1/d6/d2c/f88 x:0 0 0 2024-04-06T03:40:35.969 INFO:tasks.workunit.client.0.smithi094.stdout:3/444: rmdir d9/d1e/d26/d55 39 2024-04-06T03:40:35.969 INFO:tasks.workunit.client.0.smithi094.stdout:3/445: fdatasync d9/d1e/d26/d55/f65 0 2024-04-06T03:40:35.969 INFO:tasks.workunit.client.0.smithi094.stdout:3/446: stat d9/d10/d27/d97 0 2024-04-06T03:40:35.970 INFO:tasks.workunit.client.0.smithi094.stdout:0/396: dread d1/d9/fc [0,4194304] 0 2024-04-06T03:40:35.972 INFO:tasks.workunit.client.1.smithi136.stdout:5/451: link d1/d16/f63 d1/dc/d15/d20/d66/f88 0 2024-04-06T03:40:35.974 INFO:tasks.workunit.client.1.smithi136.stdout:5/452: dread d1/dc/f6e [0,4194304] 0 2024-04-06T03:40:35.974 INFO:tasks.workunit.client.1.smithi136.stdout:5/453: readlink d1/dc/d1c/l87 0 2024-04-06T03:40:35.977 INFO:tasks.workunit.client.1.smithi136.stdout:4/369: link d1d/d31/d3e/d56/d7b/c7c d1d/d31/d3e/d56/c8c 0 2024-04-06T03:40:35.978 INFO:tasks.workunit.client.1.smithi136.stdout:7/394: write d5/df/d1d/f1f [51146,89275] 0 2024-04-06T03:40:35.978 INFO:tasks.workunit.client.1.smithi136.stdout:7/395: readlink d5/df/d12/d59/l65 0 2024-04-06T03:40:35.980 INFO:tasks.workunit.client.1.smithi136.stdout:1/400: mknod d12/d53/c81 0 2024-04-06T03:40:35.981 INFO:tasks.workunit.client.0.smithi094.stdout:9/358: rename d2/dc/d21/d23/d47/d51/f52 to d2/d2c/f85 0 2024-04-06T03:40:35.981 INFO:tasks.workunit.client.0.smithi094.stdout:9/359: fdatasync d2/dc/d21/d23/d2f/f5f 0 2024-04-06T03:40:35.983 INFO:tasks.workunit.client.1.smithi136.stdout:3/368: sync 2024-04-06T03:40:35.984 INFO:tasks.workunit.client.1.smithi136.stdout:3/369: write d0/d4/d12/d19/d55/d3c/d5a/f71 [272228,87238] 0 2024-04-06T03:40:35.984 INFO:tasks.workunit.client.1.smithi136.stdout:3/370: dread - d0/d4/d12/d19/d2c/f67 zero size 2024-04-06T03:40:35.987 INFO:tasks.workunit.client.0.smithi094.stdout:3/447: dwrite d9/d20/d76/f5c [0,4194304] 0 2024-04-06T03:40:35.987 INFO:tasks.workunit.client.0.smithi094.stdout:3/448: readlink d9/l13 0 2024-04-06T03:40:35.987 INFO:tasks.workunit.client.0.smithi094.stdout:6/394: dwrite dc/d11/d22/f39 [0,4194304] 0 2024-04-06T03:40:35.989 INFO:tasks.workunit.client.1.smithi136.stdout:2/492: dwrite d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:35.989 INFO:tasks.workunit.client.0.smithi094.stdout:1/436: symlink d0/d4/d1b/d2f/d76/d7d/l9a 0 2024-04-06T03:40:35.995 INFO:tasks.workunit.client.1.smithi136.stdout:4/370: dwrite d1d/d31/f3b [0,4194304] 0 2024-04-06T03:40:35.997 INFO:tasks.workunit.client.1.smithi136.stdout:0/387: link d1/d6/d2c/d1a/l1b d1/d6/d2c/d57/d43/d6d/d85/d86/l89 0 2024-04-06T03:40:35.999 INFO:tasks.workunit.client.1.smithi136.stdout:5/454: rmdir d1/d16/d4b 39 2024-04-06T03:40:36.001 INFO:tasks.workunit.client.1.smithi136.stdout:1/401: dwrite d12/d1b/d28/f5e [0,4194304] 0 2024-04-06T03:40:36.002 INFO:tasks.workunit.client.1.smithi136.stdout:7/396: mkdir d5/df/d53/d8a 0 2024-04-06T03:40:36.002 INFO:tasks.workunit.client.1.smithi136.stdout:7/397: chown d5/df/d3d/f3f 25550438 1 2024-04-06T03:40:36.004 INFO:tasks.workunit.client.0.smithi094.stdout:5/379: getdents db/d19/d2c/d6d 0 2024-04-06T03:40:36.004 INFO:tasks.workunit.client.0.smithi094.stdout:5/380: dread - db/d7c/f7e zero size 2024-04-06T03:40:36.004 INFO:tasks.workunit.client.0.smithi094.stdout:5/381: truncate db/d19/d21/f49 257081 0 2024-04-06T03:40:36.005 INFO:tasks.workunit.client.1.smithi136.stdout:1/402: dread d12/d3b/d44/f67 [0,4194304] 0 2024-04-06T03:40:36.005 INFO:tasks.workunit.client.1.smithi136.stdout:3/371: symlink d0/d4/d12/d3a/d43/d5d/d6a/l77 0 2024-04-06T03:40:36.010 INFO:tasks.workunit.client.0.smithi094.stdout:0/397: rename d1/d9/dd/d18/d32/f6c to d1/d9/dd/d18/d5d/d60/d6b/f81 0 2024-04-06T03:40:36.010 INFO:tasks.workunit.client.0.smithi094.stdout:0/398: write d1/d10/f73 [750601,5341] 0 2024-04-06T03:40:36.010 INFO:tasks.workunit.client.0.smithi094.stdout:0/399: chown d1/d9/dd/d18/d1a/d44/c79 3434881 1 2024-04-06T03:40:36.011 INFO:tasks.workunit.client.0.smithi094.stdout:9/360: symlink d2/dc/d21/d26/d78/d69/l86 0 2024-04-06T03:40:36.011 INFO:tasks.workunit.client.0.smithi094.stdout:9/361: chown d2/dc/d10 1660015 1 2024-04-06T03:40:36.014 INFO:tasks.workunit.client.1.smithi136.stdout:4/371: symlink d1d/d31/d3e/d55/l8d 0 2024-04-06T03:40:36.014 INFO:tasks.workunit.client.1.smithi136.stdout:4/372: readlink d1d/d31/d3e/l63 0 2024-04-06T03:40:36.015 INFO:tasks.workunit.client.1.smithi136.stdout:3/372: dread d0/d4/d12/f20 [4194304,4194304] 0 2024-04-06T03:40:36.015 INFO:tasks.workunit.client.1.smithi136.stdout:2/493: dwrite d3/d15/f73 [0,4194304] 0 2024-04-06T03:40:36.015 INFO:tasks.workunit.client.1.smithi136.stdout:6/338: sync 2024-04-06T03:40:36.017 INFO:tasks.workunit.client.0.smithi094.stdout:5/382: dwrite db/d19/d24/d2e/f36 [0,4194304] 0 2024-04-06T03:40:36.017 INFO:tasks.workunit.client.0.smithi094.stdout:5/383: chown db/d19/d24/f37 0 1 2024-04-06T03:40:36.017 INFO:tasks.workunit.client.0.smithi094.stdout:3/449: mkdir d9/d9d 0 2024-04-06T03:40:36.017 INFO:tasks.workunit.client.0.smithi094.stdout:3/450: getdents d9/d20/d76/d6b 0 2024-04-06T03:40:36.018 INFO:tasks.workunit.client.1.smithi136.stdout:2/494: dread d3/d15/d19/f60 [0,4194304] 0 2024-04-06T03:40:36.018 INFO:tasks.workunit.client.1.smithi136.stdout:2/495: creat d3/d9/d3b/d87/fa5 x:0 0 0 2024-04-06T03:40:36.018 INFO:tasks.workunit.client.1.smithi136.stdout:0/388: truncate d1/d6/fd 62866 0 2024-04-06T03:40:36.018 INFO:tasks.workunit.client.1.smithi136.stdout:0/389: truncate d1/d6/d2c/d1a/d36/f5d 877328 0 2024-04-06T03:40:36.021 INFO:tasks.workunit.client.1.smithi136.stdout:5/455: mkdir d1/d16/d89 0 2024-04-06T03:40:36.021 INFO:tasks.workunit.client.1.smithi136.stdout:5/456: dread - d1/d16/f63 zero size 2024-04-06T03:40:36.021 INFO:tasks.workunit.client.0.smithi094.stdout:9/362: dread d2/dc/d21/d23/d2f/f5f [4194304,4194304] 0 2024-04-06T03:40:36.021 INFO:tasks.workunit.client.0.smithi094.stdout:9/363: chown d2/dc/d21/d23/d2f/l38 2 1 2024-04-06T03:40:36.022 INFO:tasks.workunit.client.0.smithi094.stdout:9/364: chown d2/dc/d21/c75 208375137 1 2024-04-06T03:40:36.022 INFO:tasks.workunit.client.0.smithi094.stdout:9/365: dread - d2/dc/d21/d26/d78/d69/d6d/d6b/f73 zero size 2024-04-06T03:40:36.022 INFO:tasks.workunit.client.0.smithi094.stdout:9/366: chown d2/d2c/d33/d64/c71 3 1 2024-04-06T03:40:36.022 INFO:tasks.workunit.client.0.smithi094.stdout:6/395: truncate dc/d11/d1b/f1f 537284 0 2024-04-06T03:40:36.024 INFO:tasks.workunit.client.0.smithi094.stdout:1/437: creat d0/d7/d1a/f9b x:0 0 0 2024-04-06T03:40:36.028 INFO:tasks.workunit.client.0.smithi094.stdout:8/449: truncate d18/f1b 712004 0 2024-04-06T03:40:36.031 INFO:tasks.workunit.client.0.smithi094.stdout:7/394: getdents d2/d6/dc/d12/d29/d35 0 2024-04-06T03:40:36.031 INFO:tasks.workunit.client.1.smithi136.stdout:6/339: dwrite d2/d7/d13/f44 [0,4194304] 0 2024-04-06T03:40:36.031 INFO:tasks.workunit.client.1.smithi136.stdout:6/340: fsync d2/d7/d26/d30/f3a 0 2024-04-06T03:40:36.032 INFO:tasks.workunit.client.1.smithi136.stdout:6/341: write d2/d7/d26/d30/d6e/f6b [239300,116368] 0 2024-04-06T03:40:36.032 INFO:tasks.workunit.client.1.smithi136.stdout:6/342: write d2/fa [4567465,6937] 0 2024-04-06T03:40:36.033 INFO:tasks.workunit.client.0.smithi094.stdout:0/400: symlink d1/d9/dd/d18/d5d/d60/d6b/d67/l82 0 2024-04-06T03:40:36.033 INFO:tasks.workunit.client.0.smithi094.stdout:0/401: fdatasync d1/d9/dd/d18/d1a/f7e 0 2024-04-06T03:40:36.033 INFO:tasks.workunit.client.0.smithi094.stdout:0/402: chown d1/d9/dd/d18/d1d/f57 558 1 2024-04-06T03:40:36.033 INFO:tasks.workunit.client.0.smithi094.stdout:0/403: fdatasync d1/d9/f5e 0 2024-04-06T03:40:36.034 INFO:tasks.workunit.client.1.smithi136.stdout:5/457: write d1/dc/d1c/d32/f6a [1964243,43154] 0 2024-04-06T03:40:36.039 INFO:tasks.workunit.client.1.smithi136.stdout:7/398: symlink d5/df/d1d/l8b 0 2024-04-06T03:40:36.041 INFO:tasks.workunit.client.1.smithi136.stdout:1/403: symlink d12/d3b/d74/l82 0 2024-04-06T03:40:36.045 INFO:tasks.workunit.client.1.smithi136.stdout:2/496: dwrite d3/d4a/d9d/d18/f74 [0,4194304] 0 2024-04-06T03:40:36.045 INFO:tasks.workunit.client.1.smithi136.stdout:2/497: chown d3/d15/f2b 4 1 2024-04-06T03:40:36.045 INFO:tasks.workunit.client.1.smithi136.stdout:4/373: mkdir d1d/d31/d8e 0 2024-04-06T03:40:36.046 INFO:tasks.workunit.client.0.smithi094.stdout:6/396: mknod dc/d3a/d4a/d54/c8e 0 2024-04-06T03:40:36.050 INFO:tasks.workunit.client.0.smithi094.stdout:1/438: creat d0/d4/d1b/d2f/d3c/d48/d67/f9c x:0 0 0 2024-04-06T03:40:36.050 INFO:tasks.workunit.client.0.smithi094.stdout:0/404: dread d1/d9/dd/d18/d5d/f65 [0,4194304] 0 2024-04-06T03:40:36.050 INFO:tasks.workunit.client.0.smithi094.stdout:0/405: readlink d1/d9/dd/d18/d5d/d60/d6b/l75 0 2024-04-06T03:40:36.052 INFO:tasks.workunit.client.0.smithi094.stdout:8/450: rmdir d18/d3c/d8b 39 2024-04-06T03:40:36.054 INFO:tasks.workunit.client.1.smithi136.stdout:9/406: sync 2024-04-06T03:40:36.057 INFO:tasks.workunit.client.0.smithi094.stdout:5/384: rename db/d19/f1f to db/d19/d21/f8a 0 2024-04-06T03:40:36.058 INFO:tasks.workunit.client.1.smithi136.stdout:2/498: dread d3/d4a/d9d/f6b [0,4194304] 0 2024-04-06T03:40:36.058 INFO:tasks.workunit.client.1.smithi136.stdout:5/458: dwrite d1/d16/d4b/f4f [0,4194304] 0 2024-04-06T03:40:36.058 INFO:tasks.workunit.client.0.smithi094.stdout:3/451: dwrite d9/d20/d76/f5c [4194304,4194304] 0 2024-04-06T03:40:36.058 INFO:tasks.workunit.client.1.smithi136.stdout:3/373: creat d0/d4/d12/d19/d55/d3c/d46/f78 x:0 0 0 2024-04-06T03:40:36.059 INFO:tasks.workunit.client.1.smithi136.stdout:3/374: dread - d0/d4/d12/d3a/f4b zero size 2024-04-06T03:40:36.059 INFO:tasks.workunit.client.1.smithi136.stdout:3/375: chown d0/d4/d12/d3a/c59 52 1 2024-04-06T03:40:36.059 INFO:tasks.workunit.client.0.smithi094.stdout:3/452: read d9/d1d/f39 [102804,121384] 0 2024-04-06T03:40:36.059 INFO:tasks.workunit.client.0.smithi094.stdout:3/453: fdatasync d9/d10/d7e/f86 0 2024-04-06T03:40:36.059 INFO:tasks.workunit.client.0.smithi094.stdout:8/451: link d18/d2c/d37/d60/d70/c98 d18/d30/d68/c9d 0 2024-04-06T03:40:36.060 INFO:tasks.workunit.client.0.smithi094.stdout:8/452: write d18/d30/f40 [743790,124270] 0 2024-04-06T03:40:36.060 INFO:tasks.workunit.client.0.smithi094.stdout:8/453: creat d18/d3c/d76/f9e x:0 0 0 2024-04-06T03:40:36.062 INFO:tasks.workunit.client.1.smithi136.stdout:4/374: read d1d/f27 [492136,74685] 0 2024-04-06T03:40:36.062 INFO:tasks.workunit.client.0.smithi094.stdout:9/367: rename d2/dc/d21/l22 to d2/dc/d10/d2a/l87 0 2024-04-06T03:40:36.069 INFO:tasks.workunit.client.0.smithi094.stdout:0/406: write d1/f23 [921946,92911] 0 2024-04-06T03:40:36.073 INFO:tasks.workunit.client.0.smithi094.stdout:8/454: write d18/d34/f3d [1792132,118366] 0 2024-04-06T03:40:36.075 INFO:tasks.workunit.client.0.smithi094.stdout:8/455: dread d18/d2c/d37/d60/d70/f84 [0,4194304] 0 2024-04-06T03:40:36.078 INFO:tasks.workunit.client.1.smithi136.stdout:7/399: dwrite d5/df/d3d/f3f [0,4194304] 0 2024-04-06T03:40:36.080 INFO:tasks.workunit.client.0.smithi094.stdout:1/439: dwrite d0/d4/d1b/d2f/d76/d30/f32 [0,4194304] 0 2024-04-06T03:40:36.096 INFO:tasks.workunit.client.0.smithi094.stdout:1/440: dwrite d0/d1/d16/d3f/f5c [0,4194304] 0 2024-04-06T03:40:36.096 INFO:tasks.workunit.client.0.smithi094.stdout:1/441: write d0/d7/f86 [501159,33755] 0 2024-04-06T03:40:36.164 INFO:tasks.workunit.client.1.smithi136.stdout:0/390: symlink d1/d6/d2c/d1a/d82/l8a 0 2024-04-06T03:40:36.166 INFO:tasks.workunit.client.1.smithi136.stdout:2/499: dwrite d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:36.169 INFO:tasks.workunit.client.0.smithi094.stdout:1/442: dwrite d0/d4/d1b/d2f/d3c/d48/f88 [0,4194304] 0 2024-04-06T03:40:36.169 INFO:tasks.workunit.client.0.smithi094.stdout:1/443: readlink d0/le 0 2024-04-06T03:40:36.169 INFO:tasks.workunit.client.0.smithi094.stdout:1/444: stat d0/d4/d1b/d2f/f77 0 2024-04-06T03:40:36.169 INFO:tasks.workunit.client.1.smithi136.stdout:8/456: sync 2024-04-06T03:40:36.171 INFO:tasks.workunit.client.1.smithi136.stdout:1/404: rmdir d12/d3b/d60/d7e 39 2024-04-06T03:40:36.178 INFO:tasks.workunit.client.0.smithi094.stdout:3/454: mknod d9/d20/d76/d6b/c9e 0 2024-04-06T03:40:36.180 INFO:tasks.workunit.client.1.smithi136.stdout:5/459: mkdir d1/d16/d89/d8a 0 2024-04-06T03:40:36.180 INFO:tasks.workunit.client.1.smithi136.stdout:5/460: chown d1/dc/f5b 1827 1 2024-04-06T03:40:36.181 INFO:tasks.workunit.client.0.smithi094.stdout:7/395: rename d2/f6c to d2/d6/f73 0 2024-04-06T03:40:36.184 INFO:tasks.workunit.client.1.smithi136.stdout:3/376: mkdir d0/d4/d12/d19/d55/d3c/d5a/d79 0 2024-04-06T03:40:36.184 INFO:tasks.workunit.client.1.smithi136.stdout:3/377: dread - d0/d4/d12/d3a/f4b zero size 2024-04-06T03:40:36.184 INFO:tasks.workunit.client.1.smithi136.stdout:3/378: chown d0/d4/d12/d19/d2e/f75 1042 1 2024-04-06T03:40:36.184 INFO:tasks.workunit.client.1.smithi136.stdout:3/379: stat d0/d57 0 2024-04-06T03:40:36.185 INFO:tasks.workunit.client.1.smithi136.stdout:3/380: write d0/d4/d12/f15 [5622388,101895] 0 2024-04-06T03:40:36.185 INFO:tasks.workunit.client.1.smithi136.stdout:4/375: mknod d1d/d31/d3e/d55/d89/c8f 0 2024-04-06T03:40:36.186 INFO:tasks.workunit.client.1.smithi136.stdout:4/376: dread d1d/d31/d3e/d56/f5e [4194304,4194304] 0 2024-04-06T03:40:36.186 INFO:tasks.workunit.client.1.smithi136.stdout:4/377: read d1d/d31/d3e/d56/d7b/f77 [893729,97065] 0 2024-04-06T03:40:36.186 INFO:tasks.workunit.client.0.smithi094.stdout:9/368: symlink d2/dc/d10/d6a/l88 0 2024-04-06T03:40:36.186 INFO:tasks.workunit.client.0.smithi094.stdout:9/369: fsync d2/dc/d10/f12 0 2024-04-06T03:40:36.186 INFO:tasks.workunit.client.0.smithi094.stdout:0/407: rename d1/f55 to d1/f83 0 2024-04-06T03:40:36.186 INFO:tasks.workunit.client.0.smithi094.stdout:0/408: fdatasync d1/d9/f5e 0 2024-04-06T03:40:36.186 INFO:tasks.workunit.client.0.smithi094.stdout:2/473: sync 2024-04-06T03:40:36.186 INFO:tasks.workunit.client.0.smithi094.stdout:4/482: sync 2024-04-06T03:40:36.187 INFO:tasks.workunit.client.1.smithi136.stdout:7/400: symlink d5/df/d1d/d6f/d2c/l8c 0 2024-04-06T03:40:36.187 INFO:tasks.workunit.client.1.smithi136.stdout:7/401: truncate d5/df/d3d/f44 1834744 0 2024-04-06T03:40:36.189 INFO:tasks.workunit.client.1.smithi136.stdout:0/391: rmdir d1/d6/d2c/d1a/d7c 0 2024-04-06T03:40:36.189 INFO:tasks.workunit.client.1.smithi136.stdout:5/461: dread d1/dc/d15/d20/f84 [0,4194304] 0 2024-04-06T03:40:36.189 INFO:tasks.workunit.client.1.smithi136.stdout:5/462: chown d1/ca 9620 1 2024-04-06T03:40:36.189 INFO:tasks.workunit.client.0.smithi094.stdout:8/456: mkdir d18/d3c/d49/d9f 0 2024-04-06T03:40:36.194 INFO:tasks.workunit.client.1.smithi136.stdout:2/500: truncate d3/d4a/d9d/d18/d1d/d3c/fa0 8078105 0 2024-04-06T03:40:36.194 INFO:tasks.workunit.client.1.smithi136.stdout:2/501: dread - d3/d9b/f86 zero size 2024-04-06T03:40:36.194 INFO:tasks.workunit.client.1.smithi136.stdout:2/502: stat d3/d9/d3b 0 2024-04-06T03:40:36.194 INFO:tasks.workunit.client.1.smithi136.stdout:2/503: dread - d3/d9b/f86 zero size 2024-04-06T03:40:36.194 INFO:tasks.workunit.client.1.smithi136.stdout:2/504: chown d3/d4a/d9d/d18/d1d/c21 44021 1 2024-04-06T03:40:36.194 INFO:tasks.workunit.client.1.smithi136.stdout:2/505: chown d3/d15/c7d 14 1 2024-04-06T03:40:36.196 INFO:tasks.workunit.client.1.smithi136.stdout:8/457: rename d3/da/d26/d35/d39/d70/c8a to d3/d2d/c8c 0 2024-04-06T03:40:36.197 INFO:tasks.workunit.client.0.smithi094.stdout:5/385: getdents db/d19/d24 0 2024-04-06T03:40:36.197 INFO:tasks.workunit.client.0.smithi094.stdout:5/386: chown db/d19/d24/f37 167497 1 2024-04-06T03:40:36.198 INFO:tasks.workunit.client.1.smithi136.stdout:1/405: getdents d12/d3b/d44/d5f/d5a 0 2024-04-06T03:40:36.200 INFO:tasks.workunit.client.1.smithi136.stdout:9/407: rmdir d17/d20/d25/d3b 39 2024-04-06T03:40:36.200 INFO:tasks.workunit.client.1.smithi136.stdout:8/458: dread d3/de/ff [0,4194304] 0 2024-04-06T03:40:36.201 INFO:tasks.workunit.client.1.smithi136.stdout:7/402: getdents d5/df/d12/d1b/d3a 0 2024-04-06T03:40:36.201 INFO:tasks.workunit.client.1.smithi136.stdout:7/403: dread - d5/df/d1d/f7f zero size 2024-04-06T03:40:36.203 INFO:tasks.workunit.client.1.smithi136.stdout:4/378: dwrite d1d/d2f/f32 [4194304,4194304] 0 2024-04-06T03:40:36.204 INFO:tasks.workunit.client.1.smithi136.stdout:0/392: symlink d1/d6/d2c/d57/d43/d6d/d85/d86/l8b 0 2024-04-06T03:40:36.204 INFO:tasks.workunit.client.0.smithi094.stdout:3/455: unlink d9/d20/d76/d6b/l73 0 2024-04-06T03:40:36.205 INFO:tasks.workunit.client.0.smithi094.stdout:3/456: write d9/d1e/f59 [305242,114224] 0 2024-04-06T03:40:36.205 INFO:tasks.workunit.client.1.smithi136.stdout:5/463: truncate d1/d16/d17/f1a 1053807 0 2024-04-06T03:40:36.207 INFO:tasks.workunit.client.0.smithi094.stdout:7/396: rename d2/d6/dc/dd/f27 to d2/d6/dc/f74 0 2024-04-06T03:40:36.209 INFO:tasks.workunit.client.1.smithi136.stdout:2/506: mknod d3/d4a/d9d/d18/d1d/d3c/ca6 0 2024-04-06T03:40:36.210 INFO:tasks.workunit.client.0.smithi094.stdout:9/370: creat d2/dc/d21/d23/d5b/f89 x:0 0 0 2024-04-06T03:40:36.210 INFO:tasks.workunit.client.1.smithi136.stdout:5/464: dread d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:36.211 INFO:tasks.workunit.client.1.smithi136.stdout:4/379: read fc [2800095,15811] 0 2024-04-06T03:40:36.211 INFO:tasks.workunit.client.1.smithi136.stdout:4/380: stat d1d/d70/f6a 0 2024-04-06T03:40:36.211 INFO:tasks.workunit.client.1.smithi136.stdout:3/381: rename d0/d4/d12/c4d to d0/d4/d12/d19/d2e/d3b/c7a 0 2024-04-06T03:40:36.212 INFO:tasks.workunit.client.0.smithi094.stdout:0/409: symlink d1/d9/dd/d18/d78/l84 0 2024-04-06T03:40:36.213 INFO:tasks.workunit.client.1.smithi136.stdout:1/406: symlink d12/d1b/d1e/d6f/l83 0 2024-04-06T03:40:36.218 INFO:tasks.workunit.client.1.smithi136.stdout:8/459: creat d3/de/f8d x:0 0 0 2024-04-06T03:40:36.218 INFO:tasks.workunit.client.0.smithi094.stdout:4/483: creat d0/df/d1e/fa7 x:0 0 0 2024-04-06T03:40:36.218 INFO:tasks.workunit.client.0.smithi094.stdout:8/457: dwrite d18/d30/d68/f97 [0,4194304] 0 2024-04-06T03:40:36.219 INFO:tasks.workunit.client.0.smithi094.stdout:2/474: mknod d0/dd/d19/d31/c99 0 2024-04-06T03:40:36.219 INFO:tasks.workunit.client.1.smithi136.stdout:6/343: sync 2024-04-06T03:40:36.219 INFO:tasks.workunit.client.1.smithi136.stdout:6/344: fsync d2/d7/d26/d30/d6e/d3d/f6a 0 2024-04-06T03:40:36.220 INFO:tasks.workunit.client.1.smithi136.stdout:7/404: mkdir d5/df/d12/d1b/d3a/d8d 0 2024-04-06T03:40:36.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/387: mkdir db/d7c/d8b 0 2024-04-06T03:40:36.221 INFO:tasks.workunit.client.0.smithi094.stdout:5/388: truncate db/d19/d21/f5e 593076 0 2024-04-06T03:40:36.225 INFO:tasks.workunit.client.0.smithi094.stdout:3/457: link d9/d20/d76/f5c d9/d1e/f9f 0 2024-04-06T03:40:36.227 INFO:tasks.workunit.client.0.smithi094.stdout:0/410: dread d1/f23 [0,4194304] 0 2024-04-06T03:40:36.228 INFO:tasks.workunit.client.1.smithi136.stdout:4/381: dwrite d1d/f2b [0,4194304] 0 2024-04-06T03:40:36.228 INFO:tasks.workunit.client.1.smithi136.stdout:4/382: readlink d1d/d31/l47 0 2024-04-06T03:40:36.228 INFO:tasks.workunit.client.1.smithi136.stdout:0/393: unlink d1/d6/d2c/d57/d43/f60 0 2024-04-06T03:40:36.230 INFO:tasks.workunit.client.1.smithi136.stdout:0/394: read d1/d6/fd [3168,39223] 0 2024-04-06T03:40:36.230 INFO:tasks.workunit.client.1.smithi136.stdout:0/395: chown d1/d6/d2c/d1a/d36/f5d 7 1 2024-04-06T03:40:36.230 INFO:tasks.workunit.client.1.smithi136.stdout:0/396: write d1/d6/d2c/f27 [4992844,119843] 0 2024-04-06T03:40:36.230 INFO:tasks.workunit.client.1.smithi136.stdout:0/397: stat d1/f54 0 2024-04-06T03:40:36.234 INFO:tasks.workunit.client.0.smithi094.stdout:7/397: write d2/d6/f8 [47916,122931] 0 2024-04-06T03:40:36.234 INFO:tasks.workunit.client.0.smithi094.stdout:7/398: chown d2/d6/dc/d12/f5f 74173 1 2024-04-06T03:40:36.235 INFO:tasks.workunit.client.1.smithi136.stdout:2/507: symlink d3/d4a/la7 0 2024-04-06T03:40:36.235 INFO:tasks.workunit.client.1.smithi136.stdout:2/508: truncate d3/d9b/f86 403194 0 2024-04-06T03:40:36.239 INFO:tasks.workunit.client.0.smithi094.stdout:9/371: unlink d2/d2c/d33/l49 0 2024-04-06T03:40:36.240 INFO:tasks.workunit.client.1.smithi136.stdout:5/465: truncate d1/dc/d1c/d32/f6a 3640794 0 2024-04-06T03:40:36.241 INFO:tasks.workunit.client.1.smithi136.stdout:9/408: rename d17/d19/f27 to d17/d20/d28/f82 0 2024-04-06T03:40:36.243 INFO:tasks.workunit.client.1.smithi136.stdout:3/382: write d0/d4/f18 [1315604,65714] 0 2024-04-06T03:40:36.245 INFO:tasks.workunit.client.0.smithi094.stdout:8/458: symlink d18/d2c/d37/d60/la0 0 2024-04-06T03:40:36.246 INFO:tasks.workunit.client.1.smithi136.stdout:1/407: getdents d12/d1b 0 2024-04-06T03:40:36.246 INFO:tasks.workunit.client.1.smithi136.stdout:1/408: creat d12/d3b/f84 x:0 0 0 2024-04-06T03:40:36.246 INFO:tasks.workunit.client.1.smithi136.stdout:1/409: readlink d12/d1b/d1e/l36 0 2024-04-06T03:40:36.247 INFO:tasks.workunit.client.1.smithi136.stdout:1/410: fdatasync d12/d3b/d74/d7a/f50 0 2024-04-06T03:40:36.248 INFO:tasks.workunit.client.0.smithi094.stdout:2/475: unlink d0/dd/d1b/f3e 0 2024-04-06T03:40:36.250 INFO:tasks.workunit.client.1.smithi136.stdout:7/405: creat d5/df/d3d/f8e x:0 0 0 2024-04-06T03:40:36.250 INFO:tasks.workunit.client.1.smithi136.stdout:7/406: truncate d5/df/f62 295590 0 2024-04-06T03:40:36.250 INFO:tasks.workunit.client.1.smithi136.stdout:7/407: fdatasync d5/df/d53/f39 0 2024-04-06T03:40:36.251 INFO:tasks.workunit.client.0.smithi094.stdout:3/458: dwrite d9/d20/f33 [4194304,4194304] 0 2024-04-06T03:40:36.252 INFO:tasks.workunit.client.1.smithi136.stdout:6/345: dwrite d2/d7/d26/d30/d6e/f27 [0,4194304] 0 2024-04-06T03:40:36.256 INFO:tasks.workunit.client.0.smithi094.stdout:0/411: symlink d1/d9/dd/d18/l85 0 2024-04-06T03:40:36.256 INFO:tasks.workunit.client.0.smithi094.stdout:0/412: truncate d1/d9/dd/d18/d1e/d22/f37 2511183 0 2024-04-06T03:40:36.258 INFO:tasks.workunit.client.0.smithi094.stdout:9/372: rmdir d2/dc/d21/d26/d78 39 2024-04-06T03:40:36.261 INFO:tasks.workunit.client.1.smithi136.stdout:4/383: mknod d1d/d31/d78/c90 0 2024-04-06T03:40:36.261 INFO:tasks.workunit.client.1.smithi136.stdout:4/384: write d1d/d2f/d6c/f8a [788687,46700] 0 2024-04-06T03:40:36.262 INFO:tasks.workunit.client.0.smithi094.stdout:8/459: rename d18/d30/f4b to d18/d2c/d37/fa1 0 2024-04-06T03:40:36.264 INFO:tasks.workunit.client.0.smithi094.stdout:2/476: mknod d0/d18/d3f/c9a 0 2024-04-06T03:40:36.265 INFO:tasks.workunit.client.1.smithi136.stdout:0/398: getdents d1/d6/d2c/d1a/d82 0 2024-04-06T03:40:36.266 INFO:tasks.workunit.client.1.smithi136.stdout:1/411: dwrite d12/d1b/d1e/d4d/f68 [0,4194304] 0 2024-04-06T03:40:36.266 INFO:tasks.workunit.client.1.smithi136.stdout:1/412: write d12/d1b/d22/f43 [5028224,24221] 0 2024-04-06T03:40:36.267 INFO:tasks.workunit.client.0.smithi094.stdout:7/399: dwrite d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:36.267 INFO:tasks.workunit.client.0.smithi094.stdout:7/400: fdatasync d2/d6/dc/dd/df/d36/f69 0 2024-04-06T03:40:36.273 INFO:tasks.workunit.client.0.smithi094.stdout:3/459: creat d9/d1e/d6e/d75/d90/fa0 x:0 0 0 2024-04-06T03:40:36.273 INFO:tasks.workunit.client.0.smithi094.stdout:3/460: dread - d9/d10/d27/f98 zero size 2024-04-06T03:40:36.273 INFO:tasks.workunit.client.0.smithi094.stdout:3/461: chown d9/d10/d27/d3a/c72 521051629 1 2024-04-06T03:40:36.273 INFO:tasks.workunit.client.0.smithi094.stdout:5/389: dwrite f6 [0,4194304] 0 2024-04-06T03:40:36.276 INFO:tasks.workunit.client.1.smithi136.stdout:2/509: rename d3/d4a/d9d/f22 to d3/d15/d19/d43/d54/fa8 0 2024-04-06T03:40:36.276 INFO:tasks.workunit.client.1.smithi136.stdout:2/510: readlink d3/d15/l34 0 2024-04-06T03:40:36.277 INFO:tasks.workunit.client.1.smithi136.stdout:8/460: dwrite d3/de/f8d [0,4194304] 0 2024-04-06T03:40:36.277 INFO:tasks.workunit.client.1.smithi136.stdout:8/461: fsync d3/f21 0 2024-04-06T03:40:36.277 INFO:tasks.workunit.client.1.smithi136.stdout:8/462: creat d3/de/f8e x:0 0 0 2024-04-06T03:40:36.277 INFO:tasks.workunit.client.1.smithi136.stdout:8/463: readlink d3/d2d/l51 0 2024-04-06T03:40:36.278 INFO:tasks.workunit.client.1.smithi136.stdout:8/464: creat d3/da/d26/d35/d53/d86/f8f x:0 0 0 2024-04-06T03:40:36.278 INFO:tasks.workunit.client.0.smithi094.stdout:0/413: dwrite d1/d9/dd/d18/d1e/d22/f37 [0,4194304] 0 2024-04-06T03:40:36.281 INFO:tasks.workunit.client.0.smithi094.stdout:9/373: dwrite d2/dc/d10/f66 [0,4194304] 0 2024-04-06T03:40:36.281 INFO:tasks.workunit.client.0.smithi094.stdout:9/374: dread - d2/f7b zero size 2024-04-06T03:40:36.283 INFO:tasks.workunit.client.1.smithi136.stdout:9/409: creat d17/d20/d28/d81/f83 x:0 0 0 2024-04-06T03:40:36.283 INFO:tasks.workunit.client.1.smithi136.stdout:9/410: creat d17/d20/d25/f84 x:0 0 0 2024-04-06T03:40:36.285 INFO:tasks.workunit.client.0.smithi094.stdout:8/460: creat d18/d2c/d37/fa2 x:0 0 0 2024-04-06T03:40:36.288 INFO:tasks.workunit.client.0.smithi094.stdout:7/401: mknod d2/d6/dc/dd/c75 0 2024-04-06T03:40:36.289 INFO:tasks.workunit.client.0.smithi094.stdout:8/461: dread d18/d2c/d62/f6d [0,4194304] 0 2024-04-06T03:40:36.290 INFO:tasks.workunit.client.1.smithi136.stdout:8/465: write d3/da/d26/f7a [3235001,40202] 0 2024-04-06T03:40:36.291 INFO:tasks.workunit.client.1.smithi136.stdout:7/408: mkdir d5/df/d53/d68/d8f 0 2024-04-06T03:40:36.294 INFO:tasks.workunit.client.1.smithi136.stdout:6/346: getdents d2/d7/d26/d30 0 2024-04-06T03:40:36.297 INFO:tasks.workunit.client.1.smithi136.stdout:0/399: unlink d1/d6/d2c/c17 0 2024-04-06T03:40:36.299 INFO:tasks.workunit.client.0.smithi094.stdout:9/375: mkdir d2/dc/d21/d26/d8a 0 2024-04-06T03:40:36.300 INFO:tasks.workunit.client.1.smithi136.stdout:1/413: mknod d12/d3b/d74/d7a/c85 0 2024-04-06T03:40:36.305 INFO:tasks.workunit.client.0.smithi094.stdout:2/477: dwrite d0/d18/f24 [0,4194304] 0 2024-04-06T03:40:36.305 INFO:tasks.workunit.client.0.smithi094.stdout:2/478: fdatasync d0/d18/f22 0 2024-04-06T03:40:36.305 INFO:tasks.workunit.client.0.smithi094.stdout:0/414: dwrite d1/d9/dd/d18/d5d/d60/d6b/f59 [0,4194304] 0 2024-04-06T03:40:36.306 INFO:tasks.workunit.client.1.smithi136.stdout:5/466: rename d1/dc/d15 to d1/dc/d1c/d32/d8b 0 2024-04-06T03:40:36.306 INFO:tasks.workunit.client.1.smithi136.stdout:5/467: write d1/dc/d1c/d32/d8b/d20/d66/f88 [675733,26799] 0 2024-04-06T03:40:36.306 INFO:tasks.workunit.client.0.smithi094.stdout:5/390: dwrite db/fc [0,4194304] 0 2024-04-06T03:40:36.307 INFO:tasks.workunit.client.0.smithi094.stdout:8/462: dread d18/f22 [0,4194304] 0 2024-04-06T03:40:36.307 INFO:tasks.workunit.client.0.smithi094.stdout:5/391: creat db/d19/d24/d28/f8c x:0 0 0 2024-04-06T03:40:36.307 INFO:tasks.workunit.client.0.smithi094.stdout:8/463: write d18/d2c/d37/fa1 [2365664,51618] 0 2024-04-06T03:40:36.307 INFO:tasks.workunit.client.0.smithi094.stdout:9/376: mknod d2/dc/d21/d26/d78/c8b 0 2024-04-06T03:40:36.309 INFO:tasks.workunit.client.1.smithi136.stdout:2/511: creat d3/d4a/d9d/d8a/d93/fa9 x:0 0 0 2024-04-06T03:40:36.313 INFO:tasks.workunit.client.0.smithi094.stdout:2/479: truncate d0/d8/f33 322119 0 2024-04-06T03:40:36.320 INFO:tasks.workunit.client.0.smithi094.stdout:0/415: mkdir d1/d9/dd/d18/d1e/d86 0 2024-04-06T03:40:36.320 INFO:tasks.workunit.client.0.smithi094.stdout:8/464: dwrite fd [0,4194304] 0 2024-04-06T03:40:36.325 INFO:tasks.workunit.client.1.smithi136.stdout:5/468: dwrite d1/dc/d10/d1e/f69 [4194304,4194304] 0 2024-04-06T03:40:36.327 INFO:tasks.workunit.client.0.smithi094.stdout:7/402: truncate d2/d6/f8 117464 0 2024-04-06T03:40:36.327 INFO:tasks.workunit.client.0.smithi094.stdout:7/403: chown d2/d6/dc/dd/df/d36/f60 683 1 2024-04-06T03:40:36.327 INFO:tasks.workunit.client.0.smithi094.stdout:7/404: dread - d2/d6/dc/d12/d29/d3f/f6b zero size 2024-04-06T03:40:36.328 INFO:tasks.workunit.client.0.smithi094.stdout:5/392: readlink db/d19/l22 0 2024-04-06T03:40:36.332 INFO:tasks.workunit.client.0.smithi094.stdout:9/377: truncate d2/d2c/d33/d37/f3a 4162571 0 2024-04-06T03:40:36.334 INFO:tasks.workunit.client.0.smithi094.stdout:8/465: dwrite d18/d5c/f8d [0,4194304] 0 2024-04-06T03:40:36.334 INFO:tasks.workunit.client.0.smithi094.stdout:8/466: stat d18/d34/f3d 0 2024-04-06T03:40:36.334 INFO:tasks.workunit.client.0.smithi094.stdout:8/467: chown d18/d3c/d76/c50 966 1 2024-04-06T03:40:36.336 INFO:tasks.workunit.client.1.smithi136.stdout:9/411: truncate f11 490557 0 2024-04-06T03:40:36.336 INFO:tasks.workunit.client.1.smithi136.stdout:9/412: dread - d17/d21/f7c zero size 2024-04-06T03:40:36.336 INFO:tasks.workunit.client.0.smithi094.stdout:2/480: symlink d0/dd/d19/d31/d67/l9b 0 2024-04-06T03:40:36.341 INFO:tasks.workunit.client.1.smithi136.stdout:8/466: unlink d3/de/c73 0 2024-04-06T03:40:36.342 INFO:tasks.workunit.client.0.smithi094.stdout:7/405: dwrite d2/d6/dc/dd/df/f2d [0,4194304] 0 2024-04-06T03:40:36.343 INFO:tasks.workunit.client.0.smithi094.stdout:5/393: creat db/d19/d21/f8d x:0 0 0 2024-04-06T03:40:36.343 INFO:tasks.workunit.client.0.smithi094.stdout:5/394: truncate db/d19/d2c/f79 481097 0 2024-04-06T03:40:36.343 INFO:tasks.workunit.client.0.smithi094.stdout:5/395: chown db/d19/d21/f49 12109132 1 2024-04-06T03:40:36.344 INFO:tasks.workunit.client.1.smithi136.stdout:6/347: mknod d2/d7/d26/c71 0 2024-04-06T03:40:36.344 INFO:tasks.workunit.client.1.smithi136.stdout:6/348: chown d2/d7/d26/d30/d6e/c36 223768 1 2024-04-06T03:40:36.344 INFO:tasks.workunit.client.1.smithi136.stdout:6/349: fdatasync d2/d7/d26/d30/d4f/f5f 0 2024-04-06T03:40:36.345 INFO:tasks.workunit.client.0.smithi094.stdout:9/378: rename d2/dc/f56 to d2/dc/d21/d23/d47/d51/f8c 0 2024-04-06T03:40:36.346 INFO:tasks.workunit.client.1.smithi136.stdout:0/400: symlink d1/d6/d2c/d1a/d36/l8c 0 2024-04-06T03:40:36.355 INFO:tasks.workunit.client.0.smithi094.stdout:7/406: dwrite d2/d6/dc/dd/d5a/f70 [0,4194304] 0 2024-04-06T03:40:36.355 INFO:tasks.workunit.client.0.smithi094.stdout:7/407: truncate d2/d6/f73 387400 0 2024-04-06T03:40:36.358 INFO:tasks.workunit.client.1.smithi136.stdout:1/414: creat d12/d1b/f86 x:0 0 0 2024-04-06T03:40:36.359 INFO:tasks.workunit.client.0.smithi094.stdout:2/481: symlink d0/dd/d1b/d72/l9c 0 2024-04-06T03:40:36.360 INFO:tasks.workunit.client.1.smithi136.stdout:2/512: mkdir d3/d15/d19/d43/daa 0 2024-04-06T03:40:36.362 INFO:tasks.workunit.client.0.smithi094.stdout:0/416: truncate d1/d9/dd/d18/d1a/f25 214591 0 2024-04-06T03:40:36.367 INFO:tasks.workunit.client.0.smithi094.stdout:7/408: dread d2/d6/dc/d12/d29/d35/f39 [0,4194304] 0 2024-04-06T03:40:36.371 INFO:tasks.workunit.client.0.smithi094.stdout:9/379: link d2/d2c/d33/d37/l6e d2/dc/l8d 0 2024-04-06T03:40:36.378 INFO:tasks.workunit.client.0.smithi094.stdout:2/482: link d0/d8/d1d/c7c d0/dd/d19/d31/d67/c9d 0 2024-04-06T03:40:36.378 INFO:tasks.workunit.client.1.smithi136.stdout:2/513: dwrite d3/d15/d19/f1b [0,4194304] 0 2024-04-06T03:40:36.379 INFO:tasks.workunit.client.1.smithi136.stdout:5/469: symlink d1/d1f/l8c 0 2024-04-06T03:40:36.382 INFO:tasks.workunit.client.1.smithi136.stdout:8/467: mkdir d3/da/d26/d90 0 2024-04-06T03:40:36.383 INFO:tasks.workunit.client.0.smithi094.stdout:0/417: unlink d1/d9/dd/d18/d1e/d22/f49 0 2024-04-06T03:40:36.383 INFO:tasks.workunit.client.0.smithi094.stdout:0/418: creat d1/d9/dd/d18/d1e/d22/f87 x:0 0 0 2024-04-06T03:40:36.383 INFO:tasks.workunit.client.0.smithi094.stdout:0/419: truncate d1/d9/dd/d18/d1e/f71 888946 0 2024-04-06T03:40:36.384 INFO:tasks.workunit.client.1.smithi136.stdout:7/409: mknod d5/df/d12/d1b/d3a/c90 0 2024-04-06T03:40:36.384 INFO:tasks.workunit.client.1.smithi136.stdout:7/410: truncate d5/df/d11/f38 200980 0 2024-04-06T03:40:36.389 INFO:tasks.workunit.client.1.smithi136.stdout:3/383: sync 2024-04-06T03:40:36.389 INFO:tasks.workunit.client.1.smithi136.stdout:4/385: sync 2024-04-06T03:40:36.389 INFO:tasks.workunit.client.1.smithi136.stdout:4/386: chown d1d/d31/d3e/l63 110 1 2024-04-06T03:40:36.389 INFO:tasks.workunit.client.1.smithi136.stdout:4/387: read d1d/d31/f48 [852500,11642] 0 2024-04-06T03:40:36.389 INFO:tasks.workunit.client.1.smithi136.stdout:6/350: rename d2/d7/d26/d30/d6e/d3d/d47/c4d to d2/d7/d26/d30/d4f/c72 0 2024-04-06T03:40:36.391 INFO:tasks.workunit.client.1.smithi136.stdout:2/514: dwrite d3/d15/f6c [0,4194304] 0 2024-04-06T03:40:36.391 INFO:tasks.workunit.client.0.smithi094.stdout:7/409: creat d2/d6/dc/d12/d29/f76 x:0 0 0 2024-04-06T03:40:36.392 INFO:tasks.workunit.client.0.smithi094.stdout:7/410: chown d2/d6/dc/dd/f3b 131 1 2024-04-06T03:40:36.392 INFO:tasks.workunit.client.1.smithi136.stdout:0/401: write d1/d6/d2c/d1a/d36/f44 [906482,97953] 0 2024-04-06T03:40:36.394 INFO:tasks.workunit.client.0.smithi094.stdout:2/483: truncate d0/dd/ff 263455 0 2024-04-06T03:40:36.397 INFO:tasks.workunit.client.1.smithi136.stdout:1/415: symlink d12/d1b/l87 0 2024-04-06T03:40:36.397 INFO:tasks.workunit.client.1.smithi136.stdout:1/416: dread - d12/d3b/f84 zero size 2024-04-06T03:40:36.397 INFO:tasks.workunit.client.1.smithi136.stdout:1/417: fsync d12/d1b/f29 0 2024-04-06T03:40:36.397 INFO:tasks.workunit.client.1.smithi136.stdout:1/418: chown d12/d1b/d1e/d23 1 1 2024-04-06T03:40:36.401 INFO:tasks.workunit.client.1.smithi136.stdout:5/470: symlink d1/d16/d4b/l8d 0 2024-04-06T03:40:36.401 INFO:tasks.workunit.client.1.smithi136.stdout:5/471: truncate d1/dc/d1c/f34 1512726 0 2024-04-06T03:40:36.402 INFO:tasks.workunit.client.0.smithi094.stdout:8/468: rename d18/d30/d68 to d18/d2c/d37/d60/da3 0 2024-04-06T03:40:36.402 INFO:tasks.workunit.client.0.smithi094.stdout:8/469: creat d18/d2c/d37/d60/fa4 x:0 0 0 2024-04-06T03:40:36.403 INFO:tasks.workunit.client.0.smithi094.stdout:0/420: dwrite d1/d9/dd/d18/d1a/f7e [0,4194304] 0 2024-04-06T03:40:36.403 INFO:tasks.workunit.client.1.smithi136.stdout:9/413: rmdir d17/d20/d25/d3d/d4b 39 2024-04-06T03:40:36.407 INFO:tasks.workunit.client.1.smithi136.stdout:8/468: symlink d3/de/d18/l91 0 2024-04-06T03:40:36.407 INFO:tasks.workunit.client.1.smithi136.stdout:7/411: dwrite d5/df/d1d/f57 [0,4194304] 0 2024-04-06T03:40:36.407 INFO:tasks.workunit.client.1.smithi136.stdout:7/412: write d5/fd [3930797,91359] 0 2024-04-06T03:40:36.407 INFO:tasks.workunit.client.1.smithi136.stdout:7/413: readlink d5/df/d53/l23 0 2024-04-06T03:40:36.407 INFO:tasks.workunit.client.0.smithi094.stdout:2/484: truncate d0/dd/d19/d31/f42 4014010 0 2024-04-06T03:40:36.407 INFO:tasks.workunit.client.0.smithi094.stdout:2/485: fsync d0/f57 0 2024-04-06T03:40:36.409 INFO:tasks.workunit.client.0.smithi094.stdout:8/470: read d18/d30/f3e [2800320,65849] 0 2024-04-06T03:40:36.410 INFO:tasks.workunit.client.1.smithi136.stdout:3/384: mknod d0/c7b 0 2024-04-06T03:40:36.411 INFO:tasks.workunit.client.0.smithi094.stdout:5/396: rename db/d19/d21/f8d to db/d19/d30/f8e 0 2024-04-06T03:40:36.413 INFO:tasks.workunit.client.1.smithi136.stdout:4/388: mkdir d1d/d2f/d6c/d91 0 2024-04-06T03:40:36.416 INFO:tasks.workunit.client.1.smithi136.stdout:6/351: link d2/l8 d2/d7/l73 0 2024-04-06T03:40:36.416 INFO:tasks.workunit.client.0.smithi094.stdout:7/411: dwrite d2/d6/dc/f74 [0,4194304] 0 2024-04-06T03:40:36.417 INFO:tasks.workunit.client.1.smithi136.stdout:2/515: mknod d3/d4a/d9d/d18/d1d/d91/cab 0 2024-04-06T03:40:36.419 INFO:tasks.workunit.client.1.smithi136.stdout:0/402: chown d1/d6/d2c/c68 95 1 2024-04-06T03:40:36.422 INFO:tasks.workunit.client.0.smithi094.stdout:0/421: mkdir d1/d9/dd/d18/d5d/d88 0 2024-04-06T03:40:36.422 INFO:tasks.workunit.client.0.smithi094.stdout:0/422: creat d1/d9/dd/d18/d1a/f89 x:0 0 0 2024-04-06T03:40:36.422 INFO:tasks.workunit.client.0.smithi094.stdout:0/423: fsync d1/d9/dd/d18/d32/f5f 0 2024-04-06T03:40:36.422 INFO:tasks.workunit.client.0.smithi094.stdout:0/424: chown d1/d10/d4e/l51 26 1 2024-04-06T03:40:36.422 INFO:tasks.workunit.client.0.smithi094.stdout:0/425: read - d1/d9/dd/d18/d32/f7d zero size 2024-04-06T03:40:36.425 INFO:tasks.workunit.client.0.smithi094.stdout:8/471: symlink d18/d3c/d76/la5 0 2024-04-06T03:40:36.428 INFO:tasks.workunit.client.1.smithi136.stdout:5/472: creat d1/d16/d4b/f8e x:0 0 0 2024-04-06T03:40:36.428 INFO:tasks.workunit.client.0.smithi094.stdout:9/380: rename d2/dc/d21/c3d to d2/dc/d21/d26/d78/c8e 0 2024-04-06T03:40:36.429 INFO:tasks.workunit.client.1.smithi136.stdout:8/469: dwrite d3/da/d26/d35/f38 [0,4194304] 0 2024-04-06T03:40:36.429 INFO:tasks.workunit.client.0.smithi094.stdout:2/486: dwrite d0/d8/d1d/f79 [0,4194304] 0 2024-04-06T03:40:36.431 INFO:tasks.workunit.client.1.smithi136.stdout:9/414: rename d17/d20/d25/c50 to d17/c85 0 2024-04-06T03:40:36.432 INFO:tasks.workunit.client.0.smithi094.stdout:5/397: mkdir db/d19/d2c/d57/d8f 0 2024-04-06T03:40:36.432 INFO:tasks.workunit.client.0.smithi094.stdout:5/398: write db/d19/d30/d83/f87 [704349,101250] 0 2024-04-06T03:40:36.434 INFO:tasks.workunit.client.1.smithi136.stdout:7/414: rename d5/df/d1d/d6f/c4e to d5/df/d1d/d6f/c91 0 2024-04-06T03:40:36.437 INFO:tasks.workunit.client.1.smithi136.stdout:3/385: link d0/d4/d12/d3a/d43/d5d/d6a/l77 d0/d4/d12/d19/d55/d3c/d46/l7c 0 2024-04-06T03:40:36.437 INFO:tasks.workunit.client.1.smithi136.stdout:3/386: truncate d0/f11 902216 0 2024-04-06T03:40:36.437 INFO:tasks.workunit.client.1.smithi136.stdout:8/470: dread d3/de/d18/d6d/f7b [0,4194304] 0 2024-04-06T03:40:36.438 INFO:tasks.workunit.client.0.smithi094.stdout:0/426: creat d1/d9/dd/d18/d5d/d60/d6b/f8a x:0 0 0 2024-04-06T03:40:36.440 INFO:tasks.workunit.client.1.smithi136.stdout:2/516: mkdir d3/d15/d19/dac 0 2024-04-06T03:40:36.440 INFO:tasks.workunit.client.1.smithi136.stdout:2/517: chown d3/d15/f6c 134617620 1 2024-04-06T03:40:36.441 INFO:tasks.workunit.client.0.smithi094.stdout:9/381: mknod d2/dc/d21/c8f 0 2024-04-06T03:40:36.441 INFO:tasks.workunit.client.0.smithi094.stdout:9/382: creat d2/dc/d21/d23/d2f/f90 x:0 0 0 2024-04-06T03:40:36.442 INFO:tasks.workunit.client.1.smithi136.stdout:1/419: getdents d12/d1b/d1e/d23 0 2024-04-06T03:40:36.444 INFO:tasks.workunit.client.1.smithi136.stdout:4/389: dwrite d1d/d31/f3a [0,4194304] 0 2024-04-06T03:40:36.444 INFO:tasks.workunit.client.1.smithi136.stdout:0/403: dwrite d1/d4e/f55 [0,4194304] 0 2024-04-06T03:40:36.446 INFO:tasks.workunit.client.1.smithi136.stdout:5/473: creat d1/dc/d1c/d32/f8f x:0 0 0 2024-04-06T03:40:36.447 INFO:tasks.workunit.client.1.smithi136.stdout:4/390: dread d1d/d70/f68 [0,4194304] 0 2024-04-06T03:40:36.448 INFO:tasks.workunit.client.0.smithi094.stdout:8/472: dwrite d18/d3c/d8b/f69 [4194304,4194304] 0 2024-04-06T03:40:36.449 INFO:tasks.workunit.client.1.smithi136.stdout:9/415: mknod d17/d21/d29/c86 0 2024-04-06T03:40:36.449 INFO:tasks.workunit.client.0.smithi094.stdout:7/412: dwrite d2/d6/dc/dd/df/d36/f4d [0,4194304] 0 2024-04-06T03:40:36.450 INFO:tasks.workunit.client.0.smithi094.stdout:2/487: symlink d0/d8/d27/d2b/d76/l9e 0 2024-04-06T03:40:36.451 INFO:tasks.workunit.client.0.smithi094.stdout:2/488: read d0/f5 [7994296,18700] 0 2024-04-06T03:40:36.451 INFO:tasks.workunit.client.0.smithi094.stdout:2/489: truncate d0/dd/d19/d31/f6e 270470 0 2024-04-06T03:40:36.451 INFO:tasks.workunit.client.1.smithi136.stdout:7/415: symlink d5/df/l92 0 2024-04-06T03:40:36.452 INFO:tasks.workunit.client.1.smithi136.stdout:1/420: dread d12/d3b/d60/d7e/d37/f4e [0,4194304] 0 2024-04-06T03:40:36.452 INFO:tasks.workunit.client.1.smithi136.stdout:6/352: dwrite d2/d25/f17 [0,4194304] 0 2024-04-06T03:40:36.452 INFO:tasks.workunit.client.1.smithi136.stdout:6/353: fdatasync d2/d7/d26/d30/d4f/f6d 0 2024-04-06T03:40:36.456 INFO:tasks.workunit.client.1.smithi136.stdout:3/387: unlink d0/d4/d12/f20 0 2024-04-06T03:40:36.457 INFO:tasks.workunit.client.0.smithi094.stdout:9/383: dwrite d2/f30 [0,4194304] 0 2024-04-06T03:40:36.457 INFO:tasks.workunit.client.0.smithi094.stdout:9/384: creat d2/dc/d21/d23/d2f/f91 x:0 0 0 2024-04-06T03:40:36.458 INFO:tasks.workunit.client.1.smithi136.stdout:8/471: getdents d3/da/d26/d35/d3d/d48/d6c 0 2024-04-06T03:40:36.459 INFO:tasks.workunit.client.0.smithi094.stdout:0/427: symlink d1/d9/dd/d18/d1a/d44/l8b 0 2024-04-06T03:40:36.461 INFO:tasks.workunit.client.1.smithi136.stdout:2/518: unlink d3/l97 0 2024-04-06T03:40:36.462 INFO:tasks.workunit.client.1.smithi136.stdout:1/421: dread d12/f39 [0,4194304] 0 2024-04-06T03:40:36.467 INFO:tasks.workunit.client.0.smithi094.stdout:2/490: dwrite d0/f57 [0,4194304] 0 2024-04-06T03:40:36.469 INFO:tasks.workunit.client.0.smithi094.stdout:9/385: read d2/d2c/d33/f4a [4176513,105109] 0 2024-04-06T03:40:36.469 INFO:tasks.workunit.client.0.smithi094.stdout:9/386: write d2/d2c/d33/d37/f55 [400273,41004] 0 2024-04-06T03:40:36.469 INFO:tasks.workunit.client.0.smithi094.stdout:9/387: read - d2/dc/d21/d23/d2f/f83 zero size 2024-04-06T03:40:36.473 INFO:tasks.workunit.client.1.smithi136.stdout:5/474: symlink d1/d16/d4b/d7b/l90 0 2024-04-06T03:40:36.474 INFO:tasks.workunit.client.0.smithi094.stdout:5/399: write db/fd [2305355,42742] 0 2024-04-06T03:40:36.475 INFO:tasks.workunit.client.1.smithi136.stdout:5/475: fdatasync d1/dc/d1c/d32/d8b/d20/d66/f88 0 2024-04-06T03:40:36.475 INFO:tasks.workunit.client.1.smithi136.stdout:4/391: rmdir d1d/d31/d78 39 2024-04-06T03:40:36.478 INFO:tasks.workunit.client.1.smithi136.stdout:9/416: rename l14 to d17/d19/l87 0 2024-04-06T03:40:36.480 INFO:tasks.workunit.client.1.smithi136.stdout:7/416: creat d5/df/d12/d1b/d58/f93 x:0 0 0 2024-04-06T03:40:36.483 INFO:tasks.workunit.client.1.smithi136.stdout:3/388: mknod d0/d4/d12/d3a/d43/c7d 0 2024-04-06T03:40:36.484 INFO:tasks.workunit.client.1.smithi136.stdout:8/472: unlink d3/da/d26/d35/d3d/f6b 0 2024-04-06T03:40:36.486 INFO:tasks.workunit.client.1.smithi136.stdout:2/519: creat d3/d9/fad x:0 0 0 2024-04-06T03:40:36.487 INFO:tasks.workunit.client.1.smithi136.stdout:1/422: symlink d12/d53/l88 0 2024-04-06T03:40:36.488 INFO:tasks.workunit.client.1.smithi136.stdout:0/404: rmdir d1/d6/d2c/d1a/d36 39 2024-04-06T03:40:36.489 INFO:tasks.workunit.client.1.smithi136.stdout:5/476: creat d1/dc/f91 x:0 0 0 2024-04-06T03:40:36.490 INFO:tasks.workunit.client.1.smithi136.stdout:4/392: mknod d1d/d31/d3e/d56/c92 0 2024-04-06T03:40:36.490 INFO:tasks.workunit.client.1.smithi136.stdout:1/423: dread d12/d3b/f7b [0,4194304] 0 2024-04-06T03:40:36.492 INFO:tasks.workunit.client.1.smithi136.stdout:6/354: rename d2/d7/d13/f44 to d2/d7/d26/d30/d6e/d58/d5d/d62/d53/f74 0 2024-04-06T03:40:36.492 INFO:tasks.workunit.client.1.smithi136.stdout:5/477: dread d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:36.493 INFO:tasks.workunit.client.1.smithi136.stdout:9/417: mkdir d17/d20/d28/d81/d88 0 2024-04-06T03:40:36.494 INFO:tasks.workunit.client.1.smithi136.stdout:7/417: fdatasync d5/df/d12/d1b/d3a/d4a/f64 0 2024-04-06T03:40:36.494 INFO:tasks.workunit.client.1.smithi136.stdout:7/418: dread - d5/df/d12/d1b/d3a/d4a/f86 zero size 2024-04-06T03:40:36.495 INFO:tasks.workunit.client.1.smithi136.stdout:3/389: mkdir d0/d4/d12/d19/d55/d3c/d5a/d79/d7e 0 2024-04-06T03:40:36.496 INFO:tasks.workunit.client.0.smithi094.stdout:0/428: creat d1/d9/dd/d18/d1e/f8c x:0 0 0 2024-04-06T03:40:36.497 INFO:tasks.workunit.client.1.smithi136.stdout:8/473: link d3/da/d26/f85 d3/da/d26/d35/d3d/d48/f92 0 2024-04-06T03:40:36.498 INFO:tasks.workunit.client.0.smithi094.stdout:8/473: getdents d18 0 2024-04-06T03:40:36.499 INFO:tasks.workunit.client.1.smithi136.stdout:2/520: creat d3/d15/d19/d4e/fae x:0 0 0 2024-04-06T03:40:36.499 INFO:tasks.workunit.client.1.smithi136.stdout:2/521: write d3/d4a/d9d/d18/f2a [1034783,64660] 0 2024-04-06T03:40:36.499 INFO:tasks.workunit.client.0.smithi094.stdout:2/491: mknod d0/dd/d19/c9f 0 2024-04-06T03:40:36.504 INFO:tasks.workunit.client.1.smithi136.stdout:6/355: write d2/d7/d26/d30/d6e/d3d/f46 [657870,127030] 0 2024-04-06T03:40:36.504 INFO:tasks.workunit.client.1.smithi136.stdout:6/356: dread - d2/d7/d26/f54 zero size 2024-04-06T03:40:36.506 INFO:tasks.workunit.client.1.smithi136.stdout:1/424: dwrite f10 [4194304,4194304] 0 2024-04-06T03:40:36.507 INFO:tasks.workunit.client.0.smithi094.stdout:9/388: creat d2/dc/d21/d23/d47/d51/f92 x:0 0 0 2024-04-06T03:40:36.507 INFO:tasks.workunit.client.0.smithi094.stdout:9/389: fsync d2/d2c/d33/f4c 0 2024-04-06T03:40:36.508 INFO:tasks.workunit.client.1.smithi136.stdout:4/393: mknod d1d/d2f/d6c/d91/c93 0 2024-04-06T03:40:36.509 INFO:tasks.workunit.client.0.smithi094.stdout:7/413: truncate d2/d6/dc/dd/df/d36/f4d 193443 0 2024-04-06T03:40:36.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/400: creat db/d19/d2c/d57/d8f/f90 x:0 0 0 2024-04-06T03:40:36.512 INFO:tasks.workunit.client.1.smithi136.stdout:6/357: dread d2/d7/d26/d30/d4f/f67 [0,4194304] 0 2024-04-06T03:40:36.512 INFO:tasks.workunit.client.1.smithi136.stdout:6/358: chown d2/f3c 16802 1 2024-04-06T03:40:36.512 INFO:tasks.workunit.client.1.smithi136.stdout:6/359: truncate d2/d7/d26/d30/d6e/f5e 1032134 0 2024-04-06T03:40:36.512 INFO:tasks.workunit.client.1.smithi136.stdout:5/478: symlink d1/dc/d1c/d32/d8b/d20/l92 0 2024-04-06T03:40:36.515 INFO:tasks.workunit.client.0.smithi094.stdout:9/390: dread d2/dc/d10/f66 [0,4194304] 0 2024-04-06T03:40:36.515 INFO:tasks.workunit.client.0.smithi094.stdout:9/391: write d2/f7b [806201,53773] 0 2024-04-06T03:40:36.516 INFO:tasks.workunit.client.1.smithi136.stdout:7/419: rmdir d5/df/d1d/d6f 39 2024-04-06T03:40:36.518 INFO:tasks.workunit.client.1.smithi136.stdout:7/420: dread d5/df/d1d/d6f/d2a/f5d [0,4194304] 0 2024-04-06T03:40:36.518 INFO:tasks.workunit.client.1.smithi136.stdout:0/405: dwrite d1/d6/d2c/d57/d43/d4a/f67 [0,4194304] 0 2024-04-06T03:40:36.519 INFO:tasks.workunit.client.1.smithi136.stdout:3/390: creat d0/d4/d12/d19/d2c/f7f x:0 0 0 2024-04-06T03:40:36.520 INFO:tasks.workunit.client.1.smithi136.stdout:3/391: chown d0/d4/d12/d3a/f4b 24027 1 2024-04-06T03:40:36.520 INFO:tasks.workunit.client.1.smithi136.stdout:3/392: stat d0/d4/d12/d3a/d43/d5d/d6a/l6e 0 2024-04-06T03:40:36.520 INFO:tasks.workunit.client.1.smithi136.stdout:3/393: read d0/d4/f18 [877985,121800] 0 2024-04-06T03:40:36.520 INFO:tasks.workunit.client.1.smithi136.stdout:3/394: dread - d0/d4/d12/d19/d2c/f7f zero size 2024-04-06T03:40:36.520 INFO:tasks.workunit.client.1.smithi136.stdout:3/395: fsync d0/d4/d12/d2d/f39 0 2024-04-06T03:40:36.521 INFO:tasks.workunit.client.1.smithi136.stdout:4/394: write d1d/d70/f64 [1529042,112876] 0 2024-04-06T03:40:36.521 INFO:tasks.workunit.client.1.smithi136.stdout:8/474: dwrite d3/da/d26/d35/f1d [4194304,4194304] 0 2024-04-06T03:40:36.522 INFO:tasks.workunit.client.1.smithi136.stdout:6/360: dread d2/d25/f56 [0,4194304] 0 2024-04-06T03:40:36.522 INFO:tasks.workunit.client.1.smithi136.stdout:1/425: rename d12/d3b/f7b to d12/f89 0 2024-04-06T03:40:36.524 INFO:tasks.workunit.client.1.smithi136.stdout:5/479: mknod d1/dc/d1c/d32/d8b/d20/d7c/c93 0 2024-04-06T03:40:36.526 INFO:tasks.workunit.client.1.smithi136.stdout:0/406: read d1/d4e/f55 [451483,20413] 0 2024-04-06T03:40:36.528 INFO:tasks.workunit.client.0.smithi094.stdout:8/474: unlink d18/d2c/d62/c85 0 2024-04-06T03:40:36.528 INFO:tasks.workunit.client.1.smithi136.stdout:9/418: rmdir d17/d20/d28/d81 39 2024-04-06T03:40:36.528 INFO:tasks.workunit.client.1.smithi136.stdout:9/419: dread - d17/d20/d25/f31 zero size 2024-04-06T03:40:36.530 INFO:tasks.workunit.client.1.smithi136.stdout:7/421: creat d5/df/d1d/d6f/d4d/d69/f94 x:0 0 0 2024-04-06T03:40:36.533 INFO:tasks.workunit.client.0.smithi094.stdout:7/414: symlink d2/d6/dc/d12/d3a/d56/l77 0 2024-04-06T03:40:36.536 INFO:tasks.workunit.client.0.smithi094.stdout:5/401: rename db/d19/d30/l4c to db/d19/d24/d28/d85/l91 0 2024-04-06T03:40:36.536 INFO:tasks.workunit.client.0.smithi094.stdout:5/402: truncate db/f69 394020 0 2024-04-06T03:40:36.537 INFO:tasks.workunit.client.0.smithi094.stdout:5/403: write db/f69 [966620,1211] 0 2024-04-06T03:40:36.537 INFO:tasks.workunit.client.0.smithi094.stdout:5/404: dread - db/d19/d2c/f73 zero size 2024-04-06T03:40:36.538 INFO:tasks.workunit.client.1.smithi136.stdout:8/475: dread d3/f2a [0,4194304] 0 2024-04-06T03:40:36.538 INFO:tasks.workunit.client.1.smithi136.stdout:8/476: readlink d3/d2d/l5d 0 2024-04-06T03:40:36.538 INFO:tasks.workunit.client.1.smithi136.stdout:3/396: mknod d0/d4/d12/d2d/d37/c80 0 2024-04-06T03:40:36.538 INFO:tasks.workunit.client.1.smithi136.stdout:3/397: dread - d0/d4/d12/d19/d2e/f75 zero size 2024-04-06T03:40:36.538 INFO:tasks.workunit.client.1.smithi136.stdout:4/395: write d1d/d31/f33 [1339224,40818] 0 2024-04-06T03:40:36.539 INFO:tasks.workunit.client.1.smithi136.stdout:1/426: write d12/f2c [835751,58105] 0 2024-04-06T03:40:36.539 INFO:tasks.workunit.client.1.smithi136.stdout:1/427: write d12/d3b/f4c [1245726,54893] 0 2024-04-06T03:40:36.540 INFO:tasks.workunit.client.1.smithi136.stdout:2/522: getdents d3/d4a/d9d/d8a/d93 0 2024-04-06T03:40:36.541 INFO:tasks.workunit.client.0.smithi094.stdout:8/475: truncate d18/d2c/f9b 75280 0 2024-04-06T03:40:36.543 INFO:tasks.workunit.client.1.smithi136.stdout:6/361: mkdir d2/d7/d26/d30/d4f/d75 0 2024-04-06T03:40:36.546 INFO:tasks.workunit.client.1.smithi136.stdout:5/480: rename d1/d16/d17/f1a to d1/dc/d10/f94 0 2024-04-06T03:40:36.546 INFO:tasks.workunit.client.1.smithi136.stdout:5/481: fsync d1/dc/d10/d1e/f2f 0 2024-04-06T03:40:36.546 INFO:tasks.workunit.client.1.smithi136.stdout:5/482: fdatasync d1/dc/d1c/d32/d8b/f73 0 2024-04-06T03:40:36.546 INFO:tasks.workunit.client.1.smithi136.stdout:5/483: creat d1/d16/d4b/f95 x:0 0 0 2024-04-06T03:40:36.547 INFO:tasks.workunit.client.1.smithi136.stdout:5/484: chown d1/d16/d4b/d7b/f83 4145 1 2024-04-06T03:40:36.548 INFO:tasks.workunit.client.1.smithi136.stdout:0/407: unlink d1/d6/c26 0 2024-04-06T03:40:36.548 INFO:tasks.workunit.client.1.smithi136.stdout:9/420: dwrite d17/d20/d28/d49/d75/d60/f70 [0,4194304] 0 2024-04-06T03:40:36.548 INFO:tasks.workunit.client.1.smithi136.stdout:9/421: dread - d17/d20/d25/f84 zero size 2024-04-06T03:40:36.548 INFO:tasks.workunit.client.1.smithi136.stdout:9/422: write d17/d20/d25/d3c/f73 [399680,34180] 0 2024-04-06T03:40:36.549 INFO:tasks.workunit.client.1.smithi136.stdout:9/423: chown d17/d20/d28/d49/d75/d60/f7a 951060470 1 2024-04-06T03:40:36.549 INFO:tasks.workunit.client.0.smithi094.stdout:2/492: rmdir d0/d8/d27 39 2024-04-06T03:40:36.549 INFO:tasks.workunit.client.0.smithi094.stdout:3/462: sync 2024-04-06T03:40:36.549 INFO:tasks.workunit.client.0.smithi094.stdout:1/445: sync 2024-04-06T03:40:36.549 INFO:tasks.workunit.client.0.smithi094.stdout:6/397: sync 2024-04-06T03:40:36.549 INFO:tasks.workunit.client.0.smithi094.stdout:4/484: sync 2024-04-06T03:40:36.549 INFO:tasks.workunit.client.0.smithi094.stdout:6/398: creat dc/d11/d22/d50/f8f x:0 0 0 2024-04-06T03:40:36.550 INFO:tasks.workunit.client.1.smithi136.stdout:0/408: dread d1/f63 [0,4194304] 0 2024-04-06T03:40:36.550 INFO:tasks.workunit.client.1.smithi136.stdout:7/422: symlink d5/df/d12/d1b/d3a/d8d/l95 0 2024-04-06T03:40:36.550 INFO:tasks.workunit.client.0.smithi094.stdout:4/485: dread d0/d15/d3a/d3e/d49/f74 [0,4194304] 0 2024-04-06T03:40:36.551 INFO:tasks.workunit.client.0.smithi094.stdout:7/415: creat d2/d6/dc/d12/f78 x:0 0 0 2024-04-06T03:40:36.554 INFO:tasks.workunit.client.1.smithi136.stdout:8/477: creat d3/da/d26/d35/d39/d70/f93 x:0 0 0 2024-04-06T03:40:36.554 INFO:tasks.workunit.client.1.smithi136.stdout:8/478: write d3/da/d26/d35/d3d/d48/f72 [1547294,78788] 0 2024-04-06T03:40:36.557 INFO:tasks.workunit.client.1.smithi136.stdout:3/398: link d0/d4/d12/d2d/d37/c80 d0/d4/d12/d3a/d51/c81 0 2024-04-06T03:40:36.557 INFO:tasks.workunit.client.1.smithi136.stdout:3/399: write d0/d4/d12/d3a/d43/d5d/f73 [392019,108662] 0 2024-04-06T03:40:36.558 INFO:tasks.workunit.client.1.smithi136.stdout:3/400: read d0/d4/fa [195345,87657] 0 2024-04-06T03:40:36.561 INFO:tasks.workunit.client.0.smithi094.stdout:8/476: mknod d18/d3c/ca6 0 2024-04-06T03:40:36.562 INFO:tasks.workunit.client.0.smithi094.stdout:1/446: dwrite d0/d4/d1b/d2f/d4b/f91 [0,4194304] 0 2024-04-06T03:40:36.563 INFO:tasks.workunit.client.1.smithi136.stdout:9/424: dwrite f13 [0,4194304] 0 2024-04-06T03:40:36.563 INFO:tasks.workunit.client.1.smithi136.stdout:1/428: link d12/d3b/d44/d5f/c52 d12/d3b/d44/d5f/d5a/c8a 0 2024-04-06T03:40:36.563 INFO:tasks.workunit.client.0.smithi094.stdout:8/477: write d18/d34/d52/f79 [533446,86426] 0 2024-04-06T03:40:36.563 INFO:tasks.workunit.client.0.smithi094.stdout:8/478: write d18/f1b [1475735,4456] 0 2024-04-06T03:40:36.563 INFO:tasks.workunit.client.0.smithi094.stdout:2/493: mkdir d0/d8/da0 0 2024-04-06T03:40:36.563 INFO:tasks.workunit.client.0.smithi094.stdout:2/494: truncate d0/dd/d3d/f88 404813 0 2024-04-06T03:40:36.565 INFO:tasks.workunit.client.1.smithi136.stdout:2/523: symlink d3/d9/laf 0 2024-04-06T03:40:36.565 INFO:tasks.workunit.client.1.smithi136.stdout:2/524: write d3/d4a/d9d/d18/d1d/f35 [4702603,39431] 0 2024-04-06T03:40:36.567 INFO:tasks.workunit.client.0.smithi094.stdout:3/463: mknod d9/d20/d76/ca1 0 2024-04-06T03:40:36.570 INFO:tasks.workunit.client.1.smithi136.stdout:6/362: rename d2/fd to d2/d7/d13/f76 0 2024-04-06T03:40:36.574 INFO:tasks.workunit.client.0.smithi094.stdout:6/399: creat dc/d3a/d4a/d54/d71/f90 x:0 0 0 2024-04-06T03:40:36.574 INFO:tasks.workunit.client.0.smithi094.stdout:6/400: fdatasync dc/d11/d1b/d24/f2e 0 2024-04-06T03:40:36.576 INFO:tasks.workunit.client.0.smithi094.stdout:4/486: mkdir d0/d15/d3a/d3e/d46/da8 0 2024-04-06T03:40:36.576 INFO:tasks.workunit.client.0.smithi094.stdout:4/487: fdatasync d0/d77/f9e 0 2024-04-06T03:40:36.577 INFO:tasks.workunit.client.0.smithi094.stdout:4/488: fdatasync d0/d15/f8f 0 2024-04-06T03:40:36.577 INFO:tasks.workunit.client.0.smithi094.stdout:4/489: stat d0/d4/f14 0 2024-04-06T03:40:36.577 INFO:tasks.workunit.client.0.smithi094.stdout:4/490: read d0/d4/fb [899363,73492] 0 2024-04-06T03:40:36.577 INFO:tasks.workunit.client.0.smithi094.stdout:4/491: write d0/df/d13/f8c [1015976,101708] 0 2024-04-06T03:40:36.577 INFO:tasks.workunit.client.0.smithi094.stdout:4/492: write d0/d4/f7c [1024243,105631] 0 2024-04-06T03:40:36.578 INFO:tasks.workunit.client.1.smithi136.stdout:0/409: creat d1/d6/d2c/d1a/d82/f8d x:0 0 0 2024-04-06T03:40:36.579 INFO:tasks.workunit.client.0.smithi094.stdout:7/416: mkdir d2/d6/dc/dd/df/d36/d79 0 2024-04-06T03:40:36.579 INFO:tasks.workunit.client.0.smithi094.stdout:7/417: readlink d2/d6/dc/l1e 0 2024-04-06T03:40:36.581 INFO:tasks.workunit.client.1.smithi136.stdout:7/423: getdents d5/df/d12/d1b 0 2024-04-06T03:40:36.581 INFO:tasks.workunit.client.1.smithi136.stdout:7/424: stat d5/df/f62 0 2024-04-06T03:40:36.581 INFO:tasks.workunit.client.1.smithi136.stdout:7/425: chown d5/df/d1d/l8b 3 1 2024-04-06T03:40:36.581 INFO:tasks.workunit.client.1.smithi136.stdout:7/426: dread - d5/f80 zero size 2024-04-06T03:40:36.581 INFO:tasks.workunit.client.1.smithi136.stdout:7/427: dread - d5/df/d11/f2b zero size 2024-04-06T03:40:36.581 INFO:tasks.workunit.client.1.smithi136.stdout:7/428: fsync d5/df/d1d/f7f 0 2024-04-06T03:40:36.582 INFO:tasks.workunit.client.0.smithi094.stdout:5/405: rmdir db/d19/d24/d28 39 2024-04-06T03:40:36.582 INFO:tasks.workunit.client.0.smithi094.stdout:3/464: dwrite d9/d20/d2d/f67 [0,4194304] 0 2024-04-06T03:40:36.582 INFO:tasks.workunit.client.0.smithi094.stdout:3/465: write d9/d1e/d6e/d75/d90/fa0 [800985,1451] 0 2024-04-06T03:40:36.582 INFO:tasks.workunit.client.1.smithi136.stdout:8/479: symlink d3/da/d26/d35/d3d/d48/d6c/d7f/l94 0 2024-04-06T03:40:36.585 INFO:tasks.workunit.client.1.smithi136.stdout:4/396: getdents d1d 0 2024-04-06T03:40:36.588 INFO:tasks.workunit.client.1.smithi136.stdout:9/425: link d17/d20/d25/d3c/d55/f79 d17/d20/d28/f89 0 2024-04-06T03:40:36.588 INFO:tasks.workunit.client.1.smithi136.stdout:9/426: chown d17/d20/d28 1243 1 2024-04-06T03:40:36.588 INFO:tasks.workunit.client.1.smithi136.stdout:9/427: readlink d17/d19/l33 0 2024-04-06T03:40:36.588 INFO:tasks.workunit.client.1.smithi136.stdout:9/428: fsync d17/d21/d32/d3f/f44 0 2024-04-06T03:40:36.590 INFO:tasks.workunit.client.1.smithi136.stdout:2/525: symlink d3/d9/d3b/d87/lb0 0 2024-04-06T03:40:36.590 INFO:tasks.workunit.client.1.smithi136.stdout:2/526: write d3/d4a/d9d/d18/fa4 [256084,1212] 0 2024-04-06T03:40:36.590 INFO:tasks.workunit.client.1.smithi136.stdout:2/527: dread - d3/d4a/d9d/d18/d1d/d3c/fa1 zero size 2024-04-06T03:40:36.590 INFO:tasks.workunit.client.1.smithi136.stdout:8/480: dread d3/da/d26/f28 [0,4194304] 0 2024-04-06T03:40:36.590 INFO:tasks.workunit.client.0.smithi094.stdout:2/495: creat d0/d8/d1d/d77/fa1 x:0 0 0 2024-04-06T03:40:36.592 INFO:tasks.workunit.client.0.smithi094.stdout:6/401: mknod dc/d11/d1b/d24/d46/c91 0 2024-04-06T03:40:36.594 INFO:tasks.workunit.client.0.smithi094.stdout:4/493: unlink d0/df/d13/d4a/c71 0 2024-04-06T03:40:36.600 INFO:tasks.workunit.client.0.smithi094.stdout:5/406: mknod db/d19/d2c/d57/d8f/c92 0 2024-04-06T03:40:36.603 INFO:tasks.workunit.client.0.smithi094.stdout:3/466: rmdir d9/d10/d27/d97 39 2024-04-06T03:40:36.603 INFO:tasks.workunit.client.0.smithi094.stdout:3/467: write d9/f8c [100773,80614] 0 2024-04-06T03:40:36.605 INFO:tasks.workunit.client.0.smithi094.stdout:8/479: rename d18/l8f to d18/la7 0 2024-04-06T03:40:36.605 INFO:tasks.workunit.client.0.smithi094.stdout:8/480: fdatasync d18/d34/f3d 0 2024-04-06T03:40:36.607 INFO:tasks.workunit.client.0.smithi094.stdout:2/496: truncate d0/d8/d1d/f38 4011867 0 2024-04-06T03:40:36.607 INFO:tasks.workunit.client.0.smithi094.stdout:2/497: write d0/d18/f55 [1866449,5261] 0 2024-04-06T03:40:36.607 INFO:tasks.workunit.client.0.smithi094.stdout:2/498: chown d0/d18/d3f/f5f 186140268 1 2024-04-06T03:40:36.609 INFO:tasks.workunit.client.0.smithi094.stdout:6/402: creat dc/d3a/d4a/f92 x:0 0 0 2024-04-06T03:40:36.609 INFO:tasks.workunit.client.0.smithi094.stdout:4/494: dwrite d0/d4/f76 [4194304,4194304] 0 2024-04-06T03:40:36.610 INFO:tasks.workunit.client.0.smithi094.stdout:5/407: dread db/d19/d24/d2e/f36 [0,4194304] 0 2024-04-06T03:40:36.610 INFO:tasks.workunit.client.0.smithi094.stdout:5/408: creat db/d19/d21/f93 x:0 0 0 2024-04-06T03:40:36.611 INFO:tasks.workunit.client.0.smithi094.stdout:7/418: getdents d2/d6/dc/dd/d5a 0 2024-04-06T03:40:36.611 INFO:tasks.workunit.client.0.smithi094.stdout:7/419: write d2/d6/dc/d12/f78 [148424,74764] 0 2024-04-06T03:40:36.611 INFO:tasks.workunit.client.0.smithi094.stdout:4/495: dread d0/f44 [0,4194304] 0 2024-04-06T03:40:36.611 INFO:tasks.workunit.client.0.smithi094.stdout:7/420: getdents d2/d6/dc/d12/d29/d35 0 2024-04-06T03:40:36.611 INFO:tasks.workunit.client.0.smithi094.stdout:4/496: write d0/d15/d3a/d3e/d46/f72 [28979,55302] 0 2024-04-06T03:40:36.611 INFO:tasks.workunit.client.1.smithi136.stdout:0/410: symlink d1/d6/d2c/d1a/d36/d59/l8e 0 2024-04-06T03:40:36.612 INFO:tasks.workunit.client.0.smithi094.stdout:8/481: mkdir d18/d3c/d49/d91/d94/da8 0 2024-04-06T03:40:36.612 INFO:tasks.workunit.client.0.smithi094.stdout:8/482: stat d18/f22 0 2024-04-06T03:40:36.613 INFO:tasks.workunit.client.1.smithi136.stdout:7/429: symlink d5/d6d/l96 0 2024-04-06T03:40:36.613 INFO:tasks.workunit.client.1.smithi136.stdout:7/430: stat d5/df/d1d/d6f/d4d/c4f 0 2024-04-06T03:40:36.615 INFO:tasks.workunit.client.0.smithi094.stdout:2/499: mknod d0/dd/d19/ca2 0 2024-04-06T03:40:36.615 INFO:tasks.workunit.client.0.smithi094.stdout:2/500: chown d0/d8/d1d/f38 1 1 2024-04-06T03:40:36.616 INFO:tasks.workunit.client.0.smithi094.stdout:3/468: dread d9/d10/f81 [0,4194304] 0 2024-04-06T03:40:36.616 INFO:tasks.workunit.client.0.smithi094.stdout:3/469: read d9/d1d/f6a [723460,26419] 0 2024-04-06T03:40:36.616 INFO:tasks.workunit.client.0.smithi094.stdout:5/409: dread db/d19/d21/f49 [0,4194304] 0 2024-04-06T03:40:36.618 INFO:tasks.workunit.client.0.smithi094.stdout:6/403: mknod dc/d11/d22/c93 0 2024-04-06T03:40:36.619 INFO:tasks.workunit.client.0.smithi094.stdout:6/404: dread dc/f40 [0,4194304] 0 2024-04-06T03:40:36.619 INFO:tasks.workunit.client.0.smithi094.stdout:6/405: read - dc/d3a/d4a/d54/d71/f90 zero size 2024-04-06T03:40:36.624 INFO:tasks.workunit.client.0.smithi094.stdout:4/497: rename d0/d4/f14 to d0/d4/d67/d9f/fa9 0 2024-04-06T03:40:36.624 INFO:tasks.workunit.client.0.smithi094.stdout:4/498: chown d0/d4/d67/l69 192664 1 2024-04-06T03:40:36.625 INFO:tasks.workunit.client.0.smithi094.stdout:8/483: symlink d18/d3c/d8b/la9 0 2024-04-06T03:40:36.630 INFO:tasks.workunit.client.0.smithi094.stdout:3/470: mkdir d9/d10/d27/d7c/da2 0 2024-04-06T03:40:36.630 INFO:tasks.workunit.client.0.smithi094.stdout:3/471: truncate d9/d1e/d26/d55/d62/f68 366640 0 2024-04-06T03:40:36.631 INFO:tasks.workunit.client.1.smithi136.stdout:4/397: dread d1d/d31/d3e/f5f [0,4194304] 0 2024-04-06T03:40:36.631 INFO:tasks.workunit.client.1.smithi136.stdout:4/398: creat d1d/d70/f94 x:0 0 0 2024-04-06T03:40:36.631 INFO:tasks.workunit.client.1.smithi136.stdout:4/399: fdatasync d1d/f22 0 2024-04-06T03:40:36.633 INFO:tasks.workunit.client.0.smithi094.stdout:4/499: symlink d0/df/d91/laa 0 2024-04-06T03:40:36.635 INFO:tasks.workunit.client.0.smithi094.stdout:5/410: rmdir db/d19/d2c/d57/d8f 39 2024-04-06T03:40:36.635 INFO:tasks.workunit.client.0.smithi094.stdout:5/411: chown db/d19/d24/d28/d85 5 1 2024-04-06T03:40:36.636 INFO:tasks.workunit.client.0.smithi094.stdout:3/472: mkdir d9/d10/d7e/da3 0 2024-04-06T03:40:36.637 INFO:tasks.workunit.client.0.smithi094.stdout:6/406: getdents dc/d11/d1b/d72 0 2024-04-06T03:40:36.638 INFO:tasks.workunit.client.0.smithi094.stdout:7/421: dwrite d2/d6/f30 [0,4194304] 0 2024-04-06T03:40:36.639 INFO:tasks.workunit.client.1.smithi136.stdout:7/431: dwrite d5/df/f10 [0,4194304] 0 2024-04-06T03:40:36.640 INFO:tasks.workunit.client.0.smithi094.stdout:4/500: truncate d0/df/d13/f58 593501 0 2024-04-06T03:40:36.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/501: stat d0/d15/d3a/d3e/d46/l47 0 2024-04-06T03:40:36.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/502: creat d0/d15/d3a/d61/d81/fab x:0 0 0 2024-04-06T03:40:36.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/503: creat d0/d15/d3a/d3e/d50/fac x:0 0 0 2024-04-06T03:40:36.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/504: write d0/d15/d3a/d3e/d50/fac [741981,28443] 0 2024-04-06T03:40:36.641 INFO:tasks.workunit.client.0.smithi094.stdout:2/501: dwrite d0/d8/d27/f45 [0,4194304] 0 2024-04-06T03:40:36.644 INFO:tasks.workunit.client.0.smithi094.stdout:5/412: rename db/d19/c4d to db/d19/d2c/c94 0 2024-04-06T03:40:36.644 INFO:tasks.workunit.client.0.smithi094.stdout:5/413: write db/d19/d24/d2e/f75 [960462,1161] 0 2024-04-06T03:40:36.647 INFO:tasks.workunit.client.1.smithi136.stdout:9/429: mkdir d17/d20/d25/d3c/d55/d8a 0 2024-04-06T03:40:36.647 INFO:tasks.workunit.client.1.smithi136.stdout:7/432: dread d5/df/d1d/d6f/d2a/f48 [0,4194304] 0 2024-04-06T03:40:36.647 INFO:tasks.workunit.client.1.smithi136.stdout:7/433: write d5/df/d1d/f1f [510603,15924] 0 2024-04-06T03:40:36.647 INFO:tasks.workunit.client.1.smithi136.stdout:7/434: fdatasync d5/df/d12/d59/f5e 0 2024-04-06T03:40:36.647 INFO:tasks.workunit.client.1.smithi136.stdout:7/435: write d5/df/d12/d1b/d3a/d4a/f64 [878015,79230] 0 2024-04-06T03:40:36.647 INFO:tasks.workunit.client.0.smithi094.stdout:6/407: mkdir dc/d3a/d4a/d94 0 2024-04-06T03:40:36.649 INFO:tasks.workunit.client.1.smithi136.stdout:2/528: mkdir d3/d9/d3b/d87/db1 0 2024-04-06T03:40:36.650 INFO:tasks.workunit.client.0.smithi094.stdout:7/422: mknod d2/d6/dc/dd/df/d36/d79/c7a 0 2024-04-06T03:40:36.654 INFO:tasks.workunit.client.1.smithi136.stdout:5/485: rename d1/dc/d1c/d32/d8b/d20/d66/f88 to d1/f96 0 2024-04-06T03:40:36.654 INFO:tasks.workunit.client.1.smithi136.stdout:5/486: truncate d1/d16/d4b/f7e 635594 0 2024-04-06T03:40:36.654 INFO:tasks.workunit.client.1.smithi136.stdout:5/487: chown d1/dc/d1c/d32/d8b/d20/l26 648 1 2024-04-06T03:40:36.654 INFO:tasks.workunit.client.1.smithi136.stdout:5/488: fdatasync d1/d16/d4b/f76 0 2024-04-06T03:40:36.656 INFO:tasks.workunit.client.1.smithi136.stdout:0/411: creat d1/d6/d2c/d1a/d36/f8f x:0 0 0 2024-04-06T03:40:36.656 INFO:tasks.workunit.client.1.smithi136.stdout:0/412: chown d1/d6/fb 16 1 2024-04-06T03:40:36.660 INFO:tasks.workunit.client.1.smithi136.stdout:7/436: unlink d5/df/d12/l4b 0 2024-04-06T03:40:36.660 INFO:tasks.workunit.client.1.smithi136.stdout:7/437: creat d5/df/d3d/f97 x:0 0 0 2024-04-06T03:40:36.662 INFO:tasks.workunit.client.0.smithi094.stdout:3/473: dwrite d9/d20/d76/f5c [4194304,4194304] 0 2024-04-06T03:40:36.662 INFO:tasks.workunit.client.0.smithi094.stdout:3/474: chown d9/d20/d76/l4c 34 1 2024-04-06T03:40:36.663 INFO:tasks.workunit.client.1.smithi136.stdout:2/529: link d3/d15/f90 d3/d4a/d9d/d8a/fb2 0 2024-04-06T03:40:36.663 INFO:tasks.workunit.client.0.smithi094.stdout:7/423: read d2/d6/dc/f74 [1166519,89512] 0 2024-04-06T03:40:36.664 INFO:tasks.workunit.client.0.smithi094.stdout:7/424: write d2/f23 [4795815,22305] 0 2024-04-06T03:40:36.664 INFO:tasks.workunit.client.0.smithi094.stdout:4/505: dwrite d0/d15/d3a/d3e/d46/fa3 [0,4194304] 0 2024-04-06T03:40:36.664 INFO:tasks.workunit.client.0.smithi094.stdout:4/506: write d0/d15/d3a/d3e/d46/f95 [165061,90941] 0 2024-04-06T03:40:36.664 INFO:tasks.workunit.client.0.smithi094.stdout:4/507: chown d0/d15/c97 1073912943 1 2024-04-06T03:40:36.667 INFO:tasks.workunit.client.1.smithi136.stdout:5/489: creat d1/dc/d10/d1e/f97 x:0 0 0 2024-04-06T03:40:36.669 INFO:tasks.workunit.client.1.smithi136.stdout:0/413: symlink d1/d6/d2c/d57/d62/l90 0 2024-04-06T03:40:36.669 INFO:tasks.workunit.client.1.smithi136.stdout:0/414: creat d1/d6/d2c/d57/d43/d4a/d5c/f91 x:0 0 0 2024-04-06T03:40:36.670 INFO:tasks.workunit.client.1.smithi136.stdout:2/530: dread d3/d4a/d9d/d18/d1d/d3c/f51 [0,4194304] 0 2024-04-06T03:40:36.670 INFO:tasks.workunit.client.1.smithi136.stdout:2/531: dread - d3/d4a/d9d/d18/d4b/f9c zero size 2024-04-06T03:40:36.672 INFO:tasks.workunit.client.0.smithi094.stdout:2/502: symlink d0/d8/d1d/la3 0 2024-04-06T03:40:36.677 INFO:tasks.workunit.client.0.smithi094.stdout:5/414: symlink db/d19/d2c/l95 0 2024-04-06T03:40:36.681 INFO:tasks.workunit.client.0.smithi094.stdout:6/408: rename dc/d11/d22/f44 to dc/f95 0 2024-04-06T03:40:36.681 INFO:tasks.workunit.client.0.smithi094.stdout:6/409: write dc/d11/d22/d50/d61/f73 [385195,20181] 0 2024-04-06T03:40:36.682 INFO:tasks.workunit.client.1.smithi136.stdout:7/438: mknod d5/df/d12/d1b/d3a/d8d/c98 0 2024-04-06T03:40:36.685 INFO:tasks.workunit.client.0.smithi094.stdout:3/475: mknod d9/d10/d27/ca4 0 2024-04-06T03:40:36.685 INFO:tasks.workunit.client.0.smithi094.stdout:3/476: chown d9/d1e/d26/d6c/d8e 58987 1 2024-04-06T03:40:36.692 INFO:tasks.workunit.client.0.smithi094.stdout:4/508: getdents d0/d15/d3a/d3e 0 2024-04-06T03:40:36.694 INFO:tasks.workunit.client.1.smithi136.stdout:0/415: rename d1/d6/d2c/d1a/d36/d59/f23 to d1/d6/d2c/d1a/d36/d59/f92 0 2024-04-06T03:40:36.694 INFO:tasks.workunit.client.1.smithi136.stdout:0/416: chown d1/d6/d2c/d1a/d36/d59 47091 1 2024-04-06T03:40:36.696 INFO:tasks.workunit.client.0.smithi094.stdout:7/425: rename d2/d6/dc/d12/d29/d35/f39 to d2/d6/dc/dd/df/d36/f7b 0 2024-04-06T03:40:36.698 INFO:tasks.workunit.client.0.smithi094.stdout:6/410: truncate dc/d11/f20 6800625 0 2024-04-06T03:40:36.698 INFO:tasks.workunit.client.0.smithi094.stdout:5/415: dwrite db/f6f [0,4194304] 0 2024-04-06T03:40:36.699 INFO:tasks.workunit.client.1.smithi136.stdout:7/439: mknod d5/df/d12/d1b/d3a/d8d/c99 0 2024-04-06T03:40:36.699 INFO:tasks.workunit.client.0.smithi094.stdout:7/426: write d2/fb [1220511,49136] 0 2024-04-06T03:40:36.700 INFO:tasks.workunit.client.0.smithi094.stdout:3/477: mknod d9/d10/d27/ca5 0 2024-04-06T03:40:36.701 INFO:tasks.workunit.client.1.smithi136.stdout:3/401: rename d0/d4/d12/d3a/d43 to d0/d4/d12/d19/d55/d6f/d82 0 2024-04-06T03:40:36.703 INFO:tasks.workunit.client.0.smithi094.stdout:2/503: rename d0/d18/d3f to d0/dd/da4 0 2024-04-06T03:40:36.703 INFO:tasks.workunit.client.0.smithi094.stdout:4/509: dwrite d0/f4e [0,4194304] 0 2024-04-06T03:40:36.704 INFO:tasks.workunit.client.1.smithi136.stdout:5/490: mkdir d1/d98 0 2024-04-06T03:40:36.704 INFO:tasks.workunit.client.0.smithi094.stdout:6/411: truncate dc/d3a/d4a/f6b 3511017 0 2024-04-06T03:40:36.707 INFO:tasks.workunit.client.1.smithi136.stdout:0/417: link d1/d6/d2c/d1a/l32 d1/d53/l93 0 2024-04-06T03:40:36.707 INFO:tasks.workunit.client.1.smithi136.stdout:0/418: fsync d1/d6/d2c/d57/d43/d6d/d85/d78/f7a 0 2024-04-06T03:40:36.707 INFO:tasks.workunit.client.1.smithi136.stdout:0/419: fdatasync d1/d6/fb 0 2024-04-06T03:40:36.709 INFO:tasks.workunit.client.1.smithi136.stdout:7/440: mkdir d5/df/d12/d1b/d3a/d4a/d9a 0 2024-04-06T03:40:36.713 INFO:tasks.workunit.client.1.smithi136.stdout:6/363: rename d2/d7/c1c to d2/d25/c77 0 2024-04-06T03:40:36.713 INFO:tasks.workunit.client.1.smithi136.stdout:5/491: write d1/dc/f5b [2781303,126493] 0 2024-04-06T03:40:36.714 INFO:tasks.workunit.client.0.smithi094.stdout:5/416: symlink db/d19/d24/d2e/l96 0 2024-04-06T03:40:36.714 INFO:tasks.workunit.client.0.smithi094.stdout:2/504: write d0/d8/d1d/f79 [3876347,2404] 0 2024-04-06T03:40:36.715 INFO:tasks.workunit.client.1.smithi136.stdout:3/402: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/f83 x:0 0 0 2024-04-06T03:40:36.715 INFO:tasks.workunit.client.1.smithi136.stdout:3/403: dread - d0/d4/d12/d3a/f4b zero size 2024-04-06T03:40:36.716 INFO:tasks.workunit.client.0.smithi094.stdout:7/427: mknod d2/d6/dc/d12/d29/c7c 0 2024-04-06T03:40:36.718 INFO:tasks.workunit.client.0.smithi094.stdout:5/417: dread db/f69 [0,4194304] 0 2024-04-06T03:40:36.719 INFO:tasks.workunit.client.1.smithi136.stdout:0/420: link d1/d6/fd d1/d6/d2c/d1a/f94 0 2024-04-06T03:40:36.723 INFO:tasks.workunit.client.0.smithi094.stdout:3/478: mkdir d9/d20/da6 0 2024-04-06T03:40:36.724 INFO:tasks.workunit.client.0.smithi094.stdout:6/412: dwrite dc/d11/d1b/f52 [0,4194304] 0 2024-04-06T03:40:36.726 INFO:tasks.workunit.client.0.smithi094.stdout:3/479: dread d9/d20/f4e [4194304,4194304] 0 2024-04-06T03:40:36.727 INFO:tasks.workunit.client.1.smithi136.stdout:4/400: rename d1d/f1f to d1d/d70/d8b/f95 0 2024-04-06T03:40:36.727 INFO:tasks.workunit.client.1.smithi136.stdout:7/441: dread d5/df/d11/f26 [0,4194304] 0 2024-04-06T03:40:36.727 INFO:tasks.workunit.client.1.smithi136.stdout:7/442: write d5/f80 [182269,84731] 0 2024-04-06T03:40:36.727 INFO:tasks.workunit.client.1.smithi136.stdout:7/443: write d5/df/d1d/d6f/d2c/f72 [472529,111127] 0 2024-04-06T03:40:36.729 INFO:tasks.workunit.client.1.smithi136.stdout:8/481: sync 2024-04-06T03:40:36.729 INFO:tasks.workunit.client.1.smithi136.stdout:1/429: sync 2024-04-06T03:40:36.730 INFO:tasks.workunit.client.1.smithi136.stdout:0/421: write d1/d6/d2c/d57/d43/d4a/f67 [2794134,87078] 0 2024-04-06T03:40:36.730 INFO:tasks.workunit.client.1.smithi136.stdout:0/422: readlink d1/d6/d2c/d1a/d36/d59/l24 0 2024-04-06T03:40:36.732 INFO:tasks.workunit.client.1.smithi136.stdout:6/364: symlink d2/l78 0 2024-04-06T03:40:36.732 INFO:tasks.workunit.client.1.smithi136.stdout:6/365: chown d2/l8 273225357 1 2024-04-06T03:40:36.733 INFO:tasks.workunit.client.0.smithi094.stdout:6/413: dread f5 [0,4194304] 0 2024-04-06T03:40:36.733 INFO:tasks.workunit.client.0.smithi094.stdout:6/414: chown dc/d11/d1b/d24 20 1 2024-04-06T03:40:36.734 INFO:tasks.workunit.client.0.smithi094.stdout:4/510: rename d0/d15/d3a/d3e/d46/da8 to d0/d15/d3a/d3e/d49/dad 0 2024-04-06T03:40:36.734 INFO:tasks.workunit.client.1.smithi136.stdout:5/492: dwrite d1/d16/d4b/f7e [0,4194304] 0 2024-04-06T03:40:36.735 INFO:tasks.workunit.client.1.smithi136.stdout:5/493: chown d1/dc/d1c/d32/d70 911 1 2024-04-06T03:40:36.735 INFO:tasks.workunit.client.1.smithi136.stdout:5/494: truncate d1/d16/f85 410930 0 2024-04-06T03:40:36.738 INFO:tasks.workunit.client.0.smithi094.stdout:2/505: mknod d0/dd/d19/ca5 0 2024-04-06T03:40:36.738 INFO:tasks.workunit.client.0.smithi094.stdout:2/506: truncate d0/dd/d3d/f88 869135 0 2024-04-06T03:40:36.738 INFO:tasks.workunit.client.0.smithi094.stdout:2/507: readlink d0/d18/l26 0 2024-04-06T03:40:36.738 INFO:tasks.workunit.client.0.smithi094.stdout:2/508: creat d0/d8/d1d/d77/fa6 x:0 0 0 2024-04-06T03:40:36.740 INFO:tasks.workunit.client.0.smithi094.stdout:7/428: mknod d2/d6/dc/d12/d29/d3f/c7d 0 2024-04-06T03:40:36.741 INFO:tasks.workunit.client.1.smithi136.stdout:4/401: symlink d1d/d31/d3e/l96 0 2024-04-06T03:40:36.742 INFO:tasks.workunit.client.0.smithi094.stdout:2/509: dread d0/dd/f8b [0,4194304] 0 2024-04-06T03:40:36.742 INFO:tasks.workunit.client.0.smithi094.stdout:5/418: unlink f6 0 2024-04-06T03:40:36.743 INFO:tasks.workunit.client.1.smithi136.stdout:7/444: symlink d5/df/d12/d1b/d58/l9b 0 2024-04-06T03:40:36.744 INFO:tasks.workunit.client.1.smithi136.stdout:1/430: stat d12/d1b/d28/l2d 0 2024-04-06T03:40:36.747 INFO:tasks.workunit.client.1.smithi136.stdout:8/482: symlink d3/da/d26/d35/d3d/d48/d6c/d79/l95 0 2024-04-06T03:40:36.747 INFO:tasks.workunit.client.1.smithi136.stdout:8/483: chown d3/c16 20 1 2024-04-06T03:40:36.750 INFO:tasks.workunit.client.0.smithi094.stdout:5/419: write db/d19/d2c/f2d [3683468,15737] 0 2024-04-06T03:40:36.751 INFO:tasks.workunit.client.1.smithi136.stdout:6/366: getdents d2/d7/d26/d30/d6e/d58/d5d 0 2024-04-06T03:40:36.753 INFO:tasks.workunit.client.1.smithi136.stdout:6/367: read d2/d7/f5a [430396,95253] 0 2024-04-06T03:40:36.755 INFO:tasks.workunit.client.0.smithi094.stdout:3/480: creat d9/d20/d30/d88/fa7 x:0 0 0 2024-04-06T03:40:36.757 INFO:tasks.workunit.client.1.smithi136.stdout:1/431: dread d12/d3b/d60/d7e/d37/f4e [0,4194304] 0 2024-04-06T03:40:36.757 INFO:tasks.workunit.client.0.smithi094.stdout:3/481: dread d9/d1e/f8f [0,4194304] 0 2024-04-06T03:40:36.758 INFO:tasks.workunit.client.1.smithi136.stdout:5/495: mkdir d1/dc/d10/d99 0 2024-04-06T03:40:36.760 INFO:tasks.workunit.client.1.smithi136.stdout:4/402: dwrite d1d/f20 [0,4194304] 0 2024-04-06T03:40:36.760 INFO:tasks.workunit.client.1.smithi136.stdout:4/403: write d1d/d31/d3e/f62 [760761,67061] 0 2024-04-06T03:40:36.760 INFO:tasks.workunit.client.1.smithi136.stdout:4/404: chown d1d/d31/d3e/d55/d89/c8f 1 1 2024-04-06T03:40:36.765 INFO:tasks.workunit.client.1.smithi136.stdout:6/368: dread d2/d7/d26/d30/d6e/f27 [0,4194304] 0 2024-04-06T03:40:36.765 INFO:tasks.workunit.client.1.smithi136.stdout:6/369: write d2/f5 [6804125,14751] 0 2024-04-06T03:40:36.767 INFO:tasks.workunit.client.1.smithi136.stdout:9/430: rename d17/d21/d29/d34/l45 to d17/d20/d25/d3c/l8b 0 2024-04-06T03:40:36.767 INFO:tasks.workunit.client.1.smithi136.stdout:9/431: fsync d17/d20/d25/f31 0 2024-04-06T03:40:36.769 INFO:tasks.workunit.client.1.smithi136.stdout:7/445: symlink d5/df/d12/l9c 0 2024-04-06T03:40:36.769 INFO:tasks.workunit.client.1.smithi136.stdout:5/496: dwrite d1/d16/d4b/f80 [0,4194304] 0 2024-04-06T03:40:36.770 INFO:tasks.workunit.client.1.smithi136.stdout:8/484: mknod d3/da/d26/d35/d53/d86/c96 0 2024-04-06T03:40:36.771 INFO:tasks.workunit.client.1.smithi136.stdout:8/485: fsync d3/d2d/f55 0 2024-04-06T03:40:36.772 INFO:tasks.workunit.client.1.smithi136.stdout:1/432: creat d12/d3b/d60/d7e/d37/f8b x:0 0 0 2024-04-06T03:40:36.773 INFO:tasks.workunit.client.1.smithi136.stdout:4/405: truncate d1d/d31/d3e/d56/f65 389421 0 2024-04-06T03:40:36.774 INFO:tasks.workunit.client.0.smithi094.stdout:7/429: mknod d2/d6/dc/dd/d5a/c7e 0 2024-04-06T03:40:36.775 INFO:tasks.workunit.client.1.smithi136.stdout:6/370: mkdir d2/d7/d13/d79 0 2024-04-06T03:40:36.775 INFO:tasks.workunit.client.1.smithi136.stdout:6/371: write d2/d7/d26/d30/d6e/f6b [1470091,20417] 0 2024-04-06T03:40:36.776 INFO:tasks.workunit.client.0.smithi094.stdout:7/430: read d2/d6/dc/d12/d29/d35/f3d [691185,67366] 0 2024-04-06T03:40:36.776 INFO:tasks.workunit.client.0.smithi094.stdout:7/431: readlink d2/d6/dc/d12/d61/l6a 0 2024-04-06T03:40:36.777 INFO:tasks.workunit.client.0.smithi094.stdout:2/510: rename d0/d8/d1d/la3 to d0/d8/da0/la7 0 2024-04-06T03:40:36.779 INFO:tasks.workunit.client.1.smithi136.stdout:0/423: rename d1/le to d1/d6/d2c/d57/d62/l95 0 2024-04-06T03:40:36.779 INFO:tasks.workunit.client.1.smithi136.stdout:0/424: write d1/d4e/f55 [2405455,50493] 0 2024-04-06T03:40:36.779 INFO:tasks.workunit.client.1.smithi136.stdout:0/425: chown d1/d6/d2c/d57/f3f 10161 1 2024-04-06T03:40:36.780 INFO:tasks.workunit.client.0.smithi094.stdout:5/420: link db/d40/f55 db/d80/f97 0 2024-04-06T03:40:36.780 INFO:tasks.workunit.client.0.smithi094.stdout:5/421: fsync db/d40/d77/f66 0 2024-04-06T03:40:36.782 INFO:tasks.workunit.client.0.smithi094.stdout:3/482: creat d9/d10/d27/d3a/fa8 x:0 0 0 2024-04-06T03:40:36.782 INFO:tasks.workunit.client.0.smithi094.stdout:3/483: truncate d9/d1e/f59 1333025 0 2024-04-06T03:40:36.784 INFO:tasks.workunit.client.0.smithi094.stdout:6/415: getdents dc/d3a 0 2024-04-06T03:40:36.786 INFO:tasks.workunit.client.1.smithi136.stdout:0/426: dread d1/d4e/f77 [0,4194304] 0 2024-04-06T03:40:36.786 INFO:tasks.workunit.client.1.smithi136.stdout:0/427: write d1/d6/d2c/d57/d43/d4a/d5c/f71 [329793,85867] 0 2024-04-06T03:40:36.787 INFO:tasks.workunit.client.0.smithi094.stdout:5/422: dread db/d19/d21/f8a [0,4194304] 0 2024-04-06T03:40:36.787 INFO:tasks.workunit.client.0.smithi094.stdout:5/423: write db/f2f [2235896,7762] 0 2024-04-06T03:40:36.788 INFO:tasks.workunit.client.0.smithi094.stdout:4/511: getdents d0/d15/d3a/d3e/d46 0 2024-04-06T03:40:36.788 INFO:tasks.workunit.client.0.smithi094.stdout:4/512: write d0/df/d57/f86 [426342,3723] 0 2024-04-06T03:40:36.788 INFO:tasks.workunit.client.0.smithi094.stdout:4/513: chown d0/d15/d3a/d3e/f7b 1221 1 2024-04-06T03:40:36.792 INFO:tasks.workunit.client.0.smithi094.stdout:7/432: rename d2/d6/dc/d12/d3a/d56/c5c to d2/d6/dc/dd/d5a/c7f 0 2024-04-06T03:40:36.793 INFO:tasks.workunit.client.0.smithi094.stdout:2/511: truncate d0/d8/d1d/f79 3780691 0 2024-04-06T03:40:36.795 INFO:tasks.workunit.client.1.smithi136.stdout:8/486: mknod d3/da/d26/d35/d3d/d48/d6c/d79/c97 0 2024-04-06T03:40:36.795 INFO:tasks.workunit.client.1.smithi136.stdout:8/487: chown d3/da/l3e 53 1 2024-04-06T03:40:36.797 INFO:tasks.workunit.client.1.smithi136.stdout:1/433: symlink d12/d3b/d44/d5f/l8c 0 2024-04-06T03:40:36.798 INFO:tasks.workunit.client.0.smithi094.stdout:3/484: getdents d9/d1e/d26/d55/d62 0 2024-04-06T03:40:36.798 INFO:tasks.workunit.client.1.smithi136.stdout:2/532: sync 2024-04-06T03:40:36.799 INFO:tasks.workunit.client.1.smithi136.stdout:4/406: truncate d1d/d31/f3b 275648 0 2024-04-06T03:40:36.800 INFO:tasks.workunit.client.0.smithi094.stdout:5/424: mkdir db/d7c/d98 0 2024-04-06T03:40:36.803 INFO:tasks.workunit.client.0.smithi094.stdout:4/514: mknod d0/d4/d67/cae 0 2024-04-06T03:40:36.803 INFO:tasks.workunit.client.0.smithi094.stdout:4/515: truncate d0/d15/d3a/d3e/d46/f62 1655951 0 2024-04-06T03:40:36.807 INFO:tasks.workunit.client.0.smithi094.stdout:6/416: rename dc/fd to dc/d11/d1b/d72/d88/f96 0 2024-04-06T03:40:36.807 INFO:tasks.workunit.client.0.smithi094.stdout:6/417: fsync dc/d3a/f41 0 2024-04-06T03:40:36.809 INFO:tasks.workunit.client.1.smithi136.stdout:9/432: rename d17/d20/d28/d49/d75/d60/f70 to d17/d21/d32/d43/f8c 0 2024-04-06T03:40:36.810 INFO:tasks.workunit.client.0.smithi094.stdout:7/433: rename d2/d6/dc/d12/d3a/d56/c58 to d2/d6/dc/d12/d3a/d56/c80 0 2024-04-06T03:40:36.812 INFO:tasks.workunit.client.0.smithi094.stdout:2/512: mkdir d0/dd/d1b/d72/da8 0 2024-04-06T03:40:36.813 INFO:tasks.workunit.client.0.smithi094.stdout:5/425: dwrite db/d19/d2c/f73 [0,4194304] 0 2024-04-06T03:40:36.814 INFO:tasks.workunit.client.1.smithi136.stdout:0/428: mkdir d1/d6/d2c/d57/d43/d4a/d5c/d96 0 2024-04-06T03:40:36.816 INFO:tasks.workunit.client.0.smithi094.stdout:3/485: unlink d9/d10/d27/d3a/f5a 0 2024-04-06T03:40:36.816 INFO:tasks.workunit.client.1.smithi136.stdout:3/404: sync 2024-04-06T03:40:36.819 INFO:tasks.workunit.client.1.smithi136.stdout:1/434: creat d12/d3b/d74/d7a/f8d x:0 0 0 2024-04-06T03:40:36.820 INFO:tasks.workunit.client.0.smithi094.stdout:4/516: unlink d0/d15/d3a/d3e/f7b 0 2024-04-06T03:40:36.820 INFO:tasks.workunit.client.0.smithi094.stdout:4/517: chown d0/df 16488 1 2024-04-06T03:40:36.822 INFO:tasks.workunit.client.1.smithi136.stdout:2/533: link d3/f61 d3/d4a/d9d/d18/d4b/fb3 0 2024-04-06T03:40:36.823 INFO:tasks.workunit.client.1.smithi136.stdout:4/407: unlink ld 0 2024-04-06T03:40:36.825 INFO:tasks.workunit.client.0.smithi094.stdout:6/418: mkdir dc/d11/d97 0 2024-04-06T03:40:36.826 INFO:tasks.workunit.client.1.smithi136.stdout:7/446: rename d5/df/d1d/d6f/f61 to d5/df/d53/f9d 0 2024-04-06T03:40:36.826 INFO:tasks.workunit.client.1.smithi136.stdout:7/447: dread - d5/df/d11/f2b zero size 2024-04-06T03:40:36.827 INFO:tasks.workunit.client.0.smithi094.stdout:7/434: creat d2/d6/dc/dd/df/d36/d79/f81 x:0 0 0 2024-04-06T03:40:36.828 INFO:tasks.workunit.client.1.smithi136.stdout:9/433: mkdir d17/d20/d28/d81/d88/d8d 0 2024-04-06T03:40:36.829 INFO:tasks.workunit.client.0.smithi094.stdout:2/513: mknod d0/d8/d4f/ca9 0 2024-04-06T03:40:36.830 INFO:tasks.workunit.client.1.smithi136.stdout:0/429: creat d1/d6/d2c/d57/d43/d6d/d85/d78/f97 x:0 0 0 2024-04-06T03:40:36.835 INFO:tasks.workunit.client.1.smithi136.stdout:3/405: creat d0/d57/f84 x:0 0 0 2024-04-06T03:40:36.835 INFO:tasks.workunit.client.1.smithi136.stdout:1/435: mkdir d12/d3b/d8e 0 2024-04-06T03:40:36.835 INFO:tasks.workunit.client.1.smithi136.stdout:1/436: readlink d12/d1b/d1e/l36 0 2024-04-06T03:40:36.835 INFO:tasks.workunit.client.1.smithi136.stdout:1/437: chown f10 10 1 2024-04-06T03:40:36.836 INFO:tasks.workunit.client.0.smithi094.stdout:3/486: mkdir d9/d20/d2d/da9 0 2024-04-06T03:40:36.836 INFO:tasks.workunit.client.0.smithi094.stdout:5/426: dwrite db/f11 [4194304,4194304] 0 2024-04-06T03:40:36.836 INFO:tasks.workunit.client.0.smithi094.stdout:5/427: read - db/f7a zero size 2024-04-06T03:40:36.836 INFO:tasks.workunit.client.0.smithi094.stdout:5/428: dread - db/d40/d77/f82 zero size 2024-04-06T03:40:36.836 INFO:tasks.workunit.client.0.smithi094.stdout:5/429: stat db/d19/d24/d2e/f75 0 2024-04-06T03:40:36.837 INFO:tasks.workunit.client.1.smithi136.stdout:2/534: unlink d3/d9/f89 0 2024-04-06T03:40:36.837 INFO:tasks.workunit.client.1.smithi136.stdout:5/497: rename d1/dc/d1c/l23 to d1/d16/d89/l9a 0 2024-04-06T03:40:36.840 INFO:tasks.workunit.client.1.smithi136.stdout:7/448: creat d5/df/d12/d1b/d3a/d8d/f9e x:0 0 0 2024-04-06T03:40:36.842 INFO:tasks.workunit.client.0.smithi094.stdout:7/435: dread d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:36.843 INFO:tasks.workunit.client.1.smithi136.stdout:9/434: link d17/d20/d25/d3c/f73 d17/d19/f8e 0 2024-04-06T03:40:36.845 INFO:tasks.workunit.client.1.smithi136.stdout:0/430: mkdir d1/d6/d2c/d57/d62/d98 0 2024-04-06T03:40:36.847 INFO:tasks.workunit.client.0.smithi094.stdout:4/518: rmdir d0/df/d13/d82 0 2024-04-06T03:40:36.849 INFO:tasks.workunit.client.0.smithi094.stdout:7/436: dread d2/f23 [0,4194304] 0 2024-04-06T03:40:36.849 INFO:tasks.workunit.client.1.smithi136.stdout:1/438: unlink d12/c6e 0 2024-04-06T03:40:36.850 INFO:tasks.workunit.client.0.smithi094.stdout:6/419: mknod dc/d3a/d4a/c98 0 2024-04-06T03:40:36.850 INFO:tasks.workunit.client.1.smithi136.stdout:2/535: rmdir d3/d4a/d9d/d18/d1d/d3c 39 2024-04-06T03:40:36.852 INFO:tasks.workunit.client.0.smithi094.stdout:2/514: mknod d0/d8/d1d/d77/d87/caa 0 2024-04-06T03:40:36.852 INFO:tasks.workunit.client.0.smithi094.stdout:2/515: chown d0/dd/d19/d31/d67/f6c 94 1 2024-04-06T03:40:36.853 INFO:tasks.workunit.client.1.smithi136.stdout:8/488: rename d3/f8 to d3/de/d60/f98 0 2024-04-06T03:40:36.854 INFO:tasks.workunit.client.1.smithi136.stdout:8/489: dread - d3/da/d26/d35/d39/d70/f93 zero size 2024-04-06T03:40:36.854 INFO:tasks.workunit.client.1.smithi136.stdout:6/372: sync 2024-04-06T03:40:36.855 INFO:tasks.workunit.client.0.smithi094.stdout:3/487: rename d9/d1e/d26/d55/d62/f80 to d9/d10/d27/d7c/faa 0 2024-04-06T03:40:36.857 INFO:tasks.workunit.client.0.smithi094.stdout:5/430: unlink db/d19/d2c/d6d/l6c 0 2024-04-06T03:40:36.857 INFO:tasks.workunit.client.0.smithi094.stdout:5/431: write db/d40/f55 [1421358,30726] 0 2024-04-06T03:40:36.859 INFO:tasks.workunit.client.1.smithi136.stdout:7/449: symlink d5/df/d1d/d6f/d2a/l9f 0 2024-04-06T03:40:36.859 INFO:tasks.workunit.client.1.smithi136.stdout:7/450: chown d5/c6 2177 1 2024-04-06T03:40:36.860 INFO:tasks.workunit.client.0.smithi094.stdout:3/488: dread d9/f3f [0,4194304] 0 2024-04-06T03:40:36.861 INFO:tasks.workunit.client.0.smithi094.stdout:4/519: symlink d0/d15/d3a/d61/d81/laf 0 2024-04-06T03:40:36.861 INFO:tasks.workunit.client.0.smithi094.stdout:4/520: fdatasync d0/d15/d3a/f75 0 2024-04-06T03:40:36.865 INFO:tasks.workunit.client.0.smithi094.stdout:7/437: creat d2/d6/dc/d12/d29/d35/d57/f82 x:0 0 0 2024-04-06T03:40:36.867 INFO:tasks.workunit.client.0.smithi094.stdout:6/420: mknod dc/d11/d1b/d24/d46/d7b/c99 0 2024-04-06T03:40:36.869 INFO:tasks.workunit.client.1.smithi136.stdout:0/431: creat d1/d6/d2c/d57/d43/d6d/f99 x:0 0 0 2024-04-06T03:40:36.870 INFO:tasks.workunit.client.0.smithi094.stdout:2/516: mkdir d0/d8/d4f/dab 0 2024-04-06T03:40:36.871 INFO:tasks.workunit.client.1.smithi136.stdout:1/439: unlink d12/l55 0 2024-04-06T03:40:36.872 INFO:tasks.workunit.client.1.smithi136.stdout:2/536: rmdir d3/d15 39 2024-04-06T03:40:36.873 INFO:tasks.workunit.client.1.smithi136.stdout:8/490: dwrite d3/da/d26/d35/d53/d86/f87 [0,4194304] 0 2024-04-06T03:40:36.873 INFO:tasks.workunit.client.1.smithi136.stdout:2/537: write d3/d4a/d9d/d18/fa4 [48253,44217] 0 2024-04-06T03:40:36.873 INFO:tasks.workunit.client.1.smithi136.stdout:2/538: stat d3/d4a/d9d/d18/f81 0 2024-04-06T03:40:36.874 INFO:tasks.workunit.client.0.smithi094.stdout:3/489: rename d9/d1e/d26/d54/l6f to d9/d1e/d6e/d75/lab 0 2024-04-06T03:40:36.874 INFO:tasks.workunit.client.0.smithi094.stdout:3/490: chown d9/d1e/d6e/d75 994 1 2024-04-06T03:40:36.874 INFO:tasks.workunit.client.0.smithi094.stdout:5/432: dwrite db/d40/d77/f6a [0,4194304] 0 2024-04-06T03:40:36.875 INFO:tasks.workunit.client.1.smithi136.stdout:4/408: rename f14 to d1d/d2f/f97 0 2024-04-06T03:40:36.877 INFO:tasks.workunit.client.1.smithi136.stdout:6/373: symlink d2/d25/l7a 0 2024-04-06T03:40:36.879 INFO:tasks.workunit.client.0.smithi094.stdout:2/517: read d0/d8/d4f/f5b [2517754,96537] 0 2024-04-06T03:40:36.879 INFO:tasks.workunit.client.1.smithi136.stdout:5/498: mkdir d1/d9b 0 2024-04-06T03:40:36.879 INFO:tasks.workunit.client.1.smithi136.stdout:5/499: dread - d1/dc/d1c/d32/d8b/d20/f42 zero size 2024-04-06T03:40:36.882 INFO:tasks.workunit.client.1.smithi136.stdout:7/451: symlink d5/df/d53/d68/la0 0 2024-04-06T03:40:36.882 INFO:tasks.workunit.client.1.smithi136.stdout:7/452: dread - d5/df/d12/d1b/d58/f93 zero size 2024-04-06T03:40:36.882 INFO:tasks.workunit.client.1.smithi136.stdout:7/453: write d5/df/f41 [4456792,105709] 0 2024-04-06T03:40:36.886 INFO:tasks.workunit.client.0.smithi094.stdout:6/421: symlink dc/d11/d1b/d24/d46/d7b/l9a 0 2024-04-06T03:40:36.886 INFO:tasks.workunit.client.0.smithi094.stdout:6/422: stat dc/c6f 0 2024-04-06T03:40:36.886 INFO:tasks.workunit.client.0.smithi094.stdout:6/423: write dc/d11/d1b/d72/f8c [83040,72441] 0 2024-04-06T03:40:36.888 INFO:tasks.workunit.client.1.smithi136.stdout:9/435: mknod d17/d21/d29/d34/c8f 0 2024-04-06T03:40:36.890 INFO:tasks.workunit.client.1.smithi136.stdout:0/432: fsync d1/d6/d2c/f27 0 2024-04-06T03:40:36.890 INFO:tasks.workunit.client.1.smithi136.stdout:0/433: truncate d1/d53/f6e 295589 0 2024-04-06T03:40:36.891 INFO:tasks.workunit.client.1.smithi136.stdout:8/491: dwrite d3/da/d26/d35/d3d/d48/f66 [4194304,4194304] 0 2024-04-06T03:40:36.891 INFO:tasks.workunit.client.0.smithi094.stdout:2/518: write d0/f57 [1478902,60943] 0 2024-04-06T03:40:36.891 INFO:tasks.workunit.client.0.smithi094.stdout:5/433: mknod db/d7c/d8b/c99 0 2024-04-06T03:40:36.891 INFO:tasks.workunit.client.0.smithi094.stdout:5/434: truncate db/d19/d21/f93 614532 0 2024-04-06T03:40:36.892 INFO:tasks.workunit.client.0.smithi094.stdout:5/435: write db/d19/d2c/f79 [594618,115001] 0 2024-04-06T03:40:36.895 INFO:tasks.workunit.client.0.smithi094.stdout:4/521: getdents d0/df/d13/d4a 0 2024-04-06T03:40:36.897 INFO:tasks.workunit.client.1.smithi136.stdout:2/539: mknod d3/d9/d3b/d87/cb4 0 2024-04-06T03:40:36.897 INFO:tasks.workunit.client.1.smithi136.stdout:2/540: creat d3/d9/fb5 x:0 0 0 2024-04-06T03:40:36.899 INFO:tasks.workunit.client.0.smithi094.stdout:7/438: getdents d2/d6/dc/dd/df 0 2024-04-06T03:40:36.902 INFO:tasks.workunit.client.0.smithi094.stdout:6/424: link dc/l15 dc/d11/d1b/d72/d88/l9b 0 2024-04-06T03:40:36.903 INFO:tasks.workunit.client.0.smithi094.stdout:3/491: getdents d9/d10/d27 0 2024-04-06T03:40:36.904 INFO:tasks.workunit.client.0.smithi094.stdout:3/492: creat d9/d20/d76/fac x:0 0 0 2024-04-06T03:40:36.904 INFO:tasks.workunit.client.0.smithi094.stdout:3/493: stat d9/d1d 0 2024-04-06T03:40:36.904 INFO:tasks.workunit.client.0.smithi094.stdout:3/494: dread - d9/d1d/d7d/f9c zero size 2024-04-06T03:40:36.904 INFO:tasks.workunit.client.1.smithi136.stdout:5/500: mkdir d1/d9b/d9c 0 2024-04-06T03:40:36.905 INFO:tasks.workunit.client.1.smithi136.stdout:7/454: dwrite d5/df/d11/f26 [8388608,4194304] 0 2024-04-06T03:40:36.905 INFO:tasks.workunit.client.1.smithi136.stdout:7/455: stat d5/df/d12/d1b/d3a/d4a/d70 0 2024-04-06T03:40:36.906 INFO:tasks.workunit.client.1.smithi136.stdout:0/434: dwrite d1/d6/d2c/d57/f3f [0,4194304] 0 2024-04-06T03:40:36.910 INFO:tasks.workunit.client.1.smithi136.stdout:2/541: dread d3/d4a/d9d/d18/f81 [0,4194304] 0 2024-04-06T03:40:36.911 INFO:tasks.workunit.client.0.smithi094.stdout:2/519: dread d0/d8/f33 [0,4194304] 0 2024-04-06T03:40:36.913 INFO:tasks.workunit.client.0.smithi094.stdout:5/436: mknod db/d19/d24/d28/c9a 0 2024-04-06T03:40:36.915 INFO:tasks.workunit.client.1.smithi136.stdout:8/492: creat d3/da/d26/d35/d39/f99 x:0 0 0 2024-04-06T03:40:36.917 INFO:tasks.workunit.client.0.smithi094.stdout:4/522: rmdir d0/d15/d3a/d3e/d49/dad 0 2024-04-06T03:40:36.919 INFO:tasks.workunit.client.1.smithi136.stdout:5/501: dwrite d1/d16/d4b/f7e [0,4194304] 0 2024-04-06T03:40:36.921 INFO:tasks.workunit.client.0.smithi094.stdout:7/439: creat d2/d6/dc/d12/d61/f83 x:0 0 0 2024-04-06T03:40:36.922 INFO:tasks.workunit.client.1.smithi136.stdout:3/406: rename d0/d4/l14 to d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/l85 0 2024-04-06T03:40:36.922 INFO:tasks.workunit.client.1.smithi136.stdout:3/407: write d0/d4/d12/d19/d55/d3c/d5a/f71 [969995,79715] 0 2024-04-06T03:40:36.923 INFO:tasks.workunit.client.0.smithi094.stdout:6/425: rename dc/d11/c7d to dc/d11/c9c 0 2024-04-06T03:40:36.926 INFO:tasks.workunit.client.1.smithi136.stdout:4/409: rename d1d/d2f/d6c/f8a to d1d/d2f/d6c/f98 0 2024-04-06T03:40:36.927 INFO:tasks.workunit.client.1.smithi136.stdout:7/456: creat d5/fa1 x:0 0 0 2024-04-06T03:40:36.927 INFO:tasks.workunit.client.1.smithi136.stdout:7/457: dread - d5/df/d11/f89 zero size 2024-04-06T03:40:36.927 INFO:tasks.workunit.client.1.smithi136.stdout:7/458: fsync d5/df/d11/f89 0 2024-04-06T03:40:36.928 INFO:tasks.workunit.client.1.smithi136.stdout:7/459: readlink d5/df/d12/d1b/d3a/d4a/l4c 0 2024-04-06T03:40:36.928 INFO:tasks.workunit.client.1.smithi136.stdout:7/460: creat d5/df/d11/fa2 x:0 0 0 2024-04-06T03:40:36.928 INFO:tasks.workunit.client.0.smithi094.stdout:3/495: truncate d9/f8d 328911 0 2024-04-06T03:40:36.928 INFO:tasks.workunit.client.0.smithi094.stdout:2/520: creat d0/d8/d27/d2b/d8e/fac x:0 0 0 2024-04-06T03:40:36.932 INFO:tasks.workunit.client.1.smithi136.stdout:0/435: getdents d1/d6/d2c/d84 0 2024-04-06T03:40:36.932 INFO:tasks.workunit.client.1.smithi136.stdout:0/436: write d1/d6/d2c/d57/d43/d6b/f80 [432549,58024] 0 2024-04-06T03:40:36.934 INFO:tasks.workunit.client.1.smithi136.stdout:7/461: write d5/fd [6874284,107903] 0 2024-04-06T03:40:36.937 INFO:tasks.workunit.client.0.smithi094.stdout:4/523: rename d0/d15/l3c to d0/d4/d67/d9f/lb0 0 2024-04-06T03:40:36.942 INFO:tasks.workunit.client.1.smithi136.stdout:9/436: getdents d17/d21/d32/d43/d51 0 2024-04-06T03:40:36.943 INFO:tasks.workunit.client.1.smithi136.stdout:9/437: readlink d17/d20/l59 0 2024-04-06T03:40:36.945 INFO:tasks.workunit.client.1.smithi136.stdout:2/542: creat d3/d4a/d9d/fb6 x:0 0 0 2024-04-06T03:40:36.946 INFO:tasks.workunit.client.1.smithi136.stdout:2/543: chown d3/d15/d19/d4e/fae 9175646 1 2024-04-06T03:40:36.946 INFO:tasks.workunit.client.1.smithi136.stdout:0/437: dread d1/d6/d2c/f27 [0,4194304] 0 2024-04-06T03:40:36.946 INFO:tasks.workunit.client.1.smithi136.stdout:7/462: write d5/df/d1d/f57 [4163115,45259] 0 2024-04-06T03:40:36.946 INFO:tasks.workunit.client.0.smithi094.stdout:2/521: rmdir d0/d8/d27 39 2024-04-06T03:40:36.948 INFO:tasks.workunit.client.1.smithi136.stdout:8/493: mknod d3/de/d18/d6d/c9a 0 2024-04-06T03:40:36.948 INFO:tasks.workunit.client.1.smithi136.stdout:5/502: dwrite d1/dc/d1c/d32/d8b/d20/f84 [0,4194304] 0 2024-04-06T03:40:36.949 INFO:tasks.workunit.client.0.smithi094.stdout:6/426: dwrite dc/d11/d1b/d24/d46/f4d [0,4194304] 0 2024-04-06T03:40:36.949 INFO:tasks.workunit.client.1.smithi136.stdout:3/408: dwrite d0/d4/d12/d19/d55/d6f/d82/d5d/f73 [0,4194304] 0 2024-04-06T03:40:36.949 INFO:tasks.workunit.client.0.smithi094.stdout:5/437: getdents db/d19/d2c 0 2024-04-06T03:40:36.950 INFO:tasks.workunit.client.0.smithi094.stdout:5/438: fsync db/f12 0 2024-04-06T03:40:36.951 INFO:tasks.workunit.client.0.smithi094.stdout:4/524: symlink d0/df/d91/lb1 0 2024-04-06T03:40:36.951 INFO:tasks.workunit.client.0.smithi094.stdout:4/525: chown d0/d15/d3a/d3e/d50/fa4 1120239 1 2024-04-06T03:40:36.952 INFO:tasks.workunit.client.0.smithi094.stdout:4/526: dread d0/d15/d3a/d3e/d49/f74 [0,4194304] 0 2024-04-06T03:40:36.953 INFO:tasks.workunit.client.1.smithi136.stdout:1/440: rename d12/d53/c81 to d12/d3b/d44/d5f/d5a/d72/d77/c8f 0 2024-04-06T03:40:36.954 INFO:tasks.workunit.client.0.smithi094.stdout:3/496: rename d9/d20/d30/c46 to d9/d10/d27/d3a/cad 0 2024-04-06T03:40:36.954 INFO:tasks.workunit.client.0.smithi094.stdout:3/497: chown d9/d10/d27/d7c/f92 48 1 2024-04-06T03:40:36.957 INFO:tasks.workunit.client.1.smithi136.stdout:5/503: dread d1/dc/d10/f6d [0,4194304] 0 2024-04-06T03:40:36.963 INFO:tasks.workunit.client.1.smithi136.stdout:5/504: dread d1/d16/d4b/f76 [0,4194304] 0 2024-04-06T03:40:36.964 INFO:tasks.workunit.client.1.smithi136.stdout:2/544: rmdir d3/d4a/d9d/d18/d4b 39 2024-04-06T03:40:36.965 INFO:tasks.workunit.client.1.smithi136.stdout:4/410: dread d1d/d31/f58 [0,4194304] 0 2024-04-06T03:40:36.967 INFO:tasks.workunit.client.1.smithi136.stdout:0/438: unlink d1/d6/d2c/d1a/d82/f8d 0 2024-04-06T03:40:36.967 INFO:tasks.workunit.client.1.smithi136.stdout:0/439: chown d1/f4c 333697264 1 2024-04-06T03:40:36.969 INFO:tasks.workunit.client.1.smithi136.stdout:7/463: mkdir d5/df/d53/d68/da3 0 2024-04-06T03:40:36.971 INFO:tasks.workunit.client.1.smithi136.stdout:8/494: rmdir d3 39 2024-04-06T03:40:36.971 INFO:tasks.workunit.client.1.smithi136.stdout:8/495: stat d3/da/d26/d35/l46 0 2024-04-06T03:40:36.974 INFO:tasks.workunit.client.1.smithi136.stdout:3/409: mknod d0/d4/d12/d19/d55/d3c/c86 0 2024-04-06T03:40:36.975 INFO:tasks.workunit.client.1.smithi136.stdout:3/410: creat d0/d4/d12/d19/d2e/f87 x:0 0 0 2024-04-06T03:40:36.976 INFO:tasks.workunit.client.1.smithi136.stdout:6/374: rename d2/d7/l23 to d2/d7/d26/d30/d6e/d58/d5d/d62/d53/l7b 0 2024-04-06T03:40:36.976 INFO:tasks.workunit.client.1.smithi136.stdout:6/375: fsync d2/d7/f1b 0 2024-04-06T03:40:36.978 INFO:tasks.workunit.client.1.smithi136.stdout:1/441: getdents d12/d3b/d44/d5f/d5a/d72/d77 0 2024-04-06T03:40:36.978 INFO:tasks.workunit.client.0.smithi094.stdout:7/440: rmdir d2 39 2024-04-06T03:40:36.980 INFO:tasks.workunit.client.1.smithi136.stdout:9/438: getdents d17/d21/d29 0 2024-04-06T03:40:36.981 INFO:tasks.workunit.client.1.smithi136.stdout:8/496: dread d3/da/d26/d35/d53/d86/f87 [0,4194304] 0 2024-04-06T03:40:36.982 INFO:tasks.workunit.client.1.smithi136.stdout:5/505: dwrite d1/d16/f85 [0,4194304] 0 2024-04-06T03:40:36.984 INFO:tasks.workunit.client.1.smithi136.stdout:4/411: unlink d1d/d31/d3e/d56/c92 0 2024-04-06T03:40:36.987 INFO:tasks.workunit.client.1.smithi136.stdout:0/440: dwrite d1/d6/d2c/d1a/d36/f44 [0,4194304] 0 2024-04-06T03:40:36.987 INFO:tasks.workunit.client.1.smithi136.stdout:0/441: dread - d1/d6/d2c/d57/d43/d4a/d5c/f91 zero size 2024-04-06T03:40:36.994 INFO:tasks.workunit.client.1.smithi136.stdout:1/442: write d12/d1b/d1e/d4d/f68 [2536796,71324] 0 2024-04-06T03:40:36.994 INFO:tasks.workunit.client.1.smithi136.stdout:1/443: fsync d12/d3b/d44/d5f/f30 0 2024-04-06T03:40:36.994 INFO:tasks.workunit.client.1.smithi136.stdout:8/497: dread f1 [0,4194304] 0 2024-04-06T03:40:36.994 INFO:tasks.workunit.client.1.smithi136.stdout:8/498: write d3/da/d26/d35/d39/d70/f93 [28323,68176] 0 2024-04-06T03:40:36.994 INFO:tasks.workunit.client.1.smithi136.stdout:8/499: write d3/d2d/f67 [633622,119040] 0 2024-04-06T03:40:36.995 INFO:tasks.workunit.client.1.smithi136.stdout:6/376: dwrite d2/d25/f59 [0,4194304] 0 2024-04-06T03:40:36.995 INFO:tasks.workunit.client.0.smithi094.stdout:6/427: mknod dc/d11/d1b/d72/d88/c9d 0 2024-04-06T03:40:36.995 INFO:tasks.workunit.client.0.smithi094.stdout:6/428: chown dc/d3a/d4a/d54/d71/l83 0 1 2024-04-06T03:40:36.995 INFO:tasks.workunit.client.0.smithi094.stdout:5/439: mkdir db/d19/d21/d9b 0 2024-04-06T03:40:36.996 INFO:tasks.workunit.client.0.smithi094.stdout:7/441: dwrite d2/d6/dc/d12/d29/d3f/f6b [0,4194304] 0 2024-04-06T03:40:36.996 INFO:tasks.workunit.client.0.smithi094.stdout:4/527: mkdir d0/d15/d3a/d3e/d49/db2 0 2024-04-06T03:40:36.998 INFO:tasks.workunit.client.1.smithi136.stdout:7/464: unlink d5/l1e 0 2024-04-06T03:40:37.002 INFO:tasks.workunit.client.1.smithi136.stdout:1/444: read d12/d1b/d28/f5e [1217259,109617] 0 2024-04-06T03:40:37.002 INFO:tasks.workunit.client.1.smithi136.stdout:1/445: truncate d12/d3b/f4b 588994 0 2024-04-06T03:40:37.002 INFO:tasks.workunit.client.1.smithi136.stdout:0/442: dwrite d1/d6/f58 [0,4194304] 0 2024-04-06T03:40:37.002 INFO:tasks.workunit.client.1.smithi136.stdout:0/443: chown d1/d6/fb 57 1 2024-04-06T03:40:37.002 INFO:tasks.workunit.client.0.smithi094.stdout:2/522: symlink d0/dd/d1b/lad 0 2024-04-06T03:40:37.003 INFO:tasks.workunit.client.0.smithi094.stdout:2/523: dread - d0/dd/d19/d31/f86 zero size 2024-04-06T03:40:37.003 INFO:tasks.workunit.client.0.smithi094.stdout:2/524: chown d0/d8/d4f/l5e 105081 1 2024-04-06T03:40:37.005 INFO:tasks.workunit.client.1.smithi136.stdout:9/439: rename d17/d20/d25/d3c/f77 to d17/d20/d28/d81/d88/f90 0 2024-04-06T03:40:37.006 INFO:tasks.workunit.client.0.smithi094.stdout:5/440: unlink db/f29 0 2024-04-06T03:40:37.007 INFO:tasks.workunit.client.0.smithi094.stdout:5/441: truncate db/d40/d77/f82 340294 0 2024-04-06T03:40:37.010 INFO:tasks.workunit.client.1.smithi136.stdout:1/446: write d12/d3b/d44/d5f/f63 [3223305,91456] 0 2024-04-06T03:40:37.011 INFO:tasks.workunit.client.0.smithi094.stdout:7/442: creat d2/d6/dc/d12/f84 x:0 0 0 2024-04-06T03:40:37.012 INFO:tasks.workunit.client.0.smithi094.stdout:7/443: dread d2/d6/dc/dd/f31 [0,4194304] 0 2024-04-06T03:40:37.012 INFO:tasks.workunit.client.0.smithi094.stdout:4/528: mkdir d0/df/db3 0 2024-04-06T03:40:37.012 INFO:tasks.workunit.client.0.smithi094.stdout:4/529: dread - d0/d15/fa2 zero size 2024-04-06T03:40:37.014 INFO:tasks.workunit.client.0.smithi094.stdout:2/525: mkdir d0/dd/d19/d31/d67/dae 0 2024-04-06T03:40:37.014 INFO:tasks.workunit.client.1.smithi136.stdout:8/500: getdents d3/da/d26/d35 0 2024-04-06T03:40:37.017 INFO:tasks.workunit.client.0.smithi094.stdout:7/444: link d2/d6/dc/d12/d29/c7c d2/d6/dc/d12/d29/c85 0 2024-04-06T03:40:37.019 INFO:tasks.workunit.client.0.smithi094.stdout:4/530: symlink d0/d4/d67/lb4 0 2024-04-06T03:40:37.019 INFO:tasks.workunit.client.0.smithi094.stdout:4/531: dread - d0/d15/d3a/d61/d81/fab zero size 2024-04-06T03:40:37.020 INFO:tasks.workunit.client.1.smithi136.stdout:7/465: creat d5/df/d1d/d54/fa4 x:0 0 0 2024-04-06T03:40:37.020 INFO:tasks.workunit.client.1.smithi136.stdout:4/412: write d1d/f2b [3685164,98731] 0 2024-04-06T03:40:37.022 INFO:tasks.workunit.client.0.smithi094.stdout:2/526: rename d0/dd/d3d/d49/c54 to d0/dd/d1b/d72/d8f/caf 0 2024-04-06T03:40:37.024 INFO:tasks.workunit.client.1.smithi136.stdout:3/411: unlink d0/d4/f30 0 2024-04-06T03:40:37.025 INFO:tasks.workunit.client.0.smithi094.stdout:5/442: getdents db/d19 0 2024-04-06T03:40:37.025 INFO:tasks.workunit.client.0.smithi094.stdout:5/443: fdatasync db/d19/d24/d2e/f48 0 2024-04-06T03:40:37.026 INFO:tasks.workunit.client.0.smithi094.stdout:7/445: symlink d2/d6/l86 0 2024-04-06T03:40:37.026 INFO:tasks.workunit.client.0.smithi094.stdout:7/446: chown d2/d6/dc/dd/df/d36/d79/f81 84 1 2024-04-06T03:40:37.027 INFO:tasks.workunit.client.1.smithi136.stdout:8/501: dwrite d3/de/f56 [0,4194304] 0 2024-04-06T03:40:37.029 INFO:tasks.workunit.client.1.smithi136.stdout:5/506: rename d1/dc/d1c/f79 to d1/dc/d10/d1e/f9d 0 2024-04-06T03:40:37.029 INFO:tasks.workunit.client.1.smithi136.stdout:6/377: rename d2/d7/d26/d30/d6e/d58/d5d/d62/d53 to d2/d7/d26/d30/d6e/d58/d5d/d62/d53/d7c 22 2024-04-06T03:40:37.029 INFO:tasks.workunit.client.1.smithi136.stdout:5/507: chown d1/d16/d4b/f57 11729 1 2024-04-06T03:40:37.031 INFO:tasks.workunit.client.1.smithi136.stdout:9/440: creat d17/d20/d28/d49/d75/f91 x:0 0 0 2024-04-06T03:40:37.031 INFO:tasks.workunit.client.1.smithi136.stdout:9/441: readlink d17/d21/d32/d43/d51/l72 0 2024-04-06T03:40:37.031 INFO:tasks.workunit.client.1.smithi136.stdout:9/442: chown d17/d20/d25/d3d 0 1 2024-04-06T03:40:37.031 INFO:tasks.workunit.client.1.smithi136.stdout:9/443: dread - d17/d20/d25/d3b/f4e zero size 2024-04-06T03:40:37.032 INFO:tasks.workunit.client.0.smithi094.stdout:5/444: mknod db/d7c/d98/c9c 0 2024-04-06T03:40:37.035 INFO:tasks.workunit.client.0.smithi094.stdout:2/527: symlink d0/d18/lb0 0 2024-04-06T03:40:37.037 INFO:tasks.workunit.client.0.smithi094.stdout:5/445: creat db/d19/d24/f9d x:0 0 0 2024-04-06T03:40:37.039 INFO:tasks.workunit.client.1.smithi136.stdout:5/508: dread d1/dc/d1c/d32/f6a [0,4194304] 0 2024-04-06T03:40:37.039 INFO:tasks.workunit.client.1.smithi136.stdout:5/509: read - d1/d16/d4b/f8e zero size 2024-04-06T03:40:37.039 INFO:tasks.workunit.client.1.smithi136.stdout:5/510: chown d1/c82 0 1 2024-04-06T03:40:37.040 INFO:tasks.workunit.client.0.smithi094.stdout:2/528: link d0/d8/d27/d2b/l48 d0/dd/d3d/d49/lb1 0 2024-04-06T03:40:37.042 INFO:tasks.workunit.client.0.smithi094.stdout:5/446: creat db/d19/d24/d28/d85/f9e x:0 0 0 2024-04-06T03:40:37.045 INFO:tasks.workunit.client.0.smithi094.stdout:2/529: symlink d0/d8/d1d/d77/d87/lb2 0 2024-04-06T03:40:37.045 INFO:tasks.workunit.client.0.smithi094.stdout:2/530: chown d0/dd/d1b/c58 1 1 2024-04-06T03:40:37.045 INFO:tasks.workunit.client.0.smithi094.stdout:2/531: dread d0/dd/d19/d31/f6e [0,4194304] 0 2024-04-06T03:40:37.045 INFO:tasks.workunit.client.0.smithi094.stdout:2/532: dread - d0/dd/da4/f5f zero size 2024-04-06T03:40:37.045 INFO:tasks.workunit.client.0.smithi094.stdout:2/533: read d0/d8/d4f/f5b [3011527,49816] 0 2024-04-06T03:40:37.048 INFO:tasks.workunit.client.0.smithi094.stdout:2/534: creat d0/d8/fb3 x:0 0 0 2024-04-06T03:40:37.048 INFO:tasks.workunit.client.0.smithi094.stdout:2/535: fsync d0/dd/d3d/f88 0 2024-04-06T03:40:37.050 INFO:tasks.workunit.client.0.smithi094.stdout:7/447: dwrite d2/d6/dc/dd/df/d36/f68 [0,4194304] 0 2024-04-06T03:40:37.051 INFO:tasks.workunit.client.0.smithi094.stdout:2/536: creat d0/d8/d27/fb4 x:0 0 0 2024-04-06T03:40:37.052 INFO:tasks.workunit.client.1.smithi136.stdout:8/502: mknod d3/d2d/d5f/c9b 0 2024-04-06T03:40:37.052 INFO:tasks.workunit.client.1.smithi136.stdout:8/503: fsync d3/de/f29 0 2024-04-06T03:40:37.052 INFO:tasks.workunit.client.0.smithi094.stdout:2/537: dread d0/dd/d19/d31/d67/f6c [0,4194304] 0 2024-04-06T03:40:37.056 INFO:tasks.workunit.client.0.smithi094.stdout:2/538: mknod d0/d8/d1d/d77/d87/cb5 0 2024-04-06T03:40:37.056 INFO:tasks.workunit.client.0.smithi094.stdout:5/447: dwrite db/d19/d24/f34 [0,4194304] 0 2024-04-06T03:40:37.059 INFO:tasks.workunit.client.1.smithi136.stdout:6/378: creat d2/d7/d26/d30/d6e/d58/f7d x:0 0 0 2024-04-06T03:40:37.059 INFO:tasks.workunit.client.1.smithi136.stdout:9/444: creat d17/d21/d29/f92 x:0 0 0 2024-04-06T03:40:37.061 INFO:tasks.workunit.client.1.smithi136.stdout:6/379: dread d2/d7/f1b [0,4194304] 0 2024-04-06T03:40:37.061 INFO:tasks.workunit.client.0.smithi094.stdout:2/539: link d0/d8/d1d/d77/c7e d0/dd/d3d/cb6 0 2024-04-06T03:40:37.061 INFO:tasks.workunit.client.0.smithi094.stdout:2/540: write d0/d8/d1d/d77/fa1 [468726,2878] 0 2024-04-06T03:40:37.063 INFO:tasks.workunit.client.1.smithi136.stdout:5/511: creat d1/dc/d1c/d32/d8b/d20/d66/f9e x:0 0 0 2024-04-06T03:40:37.064 INFO:tasks.workunit.client.0.smithi094.stdout:5/448: creat db/d7c/d98/f9f x:0 0 0 2024-04-06T03:40:37.068 INFO:tasks.workunit.client.1.smithi136.stdout:2/545: sync 2024-04-06T03:40:37.068 INFO:tasks.workunit.client.1.smithi136.stdout:2/546: write d3/d4a/f8d [725966,74084] 0 2024-04-06T03:40:37.068 INFO:tasks.workunit.client.0.smithi094.stdout:2/541: creat d0/dd/d19/d31/d67/d98/fb7 x:0 0 0 2024-04-06T03:40:37.068 INFO:tasks.workunit.client.0.smithi094.stdout:7/448: dwrite d2/d6/dc/d12/d29/f76 [0,4194304] 0 2024-04-06T03:40:37.068 INFO:tasks.workunit.client.0.smithi094.stdout:7/449: fsync d2/d6/dc/d12/d3a/d56/f71 0 2024-04-06T03:40:37.068 INFO:tasks.workunit.client.0.smithi094.stdout:7/450: chown d2/d6/dc/c67 1735739 1 2024-04-06T03:40:37.069 INFO:tasks.workunit.client.0.smithi094.stdout:2/542: write d0/dd/d3d/f88 [68705,129844] 0 2024-04-06T03:40:37.070 INFO:tasks.workunit.client.1.smithi136.stdout:8/504: truncate d3/da/d26/d35/f57 3739135 0 2024-04-06T03:40:37.070 INFO:tasks.workunit.client.1.smithi136.stdout:8/505: truncate d3/da/d26/f28 9315581 0 2024-04-06T03:40:37.071 INFO:tasks.workunit.client.1.smithi136.stdout:4/413: dwrite d1d/d70/f6a [0,4194304] 0 2024-04-06T03:40:37.072 INFO:tasks.workunit.client.1.smithi136.stdout:4/414: fdatasync d1d/d31/f40 0 2024-04-06T03:40:37.074 INFO:tasks.workunit.client.1.smithi136.stdout:1/447: rename d12/d3b/d60/d7e/c76 to d12/d1b/c90 0 2024-04-06T03:40:37.075 INFO:tasks.workunit.client.1.smithi136.stdout:9/445: link d17/d21/d32/d3f/l7b d17/d20/d25/d3c/l93 0 2024-04-06T03:40:37.076 INFO:tasks.workunit.client.1.smithi136.stdout:9/446: write d17/d21/d32/f74 [59219,35921] 0 2024-04-06T03:40:37.076 INFO:tasks.workunit.client.1.smithi136.stdout:9/447: stat d17/d21/d29/d34/d46/l7e 0 2024-04-06T03:40:37.076 INFO:tasks.workunit.client.1.smithi136.stdout:9/448: chown d17/d19/f8e 10520656 1 2024-04-06T03:40:37.076 INFO:tasks.workunit.client.1.smithi136.stdout:9/449: truncate d17/d20/d25/d3d/d4b/f80 331190 0 2024-04-06T03:40:37.076 INFO:tasks.workunit.client.1.smithi136.stdout:9/450: chown d17/d20/c24 5770945 1 2024-04-06T03:40:37.077 INFO:tasks.workunit.client.1.smithi136.stdout:6/380: creat d2/d7/d26/d30/f7e x:0 0 0 2024-04-06T03:40:37.077 INFO:tasks.workunit.client.0.smithi094.stdout:7/451: rmdir d2/d6/dc/d12/d61 39 2024-04-06T03:40:37.078 INFO:tasks.workunit.client.0.smithi094.stdout:2/543: dread d0/d8/f11 [0,4194304] 0 2024-04-06T03:40:37.080 INFO:tasks.workunit.client.0.smithi094.stdout:7/452: mknod d2/d6/dc/d12/d29/d35/d57/c87 0 2024-04-06T03:40:37.080 INFO:tasks.workunit.client.0.smithi094.stdout:7/453: write d2/d6/dc/f74 [1341227,11435] 0 2024-04-06T03:40:37.080 INFO:tasks.workunit.client.0.smithi094.stdout:7/454: stat d2/d6/dc/d12/d29 0 2024-04-06T03:40:37.082 INFO:tasks.workunit.client.0.smithi094.stdout:7/455: dread d2/d6/f73 [0,4194304] 0 2024-04-06T03:40:37.083 INFO:tasks.workunit.client.1.smithi136.stdout:1/448: dread d12/d1b/d28/f3f [0,4194304] 0 2024-04-06T03:40:37.083 INFO:tasks.workunit.client.1.smithi136.stdout:1/449: fsync d12/f2c 0 2024-04-06T03:40:37.083 INFO:tasks.workunit.client.1.smithi136.stdout:1/450: creat d12/d1b/d28/f91 x:0 0 0 2024-04-06T03:40:37.084 INFO:tasks.workunit.client.1.smithi136.stdout:4/415: unlink d1d/d70/d8b/f95 0 2024-04-06T03:40:37.084 INFO:tasks.workunit.client.1.smithi136.stdout:4/416: dread - d1d/d2f/f60 zero size 2024-04-06T03:40:37.084 INFO:tasks.workunit.client.1.smithi136.stdout:4/417: dread - d1d/d31/d3e/f5a zero size 2024-04-06T03:40:37.086 INFO:tasks.workunit.client.1.smithi136.stdout:4/418: write f1b [174195,32542] 0 2024-04-06T03:40:37.086 INFO:tasks.workunit.client.1.smithi136.stdout:4/419: read - d1d/d31/d3e/f5d zero size 2024-04-06T03:40:37.087 INFO:tasks.workunit.client.1.smithi136.stdout:7/466: rename d5/df/d12/d1b/d3a/d8d/c99 to d5/df/d12/d1b/d58/ca5 0 2024-04-06T03:40:37.088 INFO:tasks.workunit.client.1.smithi136.stdout:0/444: sync 2024-04-06T03:40:37.088 INFO:tasks.workunit.client.1.smithi136.stdout:0/445: dread - d1/d6/d2c/d57/d43/d6d/f99 zero size 2024-04-06T03:40:37.088 INFO:tasks.workunit.client.1.smithi136.stdout:0/446: write d1/d6/d2c/d57/d43/f50 [547893,107431] 0 2024-04-06T03:40:37.088 INFO:tasks.workunit.client.1.smithi136.stdout:0/447: creat d1/d6/d2c/d57/d43/d6d/d85/d78/f9a x:0 0 0 2024-04-06T03:40:37.088 INFO:tasks.workunit.client.0.smithi094.stdout:5/449: dwrite db/d40/f55 [0,4194304] 0 2024-04-06T03:40:37.088 INFO:tasks.workunit.client.0.smithi094.stdout:5/450: creat db/d7c/d98/fa0 x:0 0 0 2024-04-06T03:40:37.089 INFO:tasks.workunit.client.1.smithi136.stdout:5/512: dwrite d1/dc/d1c/f78 [0,4194304] 0 2024-04-06T03:40:37.090 INFO:tasks.workunit.client.1.smithi136.stdout:6/381: mknod d2/d7/d26/d30/d4f/d75/c7f 0 2024-04-06T03:40:37.091 INFO:tasks.workunit.client.0.smithi094.stdout:2/544: dwrite d0/dd/d3d/d49/f53 [0,4194304] 0 2024-04-06T03:40:37.091 INFO:tasks.workunit.client.0.smithi094.stdout:2/545: dread - d0/d8/d1d/d77/fa6 zero size 2024-04-06T03:40:37.097 INFO:tasks.workunit.client.1.smithi136.stdout:1/451: read d12/d3b/d60/d7e/f42 [146679,107244] 0 2024-04-06T03:40:37.099 INFO:tasks.workunit.client.0.smithi094.stdout:7/456: dread d2/d6/dc/f4a [0,4194304] 0 2024-04-06T03:40:37.100 INFO:tasks.workunit.client.1.smithi136.stdout:4/420: creat d1d/d31/d3e/d55/f99 x:0 0 0 2024-04-06T03:40:37.103 INFO:tasks.workunit.client.0.smithi094.stdout:0/429: sync 2024-04-06T03:40:37.103 INFO:tasks.workunit.client.0.smithi094.stdout:1/447: sync 2024-04-06T03:40:37.103 INFO:tasks.workunit.client.0.smithi094.stdout:1/448: chown d0/d7/d1a/d74 1444780 1 2024-04-06T03:40:37.103 INFO:tasks.workunit.client.0.smithi094.stdout:1/449: dread - d0/d4/d1b/d2f/d4b/f8a zero size 2024-04-06T03:40:37.104 INFO:tasks.workunit.client.0.smithi094.stdout:9/392: sync 2024-04-06T03:40:37.104 INFO:tasks.workunit.client.0.smithi094.stdout:9/393: read d2/dc/d21/d23/d2f/f57 [311648,128432] 0 2024-04-06T03:40:37.106 INFO:tasks.workunit.client.1.smithi136.stdout:7/467: creat d5/df/d12/d1b/d3a/fa6 x:0 0 0 2024-04-06T03:40:37.106 INFO:tasks.workunit.client.1.smithi136.stdout:1/452: dread d12/f24 [0,4194304] 0 2024-04-06T03:40:37.107 INFO:tasks.workunit.client.0.smithi094.stdout:2/546: creat d0/dd/d19/fb8 x:0 0 0 2024-04-06T03:40:37.111 INFO:tasks.workunit.client.0.smithi094.stdout:7/457: unlink d2/d6/dc/dd/df/d36/f7b 0 2024-04-06T03:40:37.112 INFO:tasks.workunit.client.1.smithi136.stdout:6/382: dwrite d2/f9 [0,4194304] 0 2024-04-06T03:40:37.112 INFO:tasks.workunit.client.1.smithi136.stdout:6/383: dread - d2/d7/d26/d30/d6e/f69 zero size 2024-04-06T03:40:37.113 INFO:tasks.workunit.client.0.smithi094.stdout:5/451: dwrite db/d19/d2c/d6d/f78 [0,4194304] 0 2024-04-06T03:40:37.113 INFO:tasks.workunit.client.0.smithi094.stdout:5/452: write db/d7c/d98/fa0 [687191,62708] 0 2024-04-06T03:40:37.113 INFO:tasks.workunit.client.0.smithi094.stdout:5/453: dread - db/d19/f45 zero size 2024-04-06T03:40:37.113 INFO:tasks.workunit.client.0.smithi094.stdout:0/430: mkdir d1/d9/dd/d18/d5d/d8d 0 2024-04-06T03:40:37.114 INFO:tasks.workunit.client.1.smithi136.stdout:5/513: dwrite d1/dc/d10/f94 [0,4194304] 0 2024-04-06T03:40:37.114 INFO:tasks.workunit.client.1.smithi136.stdout:9/451: rmdir d17/d20/d25/d3c 39 2024-04-06T03:40:37.115 INFO:tasks.workunit.client.0.smithi094.stdout:1/450: mknod d0/d4/d1b/d2f/d3c/d48/c9d 0 2024-04-06T03:40:37.115 INFO:tasks.workunit.client.0.smithi094.stdout:1/451: chown d0/d4/d1b/d2f/d3c/d48/c9d 1808 1 2024-04-06T03:40:37.118 INFO:tasks.workunit.client.0.smithi094.stdout:9/394: unlink d2/dc/f18 0 2024-04-06T03:40:37.120 INFO:tasks.workunit.client.1.smithi136.stdout:4/421: creat d1d/d31/d8e/f9a x:0 0 0 2024-04-06T03:40:37.121 INFO:tasks.workunit.client.0.smithi094.stdout:7/458: mknod d2/d6/dc/d12/d29/d35/d57/c88 0 2024-04-06T03:40:37.121 INFO:tasks.workunit.client.1.smithi136.stdout:6/384: dread d2/d25/f38 [4194304,4194304] 0 2024-04-06T03:40:37.123 INFO:tasks.workunit.client.1.smithi136.stdout:7/468: link d5/df/d12/d59/l5a d5/df/d12/d1b/d3a/d4a/d70/la7 0 2024-04-06T03:40:37.123 INFO:tasks.workunit.client.1.smithi136.stdout:9/452: dread fe [0,4194304] 0 2024-04-06T03:40:37.124 INFO:tasks.workunit.client.0.smithi094.stdout:7/459: dread d2/f11 [4194304,4194304] 0 2024-04-06T03:40:37.124 INFO:tasks.workunit.client.1.smithi136.stdout:1/453: rmdir d12/d3b/d44/d5f/d5a/d72 39 2024-04-06T03:40:37.124 INFO:tasks.workunit.client.1.smithi136.stdout:5/514: dwrite d1/dc/d1c/d32/f8f [0,4194304] 0 2024-04-06T03:40:37.126 INFO:tasks.workunit.client.1.smithi136.stdout:4/422: link d1d/d31/d3e/d56/f5e d1d/d31/d3e/d56/f9b 0 2024-04-06T03:40:37.127 INFO:tasks.workunit.client.0.smithi094.stdout:5/454: rmdir db/d19/d24/d28 39 2024-04-06T03:40:37.127 INFO:tasks.workunit.client.0.smithi094.stdout:5/455: write db/d19/d30/f33 [3356194,130296] 0 2024-04-06T03:40:37.129 INFO:tasks.workunit.client.1.smithi136.stdout:6/385: creat d2/d7/d26/d68/f80 x:0 0 0 2024-04-06T03:40:37.129 INFO:tasks.workunit.client.1.smithi136.stdout:6/386: dread - d2/d7/d26/d30/d6e/f69 zero size 2024-04-06T03:40:37.132 INFO:tasks.workunit.client.1.smithi136.stdout:3/412: rename d0/d57 to d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88 0 2024-04-06T03:40:37.132 INFO:tasks.workunit.client.1.smithi136.stdout:3/413: dread - d0/d4/d12/d19/d2e/f87 zero size 2024-04-06T03:40:37.132 INFO:tasks.workunit.client.1.smithi136.stdout:3/414: truncate d0/d4/d12/d19/d55/d3c/f58 171563 0 2024-04-06T03:40:37.132 INFO:tasks.workunit.client.1.smithi136.stdout:3/415: chown d0/d4/d12/d19/d2e 28350 1 2024-04-06T03:40:37.132 INFO:tasks.workunit.client.1.smithi136.stdout:3/416: creat d0/d4/d12/d19/d55/d3c/d5a/f89 x:0 0 0 2024-04-06T03:40:37.132 INFO:tasks.workunit.client.1.smithi136.stdout:3/417: chown d0/d4/d12/d19/d2e/l42 4859 1 2024-04-06T03:40:37.132 INFO:tasks.workunit.client.1.smithi136.stdout:3/418: chown d0/d4/d12/d19/d55/d6f/d82/d5d/d6a 204 1 2024-04-06T03:40:37.133 INFO:tasks.workunit.client.1.smithi136.stdout:7/469: mknod d5/df/d12/d1b/d3a/d4a/d9a/ca8 0 2024-04-06T03:40:37.134 INFO:tasks.workunit.client.0.smithi094.stdout:0/431: creat d1/d9/dd/d18/d1d/f8e x:0 0 0 2024-04-06T03:40:37.135 INFO:tasks.workunit.client.1.smithi136.stdout:5/515: write d1/dc/d1c/f78 [2860068,42160] 0 2024-04-06T03:40:37.135 INFO:tasks.workunit.client.1.smithi136.stdout:5/516: dread - d1/dc/d10/d1e/f97 zero size 2024-04-06T03:40:37.137 INFO:tasks.workunit.client.1.smithi136.stdout:9/453: mknod d17/d21/d32/d43/d51/d57/c94 0 2024-04-06T03:40:37.137 INFO:tasks.workunit.client.0.smithi094.stdout:1/452: unlink d0/d4/cf 0 2024-04-06T03:40:37.137 INFO:tasks.workunit.client.0.smithi094.stdout:1/453: creat d0/d4/d1b/d2f/d4b/f9e x:0 0 0 2024-04-06T03:40:37.140 INFO:tasks.workunit.client.0.smithi094.stdout:9/395: symlink d2/d2c/d33/l93 0 2024-04-06T03:40:37.144 INFO:tasks.workunit.client.1.smithi136.stdout:1/454: creat d12/d1b/d1e/d23/f92 x:0 0 0 2024-04-06T03:40:37.145 INFO:tasks.workunit.client.0.smithi094.stdout:0/432: creat d1/d10/d4e/f8f x:0 0 0 2024-04-06T03:40:37.151 INFO:tasks.workunit.client.1.smithi136.stdout:6/387: creat d2/d7/d26/d30/d6e/d3d/f81 x:0 0 0 2024-04-06T03:40:37.153 INFO:tasks.workunit.client.0.smithi094.stdout:5/456: dwrite db/f69 [0,4194304] 0 2024-04-06T03:40:37.153 INFO:tasks.workunit.client.0.smithi094.stdout:5/457: dread - db/d19/d24/f89 zero size 2024-04-06T03:40:37.153 INFO:tasks.workunit.client.0.smithi094.stdout:1/454: creat d0/d1/d16/f9f x:0 0 0 2024-04-06T03:40:37.153 INFO:tasks.workunit.client.0.smithi094.stdout:1/455: fdatasync d0/d1/d16/f3a 0 2024-04-06T03:40:37.154 INFO:tasks.workunit.client.1.smithi136.stdout:5/517: dwrite d1/dc/d1c/d32/d8b/d20/f41 [0,4194304] 0 2024-04-06T03:40:37.154 INFO:tasks.workunit.client.1.smithi136.stdout:5/518: write d1/d16/d4b/f95 [147997,52848] 0 2024-04-06T03:40:37.154 INFO:tasks.workunit.client.0.smithi094.stdout:5/458: dread db/d19/d24/d28/f3f [0,4194304] 0 2024-04-06T03:40:37.155 INFO:tasks.workunit.client.0.smithi094.stdout:5/459: write db/d19/d24/d28/f8c [461469,61707] 0 2024-04-06T03:40:37.161 INFO:tasks.workunit.client.0.smithi094.stdout:9/396: mknod d2/dc/d21/d23/d47/d51/c94 0 2024-04-06T03:40:37.161 INFO:tasks.workunit.client.0.smithi094.stdout:9/397: chown d2/d2c/d33/d37/f55 7447936 1 2024-04-06T03:40:37.161 INFO:tasks.workunit.client.0.smithi094.stdout:9/398: write d2/dc/d21/d26/f72 [508495,73748] 0 2024-04-06T03:40:37.163 INFO:tasks.workunit.client.1.smithi136.stdout:6/388: dwrite d2/d7/d26/d68/f80 [0,4194304] 0 2024-04-06T03:40:37.163 INFO:tasks.workunit.client.1.smithi136.stdout:6/389: truncate d2/d7/d26/d30/d6e/f5e 2079166 0 2024-04-06T03:40:37.164 INFO:tasks.workunit.client.0.smithi094.stdout:0/433: creat d1/d9/f90 x:0 0 0 2024-04-06T03:40:37.173 INFO:tasks.workunit.client.0.smithi094.stdout:1/456: mkdir d0/d4/d1b/d2f/d3c/da0 0 2024-04-06T03:40:37.175 INFO:tasks.workunit.client.0.smithi094.stdout:5/460: unlink db/d19/f45 0 2024-04-06T03:40:37.175 INFO:tasks.workunit.client.0.smithi094.stdout:9/399: dwrite d2/d2c/d33/f4a [0,4194304] 0 2024-04-06T03:40:37.179 INFO:tasks.workunit.client.0.smithi094.stdout:1/457: rename d0/f5d to d0/d4/d1b/d2f/d4b/fa1 0 2024-04-06T03:40:37.182 INFO:tasks.workunit.client.0.smithi094.stdout:9/400: write d2/dc/d21/d23/d2f/f5f [2775594,6431] 0 2024-04-06T03:40:37.186 INFO:tasks.workunit.client.1.smithi136.stdout:3/419: mknod d0/d4/d12/d2d/c8a 0 2024-04-06T03:40:37.187 INFO:tasks.workunit.client.1.smithi136.stdout:7/470: truncate d5/df/d12/d1b/f2d 3743107 0 2024-04-06T03:40:37.187 INFO:tasks.workunit.client.1.smithi136.stdout:7/471: stat d5/df/d1d/d6f/d2a/c75 0 2024-04-06T03:40:37.190 INFO:tasks.workunit.client.0.smithi094.stdout:5/461: rename db/d19/d30/f33 to db/d19/d30/d83/fa1 0 2024-04-06T03:40:37.192 INFO:tasks.workunit.client.1.smithi136.stdout:1/455: unlink d12/d53/c56 0 2024-04-06T03:40:37.192 INFO:tasks.workunit.client.0.smithi094.stdout:9/401: getdents d2/dc/d21/d26/d78/d69 0 2024-04-06T03:40:37.193 INFO:tasks.workunit.client.0.smithi094.stdout:9/402: creat d2/dc/d46/f95 x:0 0 0 2024-04-06T03:40:37.193 INFO:tasks.workunit.client.0.smithi094.stdout:9/403: dread - d2/dc/d21/d23/d47/f77 zero size 2024-04-06T03:40:37.193 INFO:tasks.workunit.client.0.smithi094.stdout:9/404: fsync d2/d2c/d33/d37/f55 0 2024-04-06T03:40:37.195 INFO:tasks.workunit.client.0.smithi094.stdout:5/462: mknod db/d19/d24/d28/d85/ca2 0 2024-04-06T03:40:37.195 INFO:tasks.workunit.client.0.smithi094.stdout:5/463: chown db/d19/d24/d28/d85/f9e 0 1 2024-04-06T03:40:37.195 INFO:tasks.workunit.client.0.smithi094.stdout:5/464: fdatasync db/d40/d77/f66 0 2024-04-06T03:40:37.197 INFO:tasks.workunit.client.1.smithi136.stdout:5/519: creat d1/dc/d1c/d32/d8b/f9f x:0 0 0 2024-04-06T03:40:37.198 INFO:tasks.workunit.client.1.smithi136.stdout:5/520: chown d1/dc/d1c/l87 2006 1 2024-04-06T03:40:37.198 INFO:tasks.workunit.client.0.smithi094.stdout:9/405: mkdir d2/d2c/d33/d37/d96 0 2024-04-06T03:40:37.199 INFO:tasks.workunit.client.0.smithi094.stdout:9/406: write d2/d2c/d33/d37/f55 [1456378,120184] 0 2024-04-06T03:40:37.202 INFO:tasks.workunit.client.0.smithi094.stdout:0/434: dwrite d1/d9/dd/d18/d1a/f4d [4194304,4194304] 0 2024-04-06T03:40:37.202 INFO:tasks.workunit.client.0.smithi094.stdout:0/435: creat d1/d9/f91 x:0 0 0 2024-04-06T03:40:37.203 INFO:tasks.workunit.client.1.smithi136.stdout:6/390: unlink d2/d7/f1b 0 2024-04-06T03:40:37.207 INFO:tasks.workunit.client.1.smithi136.stdout:7/472: mknod d5/df/d12/d1b/d3a/d8d/ca9 0 2024-04-06T03:40:37.209 INFO:tasks.workunit.client.0.smithi094.stdout:9/407: write d2/d2c/d33/f4c [5145673,16176] 0 2024-04-06T03:40:37.212 INFO:tasks.workunit.client.1.smithi136.stdout:3/420: dwrite d0/d4/d12/f15 [0,4194304] 0 2024-04-06T03:40:37.212 INFO:tasks.workunit.client.0.smithi094.stdout:1/458: dwrite d0/d4/d1b/d2f/f54 [4194304,4194304] 0 2024-04-06T03:40:37.214 INFO:tasks.workunit.client.1.smithi136.stdout:3/421: dread d0/d4/f45 [0,4194304] 0 2024-04-06T03:40:37.218 INFO:tasks.workunit.client.0.smithi094.stdout:1/459: write d0/d4/d1b/d2f/d4b/f91 [1841445,126561] 0 2024-04-06T03:40:37.274 INFO:tasks.workunit.client.1.smithi136.stdout:5/521: mknod d1/dc/d1c/d32/d8b/d20/d7c/ca0 0 2024-04-06T03:40:37.274 INFO:tasks.workunit.client.1.smithi136.stdout:5/522: dread - d1/dc/f91 zero size 2024-04-06T03:40:37.277 INFO:tasks.workunit.client.1.smithi136.stdout:6/391: unlink d2/f5 0 2024-04-06T03:40:37.277 INFO:tasks.workunit.client.1.smithi136.stdout:6/392: fsync d2/d7/d26/d30/d6e/f6b 0 2024-04-06T03:40:37.280 INFO:tasks.workunit.client.1.smithi136.stdout:0/448: rename d1/d6/d2c/d57/d43/d4a to d1/d6/d2c/d57/d43/d6d/d9b 0 2024-04-06T03:40:37.283 INFO:tasks.workunit.client.0.smithi094.stdout:5/465: rmdir db/d19 39 2024-04-06T03:40:37.283 INFO:tasks.workunit.client.0.smithi094.stdout:5/466: write db/d19/d2c/f2d [695845,104855] 0 2024-04-06T03:40:37.285 INFO:tasks.workunit.client.1.smithi136.stdout:1/456: dwrite d12/d3b/d60/d7e/d37/f78 [0,4194304] 0 2024-04-06T03:40:37.288 INFO:tasks.workunit.client.1.smithi136.stdout:4/423: rmdir d1d/d70/d8b 0 2024-04-06T03:40:37.288 INFO:tasks.workunit.client.1.smithi136.stdout:6/393: write d2/d7/d26/d30/f50 [1578689,90396] 0 2024-04-06T03:40:37.288 INFO:tasks.workunit.client.1.smithi136.stdout:4/424: stat d1d/d31/d3e/d55/d89 0 2024-04-06T03:40:37.289 INFO:tasks.workunit.client.1.smithi136.stdout:1/457: dread d12/f89 [0,4194304] 0 2024-04-06T03:40:37.292 INFO:tasks.workunit.client.1.smithi136.stdout:9/454: rename d17/d19/f69 to d17/d20/f95 0 2024-04-06T03:40:37.293 INFO:tasks.workunit.client.1.smithi136.stdout:5/523: dwrite d1/dc/d1c/d32/f6a [0,4194304] 0 2024-04-06T03:40:37.294 INFO:tasks.workunit.client.0.smithi094.stdout:0/436: creat d1/d9/dd/d18/d5d/d60/f92 x:0 0 0 2024-04-06T03:40:37.296 INFO:tasks.workunit.client.0.smithi094.stdout:5/467: dread db/d19/d2c/d57/f58 [0,4194304] 0 2024-04-06T03:40:37.296 INFO:tasks.workunit.client.0.smithi094.stdout:5/468: creat db/d7c/d98/fa3 x:0 0 0 2024-04-06T03:40:37.297 INFO:tasks.workunit.client.1.smithi136.stdout:0/449: mkdir d1/d6/d2c/d1a/d9c 0 2024-04-06T03:40:37.302 INFO:tasks.workunit.client.1.smithi136.stdout:6/394: read d2/d7/d26/d30/d6e/f27 [1117491,4236] 0 2024-04-06T03:40:37.302 INFO:tasks.workunit.client.1.smithi136.stdout:6/395: write d2/d7/f2c [4307979,3642] 0 2024-04-06T03:40:37.302 INFO:tasks.workunit.client.0.smithi094.stdout:9/408: unlink d2/d2c/d33/f4c 0 2024-04-06T03:40:37.305 INFO:tasks.workunit.client.0.smithi094.stdout:5/469: write db/d19/d24/d2e/f36 [2226802,5283] 0 2024-04-06T03:40:37.309 INFO:tasks.workunit.client.0.smithi094.stdout:1/460: rename d0/d4/f8b to d0/d4/d1b/d2f/d4b/d95/fa2 0 2024-04-06T03:40:37.310 INFO:tasks.workunit.client.1.smithi136.stdout:1/458: mknod d12/d3b/d44/d5f/d5a/d72/d77/c93 0 2024-04-06T03:40:37.312 INFO:tasks.workunit.client.1.smithi136.stdout:8/506: sync 2024-04-06T03:40:37.312 INFO:tasks.workunit.client.1.smithi136.stdout:2/547: sync 2024-04-06T03:40:37.312 INFO:tasks.workunit.client.1.smithi136.stdout:8/507: stat d3/da/d26/d35/l3f 0 2024-04-06T03:40:37.314 INFO:tasks.workunit.client.1.smithi136.stdout:5/524: mknod d1/d16/ca1 0 2024-04-06T03:40:37.314 INFO:tasks.workunit.client.0.smithi094.stdout:8/484: sync 2024-04-06T03:40:37.317 INFO:tasks.workunit.client.1.smithi136.stdout:0/450: symlink d1/d6/d2c/d57/d43/d6d/d85/d78/l9d 0 2024-04-06T03:40:37.317 INFO:tasks.workunit.client.1.smithi136.stdout:0/451: readlink d1/d6/d2c/d57/l34 0 2024-04-06T03:40:37.319 INFO:tasks.workunit.client.1.smithi136.stdout:9/455: dwrite d17/d20/d25/f26 [0,4194304] 0 2024-04-06T03:40:37.320 INFO:tasks.workunit.client.1.smithi136.stdout:6/396: creat d2/d7/d26/d30/d4f/d75/f82 x:0 0 0 2024-04-06T03:40:37.323 INFO:tasks.workunit.client.1.smithi136.stdout:1/459: unlink d12/d1b/c5d 0 2024-04-06T03:40:37.325 INFO:tasks.workunit.client.0.smithi094.stdout:1/461: symlink d0/d4/d37/d5f/la3 0 2024-04-06T03:40:37.325 INFO:tasks.workunit.client.1.smithi136.stdout:5/525: dread d1/dc/d1c/d32/d8b/d20/f41 [0,4194304] 0 2024-04-06T03:40:37.326 INFO:tasks.workunit.client.1.smithi136.stdout:8/508: symlink d3/da/d26/d35/d53/d86/l9c 0 2024-04-06T03:40:37.327 INFO:tasks.workunit.client.1.smithi136.stdout:0/452: dread d1/d6/d2c/d1a/f4b [0,4194304] 0 2024-04-06T03:40:37.329 INFO:tasks.workunit.client.0.smithi094.stdout:8/485: mkdir d18/d3c/d49/d91/d94/daa 0 2024-04-06T03:40:37.331 INFO:tasks.workunit.client.1.smithi136.stdout:1/460: mkdir d12/d3b/d74/d94 0 2024-04-06T03:40:37.331 INFO:tasks.workunit.client.1.smithi136.stdout:1/461: write d12/f89 [1810694,95114] 0 2024-04-06T03:40:37.331 INFO:tasks.workunit.client.1.smithi136.stdout:5/526: creat d1/d75/fa2 x:0 0 0 2024-04-06T03:40:37.333 INFO:tasks.workunit.client.0.smithi094.stdout:8/486: unlink d18/d5c/d75/l7e 0 2024-04-06T03:40:37.337 INFO:tasks.workunit.client.1.smithi136.stdout:8/509: rename d3/d2d/c8c to d3/de/d18/c9d 0 2024-04-06T03:40:37.338 INFO:tasks.workunit.client.1.smithi136.stdout:8/510: readlink d3/d2d/l6e 0 2024-04-06T03:40:37.338 INFO:tasks.workunit.client.1.smithi136.stdout:8/511: dread - d3/f77 zero size 2024-04-06T03:40:37.338 INFO:tasks.workunit.client.1.smithi136.stdout:0/453: dread d1/d6/d2c/d57/f3f [0,4194304] 0 2024-04-06T03:40:37.338 INFO:tasks.workunit.client.0.smithi094.stdout:8/487: rename d18/d5c/d75/c7a to d18/d2c/cab 0 2024-04-06T03:40:37.339 INFO:tasks.workunit.client.1.smithi136.stdout:6/397: getdents d2/d7/d26/d30 0 2024-04-06T03:40:37.339 INFO:tasks.workunit.client.1.smithi136.stdout:6/398: fdatasync d2/d7/d26/d30/d4f/f57 0 2024-04-06T03:40:37.342 INFO:tasks.workunit.client.1.smithi136.stdout:0/454: mkdir d1/d6/d2c/d57/d43/d9e 0 2024-04-06T03:40:37.342 INFO:tasks.workunit.client.1.smithi136.stdout:0/455: chown d1/d6/d2c/d1a/d36/d59/f21 23107 1 2024-04-06T03:40:37.348 INFO:tasks.workunit.client.1.smithi136.stdout:5/527: dwrite d1/d16/d4b/d7b/f83 [0,4194304] 0 2024-04-06T03:40:37.350 INFO:tasks.workunit.client.1.smithi136.stdout:5/528: mkdir d1/dc/d10/da3 0 2024-04-06T03:40:37.351 INFO:tasks.workunit.client.1.smithi136.stdout:5/529: truncate d1/dc/d1c/f78 3896994 0 2024-04-06T03:40:37.351 INFO:tasks.workunit.client.1.smithi136.stdout:5/530: truncate d1/dc/d10/f11 2644572 0 2024-04-06T03:40:37.352 INFO:tasks.workunit.client.1.smithi136.stdout:5/531: mknod d1/dc/d1c/d32/d8b/d20/d66/ca4 0 2024-04-06T03:40:37.352 INFO:tasks.workunit.client.1.smithi136.stdout:5/532: chown d1/d16/d4b/f4f 3367 1 2024-04-06T03:40:37.352 INFO:tasks.workunit.client.1.smithi136.stdout:8/512: write d3/de/d18/f1e [2066960,51226] 0 2024-04-06T03:40:37.352 INFO:tasks.workunit.client.1.smithi136.stdout:8/513: write d3/da/d26/d35/d53/f7c [827198,39497] 0 2024-04-06T03:40:37.365 INFO:tasks.workunit.client.0.smithi094.stdout:8/488: dwrite d18/f1c [0,4194304] 0 2024-04-06T03:40:37.365 INFO:tasks.workunit.client.0.smithi094.stdout:8/489: dread - d18/d2c/d37/fa2 zero size 2024-04-06T03:40:37.366 INFO:tasks.workunit.client.0.smithi094.stdout:8/490: creat d18/d2c/fac x:0 0 0 2024-04-06T03:40:37.366 INFO:tasks.workunit.client.1.smithi136.stdout:8/514: dwrite d3/da/d26/d35/d53/d86/f8f [0,4194304] 0 2024-04-06T03:40:37.367 INFO:tasks.workunit.client.0.smithi094.stdout:8/491: unlink d18/d3c/d8b/f8a 0 2024-04-06T03:40:37.370 INFO:tasks.workunit.client.1.smithi136.stdout:8/515: link d3/de/f56 d3/da/d26/d35/d39/d70/f9e 0 2024-04-06T03:40:37.372 INFO:tasks.workunit.client.1.smithi136.stdout:8/516: symlink d3/da/d26/d35/d53/l9f 0 2024-04-06T03:40:37.374 INFO:tasks.workunit.client.1.smithi136.stdout:8/517: mkdir d3/da/da0 0 2024-04-06T03:40:37.374 INFO:tasks.workunit.client.1.smithi136.stdout:8/518: write d3/d2d/f67 [1331098,83674] 0 2024-04-06T03:40:37.374 INFO:tasks.workunit.client.1.smithi136.stdout:8/519: write d3/da/d26/d35/d39/f99 [127654,113706] 0 2024-04-06T03:40:37.375 INFO:tasks.workunit.client.1.smithi136.stdout:8/520: mknod d3/da/d26/d90/ca1 0 2024-04-06T03:40:37.376 INFO:tasks.workunit.client.1.smithi136.stdout:8/521: rename d3/de/d18/f76 to d3/da/d26/d88/fa2 0 2024-04-06T03:40:37.376 INFO:tasks.workunit.client.1.smithi136.stdout:8/522: chown d3/da/l3e 23006 1 2024-04-06T03:40:37.384 INFO:tasks.workunit.client.1.smithi136.stdout:8/523: dread d3/da/d26/f28 [0,4194304] 0 2024-04-06T03:40:37.385 INFO:tasks.workunit.client.1.smithi136.stdout:8/524: truncate d3/de/f27 1414068 0 2024-04-06T03:40:37.410 INFO:tasks.workunit.client.0.smithi094.stdout:9/409: dwrite d2/dc/d21/d23/d2f/f83 [0,4194304] 0 2024-04-06T03:40:37.412 INFO:tasks.workunit.client.1.smithi136.stdout:8/525: dread d3/da/d26/d35/d39/d70/f93 [0,4194304] 0 2024-04-06T03:40:37.412 INFO:tasks.workunit.client.1.smithi136.stdout:8/526: fdatasync d3/fc 0 2024-04-06T03:40:37.412 INFO:tasks.workunit.client.1.smithi136.stdout:8/527: rename d3/da/d26/d35/d3d/d48/d6c/d79 to d3/da/d26/d35/d3d/d48/d6c/d79/da3 22 2024-04-06T03:40:37.413 INFO:tasks.workunit.client.0.smithi094.stdout:0/437: dwrite d1/d9/dd/d18/d1e/d22/d35/f5c [0,4194304] 0 2024-04-06T03:40:37.413 INFO:tasks.workunit.client.0.smithi094.stdout:0/438: write d1/d9/dd/d18/d1e/d22/d35/f3c [855039,100990] 0 2024-04-06T03:40:37.414 INFO:tasks.workunit.client.1.smithi136.stdout:8/528: symlink d3/de/d60/la4 0 2024-04-06T03:40:37.417 INFO:tasks.workunit.client.0.smithi094.stdout:0/439: truncate d1/d9/dd/d18/d1e/d22/d35/f52 124322 0 2024-04-06T03:40:37.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:37 smithi094 ceph-mon[29713]: pgmap v18: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 143 MiB/s rd, 191 MiB/s wr, 258 op/s 2024-04-06T03:40:37.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:37 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:40:37.420 INFO:tasks.workunit.client.1.smithi136.stdout:2/548: dwrite d3/d15/f30 [4194304,4194304] 0 2024-04-06T03:40:37.420 INFO:tasks.workunit.client.1.smithi136.stdout:2/549: truncate d3/d4a/d9d/d18/d4b/f95 193491 0 2024-04-06T03:40:37.423 INFO:tasks.workunit.client.1.smithi136.stdout:6/399: dwrite d2/d7/d26/d30/f50 [4194304,4194304] 0 2024-04-06T03:40:37.426 INFO:tasks.workunit.client.0.smithi094.stdout:1/462: dwrite d0/d4/f39 [0,4194304] 0 2024-04-06T03:40:37.429 INFO:tasks.workunit.client.1.smithi136.stdout:2/550: dread d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:37.434 INFO:tasks.workunit.client.1.smithi136.stdout:1/462: dread d12/f89 [0,4194304] 0 2024-04-06T03:40:37.434 INFO:tasks.workunit.client.0.smithi094.stdout:3/498: sync 2024-04-06T03:40:37.435 INFO:tasks.workunit.client.0.smithi094.stdout:8/492: dwrite d18/d30/f40 [0,4194304] 0 2024-04-06T03:40:37.438 INFO:tasks.workunit.client.0.smithi094.stdout:1/463: write d0/f19 [2691114,35848] 0 2024-04-06T03:40:37.438 INFO:tasks.workunit.client.0.smithi094.stdout:3/499: mknod d9/d1e/d26/d55/d62/cae 0 2024-04-06T03:40:37.442 INFO:tasks.workunit.client.0.smithi094.stdout:9/410: dwrite d2/dc/d21/d23/f3f [0,4194304] 0 2024-04-06T03:40:37.443 INFO:tasks.workunit.client.1.smithi136.stdout:6/400: rename d2/d25/c45 to d2/d7/d26/d30/d4f/c83 0 2024-04-06T03:40:37.444 INFO:tasks.workunit.client.1.smithi136.stdout:0/456: dwrite d1/d6/d2c/d57/f66 [0,4194304] 0 2024-04-06T03:40:37.444 INFO:tasks.workunit.client.1.smithi136.stdout:8/529: dwrite d3/de/f3a [4194304,4194304] 0 2024-04-06T03:40:37.445 INFO:tasks.workunit.client.0.smithi094.stdout:3/500: mkdir d9/d10/d27/d3a/daf 0 2024-04-06T03:40:37.445 INFO:tasks.workunit.client.0.smithi094.stdout:3/501: creat d9/d1d/d7d/fb0 x:0 0 0 2024-04-06T03:40:37.445 INFO:tasks.workunit.client.0.smithi094.stdout:3/502: chown d9/d10/d27/d3a/c5b 32179 1 2024-04-06T03:40:37.447 INFO:tasks.workunit.client.0.smithi094.stdout:9/411: rename d2/dc/d21/d26/d78/c8b to d2/dc/d21/d26/d78/d69/d6d/d6b/c97 0 2024-04-06T03:40:37.450 INFO:tasks.workunit.client.0.smithi094.stdout:1/464: write d0/d1/d16/d40/f7a [2095758,32723] 0 2024-04-06T03:40:37.451 INFO:tasks.workunit.client.1.smithi136.stdout:1/463: creat d12/d53/f95 x:0 0 0 2024-04-06T03:40:37.451 INFO:tasks.workunit.client.1.smithi136.stdout:6/401: truncate d2/d25/f38 6942075 0 2024-04-06T03:40:37.451 INFO:tasks.workunit.client.0.smithi094.stdout:3/503: mknod d9/d10/d27/cb1 0 2024-04-06T03:40:37.452 INFO:tasks.workunit.client.0.smithi094.stdout:8/493: dwrite d18/f64 [0,4194304] 0 2024-04-06T03:40:37.453 INFO:tasks.workunit.client.0.smithi094.stdout:9/412: mknod d2/dc/d21/d26/d78/c98 0 2024-04-06T03:40:37.455 INFO:tasks.workunit.client.0.smithi094.stdout:3/504: write d9/f3f [1046283,46931] 0 2024-04-06T03:40:37.456 INFO:tasks.workunit.client.0.smithi094.stdout:8/494: dread d18/d2c/f41 [0,4194304] 0 2024-04-06T03:40:37.456 INFO:tasks.workunit.client.0.smithi094.stdout:8/495: write d18/f27 [281923,60837] 0 2024-04-06T03:40:37.456 INFO:tasks.workunit.client.0.smithi094.stdout:8/496: fdatasync f13 0 2024-04-06T03:40:37.457 INFO:tasks.workunit.client.1.smithi136.stdout:7/473: sync 2024-04-06T03:40:37.457 INFO:tasks.workunit.client.1.smithi136.stdout:7/474: read - d5/df/d12/d1b/d3a/d4a/f86 zero size 2024-04-06T03:40:37.457 INFO:tasks.workunit.client.1.smithi136.stdout:7/475: dread - d5/fa1 zero size 2024-04-06T03:40:37.458 INFO:tasks.workunit.client.1.smithi136.stdout:8/530: read d3/da/d26/d35/d3d/d48/f66 [886375,65121] 0 2024-04-06T03:40:37.465 INFO:tasks.workunit.client.1.smithi136.stdout:2/551: rmdir d3/d15/d19/d43/d54 39 2024-04-06T03:40:37.465 INFO:tasks.workunit.client.1.smithi136.stdout:2/552: write d3/d4a/d9d/fb6 [151899,96074] 0 2024-04-06T03:40:37.467 INFO:tasks.workunit.client.1.smithi136.stdout:1/464: mkdir d12/d3b/d44/d5f/d5a/d72/d77/d96 0 2024-04-06T03:40:37.472 INFO:tasks.workunit.client.1.smithi136.stdout:6/402: mkdir d2/d7/d26/d30/d6e/d3d/d47/d84 0 2024-04-06T03:40:37.473 INFO:tasks.workunit.client.0.smithi094.stdout:9/413: mkdir d2/d2c/d99 0 2024-04-06T03:40:37.475 INFO:tasks.workunit.client.1.smithi136.stdout:7/476: unlink d5/df/d1d/d6f/d2a/f49 0 2024-04-06T03:40:37.476 INFO:tasks.workunit.client.1.smithi136.stdout:0/457: dwrite d1/d6/d2c/f5a [0,4194304] 0 2024-04-06T03:40:37.477 INFO:tasks.workunit.client.1.smithi136.stdout:6/403: mkdir d2/d7/d26/d30/d6e/d85 0 2024-04-06T03:40:37.479 INFO:tasks.workunit.client.0.smithi094.stdout:3/505: symlink d9/d1e/d26/d6c/lb2 0 2024-04-06T03:40:37.479 INFO:tasks.workunit.client.0.smithi094.stdout:3/506: write d9/d1e/f8f [261242,98751] 0 2024-04-06T03:40:37.479 INFO:tasks.workunit.client.0.smithi094.stdout:3/507: rename d9/d10/d27 to d9/d10/d27/d3a/db3 22 2024-04-06T03:40:37.479 INFO:tasks.workunit.client.1.smithi136.stdout:2/553: dread d3/d15/f73 [0,4194304] 0 2024-04-06T03:40:37.479 INFO:tasks.workunit.client.1.smithi136.stdout:2/554: fsync d3/f61 0 2024-04-06T03:40:37.480 INFO:tasks.workunit.client.1.smithi136.stdout:2/555: chown d3/d15/d19/c38 3486 1 2024-04-06T03:40:37.480 INFO:tasks.workunit.client.1.smithi136.stdout:3/422: sync 2024-04-06T03:40:37.480 INFO:tasks.workunit.client.1.smithi136.stdout:3/423: chown d0/d4/d12/d3a 62 1 2024-04-06T03:40:37.480 INFO:tasks.workunit.client.1.smithi136.stdout:3/424: readlink d0/l9 0 2024-04-06T03:40:37.480 INFO:tasks.workunit.client.0.smithi094.stdout:1/465: rmdir d0/d1/d16/d40 39 2024-04-06T03:40:37.481 INFO:tasks.workunit.client.1.smithi136.stdout:7/477: symlink d5/d6d/laa 0 2024-04-06T03:40:37.481 INFO:tasks.workunit.client.1.smithi136.stdout:7/478: readlink d5/df/d1d/d6f/d4d/d69/l7d 0 2024-04-06T03:40:37.481 INFO:tasks.workunit.client.1.smithi136.stdout:2/556: write d3/d4a/d9d/d18/d1d/f20 [1513790,48317] 0 2024-04-06T03:40:37.482 INFO:tasks.workunit.client.0.smithi094.stdout:9/414: mknod d2/d2c/c9a 0 2024-04-06T03:40:37.485 INFO:tasks.workunit.client.1.smithi136.stdout:0/458: rename d1/d6/d2c/d57/l34 to d1/d6/d2c/d84/l9f 0 2024-04-06T03:40:37.486 INFO:tasks.workunit.client.1.smithi136.stdout:6/404: mknod d2/d7/d13/c86 0 2024-04-06T03:40:37.487 INFO:tasks.workunit.client.0.smithi094.stdout:8/497: symlink d18/lad 0 2024-04-06T03:40:37.488 INFO:tasks.workunit.client.1.smithi136.stdout:7/479: truncate d5/df/d1d/d6f/d4d/d69/f7b 3304051 0 2024-04-06T03:40:37.489 INFO:tasks.workunit.client.0.smithi094.stdout:9/415: creat d2/dc/d21/d23/d47/f9b x:0 0 0 2024-04-06T03:40:37.489 INFO:tasks.workunit.client.0.smithi094.stdout:9/416: stat d2/dc/d21/d23/d5b/f89 0 2024-04-06T03:40:37.490 INFO:tasks.workunit.client.0.smithi094.stdout:3/508: rmdir d9/d1d/d7d 39 2024-04-06T03:40:37.492 INFO:tasks.workunit.client.1.smithi136.stdout:6/405: symlink d2/d25/l87 0 2024-04-06T03:40:37.493 INFO:tasks.workunit.client.0.smithi094.stdout:9/417: creat d2/dc/d21/f9c x:0 0 0 2024-04-06T03:40:37.494 INFO:tasks.workunit.client.1.smithi136.stdout:3/425: dread d0/d4/d12/d19/d2c/f31 [0,4194304] 0 2024-04-06T03:40:37.494 INFO:tasks.workunit.client.1.smithi136.stdout:8/531: dwrite d3/f5e [0,4194304] 0 2024-04-06T03:40:37.494 INFO:tasks.workunit.client.1.smithi136.stdout:8/532: dread - d3/da/d26/d35/d39/f65 zero size 2024-04-06T03:40:37.495 INFO:tasks.workunit.client.1.smithi136.stdout:0/459: dread d1/d6/d2c/d57/d43/d6d/d9b/f67 [0,4194304] 0 2024-04-06T03:40:37.495 INFO:tasks.workunit.client.1.smithi136.stdout:0/460: creat d1/d6/d2c/d1a/fa0 x:0 0 0 2024-04-06T03:40:37.495 INFO:tasks.workunit.client.1.smithi136.stdout:0/461: chown d1/d6/d2c/d57/d43 0 1 2024-04-06T03:40:37.495 INFO:tasks.workunit.client.0.smithi094.stdout:8/498: write d18/f6b [1690695,93567] 0 2024-04-06T03:40:37.496 INFO:tasks.workunit.client.1.smithi136.stdout:3/426: dread d0/d4/d12/d19/d55/d3c/d46/f72 [0,4194304] 0 2024-04-06T03:40:37.496 INFO:tasks.workunit.client.1.smithi136.stdout:6/406: symlink d2/d7/d26/d30/d6e/d58/d5d/d62/l88 0 2024-04-06T03:40:37.496 INFO:tasks.workunit.client.1.smithi136.stdout:6/407: write d2/d25/f38 [7832477,30612] 0 2024-04-06T03:40:37.496 INFO:tasks.workunit.client.1.smithi136.stdout:3/427: dread d0/d4/d12/d3a/f3e [0,4194304] 0 2024-04-06T03:40:37.497 INFO:tasks.workunit.client.1.smithi136.stdout:3/428: stat d0/l9 0 2024-04-06T03:40:37.498 INFO:tasks.workunit.client.0.smithi094.stdout:9/418: fsync d2/dc/d10/f66 0 2024-04-06T03:40:37.499 INFO:tasks.workunit.client.1.smithi136.stdout:2/557: write d3/d4a/d9d/d18/d4b/fb3 [3402452,33697] 0 2024-04-06T03:40:37.499 INFO:tasks.workunit.client.1.smithi136.stdout:2/558: write d3/d9/fad [165118,128253] 0 2024-04-06T03:40:37.501 INFO:tasks.workunit.client.1.smithi136.stdout:8/533: rename d3/f9 to d3/da/d26/d35/d53/fa5 0 2024-04-06T03:40:37.501 INFO:tasks.workunit.client.1.smithi136.stdout:8/534: fsync d3/de/f4b 0 2024-04-06T03:40:37.501 INFO:tasks.workunit.client.0.smithi094.stdout:1/466: dread d0/d4/f39 [0,4194304] 0 2024-04-06T03:40:37.501 INFO:tasks.workunit.client.0.smithi094.stdout:1/467: fsync d0/d4/d1b/d2f/d4b/f91 0 2024-04-06T03:40:37.502 INFO:tasks.workunit.client.0.smithi094.stdout:9/419: read d2/f7b [763163,33845] 0 2024-04-06T03:40:37.502 INFO:tasks.workunit.client.0.smithi094.stdout:3/509: rmdir d9/d20/d30 39 2024-04-06T03:40:37.506 INFO:tasks.workunit.client.0.smithi094.stdout:8/499: dread d18/f63 [0,4194304] 0 2024-04-06T03:40:37.507 INFO:tasks.workunit.client.0.smithi094.stdout:8/500: truncate d18/d2c/d62/f72 345188 0 2024-04-06T03:40:37.508 INFO:tasks.workunit.client.1.smithi136.stdout:0/462: creat d1/d6/fa1 x:0 0 0 2024-04-06T03:40:37.508 INFO:tasks.workunit.client.1.smithi136.stdout:0/463: creat d1/d6/d2c/d57/d43/fa2 x:0 0 0 2024-04-06T03:40:37.509 INFO:tasks.workunit.client.0.smithi094.stdout:9/420: mknod d2/d2c/d33/d37/d96/c9d 0 2024-04-06T03:40:37.509 INFO:tasks.workunit.client.0.smithi094.stdout:9/421: chown d2/dc/d10 8880 1 2024-04-06T03:40:37.511 INFO:tasks.workunit.client.0.smithi094.stdout:3/510: symlink d9/d10/d27/d7c/da2/lb4 0 2024-04-06T03:40:37.512 INFO:tasks.workunit.client.1.smithi136.stdout:6/408: mkdir d2/d7/d13/d89 0 2024-04-06T03:40:37.514 INFO:tasks.workunit.client.0.smithi094.stdout:8/501: symlink d18/d3c/d76/lae 0 2024-04-06T03:40:37.515 INFO:tasks.workunit.client.1.smithi136.stdout:2/559: truncate d3/d4a/d9d/d18/d1d/d3c/f51 2849426 0 2024-04-06T03:40:37.515 INFO:tasks.workunit.client.1.smithi136.stdout:7/480: dwrite d5/df/d11/f79 [0,4194304] 0 2024-04-06T03:40:37.515 INFO:tasks.workunit.client.1.smithi136.stdout:7/481: readlink d5/df/d12/l9c 0 2024-04-06T03:40:37.516 INFO:tasks.workunit.client.1.smithi136.stdout:7/482: creat d5/df/d3d/fab x:0 0 0 2024-04-06T03:40:37.516 INFO:tasks.workunit.client.1.smithi136.stdout:7/483: read - d5/df/d3d/f8e zero size 2024-04-06T03:40:37.517 INFO:tasks.workunit.client.1.smithi136.stdout:0/464: dread d1/d6/f58 [0,4194304] 0 2024-04-06T03:40:37.518 INFO:tasks.workunit.client.0.smithi094.stdout:3/511: unlink d9/d20/d30/d88/fa7 0 2024-04-06T03:40:37.518 INFO:tasks.workunit.client.0.smithi094.stdout:4/532: sync 2024-04-06T03:40:37.518 INFO:tasks.workunit.client.0.smithi094.stdout:6/429: sync 2024-04-06T03:40:37.518 INFO:tasks.workunit.client.0.smithi094.stdout:6/430: chown dc/d3a/f5e 0 1 2024-04-06T03:40:37.518 INFO:tasks.workunit.client.0.smithi094.stdout:4/533: truncate d0/d15/d3a/fa6 498071 0 2024-04-06T03:40:37.518 INFO:tasks.workunit.client.0.smithi094.stdout:4/534: readlink d0/df/d1e/l7e 0 2024-04-06T03:40:37.518 INFO:tasks.workunit.client.0.smithi094.stdout:4/535: chown d0/d15/d3a/d3e/d50/l5b 6436315 1 2024-04-06T03:40:37.520 INFO:tasks.workunit.client.0.smithi094.stdout:8/502: creat d18/d3c/d49/d80/faf x:0 0 0 2024-04-06T03:40:37.523 INFO:tasks.workunit.client.1.smithi136.stdout:3/429: dwrite d0/d4/d12/f15 [4194304,4194304] 0 2024-04-06T03:40:37.523 INFO:tasks.workunit.client.0.smithi094.stdout:9/422: link d2/dc/d21/d23/d47/f9b d2/d2c/d99/f9e 0 2024-04-06T03:40:37.525 INFO:tasks.workunit.client.0.smithi094.stdout:6/431: creat dc/d11/d22/f9e x:0 0 0 2024-04-06T03:40:37.525 INFO:tasks.workunit.client.0.smithi094.stdout:6/432: chown dc/d11/d1b/d72/d88/f96 28 1 2024-04-06T03:40:37.525 INFO:tasks.workunit.client.1.smithi136.stdout:8/535: dwrite d3/de/f8e [0,4194304] 0 2024-04-06T03:40:37.526 INFO:tasks.workunit.client.0.smithi094.stdout:1/468: dwrite d0/d1/d16/f1c [0,4194304] 0 2024-04-06T03:40:37.526 INFO:tasks.workunit.client.0.smithi094.stdout:1/469: dread - d0/d1/f73 zero size 2024-04-06T03:40:37.526 INFO:tasks.workunit.client.0.smithi094.stdout:1/470: write d0/d7/d2d/f6e [682500,118597] 0 2024-04-06T03:40:37.526 INFO:tasks.workunit.client.1.smithi136.stdout:7/484: symlink d5/df/d12/d59/d71/lac 0 2024-04-06T03:40:37.527 INFO:tasks.workunit.client.1.smithi136.stdout:2/560: write d3/d4a/d9d/d18/d4b/f85 [1351438,43822] 0 2024-04-06T03:40:37.530 INFO:tasks.workunit.client.1.smithi136.stdout:0/465: fsync d1/d6/d2c/f88 0 2024-04-06T03:40:37.530 INFO:tasks.workunit.client.1.smithi136.stdout:0/466: fsync d1/d6/d2c/d1a/d36/f44 0 2024-04-06T03:40:37.530 INFO:tasks.workunit.client.1.smithi136.stdout:0/467: truncate d1/d6/d2c/d57/f5f 50117 0 2024-04-06T03:40:37.531 INFO:tasks.workunit.client.0.smithi094.stdout:9/423: creat d2/d2c/d33/d37/d96/f9f x:0 0 0 2024-04-06T03:40:37.532 INFO:tasks.workunit.client.1.smithi136.stdout:6/409: getdents d2/d7/d26 0 2024-04-06T03:40:37.534 INFO:tasks.workunit.client.1.smithi136.stdout:8/536: dread d3/de/f4c [0,4194304] 0 2024-04-06T03:40:37.535 INFO:tasks.workunit.client.1.smithi136.stdout:3/430: unlink d0/d4/d12/d19/d2e/f75 0 2024-04-06T03:40:37.537 INFO:tasks.workunit.client.0.smithi094.stdout:1/471: link d0/d1/d27/c47 d0/d1/ca4 0 2024-04-06T03:40:37.539 INFO:tasks.workunit.client.0.smithi094.stdout:6/433: dread dc/d11/d22/f57 [0,4194304] 0 2024-04-06T03:40:37.539 INFO:tasks.workunit.client.0.smithi094.stdout:8/503: dwrite d18/d34/f3b [0,4194304] 0 2024-04-06T03:40:37.540 INFO:tasks.workunit.client.0.smithi094.stdout:4/536: getdents d0/d4/d67 0 2024-04-06T03:40:37.540 INFO:tasks.workunit.client.0.smithi094.stdout:4/537: write d0/df/d1e/f24 [7217948,28845] 0 2024-04-06T03:40:37.541 INFO:tasks.workunit.client.0.smithi094.stdout:4/538: dread d0/d15/d3a/d3e/d50/f5e [0,4194304] 0 2024-04-06T03:40:37.542 INFO:tasks.workunit.client.1.smithi136.stdout:0/468: write d1/d6/d2c/f5a [1338283,31235] 0 2024-04-06T03:40:37.542 INFO:tasks.workunit.client.1.smithi136.stdout:0/469: fsync d1/d6/d2c/d57/d62/f65 0 2024-04-06T03:40:37.542 INFO:tasks.workunit.client.0.smithi094.stdout:9/424: rename d2/d2c/f41 to d2/d2c/d33/d37/d96/fa0 0 2024-04-06T03:40:37.543 INFO:tasks.workunit.client.0.smithi094.stdout:3/512: dwrite d9/d10/f16 [0,4194304] 0 2024-04-06T03:40:37.543 INFO:tasks.workunit.client.0.smithi094.stdout:3/513: write d9/fd [733854,79237] 0 2024-04-06T03:40:37.544 INFO:tasks.workunit.client.1.smithi136.stdout:7/485: mknod d5/df/d53/d68/da3/cad 0 2024-04-06T03:40:37.546 INFO:tasks.workunit.client.0.smithi094.stdout:1/472: truncate d0/d4/d1b/d2f/d3c/d48/f69 1822464 0 2024-04-06T03:40:37.546 INFO:tasks.workunit.client.0.smithi094.stdout:1/473: dread - d0/d4/d1b/d2f/d76/d30/d38/f81 zero size 2024-04-06T03:40:37.548 INFO:tasks.workunit.client.1.smithi136.stdout:6/410: creat d2/d7/d26/d30/d6e/d3d/d47/d84/f8a x:0 0 0 2024-04-06T03:40:37.548 INFO:tasks.workunit.client.1.smithi136.stdout:6/411: fsync d2/d7/d26/d68/f80 0 2024-04-06T03:40:37.549 INFO:tasks.workunit.client.1.smithi136.stdout:8/537: rmdir d3/da/d26/d90 39 2024-04-06T03:40:37.552 INFO:tasks.workunit.client.1.smithi136.stdout:3/431: mknod d0/d4/d12/d19/d2e/d3b/c8b 0 2024-04-06T03:40:37.552 INFO:tasks.workunit.client.1.smithi136.stdout:2/561: dwrite d3/d4a/f8d [0,4194304] 0 2024-04-06T03:40:37.556 INFO:tasks.workunit.client.0.smithi094.stdout:8/504: symlink d18/d3c/d8b/lb0 0 2024-04-06T03:40:37.557 INFO:tasks.workunit.client.1.smithi136.stdout:0/470: mknod d1/d6/d2c/d57/d43/ca3 0 2024-04-06T03:40:37.557 INFO:tasks.workunit.client.1.smithi136.stdout:0/471: truncate d1/d6/d2c/d57/d43/d6d/d9b/d5c/f6a 229557 0 2024-04-06T03:40:37.558 INFO:tasks.workunit.client.1.smithi136.stdout:0/472: chown d1/d6/d2c/d57/d43/d6d/d9b/c4d 998 1 2024-04-06T03:40:37.558 INFO:tasks.workunit.client.1.smithi136.stdout:0/473: fsync d1/d6/d2c/d57/d43/fa2 0 2024-04-06T03:40:37.559 INFO:tasks.workunit.client.1.smithi136.stdout:6/412: write d2/d7/d26/d30/d6e/f10 [6927350,123880] 0 2024-04-06T03:40:37.561 INFO:tasks.workunit.client.0.smithi094.stdout:4/539: mkdir d0/df/d1e/d34/db5 0 2024-04-06T03:40:37.561 INFO:tasks.workunit.client.0.smithi094.stdout:4/540: truncate d0/df/d1e/d34/f88 375230 0 2024-04-06T03:40:37.564 INFO:tasks.workunit.client.0.smithi094.stdout:9/425: creat d2/dc/d80/fa1 x:0 0 0 2024-04-06T03:40:37.573 INFO:tasks.workunit.client.1.smithi136.stdout:8/538: dwrite d3/da/d26/d35/d3d/d48/f66 [0,4194304] 0 2024-04-06T03:40:37.577 INFO:tasks.workunit.client.1.smithi136.stdout:2/562: dread d3/d4a/d9d/d18/f2a [0,4194304] 0 2024-04-06T03:40:37.577 INFO:tasks.workunit.client.1.smithi136.stdout:3/432: dwrite d0/d4/d12/d3a/f3e [0,4194304] 0 2024-04-06T03:40:37.577 INFO:tasks.workunit.client.1.smithi136.stdout:2/563: write d3/f61 [2633499,3596] 0 2024-04-06T03:40:37.577 INFO:tasks.workunit.client.0.smithi094.stdout:9/426: dwrite d2/dc/d21/d23/d2f/f57 [0,4194304] 0 2024-04-06T03:40:37.583 INFO:tasks.workunit.client.0.smithi094.stdout:3/514: rename d9/d10/c95 to d9/d20/cb5 0 2024-04-06T03:40:37.586 INFO:tasks.workunit.client.0.smithi094.stdout:6/434: getdents dc/d11/d22/d50/d61 0 2024-04-06T03:40:37.586 INFO:tasks.workunit.client.0.smithi094.stdout:6/435: write dc/d3a/f89 [109722,18817] 0 2024-04-06T03:40:37.588 INFO:tasks.workunit.client.1.smithi136.stdout:6/413: symlink d2/d7/d26/d68/l8b 0 2024-04-06T03:40:37.589 INFO:tasks.workunit.client.1.smithi136.stdout:7/486: symlink d5/df/d12/d1b/d3a/lae 0 2024-04-06T03:40:37.589 INFO:tasks.workunit.client.1.smithi136.stdout:8/539: getdents d3/de/d18 0 2024-04-06T03:40:37.590 INFO:tasks.workunit.client.0.smithi094.stdout:8/505: mkdir d18/d2c/d37/d60/da3/db1 0 2024-04-06T03:40:37.592 INFO:tasks.workunit.client.0.smithi094.stdout:4/541: mkdir d0/df/d1e/d34/db6 0 2024-04-06T03:40:37.592 INFO:tasks.workunit.client.0.smithi094.stdout:4/542: readlink d0/d4/l1d 0 2024-04-06T03:40:37.592 INFO:tasks.workunit.client.0.smithi094.stdout:4/543: chown d0/d23/f32 2808 1 2024-04-06T03:40:37.597 INFO:tasks.workunit.client.0.smithi094.stdout:3/515: mknod d9/d10/d27/d3a/daf/cb6 0 2024-04-06T03:40:37.598 INFO:tasks.workunit.client.1.smithi136.stdout:6/414: write d2/d7/f2c [393492,112667] 0 2024-04-06T03:40:37.598 INFO:tasks.workunit.client.0.smithi094.stdout:8/506: creat d18/d3c/fb2 x:0 0 0 2024-04-06T03:40:37.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:37 smithi136 ceph-mon[36436]: pgmap v18: 65 pgs: 65 active+clean; 1.4 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 143 MiB/s rd, 191 MiB/s wr, 258 op/s 2024-04-06T03:40:37.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:37 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:40:37.600 INFO:tasks.workunit.client.0.smithi094.stdout:5/470: sync 2024-04-06T03:40:37.600 INFO:tasks.workunit.client.0.smithi094.stdout:2/547: sync 2024-04-06T03:40:37.600 INFO:tasks.workunit.client.0.smithi094.stdout:7/460: sync 2024-04-06T03:40:37.600 INFO:tasks.workunit.client.0.smithi094.stdout:2/548: dread - d0/dd/da4/f5f zero size 2024-04-06T03:40:37.600 INFO:tasks.workunit.client.0.smithi094.stdout:2/549: write d0/dd/d19/d31/d67/d98/fb7 [763037,86990] 0 2024-04-06T03:40:37.600 INFO:tasks.workunit.client.1.smithi136.stdout:7/487: rename d5/df/d12/d1b/d58/f93 to d5/df/d1d/d6f/d2c/faf 0 2024-04-06T03:40:37.601 INFO:tasks.workunit.client.1.smithi136.stdout:8/540: mknod d3/de/d18/d6d/ca6 0 2024-04-06T03:40:37.603 INFO:tasks.workunit.client.1.smithi136.stdout:6/415: rename d2/d7/d26/d30/d6e/d58/d5d/f63 to d2/d7/d26/d30/d6e/d3d/d47/f8c 0 2024-04-06T03:40:37.604 INFO:tasks.workunit.client.0.smithi094.stdout:3/516: truncate d9/d1e/d26/d55/d62/f68 364558 0 2024-04-06T03:40:37.606 INFO:tasks.workunit.client.1.smithi136.stdout:9/456: sync 2024-04-06T03:40:37.606 INFO:tasks.workunit.client.1.smithi136.stdout:4/425: sync 2024-04-06T03:40:37.606 INFO:tasks.workunit.client.1.smithi136.stdout:5/533: sync 2024-04-06T03:40:37.607 INFO:tasks.workunit.client.1.smithi136.stdout:9/457: creat d17/d20/d28/d49/f96 x:0 0 0 2024-04-06T03:40:37.607 INFO:tasks.workunit.client.1.smithi136.stdout:7/488: creat d5/df/d53/d68/fb0 x:0 0 0 2024-04-06T03:40:37.607 INFO:tasks.workunit.client.1.smithi136.stdout:7/489: write d5/df/d53/f9d [8878445,51839] 0 2024-04-06T03:40:37.608 INFO:tasks.workunit.client.0.smithi094.stdout:8/507: creat d18/d3c/d49/d9f/fb3 x:0 0 0 2024-04-06T03:40:37.608 INFO:tasks.workunit.client.1.smithi136.stdout:2/564: dwrite d3/d15/d19/d43/d54/f8c [4194304,4194304] 0 2024-04-06T03:40:37.609 INFO:tasks.workunit.client.1.smithi136.stdout:2/565: stat d3/d15/d19/c8e 0 2024-04-06T03:40:37.609 INFO:tasks.workunit.client.1.smithi136.stdout:2/566: fdatasync d3/d4a/d9d/d18/fa4 0 2024-04-06T03:40:37.609 INFO:tasks.workunit.client.1.smithi136.stdout:8/541: rmdir d3/da 39 2024-04-06T03:40:37.609 INFO:tasks.workunit.client.1.smithi136.stdout:8/542: stat d3/l15 0 2024-04-06T03:40:37.609 INFO:tasks.workunit.client.0.smithi094.stdout:9/427: dwrite d2/f7 [0,4194304] 0 2024-04-06T03:40:37.610 INFO:tasks.workunit.client.0.smithi094.stdout:7/461: creat d2/d6/dc/d12/d29/f89 x:0 0 0 2024-04-06T03:40:37.611 INFO:tasks.workunit.client.1.smithi136.stdout:3/433: dwrite d0/d4/d12/d19/d2c/f31 [0,4194304] 0 2024-04-06T03:40:37.611 INFO:tasks.workunit.client.1.smithi136.stdout:3/434: fdatasync d0/d4/d12/d3a/f4b 0 2024-04-06T03:40:37.611 INFO:tasks.workunit.client.1.smithi136.stdout:3/435: chown d0/l9 449335 1 2024-04-06T03:40:37.613 INFO:tasks.workunit.client.1.smithi136.stdout:3/436: chown d0/d4/d12/d3a/d51 29 1 2024-04-06T03:40:37.613 INFO:tasks.workunit.client.1.smithi136.stdout:3/437: stat d0/d4/d12/d3a/f4b 0 2024-04-06T03:40:37.613 INFO:tasks.workunit.client.1.smithi136.stdout:6/416: mkdir d2/d7/d26/d30/d6e/d58/d5d/d62/d8d 0 2024-04-06T03:40:37.613 INFO:tasks.workunit.client.1.smithi136.stdout:6/417: fdatasync d2/d25/f38 0 2024-04-06T03:40:37.613 INFO:tasks.workunit.client.1.smithi136.stdout:6/418: fsync d2/d7/d26/d30/f3a 0 2024-04-06T03:40:37.616 INFO:tasks.workunit.client.0.smithi094.stdout:1/474: rename d0/d1/d16 to d0/d1/d27/da5 0 2024-04-06T03:40:37.619 INFO:tasks.workunit.client.0.smithi094.stdout:0/440: sync 2024-04-06T03:40:37.620 INFO:tasks.workunit.client.0.smithi094.stdout:3/517: mkdir d9/d10/d9b/db7 0 2024-04-06T03:40:37.623 INFO:tasks.workunit.client.0.smithi094.stdout:7/462: mknod d2/d6/dc/d12/d61/c8a 0 2024-04-06T03:40:37.627 INFO:tasks.workunit.client.1.smithi136.stdout:8/543: mknod d3/da/d26/d35/ca7 0 2024-04-06T03:40:37.629 INFO:tasks.workunit.client.0.smithi094.stdout:4/544: dread d0/d4/d67/d9f/fa9 [4194304,4194304] 0 2024-04-06T03:40:37.631 INFO:tasks.workunit.client.0.smithi094.stdout:2/550: rmdir d0/d8/d27 39 2024-04-06T03:40:37.631 INFO:tasks.workunit.client.0.smithi094.stdout:2/551: readlink d0/dd/d3d/d49/l7f 0 2024-04-06T03:40:37.632 INFO:tasks.workunit.client.0.smithi094.stdout:2/552: dread - d0/dd/d19/fb8 zero size 2024-04-06T03:40:37.634 INFO:tasks.workunit.client.1.smithi136.stdout:3/438: dread d0/d4/d12/d19/d55/d3c/d46/f72 [0,4194304] 0 2024-04-06T03:40:37.635 INFO:tasks.workunit.client.1.smithi136.stdout:7/490: rmdir d5/df/d1d/d6f/d2a 39 2024-04-06T03:40:37.637 INFO:tasks.workunit.client.0.smithi094.stdout:6/436: rename dc/d3a/d7f/f84 to dc/d11/d1b/d72/d88/f9f 0 2024-04-06T03:40:37.637 INFO:tasks.workunit.client.0.smithi094.stdout:6/437: fdatasync dc/f17 0 2024-04-06T03:40:37.638 INFO:tasks.workunit.client.0.smithi094.stdout:6/438: stat dc/d11/d22/d50 0 2024-04-06T03:40:37.639 INFO:tasks.workunit.client.1.smithi136.stdout:8/544: symlink d3/da/d26/d35/la8 0 2024-04-06T03:40:37.641 INFO:tasks.workunit.client.0.smithi094.stdout:1/475: mknod d0/d4/d1b/d2f/d3c/d48/ca6 0 2024-04-06T03:40:37.641 INFO:tasks.workunit.client.0.smithi094.stdout:1/476: write d0/d7/d1a/f9b [105697,38357] 0 2024-04-06T03:40:37.641 INFO:tasks.workunit.client.0.smithi094.stdout:1/477: stat d0/d4/f9 0 2024-04-06T03:40:37.641 INFO:tasks.workunit.client.0.smithi094.stdout:1/478: fsync d0/d7/f78 0 2024-04-06T03:40:37.642 INFO:tasks.workunit.client.0.smithi094.stdout:5/471: dwrite db/d7c/f7e [0,4194304] 0 2024-04-06T03:40:37.642 INFO:tasks.workunit.client.1.smithi136.stdout:1/465: sync 2024-04-06T03:40:37.642 INFO:tasks.workunit.client.1.smithi136.stdout:1/466: fsync fb 0 2024-04-06T03:40:37.642 INFO:tasks.workunit.client.1.smithi136.stdout:1/467: chown d12/d3b/d74/d7a/c85 13053 1 2024-04-06T03:40:37.643 INFO:tasks.workunit.client.1.smithi136.stdout:3/439: mkdir d0/d4/d12/d19/d55/d6f/d82/d5d/d8c 0 2024-04-06T03:40:37.645 INFO:tasks.workunit.client.0.smithi094.stdout:0/441: creat d1/d9/dd/d63/f93 x:0 0 0 2024-04-06T03:40:37.645 INFO:tasks.workunit.client.0.smithi094.stdout:0/442: fsync d1/d9/dd/d18/d1a/f54 0 2024-04-06T03:40:37.645 INFO:tasks.workunit.client.0.smithi094.stdout:0/443: write d1/d9/dd/d18/d5d/d60/d6b/f81 [12618,17750] 0 2024-04-06T03:40:37.647 INFO:tasks.workunit.client.1.smithi136.stdout:5/534: dwrite d1/dc/d1c/d32/d8b/f7f [0,4194304] 0 2024-04-06T03:40:37.647 INFO:tasks.workunit.client.1.smithi136.stdout:9/458: dwrite d17/d20/d25/d3c/f73 [0,4194304] 0 2024-04-06T03:40:37.647 INFO:tasks.workunit.client.1.smithi136.stdout:9/459: chown d17/d21/d32/d3f/f63 1554525 1 2024-04-06T03:40:37.647 INFO:tasks.workunit.client.1.smithi136.stdout:8/545: rename d3/de/d18/f6a to d3/da/d26/d35/d53/fa9 0 2024-04-06T03:40:37.647 INFO:tasks.workunit.client.1.smithi136.stdout:2/567: dwrite d3/d15/d19/f9a [0,4194304] 0 2024-04-06T03:40:37.647 INFO:tasks.workunit.client.1.smithi136.stdout:2/568: write d3/d4a/d9d/d18/d1d/d3c/d58/fa2 [420782,9103] 0 2024-04-06T03:40:37.648 INFO:tasks.workunit.client.0.smithi094.stdout:4/545: mknod d0/d15/d3a/d3e/d49/db2/cb7 0 2024-04-06T03:40:37.648 INFO:tasks.workunit.client.0.smithi094.stdout:4/546: chown d0/d15/d3a/d3e/d46/l47 38151324 1 2024-04-06T03:40:37.650 INFO:tasks.workunit.client.0.smithi094.stdout:9/428: dwrite d2/d2c/d33/d37/d96/fa0 [0,4194304] 0 2024-04-06T03:40:37.650 INFO:tasks.workunit.client.0.smithi094.stdout:9/429: chown d2/l40 520996971 1 2024-04-06T03:40:37.650 INFO:tasks.workunit.client.0.smithi094.stdout:9/430: chown d2/f7b 218665515 1 2024-04-06T03:40:37.650 INFO:tasks.workunit.client.0.smithi094.stdout:9/431: write d2/dc/d21/d23/d47/f77 [838300,105330] 0 2024-04-06T03:40:37.651 INFO:tasks.workunit.client.1.smithi136.stdout:6/419: dwrite d2/d25/f38 [0,4194304] 0 2024-04-06T03:40:37.651 INFO:tasks.workunit.client.1.smithi136.stdout:6/420: write d2/d7/d26/f54 [764930,124122] 0 2024-04-06T03:40:37.651 INFO:tasks.workunit.client.1.smithi136.stdout:6/421: write d2/d7/d26/d30/d6e/d58/d5d/d62/d53/f74 [5042534,94595] 0 2024-04-06T03:40:37.651 INFO:tasks.workunit.client.1.smithi136.stdout:6/422: chown d2/d7/lc 8690070 1 2024-04-06T03:40:37.651 INFO:tasks.workunit.client.1.smithi136.stdout:6/423: creat d2/d7/d26/d30/d6e/d58/f8e x:0 0 0 2024-04-06T03:40:37.651 INFO:tasks.workunit.client.1.smithi136.stdout:3/440: symlink d0/d4/d12/d19/d2c/l8d 0 2024-04-06T03:40:37.654 INFO:tasks.workunit.client.0.smithi094.stdout:2/553: unlink d0/dd/da4/c9a 0 2024-04-06T03:40:37.655 INFO:tasks.workunit.client.1.smithi136.stdout:1/468: dread d12/d3b/d44/d5f/f30 [0,4194304] 0 2024-04-06T03:40:37.655 INFO:tasks.workunit.client.1.smithi136.stdout:1/469: write d12/d3b/d60/d7e/d37/f4e [5136304,56101] 0 2024-04-06T03:40:37.659 INFO:tasks.workunit.client.0.smithi094.stdout:6/439: mknod dc/d3a/d4a/ca0 0 2024-04-06T03:40:37.659 INFO:tasks.workunit.client.0.smithi094.stdout:6/440: read dc/d11/d1b/d72/f8c [30044,43504] 0 2024-04-06T03:40:37.660 INFO:tasks.workunit.client.1.smithi136.stdout:4/426: dwrite d1d/d31/f41 [0,4194304] 0 2024-04-06T03:40:37.660 INFO:tasks.workunit.client.1.smithi136.stdout:7/491: dwrite d5/f82 [0,4194304] 0 2024-04-06T03:40:37.661 INFO:tasks.workunit.client.1.smithi136.stdout:5/535: link d1/d16/d4b/l8d d1/d98/la5 0 2024-04-06T03:40:37.661 INFO:tasks.workunit.client.1.smithi136.stdout:5/536: truncate d1/d16/d4b/f8e 437635 0 2024-04-06T03:40:37.663 INFO:tasks.workunit.client.0.smithi094.stdout:3/518: dwrite d9/d20/d30/f4d [4194304,4194304] 0 2024-04-06T03:40:37.663 INFO:tasks.workunit.client.0.smithi094.stdout:8/508: rename d18/d2c/d37/d60/da3/l83 to d18/d2c/d37/d60/da3/db1/lb4 0 2024-04-06T03:40:37.664 INFO:tasks.workunit.client.1.smithi136.stdout:9/460: dwrite d17/d20/d28/d81/f83 [0,4194304] 0 2024-04-06T03:40:37.669 INFO:tasks.workunit.client.0.smithi094.stdout:6/441: dread dc/d3a/d4a/d54/f76 [0,4194304] 0 2024-04-06T03:40:37.671 INFO:tasks.workunit.client.0.smithi094.stdout:5/472: mknod db/d19/d2c/d57/ca4 0 2024-04-06T03:40:37.675 INFO:tasks.workunit.client.0.smithi094.stdout:0/444: link d1/d9/dd/d18/d1a/d44/l8b d1/d9/dd/d18/d5d/d88/l94 0 2024-04-06T03:40:37.675 INFO:tasks.workunit.client.0.smithi094.stdout:0/445: fdatasync d1/d9/dd/d18/d1a/f4d 0 2024-04-06T03:40:37.675 INFO:tasks.workunit.client.0.smithi094.stdout:0/446: write d1/d10/f2e [4938415,81578] 0 2024-04-06T03:40:37.675 INFO:tasks.workunit.client.0.smithi094.stdout:0/447: chown d1/d9/dd/d18/d1e/d22/c38 889538 1 2024-04-06T03:40:37.675 INFO:tasks.workunit.client.0.smithi094.stdout:0/448: dread - d1/d9/f5e zero size 2024-04-06T03:40:37.679 INFO:tasks.workunit.client.1.smithi136.stdout:2/569: getdents d3/d4a/d9d/d18/d4b 0 2024-04-06T03:40:37.679 INFO:tasks.workunit.client.1.smithi136.stdout:2/570: write d3/d4a/d9d/d18/d4b/f65 [2173545,63716] 0 2024-04-06T03:40:37.680 INFO:tasks.workunit.client.1.smithi136.stdout:2/571: chown d3/d9/l52 15093 1 2024-04-06T03:40:37.685 INFO:tasks.workunit.client.1.smithi136.stdout:6/424: mknod d2/d7/d13/d79/c8f 0 2024-04-06T03:40:37.685 INFO:tasks.workunit.client.1.smithi136.stdout:6/425: dread - d2/d7/d26/d30/f3a zero size 2024-04-06T03:40:37.686 INFO:tasks.workunit.client.1.smithi136.stdout:3/441: getdents d0/d4/d12/d19/d55/d6f 0 2024-04-06T03:40:37.688 INFO:tasks.workunit.client.0.smithi094.stdout:2/554: creat d0/dd/d3d/d49/fb9 x:0 0 0 2024-04-06T03:40:37.688 INFO:tasks.workunit.client.0.smithi094.stdout:2/555: truncate d0/dd/da4/f5f 625621 0 2024-04-06T03:40:37.688 INFO:tasks.workunit.client.0.smithi094.stdout:9/432: dread d2/dc/d10/f12 [4194304,4194304] 0 2024-04-06T03:40:37.689 INFO:tasks.workunit.client.1.smithi136.stdout:1/470: unlink d12/f39 0 2024-04-06T03:40:37.695 INFO:tasks.workunit.client.0.smithi094.stdout:3/519: unlink d9/d20/d76/ca1 0 2024-04-06T03:40:37.697 INFO:tasks.workunit.client.0.smithi094.stdout:0/449: dwrite d1/d10/f73 [0,4194304] 0 2024-04-06T03:40:37.698 INFO:tasks.workunit.client.0.smithi094.stdout:0/450: truncate d1/d9/dd/d18/d1e/d22/f6a 1168654 0 2024-04-06T03:40:37.699 INFO:tasks.workunit.client.0.smithi094.stdout:4/547: rename d0/df/d1e/l2e to d0/lb8 0 2024-04-06T03:40:37.699 INFO:tasks.workunit.client.0.smithi094.stdout:4/548: chown d0/d15/d3a/d61/d81/da0 22472235 1 2024-04-06T03:40:37.700 INFO:tasks.workunit.client.1.smithi136.stdout:3/442: dread d0/d4/d12/f15 [4194304,4194304] 0 2024-04-06T03:40:37.700 INFO:tasks.workunit.client.1.smithi136.stdout:7/492: getdents d5/d6d 0 2024-04-06T03:40:37.702 INFO:tasks.workunit.client.0.smithi094.stdout:8/509: truncate d18/f1c 3140892 0 2024-04-06T03:40:37.704 INFO:tasks.workunit.client.1.smithi136.stdout:5/537: link d1/dc/c29 d1/dc/d10/ca6 0 2024-04-06T03:40:37.705 INFO:tasks.workunit.client.0.smithi094.stdout:6/442: rmdir dc/d11/d22/d50/d61 39 2024-04-06T03:40:37.706 INFO:tasks.workunit.client.0.smithi094.stdout:0/451: dread d1/f83 [0,4194304] 0 2024-04-06T03:40:37.709 INFO:tasks.workunit.client.0.smithi094.stdout:5/473: getdents db/d80 0 2024-04-06T03:40:37.709 INFO:tasks.workunit.client.0.smithi094.stdout:5/474: truncate db/d7c/d98/fa3 223026 0 2024-04-06T03:40:37.716 INFO:tasks.workunit.client.1.smithi136.stdout:3/443: dread d0/d4/d12/d19/d55/f24 [0,4194304] 0 2024-04-06T03:40:37.717 INFO:tasks.workunit.client.0.smithi094.stdout:2/556: dwrite d0/dd/d19/d31/f6e [0,4194304] 0 2024-04-06T03:40:37.717 INFO:tasks.workunit.client.0.smithi094.stdout:8/510: dread d18/d34/f58 [0,4194304] 0 2024-04-06T03:40:37.724 INFO:tasks.workunit.client.1.smithi136.stdout:7/493: dwrite d5/df/d53/f9d [8388608,4194304] 0 2024-04-06T03:40:37.726 INFO:tasks.workunit.client.0.smithi094.stdout:6/443: dwrite dc/d11/d1b/f38 [0,4194304] 0 2024-04-06T03:40:37.728 INFO:tasks.workunit.client.1.smithi136.stdout:5/538: dwrite d1/d16/d4b/d7b/f83 [0,4194304] 0 2024-04-06T03:40:37.731 INFO:tasks.workunit.client.0.smithi094.stdout:4/549: unlink d0/d15/l2c 0 2024-04-06T03:40:37.731 INFO:tasks.workunit.client.0.smithi094.stdout:4/550: stat d0/d15/d3a/c41 0 2024-04-06T03:40:37.733 INFO:tasks.workunit.client.0.smithi094.stdout:0/452: link d1/d9/dd/d18/d1d/l21 d1/d9/dd/d18/d1e/l95 0 2024-04-06T03:40:37.736 INFO:tasks.workunit.client.0.smithi094.stdout:5/475: link db/d19/d24/c35 db/d19/d21/ca5 0 2024-04-06T03:40:37.737 INFO:tasks.workunit.client.0.smithi094.stdout:3/520: dread d9/d1e/d26/d55/d62/f68 [0,4194304] 0 2024-04-06T03:40:37.738 INFO:tasks.workunit.client.0.smithi094.stdout:3/521: fdatasync d9/d1d/f45 0 2024-04-06T03:40:37.739 INFO:tasks.workunit.client.0.smithi094.stdout:2/557: dwrite d0/dd/d3d/d49/fb9 [0,4194304] 0 2024-04-06T03:40:37.739 INFO:tasks.workunit.client.0.smithi094.stdout:8/511: dwrite fd [0,4194304] 0 2024-04-06T03:40:37.740 INFO:tasks.workunit.client.1.smithi136.stdout:9/461: unlink d17/d20/d25/d3c/d55/f79 0 2024-04-06T03:40:37.741 INFO:tasks.workunit.client.1.smithi136.stdout:8/546: getdents d3/de/d18/d7d 0 2024-04-06T03:40:37.741 INFO:tasks.workunit.client.1.smithi136.stdout:8/547: read - d3/da/d26/f50 zero size 2024-04-06T03:40:37.742 INFO:tasks.workunit.client.0.smithi094.stdout:8/512: dread d18/f4d [0,4194304] 0 2024-04-06T03:40:37.745 INFO:tasks.workunit.client.1.smithi136.stdout:2/572: mkdir d3/d9/d3b/d87/db7 0 2024-04-06T03:40:37.745 INFO:tasks.workunit.client.1.smithi136.stdout:2/573: fsync d3/f68 0 2024-04-06T03:40:37.748 INFO:tasks.workunit.client.1.smithi136.stdout:6/426: symlink d2/d7/d13/d89/l90 0 2024-04-06T03:40:37.748 INFO:tasks.workunit.client.0.smithi094.stdout:9/433: creat d2/dc/d10/d2a/fa2 x:0 0 0 2024-04-06T03:40:37.751 INFO:tasks.workunit.client.1.smithi136.stdout:1/471: creat d12/d3b/d44/d5f/d5a/d72/d77/d96/f97 x:0 0 0 2024-04-06T03:40:37.751 INFO:tasks.workunit.client.1.smithi136.stdout:1/472: read d12/d3b/f4b [339442,8541] 0 2024-04-06T03:40:37.752 INFO:tasks.workunit.client.1.smithi136.stdout:0/474: sync 2024-04-06T03:40:37.752 INFO:tasks.workunit.client.1.smithi136.stdout:0/475: write d1/d6/d2c/d57/f66 [4563484,27583] 0 2024-04-06T03:40:37.752 INFO:tasks.workunit.client.1.smithi136.stdout:0/476: chown d1/d53 164127513 1 2024-04-06T03:40:37.752 INFO:tasks.workunit.client.1.smithi136.stdout:0/477: write d1/d6/d2c/d57/d43/d6d/d85/d78/f9a [681932,32424] 0 2024-04-06T03:40:37.752 INFO:tasks.workunit.client.1.smithi136.stdout:0/478: chown d1/d6/d2c/d1a/d36/f44 1134532113 1 2024-04-06T03:40:37.755 INFO:tasks.workunit.client.1.smithi136.stdout:3/444: mkdir d0/d4/d12/d3a/d51/d8e 0 2024-04-06T03:40:37.755 INFO:tasks.workunit.client.1.smithi136.stdout:3/445: chown d0/d4/d12/d19/d55/d3c/c41 2872295 1 2024-04-06T03:40:37.756 INFO:tasks.workunit.client.0.smithi094.stdout:8/513: dread d18/d3c/d8b/f69 [4194304,4194304] 0 2024-04-06T03:40:37.756 INFO:tasks.workunit.client.0.smithi094.stdout:8/514: write d18/d2c/d37/d60/d70/f84 [15392,101901] 0 2024-04-06T03:40:37.756 INFO:tasks.workunit.client.1.smithi136.stdout:3/446: dread d0/d4/d12/d19/d55/d3c/f58 [0,4194304] 0 2024-04-06T03:40:37.760 INFO:tasks.workunit.client.1.smithi136.stdout:8/548: read d3/de/f3a [8023465,11184] 0 2024-04-06T03:40:37.761 INFO:tasks.workunit.client.1.smithi136.stdout:9/462: dwrite d17/d19/f1e [0,4194304] 0 2024-04-06T03:40:37.762 INFO:tasks.workunit.client.1.smithi136.stdout:7/494: link d5/df/d11/f89 d5/df/d1d/d6f/d2c/fb1 0 2024-04-06T03:40:37.763 INFO:tasks.workunit.client.1.smithi136.stdout:0/479: dread d1/d6/fb [0,4194304] 0 2024-04-06T03:40:37.763 INFO:tasks.workunit.client.1.smithi136.stdout:0/480: write d1/f29 [12878,6713] 0 2024-04-06T03:40:37.764 INFO:tasks.workunit.client.0.smithi094.stdout:3/522: dwrite d9/fd [0,4194304] 0 2024-04-06T03:40:37.767 INFO:tasks.workunit.client.1.smithi136.stdout:5/539: rename d1/dc/d1c/d32/d8b/d20/f84 to d1/dc/d1c/d32/d8b/fa7 0 2024-04-06T03:40:37.767 INFO:tasks.workunit.client.1.smithi136.stdout:6/427: rename d2 to d2/d7/d26/d30/d6e/d58/d5d/d62/d91 22 2024-04-06T03:40:37.767 INFO:tasks.workunit.client.1.smithi136.stdout:6/428: write d2/d7/d26/d30/d4f/f5f [2513363,116062] 0 2024-04-06T03:40:37.767 INFO:tasks.workunit.client.0.smithi094.stdout:2/558: dwrite d0/f57 [4194304,4194304] 0 2024-04-06T03:40:37.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/434: dwrite d2/d2c/d33/d37/f3a [0,4194304] 0 2024-04-06T03:40:37.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/435: getdents d2/dc/d21/d26/d78/d69 0 2024-04-06T03:40:37.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/436: creat d2/dc/d21/d23/fa3 x:0 0 0 2024-04-06T03:40:37.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/437: creat d2/dc/d21/fa4 x:0 0 0 2024-04-06T03:40:37.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/438: write d2/dc/d21/d26/d78/d69/d6d/d6b/f73 [415039,66832] 0 2024-04-06T03:40:37.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/439: creat d2/d2c/fa5 x:0 0 0 2024-04-06T03:40:37.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/440: dread - d2/dc/d46/f60 zero size 2024-04-06T03:40:37.770 INFO:tasks.workunit.client.0.smithi094.stdout:9/441: chown d2/d2c/d33/d37/d96/f9f 2730571 1 2024-04-06T03:40:37.772 INFO:tasks.workunit.client.1.smithi136.stdout:2/574: mknod d3/d4a/d9d/d18/cb8 0 2024-04-06T03:40:37.774 INFO:tasks.workunit.client.0.smithi094.stdout:6/444: creat dc/d11/d97/fa1 x:0 0 0 2024-04-06T03:40:37.775 INFO:tasks.workunit.client.1.smithi136.stdout:5/540: dread d1/dc/d10/f11 [0,4194304] 0 2024-04-06T03:40:37.776 INFO:tasks.workunit.client.1.smithi136.stdout:3/447: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d61/f8f x:0 0 0 2024-04-06T03:40:37.776 INFO:tasks.workunit.client.1.smithi136.stdout:3/448: fdatasync d0/d4/d12/d19/d2c/f40 0 2024-04-06T03:40:37.776 INFO:tasks.workunit.client.1.smithi136.stdout:7/495: dread d5/df/d53/f9d [0,4194304] 0 2024-04-06T03:40:37.777 INFO:tasks.workunit.client.0.smithi094.stdout:4/551: rename d0/d15/d3a/d3e/d50/l5b to d0/df/d1e/d34/db5/lb9 0 2024-04-06T03:40:37.781 INFO:tasks.workunit.client.1.smithi136.stdout:8/549: mkdir d3/da/d26/d35/d39/daa 0 2024-04-06T03:40:37.782 INFO:tasks.workunit.client.1.smithi136.stdout:8/550: fdatasync d3/da/d26/d35/d53/f5b 0 2024-04-06T03:40:37.782 INFO:tasks.workunit.client.1.smithi136.stdout:5/541: dread d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:37.783 INFO:tasks.workunit.client.0.smithi094.stdout:0/453: mkdir d1/d9/dd/d18/d1e/d86/d96 0 2024-04-06T03:40:37.783 INFO:tasks.workunit.client.0.smithi094.stdout:0/454: creat d1/d9/dd/d18/d1e/d22/f97 x:0 0 0 2024-04-06T03:40:37.787 INFO:tasks.workunit.client.0.smithi094.stdout:5/476: symlink db/d19/d2c/la6 0 2024-04-06T03:40:37.788 INFO:tasks.workunit.client.0.smithi094.stdout:5/477: chown db/fd 18707 1 2024-04-06T03:40:37.791 INFO:tasks.workunit.client.1.smithi136.stdout:9/463: mknod d17/d20/d25/d3c/d55/d8a/c97 0 2024-04-06T03:40:37.791 INFO:tasks.workunit.client.1.smithi136.stdout:9/464: fsync fa 0 2024-04-06T03:40:37.791 INFO:tasks.workunit.client.1.smithi136.stdout:4/427: sync 2024-04-06T03:40:37.793 INFO:tasks.workunit.client.0.smithi094.stdout:6/445: dwrite dc/d11/d22/f9e [0,4194304] 0 2024-04-06T03:40:37.795 INFO:tasks.workunit.client.1.smithi136.stdout:2/575: dwrite d3/d4a/d9d/d8a/d93/f67 [0,4194304] 0 2024-04-06T03:40:37.795 INFO:tasks.workunit.client.1.smithi136.stdout:0/481: truncate d1/d6/f1f 1638344 0 2024-04-06T03:40:37.797 INFO:tasks.workunit.client.0.smithi094.stdout:8/515: mkdir d18/d3c/d49/db5 0 2024-04-06T03:40:37.797 INFO:tasks.workunit.client.0.smithi094.stdout:8/516: write d18/d2c/d37/fa1 [2011138,58949] 0 2024-04-06T03:40:37.797 INFO:tasks.workunit.client.0.smithi094.stdout:8/517: readlink d18/d3c/d49/l5e 0 2024-04-06T03:40:37.798 INFO:tasks.workunit.client.0.smithi094.stdout:8/518: read - d18/d3c/d49/f53 zero size 2024-04-06T03:40:37.800 INFO:tasks.workunit.client.0.smithi094.stdout:2/559: mkdir d0/d8/d4f/dab/dba 0 2024-04-06T03:40:37.802 INFO:tasks.workunit.client.1.smithi136.stdout:1/473: rename d12/c20 to d12/d3b/d44/d5f/d5a/d72/d77/c98 0 2024-04-06T03:40:37.806 INFO:tasks.workunit.client.1.smithi136.stdout:7/496: dwrite d5/df/d12/d59/f5e [0,4194304] 0 2024-04-06T03:40:37.806 INFO:tasks.workunit.client.1.smithi136.stdout:7/497: readlink d5/df/d1d/d54/l77 0 2024-04-06T03:40:37.807 INFO:tasks.workunit.client.0.smithi094.stdout:4/552: dwrite d0/df/d13/f98 [0,4194304] 0 2024-04-06T03:40:37.807 INFO:tasks.workunit.client.0.smithi094.stdout:5/478: dwrite db/d19/d2c/d57/f58 [4194304,4194304] 0 2024-04-06T03:40:37.807 INFO:tasks.workunit.client.0.smithi094.stdout:5/479: read - db/f76 zero size 2024-04-06T03:40:37.817 INFO:tasks.workunit.client.1.smithi136.stdout:8/551: mkdir d3/da/d26/d35/d3d/d48/d6c/dab 0 2024-04-06T03:40:37.817 INFO:tasks.workunit.client.1.smithi136.stdout:7/498: write d5/df/d11/f26 [2527530,126733] 0 2024-04-06T03:40:37.817 INFO:tasks.workunit.client.1.smithi136.stdout:7/499: dread - d5/df/d1d/f7f zero size 2024-04-06T03:40:37.817 INFO:tasks.workunit.client.1.smithi136.stdout:7/500: chown d5/df/d12/d1b/d3a/d8d/ca9 36 1 2024-04-06T03:40:37.817 INFO:tasks.workunit.client.1.smithi136.stdout:7/501: truncate d5/df/d12/d1b/d3a/d4a/f64 1008003 0 2024-04-06T03:40:37.818 INFO:tasks.workunit.client.1.smithi136.stdout:0/482: dwrite d1/d6/d2c/d57/f66 [0,4194304] 0 2024-04-06T03:40:37.818 INFO:tasks.workunit.client.0.smithi094.stdout:8/519: dwrite d18/f64 [0,4194304] 0 2024-04-06T03:40:37.818 INFO:tasks.workunit.client.0.smithi094.stdout:8/520: stat d18/d3c/d76/f8c 0 2024-04-06T03:40:37.819 INFO:tasks.workunit.client.1.smithi136.stdout:5/542: link d1/d16/f63 d1/dc/d1c/d32/d70/fa8 0 2024-04-06T03:40:37.819 INFO:tasks.workunit.client.1.smithi136.stdout:5/543: write d1/dc/d1c/d32/f38 [2450730,85736] 0 2024-04-06T03:40:37.823 INFO:tasks.workunit.client.0.smithi094.stdout:3/523: rename d9/d1e/d6e/d75/d90/d9a to d9/d1e/d26/d55/d62/db8 0 2024-04-06T03:40:37.824 INFO:tasks.workunit.client.1.smithi136.stdout:9/465: creat d17/d20/d28/f98 x:0 0 0 2024-04-06T03:40:37.825 INFO:tasks.workunit.client.1.smithi136.stdout:3/449: dwrite d0/d4/d12/d19/f29 [0,4194304] 0 2024-04-06T03:40:37.825 INFO:tasks.workunit.client.1.smithi136.stdout:3/450: fdatasync d0/f2 0 2024-04-06T03:40:37.825 INFO:tasks.workunit.client.1.smithi136.stdout:3/451: dread - d0/d4/d12/d3a/f4b zero size 2024-04-06T03:40:37.825 INFO:tasks.workunit.client.1.smithi136.stdout:3/452: creat d0/d4/d12/d19/d2c/f90 x:0 0 0 2024-04-06T03:40:37.825 INFO:tasks.workunit.client.1.smithi136.stdout:3/453: read d0/f32 [123272,76382] 0 2024-04-06T03:40:37.826 INFO:tasks.workunit.client.1.smithi136.stdout:4/428: truncate d1d/d31/d3e/d56/f5e 1765225 0 2024-04-06T03:40:37.826 INFO:tasks.workunit.client.1.smithi136.stdout:4/429: chown d1d/d70/l84 22594 1 2024-04-06T03:40:37.828 INFO:tasks.workunit.client.1.smithi136.stdout:2/576: creat d3/d15/d19/dac/fb9 x:0 0 0 2024-04-06T03:40:37.828 INFO:tasks.workunit.client.1.smithi136.stdout:2/577: creat d3/d15/d19/d4e/fba x:0 0 0 2024-04-06T03:40:37.828 INFO:tasks.workunit.client.1.smithi136.stdout:2/578: truncate d3/d4a/d9d/d18/d1d/d3c/d58/fa2 1290307 0 2024-04-06T03:40:37.828 INFO:tasks.workunit.client.1.smithi136.stdout:2/579: write d3/d15/d19/d43/d54/fa8 [4698910,27844] 0 2024-04-06T03:40:37.829 INFO:tasks.workunit.client.1.smithi136.stdout:1/474: creat d12/d1b/d1e/f99 x:0 0 0 2024-04-06T03:40:37.833 INFO:tasks.workunit.client.0.smithi094.stdout:6/446: mkdir dc/d3a/da2 0 2024-04-06T03:40:37.833 INFO:tasks.workunit.client.0.smithi094.stdout:6/447: fsync dc/d11/d1b/d72/f81 0 2024-04-06T03:40:37.833 INFO:tasks.workunit.client.0.smithi094.stdout:6/448: write dc/d11/d1b/d72/d88/f9f [311444,23856] 0 2024-04-06T03:40:37.833 INFO:tasks.workunit.client.0.smithi094.stdout:6/449: fdatasync dc/d11/d1b/d72/d88/f96 0 2024-04-06T03:40:37.838 INFO:tasks.workunit.client.1.smithi136.stdout:7/502: symlink d5/df/d12/d1b/d3a/d4a/lb2 0 2024-04-06T03:40:37.838 INFO:tasks.workunit.client.1.smithi136.stdout:7/503: write d5/df/d1d/d6f/f56 [798055,97205] 0 2024-04-06T03:40:37.840 INFO:tasks.workunit.client.0.smithi094.stdout:4/553: getdents d0 0 2024-04-06T03:40:37.840 INFO:tasks.workunit.client.0.smithi094.stdout:4/554: write d0/d15/d3a/f75 [396003,117926] 0 2024-04-06T03:40:37.842 INFO:tasks.workunit.client.0.smithi094.stdout:5/480: getdents db/d19 0 2024-04-06T03:40:37.843 INFO:tasks.workunit.client.1.smithi136.stdout:0/483: mkdir d1/d6/d2c/d57/d43/d6d/d85/d78/da4 0 2024-04-06T03:40:37.846 INFO:tasks.workunit.client.0.smithi094.stdout:6/450: dwrite dc/d3a/f5e [0,4194304] 0 2024-04-06T03:40:37.847 INFO:tasks.workunit.client.1.smithi136.stdout:5/544: mkdir d1/d9b/d9c/da9 0 2024-04-06T03:40:37.847 INFO:tasks.workunit.client.1.smithi136.stdout:5/545: fsync d1/d16/f85 0 2024-04-06T03:40:37.849 INFO:tasks.workunit.client.1.smithi136.stdout:9/466: link d17/d20/d25/d3d/d4b/f80 d17/d21/d29/f99 0 2024-04-06T03:40:37.849 INFO:tasks.workunit.client.0.smithi094.stdout:4/555: dread d0/d15/f1a [0,4194304] 0 2024-04-06T03:40:37.851 INFO:tasks.workunit.client.0.smithi094.stdout:4/556: dread d0/d15/d3a/d3e/d46/f72 [0,4194304] 0 2024-04-06T03:40:37.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/557: read d0/d15/d3a/d3e/d50/f5e [117075,113479] 0 2024-04-06T03:40:37.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/558: chown d0/df/f1b 192246684 1 2024-04-06T03:40:37.852 INFO:tasks.workunit.client.0.smithi094.stdout:4/559: write d0/df/d13/f35 [615499,37378] 0 2024-04-06T03:40:37.854 INFO:tasks.workunit.client.1.smithi136.stdout:7/504: write d5/df/d12/d59/f5e [4177869,18565] 0 2024-04-06T03:40:37.858 INFO:tasks.workunit.client.1.smithi136.stdout:4/430: mknod d1d/d31/d3e/d56/d7b/c9c 0 2024-04-06T03:40:37.858 INFO:tasks.workunit.client.1.smithi136.stdout:0/484: dwrite d1/d6/f1f [0,4194304] 0 2024-04-06T03:40:37.858 INFO:tasks.workunit.client.1.smithi136.stdout:0/485: write d1/d6/d2c/d1a/d36/f5d [449085,122785] 0 2024-04-06T03:40:37.858 INFO:tasks.workunit.client.1.smithi136.stdout:0/486: write d1/d6/d2c/d1a/d36/f8f [1020137,96000] 0 2024-04-06T03:40:37.860 INFO:tasks.workunit.client.1.smithi136.stdout:2/580: mknod d3/d15/cbb 0 2024-04-06T03:40:37.860 INFO:tasks.workunit.client.1.smithi136.stdout:2/581: chown d3/d9b 92730786 1 2024-04-06T03:40:37.860 INFO:tasks.workunit.client.0.smithi094.stdout:9/442: rmdir d2/dc 39 2024-04-06T03:40:37.862 INFO:tasks.workunit.client.1.smithi136.stdout:1/475: symlink d12/d3b/d44/d5f/d5a/l9a 0 2024-04-06T03:40:37.866 INFO:tasks.workunit.client.0.smithi094.stdout:2/560: rename d0/dd/d3d/d49/fb9 to d0/dd/d1b/d72/d8f/fbb 0 2024-04-06T03:40:37.867 INFO:tasks.workunit.client.0.smithi094.stdout:3/524: rmdir d9/d10/d27/d3a/daf 39 2024-04-06T03:40:37.868 INFO:tasks.workunit.client.1.smithi136.stdout:9/467: dwrite d17/d20/d25/f26 [4194304,4194304] 0 2024-04-06T03:40:37.871 INFO:tasks.workunit.client.1.smithi136.stdout:8/552: getdents d3/da/d26/d35/d53 0 2024-04-06T03:40:37.873 INFO:tasks.workunit.client.0.smithi094.stdout:5/481: unlink db/f7a 0 2024-04-06T03:40:37.873 INFO:tasks.workunit.client.0.smithi094.stdout:5/482: readlink db/d19/d24/d2e/l18 0 2024-04-06T03:40:37.876 INFO:tasks.workunit.client.1.smithi136.stdout:5/546: creat d1/d9b/faa x:0 0 0 2024-04-06T03:40:37.876 INFO:tasks.workunit.client.1.smithi136.stdout:5/547: truncate d1/dc/d1c/d32/d8b/d20/f42 862616 0 2024-04-06T03:40:37.879 INFO:tasks.workunit.client.1.smithi136.stdout:3/454: mkdir d0/d4/d91 0 2024-04-06T03:40:37.879 INFO:tasks.workunit.client.1.smithi136.stdout:3/455: chown d0/d4/d12/d2d/d37/f6d 1608 1 2024-04-06T03:40:37.882 INFO:tasks.workunit.client.1.smithi136.stdout:4/431: unlink d1d/d70/c87 0 2024-04-06T03:40:37.884 INFO:tasks.workunit.client.1.smithi136.stdout:2/582: dwrite d3/d4a/d9d/d18/f48 [4194304,4194304] 0 2024-04-06T03:40:37.885 INFO:tasks.workunit.client.1.smithi136.stdout:0/487: symlink d1/d6/d2c/d57/d43/d6d/d85/la5 0 2024-04-06T03:40:37.886 INFO:tasks.workunit.client.1.smithi136.stdout:7/505: dwrite d5/df/d1d/d6f/f56 [0,4194304] 0 2024-04-06T03:40:37.889 INFO:tasks.workunit.client.1.smithi136.stdout:9/468: mkdir d17/d21/d29/d34/d46/d9a 0 2024-04-06T03:40:37.890 INFO:tasks.workunit.client.0.smithi094.stdout:2/561: dwrite d0/d18/f24 [4194304,4194304] 0 2024-04-06T03:40:37.894 INFO:tasks.workunit.client.0.smithi094.stdout:5/483: dwrite db/d19/d2c/f31 [4194304,4194304] 0 2024-04-06T03:40:37.894 INFO:tasks.workunit.client.0.smithi094.stdout:5/484: chown db/d19/d2c/d57/f58 52205134 1 2024-04-06T03:40:37.895 INFO:tasks.workunit.client.1.smithi136.stdout:8/553: truncate d3/f5e 1237526 0 2024-04-06T03:40:37.895 INFO:tasks.workunit.client.1.smithi136.stdout:6/429: sync 2024-04-06T03:40:37.898 INFO:tasks.workunit.client.1.smithi136.stdout:3/456: dwrite d0/d4/d12/d19/d2e/f87 [0,4194304] 0 2024-04-06T03:40:37.898 INFO:tasks.workunit.client.1.smithi136.stdout:3/457: stat d0/d4/d12/d19/d55/d6f/d82/c7d 0 2024-04-06T03:40:37.900 INFO:tasks.workunit.client.1.smithi136.stdout:5/548: symlink d1/dc/d10/da3/lab 0 2024-04-06T03:40:37.900 INFO:tasks.workunit.client.1.smithi136.stdout:5/549: chown d1/d16/d17 97549234 1 2024-04-06T03:40:37.900 INFO:tasks.workunit.client.1.smithi136.stdout:5/550: creat d1/d9b/fac x:0 0 0 2024-04-06T03:40:37.900 INFO:tasks.workunit.client.1.smithi136.stdout:5/551: dread - d1/dc/f91 zero size 2024-04-06T03:40:37.902 INFO:tasks.workunit.client.0.smithi094.stdout:2/562: write d0/dd/d19/d31/f42 [1025639,36167] 0 2024-04-06T03:40:37.904 INFO:tasks.workunit.client.1.smithi136.stdout:8/554: dread d3/da/fb [0,4194304] 0 2024-04-06T03:40:37.904 INFO:tasks.workunit.client.1.smithi136.stdout:2/583: rmdir d3/d9/d3b/d70 39 2024-04-06T03:40:37.906 INFO:tasks.workunit.client.1.smithi136.stdout:0/488: dwrite d1/f63 [0,4194304] 0 2024-04-06T03:40:37.906 INFO:tasks.workunit.client.0.smithi094.stdout:0/455: write d1/d9/dd/d18/f3a [690036,58756] 0 2024-04-06T03:40:37.907 INFO:tasks.workunit.client.0.smithi094.stdout:6/451: mkdir dc/d11/d1b/d72/d88/da3 0 2024-04-06T03:40:37.910 INFO:tasks.workunit.client.0.smithi094.stdout:4/560: dread - d0/df/d1e/f59 zero size 2024-04-06T03:40:37.912 INFO:tasks.workunit.client.0.smithi094.stdout:5/485: dwrite db/d19/d2c/d6d/f5f [0,4194304] 0 2024-04-06T03:40:37.913 INFO:tasks.workunit.client.0.smithi094.stdout:9/443: mkdir d2/dc/da6 0 2024-04-06T03:40:37.914 INFO:tasks.workunit.client.0.smithi094.stdout:8/521: rename d18/c2a to d18/d2c/d37/d60/d70/cb6 0 2024-04-06T03:40:37.918 INFO:tasks.workunit.client.0.smithi094.stdout:3/525: mknod d9/d1e/cb9 0 2024-04-06T03:40:37.920 INFO:tasks.workunit.client.1.smithi136.stdout:7/506: symlink d5/df/d1d/d54/lb3 0 2024-04-06T03:40:37.922 INFO:tasks.workunit.client.0.smithi094.stdout:2/563: symlink d0/d8/d27/lbc 0 2024-04-06T03:40:37.922 INFO:tasks.workunit.client.0.smithi094.stdout:7/463: sync 2024-04-06T03:40:37.922 INFO:tasks.workunit.client.0.smithi094.stdout:1/479: sync 2024-04-06T03:40:37.923 INFO:tasks.workunit.client.0.smithi094.stdout:7/464: chown d2/d6/dc/dd/df/d36/c62 79727604 1 2024-04-06T03:40:37.923 INFO:tasks.workunit.client.0.smithi094.stdout:1/480: truncate d0/d4/d1b/d2f/d76/d30/d38/f8d 931389 0 2024-04-06T03:40:37.924 INFO:tasks.workunit.client.1.smithi136.stdout:6/430: mkdir d2/d92 0 2024-04-06T03:40:37.924 INFO:tasks.workunit.client.1.smithi136.stdout:6/431: write d2/f9 [4564964,57215] 0 2024-04-06T03:40:37.924 INFO:tasks.workunit.client.1.smithi136.stdout:6/432: write d2/d7/d26/d30/d4f/f67 [1179326,103817] 0 2024-04-06T03:40:37.924 INFO:tasks.workunit.client.0.smithi094.stdout:0/456: symlink d1/d9/l98 0 2024-04-06T03:40:37.925 INFO:tasks.workunit.client.0.smithi094.stdout:4/561: dwrite d0/d15/d3a/d3e/d50/f65 [0,4194304] 0 2024-04-06T03:40:37.925 INFO:tasks.workunit.client.0.smithi094.stdout:4/562: creat d0/d15/d3a/d61/d81/fba x:0 0 0 2024-04-06T03:40:37.926 INFO:tasks.workunit.client.0.smithi094.stdout:4/563: write d0/df/d13/f5a [149164,38295] 0 2024-04-06T03:40:37.926 INFO:tasks.workunit.client.0.smithi094.stdout:4/564: readlink d0/df/d1e/l7e 0 2024-04-06T03:40:37.928 INFO:tasks.workunit.client.0.smithi094.stdout:7/465: dread d2/d6/f9 [0,4194304] 0 2024-04-06T03:40:37.928 INFO:tasks.workunit.client.1.smithi136.stdout:3/458: mknod d0/d4/d91/c92 0 2024-04-06T03:40:37.931 INFO:tasks.workunit.client.0.smithi094.stdout:7/466: dread d2/d6/dc/dd/f2f [0,4194304] 0 2024-04-06T03:40:37.931 INFO:tasks.workunit.client.0.smithi094.stdout:7/467: write d2/d6/dc/f2b [2470697,126578] 0 2024-04-06T03:40:37.931 INFO:tasks.workunit.client.0.smithi094.stdout:5/486: symlink db/d19/d21/la7 0 2024-04-06T03:40:37.932 INFO:tasks.workunit.client.1.smithi136.stdout:4/432: rmdir d1d/d31/d3e/d55/d7f 39 2024-04-06T03:40:37.934 INFO:tasks.workunit.client.1.smithi136.stdout:2/584: symlink d3/d4a/d9d/d18/d1d/d91/lbc 0 2024-04-06T03:40:37.934 INFO:tasks.workunit.client.0.smithi094.stdout:9/444: getdents d2/dc/d21/d23/d2f 0 2024-04-06T03:40:37.934 INFO:tasks.workunit.client.0.smithi094.stdout:9/445: truncate d2/dc/d21/d23/d47/d51/f92 494907 0 2024-04-06T03:40:37.938 INFO:tasks.workunit.client.0.smithi094.stdout:8/522: mknod d18/d3c/d49/d91/cb7 0 2024-04-06T03:40:37.939 INFO:tasks.workunit.client.1.smithi136.stdout:0/489: truncate d1/d6/d2c/d57/d43/d6d/d9b/d5c/f6a 157400 0 2024-04-06T03:40:37.941 INFO:tasks.workunit.client.0.smithi094.stdout:9/446: dread d2/f14 [4194304,4194304] 0 2024-04-06T03:40:37.941 INFO:tasks.workunit.client.0.smithi094.stdout:2/564: mknod d0/d8/d1d/cbd 0 2024-04-06T03:40:37.942 INFO:tasks.workunit.client.1.smithi136.stdout:7/507: rmdir d5/df 39 2024-04-06T03:40:37.943 INFO:tasks.workunit.client.1.smithi136.stdout:9/469: rmdir d17/d21/d29/d34/d46 39 2024-04-06T03:40:37.948 INFO:tasks.workunit.client.1.smithi136.stdout:3/459: rename d0/d4/d12/d19/d55/d3c/d53/l54 to d0/d4/d12/d19/d55/d6f/l93 0 2024-04-06T03:40:37.949 INFO:tasks.workunit.client.1.smithi136.stdout:3/460: dread - d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 zero size 2024-04-06T03:40:37.949 INFO:tasks.workunit.client.0.smithi094.stdout:4/565: mkdir d0/df/d91/dbb 0 2024-04-06T03:40:37.949 INFO:tasks.workunit.client.0.smithi094.stdout:4/566: chown d0/d15/d3a/f43 4828 1 2024-04-06T03:40:37.949 INFO:tasks.workunit.client.0.smithi094.stdout:4/567: chown d0/d15/d3a/d61/d81/fba 3849 1 2024-04-06T03:40:37.951 INFO:tasks.workunit.client.1.smithi136.stdout:7/508: write d5/df/d11/f79 [1932210,16186] 0 2024-04-06T03:40:37.952 INFO:tasks.workunit.client.1.smithi136.stdout:4/433: creat d1d/f9d x:0 0 0 2024-04-06T03:40:37.952 INFO:tasks.workunit.client.1.smithi136.stdout:4/434: truncate d1d/d31/d3e/d56/f61 1459651 0 2024-04-06T03:40:37.952 INFO:tasks.workunit.client.1.smithi136.stdout:4/435: stat d1d/d2f/d6c/f76 0 2024-04-06T03:40:37.952 INFO:tasks.workunit.client.1.smithi136.stdout:4/436: creat d1d/d31/d8e/f9e x:0 0 0 2024-04-06T03:40:37.952 INFO:tasks.workunit.client.1.smithi136.stdout:4/437: readlink d1d/d31/l82 0 2024-04-06T03:40:37.955 INFO:tasks.workunit.client.1.smithi136.stdout:2/585: unlink d3/d4a/d9d/d18/d1d/d3c/d58/fa2 0 2024-04-06T03:40:37.955 INFO:tasks.workunit.client.1.smithi136.stdout:2/586: fsync d3/d4a/d9d/d18/d4b/f5d 0 2024-04-06T03:40:37.956 INFO:tasks.workunit.client.0.smithi094.stdout:7/468: symlink d2/d6/dc/d12/l8b 0 2024-04-06T03:40:37.956 INFO:tasks.workunit.client.1.smithi136.stdout:2/587: read d3/d4a/d9d/fb6 [97586,121821] 0 2024-04-06T03:40:37.957 INFO:tasks.workunit.client.0.smithi094.stdout:8/523: unlink d18/d2c/c43 0 2024-04-06T03:40:37.961 INFO:tasks.workunit.client.1.smithi136.stdout:7/509: getdents d5/df/d12/d1b/d3a/d4a/d9a 0 2024-04-06T03:40:37.961 INFO:tasks.workunit.client.1.smithi136.stdout:4/438: dread d1d/d31/f33 [0,4194304] 0 2024-04-06T03:40:37.961 INFO:tasks.workunit.client.1.smithi136.stdout:7/510: write d5/df/d12/f37 [2495843,80168] 0 2024-04-06T03:40:37.961 INFO:tasks.workunit.client.1.smithi136.stdout:7/511: truncate d5/df/d1d/d6f/d2c/fb1 35274 0 2024-04-06T03:40:37.961 INFO:tasks.workunit.client.1.smithi136.stdout:7/512: chown d5/df/d12/c18 280 1 2024-04-06T03:40:37.963 INFO:tasks.workunit.client.0.smithi094.stdout:3/526: rename d9/d10/d27 to d9/d1e/d6e/d75/dba 0 2024-04-06T03:40:37.964 INFO:tasks.workunit.client.1.smithi136.stdout:3/461: dwrite d0/d4/f18 [0,4194304] 0 2024-04-06T03:40:37.965 INFO:tasks.workunit.client.0.smithi094.stdout:8/524: read d18/d3c/d8b/f69 [1317156,50514] 0 2024-04-06T03:40:37.965 INFO:tasks.workunit.client.1.smithi136.stdout:7/513: fdatasync d5/df/d12/f37 0 2024-04-06T03:40:37.967 INFO:tasks.workunit.client.0.smithi094.stdout:9/447: creat d2/dc/d21/d26/d78/d69/d6d/fa7 x:0 0 0 2024-04-06T03:40:37.968 INFO:tasks.workunit.client.1.smithi136.stdout:2/588: mkdir d3/d4a/d9d/d18/d4b/dbd 0 2024-04-06T03:40:37.968 INFO:tasks.workunit.client.1.smithi136.stdout:2/589: fdatasync d3/d4a/d9d/d18/d4b/f9c 0 2024-04-06T03:40:37.968 INFO:tasks.workunit.client.1.smithi136.stdout:2/590: dread - d3/d15/d19/d43/f94 zero size 2024-04-06T03:40:37.968 INFO:tasks.workunit.client.1.smithi136.stdout:2/591: chown d3/d4a/d9d/d18/d1d/d3c/d58/l7c 2966076 1 2024-04-06T03:40:37.969 INFO:tasks.workunit.client.0.smithi094.stdout:1/481: symlink d0/d7/d1a/la7 0 2024-04-06T03:40:37.969 INFO:tasks.workunit.client.0.smithi094.stdout:1/482: write d0/d7/f90 [301824,90442] 0 2024-04-06T03:40:37.970 INFO:tasks.workunit.client.0.smithi094.stdout:1/483: write d0/d4/d1b/d2f/d76/d30/d38/f8d [741314,22900] 0 2024-04-06T03:40:37.971 INFO:tasks.workunit.client.1.smithi136.stdout:0/490: symlink d1/la6 0 2024-04-06T03:40:37.974 INFO:tasks.workunit.client.1.smithi136.stdout:9/470: creat d17/d21/d29/d34/d66/f9b x:0 0 0 2024-04-06T03:40:37.976 INFO:tasks.workunit.client.0.smithi094.stdout:7/469: mknod d2/d6/dc/d12/d29/c8c 0 2024-04-06T03:40:37.977 INFO:tasks.workunit.client.1.smithi136.stdout:4/439: chown d1d/f27 57 1 2024-04-06T03:40:37.980 INFO:tasks.workunit.client.0.smithi094.stdout:0/457: rename d1/d9/dd/d18/d5d/d60/d6b/f8a to d1/d9/dd/d18/d5d/d88/f99 0 2024-04-06T03:40:37.981 INFO:tasks.workunit.client.0.smithi094.stdout:1/484: dwrite d0/d4/d1b/d2f/f77 [0,4194304] 0 2024-04-06T03:40:37.981 INFO:tasks.workunit.client.0.smithi094.stdout:1/485: chown d0/d7/f90 1485 1 2024-04-06T03:40:37.982 INFO:tasks.workunit.client.1.smithi136.stdout:3/462: rename d0/d4/d12/d19/d2c/f40 to d0/d4/d12/d3a/f94 0 2024-04-06T03:40:37.983 INFO:tasks.workunit.client.0.smithi094.stdout:8/525: mkdir d18/d2c/d37/d60/da3/db8 0 2024-04-06T03:40:37.985 INFO:tasks.workunit.client.1.smithi136.stdout:7/514: mknod d5/df/d12/d1b/d58/cb4 0 2024-04-06T03:40:37.987 INFO:tasks.workunit.client.0.smithi094.stdout:9/448: rmdir d2/dc/d21 39 2024-04-06T03:40:37.988 INFO:tasks.workunit.client.1.smithi136.stdout:2/592: link d3/d4a/c14 d3/d15/d19/d43/daa/cbe 0 2024-04-06T03:40:37.992 INFO:tasks.workunit.client.0.smithi094.stdout:7/470: getdents d2/d6/dc/dd/d5a 0 2024-04-06T03:40:37.997 INFO:tasks.workunit.client.0.smithi094.stdout:1/486: dwrite d0/d1/d27/da5/d3f/d8c/f8e [0,4194304] 0 2024-04-06T03:40:37.997 INFO:tasks.workunit.client.0.smithi094.stdout:5/487: rename db/d19/d2c/d57/f58 to db/fa8 0 2024-04-06T03:40:37.997 INFO:tasks.workunit.client.0.smithi094.stdout:5/488: read - db/f76 zero size 2024-04-06T03:40:37.997 INFO:tasks.workunit.client.0.smithi094.stdout:1/487: chown d0/d4/d1b/d2f/d3c/d48/ca6 86 1 2024-04-06T03:40:37.997 INFO:tasks.workunit.client.0.smithi094.stdout:1/488: fsync d0/d1/d27/da5/f3b 0 2024-04-06T03:40:37.997 INFO:tasks.workunit.client.0.smithi094.stdout:3/527: write d9/d20/f3d [1094313,53230] 0 2024-04-06T03:40:37.998 INFO:tasks.workunit.client.0.smithi094.stdout:0/458: creat d1/d9/dd/d18/d1e/d86/d96/f9a x:0 0 0 2024-04-06T03:40:38.000 INFO:tasks.workunit.client.0.smithi094.stdout:8/526: mkdir d18/d34/d96/db9 0 2024-04-06T03:40:38.000 INFO:tasks.workunit.client.0.smithi094.stdout:8/527: write d18/d5c/f8d [4781700,34209] 0 2024-04-06T03:40:38.001 INFO:tasks.workunit.client.1.smithi136.stdout:0/491: mkdir d1/d6/d2c/d57/d43/d6d/d85/da7 0 2024-04-06T03:40:38.004 INFO:tasks.workunit.client.1.smithi136.stdout:9/471: creat d17/d20/d28/f9c x:0 0 0 2024-04-06T03:40:38.007 INFO:tasks.workunit.client.0.smithi094.stdout:2/565: rename d0/d8/d1d/d77/d87 to d0/dd/d19/dbe 0 2024-04-06T03:40:38.007 INFO:tasks.workunit.client.0.smithi094.stdout:2/566: stat d0/d8/da0 0 2024-04-06T03:40:38.007 INFO:tasks.workunit.client.0.smithi094.stdout:5/489: write db/d40/f55 [3469022,99199] 0 2024-04-06T03:40:38.008 INFO:tasks.workunit.client.1.smithi136.stdout:4/440: mknod d1d/d31/d3e/d55/d89/c9f 0 2024-04-06T03:40:38.010 INFO:tasks.workunit.client.0.smithi094.stdout:3/528: mkdir d9/d10/dbb 0 2024-04-06T03:40:38.010 INFO:tasks.workunit.client.0.smithi094.stdout:7/471: dwrite d2/f11 [0,4194304] 0 2024-04-06T03:40:38.011 INFO:tasks.workunit.client.1.smithi136.stdout:3/463: rename d0/d4/d12/d19/d55/d6f/d82/c7d to d0/d4/d12/d3a/d51/d8e/c95 0 2024-04-06T03:40:38.012 INFO:tasks.workunit.client.1.smithi136.stdout:3/464: creat d0/d4/d12/d2d/f96 x:0 0 0 2024-04-06T03:40:38.012 INFO:tasks.workunit.client.1.smithi136.stdout:3/465: write d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 [974333,129253] 0 2024-04-06T03:40:38.012 INFO:tasks.workunit.client.1.smithi136.stdout:3/466: write d0/d4/d12/d19/f50 [1243430,86134] 0 2024-04-06T03:40:38.013 INFO:tasks.workunit.client.0.smithi094.stdout:8/528: link d18/d2c/d37/d60/da3/c9d d18/d34/d52/cba 0 2024-04-06T03:40:38.014 INFO:tasks.workunit.client.1.smithi136.stdout:7/515: dread d5/df/d11/f38 [0,4194304] 0 2024-04-06T03:40:38.014 INFO:tasks.workunit.client.1.smithi136.stdout:7/516: fsync d5/df/d1d/d6f/d2a/f48 0 2024-04-06T03:40:38.015 INFO:tasks.workunit.client.1.smithi136.stdout:2/593: symlink d3/d15/d19/d43/daa/lbf 0 2024-04-06T03:40:38.017 INFO:tasks.workunit.client.1.smithi136.stdout:1/476: sync 2024-04-06T03:40:38.018 INFO:tasks.workunit.client.1.smithi136.stdout:7/517: dread d5/df/d1d/d6f/d2a/f48 [0,4194304] 0 2024-04-06T03:40:38.018 INFO:tasks.workunit.client.1.smithi136.stdout:7/518: truncate d5/df/d12/d1b/d3a/fa6 734899 0 2024-04-06T03:40:38.018 INFO:tasks.workunit.client.1.smithi136.stdout:7/519: fsync d5/df/d12/d1b/d58/f7a 0 2024-04-06T03:40:38.019 INFO:tasks.workunit.client.0.smithi094.stdout:3/529: mknod d9/d1e/d26/d6c/d8e/cbc 0 2024-04-06T03:40:38.019 INFO:tasks.workunit.client.0.smithi094.stdout:3/530: chown d9/d1e/d26/d55/f93 240 1 2024-04-06T03:40:38.020 INFO:tasks.workunit.client.0.smithi094.stdout:7/472: dread d2/d6/dc/d12/d29/d35/f54 [0,4194304] 0 2024-04-06T03:40:38.020 INFO:tasks.workunit.client.0.smithi094.stdout:7/473: readlink d2/d6/l40 0 2024-04-06T03:40:38.020 INFO:tasks.workunit.client.1.smithi136.stdout:9/472: creat d17/d20/d28/d49/d75/f9d x:0 0 0 2024-04-06T03:40:38.021 INFO:tasks.workunit.client.1.smithi136.stdout:1/477: dread d12/d3b/f4c [0,4194304] 0 2024-04-06T03:40:38.021 INFO:tasks.workunit.client.1.smithi136.stdout:9/473: dread d17/d20/d25/d3d/d4b/f80 [0,4194304] 0 2024-04-06T03:40:38.021 INFO:tasks.workunit.client.1.smithi136.stdout:4/441: mkdir d1d/d31/d3e/d55/da0 0 2024-04-06T03:40:38.023 INFO:tasks.workunit.client.0.smithi094.stdout:2/567: dread d0/dd/d1b/f1c [0,4194304] 0 2024-04-06T03:40:38.026 INFO:tasks.workunit.client.0.smithi094.stdout:5/490: dwrite db/f11 [0,4194304] 0 2024-04-06T03:40:38.026 INFO:tasks.workunit.client.0.smithi094.stdout:5/491: creat db/d40/d77/fa9 x:0 0 0 2024-04-06T03:40:38.027 INFO:tasks.workunit.client.1.smithi136.stdout:3/467: mkdir d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97 0 2024-04-06T03:40:38.027 INFO:tasks.workunit.client.1.smithi136.stdout:2/594: dread d3/d4a/d9d/d18/f23 [0,4194304] 0 2024-04-06T03:40:38.034 INFO:tasks.workunit.client.0.smithi094.stdout:8/529: symlink d18/d3c/d49/d91/d94/da8/lbb 0 2024-04-06T03:40:38.034 INFO:tasks.workunit.client.0.smithi094.stdout:8/530: write d18/f21 [174194,74877] 0 2024-04-06T03:40:38.036 INFO:tasks.workunit.client.0.smithi094.stdout:9/449: rename d2/d2c/d33/d37 to d2/dc/d21/d26/d78/d69/da8 0 2024-04-06T03:40:38.036 INFO:tasks.workunit.client.0.smithi094.stdout:9/450: dread - d2/dc/d21/d23/d47/f9b zero size 2024-04-06T03:40:38.036 INFO:tasks.workunit.client.1.smithi136.stdout:5/552: sync 2024-04-06T03:40:38.037 INFO:tasks.workunit.client.0.smithi094.stdout:3/531: mknod d9/d20/d2d/da9/cbd 0 2024-04-06T03:40:38.037 INFO:tasks.workunit.client.1.smithi136.stdout:9/474: dwrite d17/d21/f7c [0,4194304] 0 2024-04-06T03:40:38.038 INFO:tasks.workunit.client.1.smithi136.stdout:7/520: rename d5/df/d12/d59/d71/lac to d5/df/d12/d59/lb5 0 2024-04-06T03:40:38.038 INFO:tasks.workunit.client.1.smithi136.stdout:7/521: write d5/df/d12/d59/f5e [4354240,90127] 0 2024-04-06T03:40:38.039 INFO:tasks.workunit.client.1.smithi136.stdout:9/475: dread f11 [0,4194304] 0 2024-04-06T03:40:38.040 INFO:tasks.workunit.client.0.smithi094.stdout:7/474: creat d2/d6/dc/d12/d29/d35/f8d x:0 0 0 2024-04-06T03:40:38.040 INFO:tasks.workunit.client.0.smithi094.stdout:7/475: chown d2/d6/dc/dd/df/f2d 8 1 2024-04-06T03:40:38.040 INFO:tasks.workunit.client.0.smithi094.stdout:3/532: dread d9/d1e/d26/f28 [0,4194304] 0 2024-04-06T03:40:38.041 INFO:tasks.workunit.client.0.smithi094.stdout:2/568: stat d0/dd/da4/c8a 0 2024-04-06T03:40:38.043 INFO:tasks.workunit.client.1.smithi136.stdout:1/478: symlink d12/d3b/l9b 0 2024-04-06T03:40:38.044 INFO:tasks.workunit.client.0.smithi094.stdout:5/492: mknod db/d19/d30/d83/caa 0 2024-04-06T03:40:38.044 INFO:tasks.workunit.client.1.smithi136.stdout:4/442: symlink d1d/d31/d3e/d55/da0/la1 0 2024-04-06T03:40:38.045 INFO:tasks.workunit.client.1.smithi136.stdout:6/433: sync 2024-04-06T03:40:38.045 INFO:tasks.workunit.client.1.smithi136.stdout:8/555: sync 2024-04-06T03:40:38.046 INFO:tasks.workunit.client.1.smithi136.stdout:3/468: mkdir d0/d4/d12/d19/d55/d3c/d5a/d79/d98 0 2024-04-06T03:40:38.047 INFO:tasks.workunit.client.0.smithi094.stdout:9/451: symlink d2/d2c/d33/d64/la9 0 2024-04-06T03:40:38.049 INFO:tasks.workunit.client.0.smithi094.stdout:7/476: unlink d2/d6/dc/d12/d29/c85 0 2024-04-06T03:40:38.049 INFO:tasks.workunit.client.1.smithi136.stdout:2/595: truncate d3/d4a/d9d/d18/d1d/d3c/f40 187729 0 2024-04-06T03:40:38.049 INFO:tasks.workunit.client.1.smithi136.stdout:2/596: chown d3/d15/cbb 5878 1 2024-04-06T03:40:38.049 INFO:tasks.workunit.client.1.smithi136.stdout:5/553: mknod d1/d16/cad 0 2024-04-06T03:40:38.053 INFO:tasks.workunit.client.0.smithi094.stdout:3/533: dread d9/f19 [0,4194304] 0 2024-04-06T03:40:38.053 INFO:tasks.workunit.client.0.smithi094.stdout:3/534: write d9/d1e/d6e/d75/dba/d7c/faa [532490,20366] 0 2024-04-06T03:40:38.053 INFO:tasks.workunit.client.0.smithi094.stdout:3/535: truncate d9/d1e/d6e/d75/dba/f38 4258044 0 2024-04-06T03:40:38.055 INFO:tasks.workunit.client.0.smithi094.stdout:5/493: symlink db/d19/d24/d28/d85/lab 0 2024-04-06T03:40:38.055 INFO:tasks.workunit.client.0.smithi094.stdout:5/494: stat db/d19/d2c/d57/d8f/f90 0 2024-04-06T03:40:38.056 INFO:tasks.workunit.client.0.smithi094.stdout:5/495: dread db/d19/d24/d28/f8c [0,4194304] 0 2024-04-06T03:40:38.061 INFO:tasks.workunit.client.0.smithi094.stdout:1/489: rename d0/d4/d1b/d2f/d4b/d95 to d0/d7/d1a/da8 0 2024-04-06T03:40:38.062 INFO:tasks.workunit.client.0.smithi094.stdout:1/490: chown d0/d7/d2d/l44 232323 1 2024-04-06T03:40:38.062 INFO:tasks.workunit.client.0.smithi094.stdout:1/491: fdatasync d0/d1/d8f/f96 0 2024-04-06T03:40:38.062 INFO:tasks.workunit.client.0.smithi094.stdout:0/459: rename d1/d9/dd/d18/d1a to d1/d9/dd/d18/d1a/d9b 22 2024-04-06T03:40:38.062 INFO:tasks.workunit.client.0.smithi094.stdout:0/460: readlink d1/d9/dd/d18/d1d/l6e 0 2024-04-06T03:40:38.062 INFO:tasks.workunit.client.1.smithi136.stdout:7/522: creat d5/df/d12/d59/d71/fb6 x:0 0 0 2024-04-06T03:40:38.063 INFO:tasks.workunit.client.0.smithi094.stdout:9/452: symlink d2/dc/d21/d26/d8a/laa 0 2024-04-06T03:40:38.063 INFO:tasks.workunit.client.0.smithi094.stdout:9/453: read - d2/dc/d21/fa4 zero size 2024-04-06T03:40:38.063 INFO:tasks.workunit.client.0.smithi094.stdout:9/454: write d2/dc/d21/f9c [449916,32183] 0 2024-04-06T03:40:38.063 INFO:tasks.workunit.client.1.smithi136.stdout:9/476: mkdir d17/d20/d25/d3b/d9e 0 2024-04-06T03:40:38.063 INFO:tasks.workunit.client.1.smithi136.stdout:9/477: dread - d17/d20/d28/f98 zero size 2024-04-06T03:40:38.065 INFO:tasks.workunit.client.0.smithi094.stdout:7/477: dwrite d2/d6/dc/d12/f2a [0,4194304] 0 2024-04-06T03:40:38.065 INFO:tasks.workunit.client.0.smithi094.stdout:7/478: stat d2/d6/dc/dd/d5a/c7f 0 2024-04-06T03:40:38.066 INFO:tasks.workunit.client.0.smithi094.stdout:7/479: write d2/d6/dc/d12/f5f [368144,29734] 0 2024-04-06T03:40:38.066 INFO:tasks.workunit.client.0.smithi094.stdout:7/480: chown d2/d6/dc/dd/df/d36/f60 245375102 1 2024-04-06T03:40:38.066 INFO:tasks.workunit.client.1.smithi136.stdout:0/492: link d1/d6/d2c/l6f d1/d6/d2c/la8 0 2024-04-06T03:40:38.067 INFO:tasks.workunit.client.1.smithi136.stdout:6/434: dwrite d2/d7/d26/d30/d6e/f5e [0,4194304] 0 2024-04-06T03:40:38.068 INFO:tasks.workunit.client.0.smithi094.stdout:3/536: creat d9/d1d/d7d/fbe x:0 0 0 2024-04-06T03:40:38.071 INFO:tasks.workunit.client.1.smithi136.stdout:1/479: unlink d12/d1b/d28/c47 0 2024-04-06T03:40:38.071 INFO:tasks.workunit.client.1.smithi136.stdout:1/480: chown d12/d1b/d28/f91 91103 1 2024-04-06T03:40:38.071 INFO:tasks.workunit.client.1.smithi136.stdout:1/481: truncate d12/d1b/d28/f91 807563 0 2024-04-06T03:40:38.073 INFO:tasks.workunit.client.0.smithi094.stdout:5/496: symlink db/d19/d24/d2e/lac 0 2024-04-06T03:40:38.073 INFO:tasks.workunit.client.1.smithi136.stdout:9/478: write d17/d20/d28/d49/d75/d60/f7a [56555,61473] 0 2024-04-06T03:40:38.075 INFO:tasks.workunit.client.1.smithi136.stdout:4/443: creat d1d/d31/fa2 x:0 0 0 2024-04-06T03:40:38.075 INFO:tasks.workunit.client.1.smithi136.stdout:4/444: dread - d1d/d31/fa2 zero size 2024-04-06T03:40:38.075 INFO:tasks.workunit.client.1.smithi136.stdout:4/445: creat d1d/d2f/d6c/fa3 x:0 0 0 2024-04-06T03:40:38.075 INFO:tasks.workunit.client.1.smithi136.stdout:0/493: dread d1/d6/d2c/d57/d43/f50 [0,4194304] 0 2024-04-06T03:40:38.075 INFO:tasks.workunit.client.0.smithi094.stdout:0/461: getdents d1/d9/dd/d18/d78 0 2024-04-06T03:40:38.076 INFO:tasks.workunit.client.0.smithi094.stdout:2/569: write d0/d8/f33 [954406,40795] 0 2024-04-06T03:40:38.076 INFO:tasks.workunit.client.1.smithi136.stdout:8/556: getdents d3/da/d26/d35/d3d/d48/d6c/d7f 0 2024-04-06T03:40:38.079 INFO:tasks.workunit.client.1.smithi136.stdout:3/469: mknod d0/d4/d12/d19/d55/d6f/d82/d5d/c99 0 2024-04-06T03:40:38.079 INFO:tasks.workunit.client.1.smithi136.stdout:3/470: creat d0/d4/d12/d19/d55/d3c/d5a/f9a x:0 0 0 2024-04-06T03:40:38.079 INFO:tasks.workunit.client.0.smithi094.stdout:9/455: creat d2/dc/d21/d26/d78/d69/da8/d96/fab x:0 0 0 2024-04-06T03:40:38.080 INFO:tasks.workunit.client.1.smithi136.stdout:2/597: truncate d3/f83 507397 0 2024-04-06T03:40:38.080 INFO:tasks.workunit.client.1.smithi136.stdout:2/598: stat d3/d9/d3b/d70 0 2024-04-06T03:40:38.080 INFO:tasks.workunit.client.1.smithi136.stdout:2/599: dread - d3/d9/d3b/d87/f88 zero size 2024-04-06T03:40:38.081 INFO:tasks.workunit.client.0.smithi094.stdout:1/492: dwrite d0/d4/d1b/d2f/d3c/d48/f89 [0,4194304] 0 2024-04-06T03:40:38.081 INFO:tasks.workunit.client.0.smithi094.stdout:1/493: creat d0/d1/d27/fa9 x:0 0 0 2024-04-06T03:40:38.081 INFO:tasks.workunit.client.0.smithi094.stdout:1/494: chown d0/d1/d27/da5/f3a 5368 1 2024-04-06T03:40:38.081 INFO:tasks.workunit.client.1.smithi136.stdout:7/523: dwrite d5/df/d11/f14 [0,4194304] 0 2024-04-06T03:40:38.082 INFO:tasks.workunit.client.1.smithi136.stdout:5/554: mknod d1/d9b/cae 0 2024-04-06T03:40:38.084 INFO:tasks.workunit.client.0.smithi094.stdout:7/481: unlink d2/d6/dc/d12/d3a/d56/l77 0 2024-04-06T03:40:38.086 INFO:tasks.workunit.client.0.smithi094.stdout:3/537: mknod d9/d1e/d26/d55/cbf 0 2024-04-06T03:40:38.087 INFO:tasks.workunit.client.1.smithi136.stdout:4/446: dread d1d/d31/f41 [0,4194304] 0 2024-04-06T03:40:38.087 INFO:tasks.workunit.client.0.smithi094.stdout:0/462: dread d1/d10/f73 [0,4194304] 0 2024-04-06T03:40:38.090 INFO:tasks.workunit.client.1.smithi136.stdout:6/435: mkdir d2/d93 0 2024-04-06T03:40:38.091 INFO:tasks.workunit.client.0.smithi094.stdout:5/497: symlink db/d7c/d98/lad 0 2024-04-06T03:40:38.091 INFO:tasks.workunit.client.0.smithi094.stdout:5/498: fdatasync db/f11 0 2024-04-06T03:40:38.091 INFO:tasks.workunit.client.0.smithi094.stdout:5/499: creat db/fae x:0 0 0 2024-04-06T03:40:38.093 INFO:tasks.workunit.client.1.smithi136.stdout:1/482: truncate d12/d1b/d22/f43 359981 0 2024-04-06T03:40:38.097 INFO:tasks.workunit.client.1.smithi136.stdout:9/479: unlink d17/d20/d25/d3c/l93 0 2024-04-06T03:40:38.100 INFO:tasks.workunit.client.0.smithi094.stdout:9/456: creat d2/dc/d80/fac x:0 0 0 2024-04-06T03:40:38.105 INFO:tasks.workunit.client.1.smithi136.stdout:0/494: rename d1/d6/l16 to d1/d6/d2c/d1a/d36/d59/la9 0 2024-04-06T03:40:38.110 INFO:tasks.workunit.client.1.smithi136.stdout:8/557: creat d3/da/d26/d35/d53/fac x:0 0 0 2024-04-06T03:40:38.113 INFO:tasks.workunit.client.0.smithi094.stdout:5/500: write db/f6f [88361,84708] 0 2024-04-06T03:40:38.114 INFO:tasks.workunit.client.0.smithi094.stdout:7/482: symlink d2/d6/dc/dd/df/l8e 0 2024-04-06T03:40:38.114 INFO:tasks.workunit.client.0.smithi094.stdout:2/570: dwrite d0/dd/d19/d31/f8c [4194304,4194304] 0 2024-04-06T03:40:38.115 INFO:tasks.workunit.client.1.smithi136.stdout:3/471: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/f9b x:0 0 0 2024-04-06T03:40:38.117 INFO:tasks.workunit.client.0.smithi094.stdout:3/538: mkdir d9/d10/d9b/db7/dc0 0 2024-04-06T03:40:38.117 INFO:tasks.workunit.client.1.smithi136.stdout:8/558: dread d3/de/f56 [0,4194304] 0 2024-04-06T03:40:38.117 INFO:tasks.workunit.client.1.smithi136.stdout:8/559: stat d3/de/d18/f1e 0 2024-04-06T03:40:38.120 INFO:tasks.workunit.client.1.smithi136.stdout:4/447: dwrite d1d/d2f/f7d [0,4194304] 0 2024-04-06T03:40:38.120 INFO:tasks.workunit.client.1.smithi136.stdout:2/600: dwrite d3/d15/d19/f1b [0,4194304] 0 2024-04-06T03:40:38.120 INFO:tasks.workunit.client.1.smithi136.stdout:7/524: creat d5/df/fb7 x:0 0 0 2024-04-06T03:40:38.122 INFO:tasks.workunit.client.0.smithi094.stdout:8/531: rename d18/d5c/d75 to d18/d34/d96/dbc 0 2024-04-06T03:40:38.122 INFO:tasks.workunit.client.0.smithi094.stdout:8/532: truncate d18/d34/d96/dbc/f93 5209280 0 2024-04-06T03:40:38.122 INFO:tasks.workunit.client.0.smithi094.stdout:0/463: dwrite d1/d9/dd/d18/d1e/d22/f97 [0,4194304] 0 2024-04-06T03:40:38.124 INFO:tasks.workunit.client.1.smithi136.stdout:5/555: truncate d1/dc/d1c/d32/f38 3519266 0 2024-04-06T03:40:38.125 INFO:tasks.workunit.client.1.smithi136.stdout:2/601: read d3/d4a/d9d/d18/d4b/f65 [434437,114445] 0 2024-04-06T03:40:38.126 INFO:tasks.workunit.client.1.smithi136.stdout:3/472: read d0/fb [3840605,66887] 0 2024-04-06T03:40:38.127 INFO:tasks.workunit.client.1.smithi136.stdout:6/436: rmdir d2/d7/d13/d79 39 2024-04-06T03:40:38.127 INFO:tasks.workunit.client.1.smithi136.stdout:6/437: readlink d2/d7/l73 0 2024-04-06T03:40:38.132 INFO:tasks.workunit.client.0.smithi094.stdout:5/501: mknod db/d19/d30/caf 0 2024-04-06T03:40:38.134 INFO:tasks.workunit.client.0.smithi094.stdout:7/483: dwrite d2/d6/dc/dd/d5a/f70 [0,4194304] 0 2024-04-06T03:40:38.134 INFO:tasks.workunit.client.0.smithi094.stdout:7/484: write d2/d6/dc/dd/f31 [51581,10188] 0 2024-04-06T03:40:38.134 INFO:tasks.workunit.client.1.smithi136.stdout:1/483: symlink d12/d1b/d1e/l9c 0 2024-04-06T03:40:38.139 INFO:tasks.workunit.client.1.smithi136.stdout:9/480: creat d17/d21/d32/d43/f9f x:0 0 0 2024-04-06T03:40:38.143 INFO:tasks.workunit.client.0.smithi094.stdout:3/539: stat d9/d1e/d6e/d75/dba/d3a/cad 0 2024-04-06T03:40:38.144 INFO:tasks.workunit.client.0.smithi094.stdout:3/540: write d9/d1d/f39 [182236,100788] 0 2024-04-06T03:40:38.144 INFO:tasks.workunit.client.0.smithi094.stdout:7/485: dread d2/d6/dc/f15 [0,4194304] 0 2024-04-06T03:40:38.144 INFO:tasks.workunit.client.1.smithi136.stdout:1/484: dread d12/d3b/f4c [0,4194304] 0 2024-04-06T03:40:38.144 INFO:tasks.workunit.client.1.smithi136.stdout:1/485: truncate d12/d3b/d74/d7a/f8d 67630 0 2024-04-06T03:40:38.145 INFO:tasks.workunit.client.0.smithi094.stdout:5/502: dread db/d19/d2c/f2d [0,4194304] 0 2024-04-06T03:40:38.145 INFO:tasks.workunit.client.0.smithi094.stdout:5/503: chown db/d19/d24/f34 364544 1 2024-04-06T03:40:38.147 INFO:tasks.workunit.client.1.smithi136.stdout:3/473: dwrite d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 [0,4194304] 0 2024-04-06T03:40:38.147 INFO:tasks.workunit.client.1.smithi136.stdout:3/474: readlink d0/d4/d12/d3a/d51/l63 0 2024-04-06T03:40:38.147 INFO:tasks.workunit.client.1.smithi136.stdout:3/475: creat d0/d4/d12/d19/d2c/f9c x:0 0 0 2024-04-06T03:40:38.147 INFO:tasks.workunit.client.1.smithi136.stdout:3/476: creat d0/d4/d12/d2d/d37/f9d x:0 0 0 2024-04-06T03:40:38.150 INFO:tasks.workunit.client.1.smithi136.stdout:6/438: dwrite d2/d7/f31 [0,4194304] 0 2024-04-06T03:40:38.152 INFO:tasks.workunit.client.1.smithi136.stdout:4/448: rmdir d1d/d31/d3e/d55/da0 39 2024-04-06T03:40:38.152 INFO:tasks.workunit.client.1.smithi136.stdout:4/449: creat d1d/d2f/d6c/fa4 x:0 0 0 2024-04-06T03:40:38.152 INFO:tasks.workunit.client.1.smithi136.stdout:4/450: write d1d/d31/f41 [4838637,43168] 0 2024-04-06T03:40:38.152 INFO:tasks.workunit.client.0.smithi094.stdout:7/486: dread d2/d6/f22 [0,4194304] 0 2024-04-06T03:40:38.153 INFO:tasks.workunit.client.0.smithi094.stdout:7/487: chown d2/d6/dc/dd/df/d36 6981973 1 2024-04-06T03:40:38.154 INFO:tasks.workunit.client.1.smithi136.stdout:4/451: fdatasync d1d/d31/f33 0 2024-04-06T03:40:38.155 INFO:tasks.workunit.client.0.smithi094.stdout:1/495: rename d0/d1/d27/da5/f1c to d0/d4/d1b/faa 0 2024-04-06T03:40:38.158 INFO:tasks.workunit.client.1.smithi136.stdout:7/525: unlink d5/df/d1d/d6f/d4d/c4f 0 2024-04-06T03:40:38.158 INFO:tasks.workunit.client.1.smithi136.stdout:4/452: dread d1d/d31/f58 [0,4194304] 0 2024-04-06T03:40:38.159 INFO:tasks.workunit.client.0.smithi094.stdout:8/533: symlink d18/d2c/d37/d60/lbd 0 2024-04-06T03:40:38.162 INFO:tasks.workunit.client.1.smithi136.stdout:5/556: truncate d1/d16/d4b/f76 1976302 0 2024-04-06T03:40:38.162 INFO:tasks.workunit.client.1.smithi136.stdout:5/557: chown d1/dc/d1c/d32/d8b/d20/d66/l81 47 1 2024-04-06T03:40:38.162 INFO:tasks.workunit.client.1.smithi136.stdout:5/558: chown d1/d16/ca1 18056277 1 2024-04-06T03:40:38.165 INFO:tasks.workunit.client.0.smithi094.stdout:7/488: dwrite d2/d6/dc/d12/d61/f83 [0,4194304] 0 2024-04-06T03:40:38.166 INFO:tasks.workunit.client.0.smithi094.stdout:0/464: unlink d1/d9/dd/d18/d5d/f65 0 2024-04-06T03:40:38.166 INFO:tasks.workunit.client.0.smithi094.stdout:0/465: creat d1/d9/dd/d18/d32/f9c x:0 0 0 2024-04-06T03:40:38.171 INFO:tasks.workunit.client.0.smithi094.stdout:8/534: write f17 [1874628,119234] 0 2024-04-06T03:40:38.171 INFO:tasks.workunit.client.0.smithi094.stdout:8/535: fdatasync d18/d2c/f2e 0 2024-04-06T03:40:38.171 INFO:tasks.workunit.client.0.smithi094.stdout:8/536: readlink d18/d3c/d49/l5e 0 2024-04-06T03:40:38.171 INFO:tasks.workunit.client.0.smithi094.stdout:8/537: write d18/d2c/d62/f72 [1269307,80713] 0 2024-04-06T03:40:38.171 INFO:tasks.workunit.client.0.smithi094.stdout:8/538: fsync d18/d2c/d62/f6d 0 2024-04-06T03:40:38.174 INFO:tasks.workunit.client.0.smithi094.stdout:8/539: dread d18/f27 [0,4194304] 0 2024-04-06T03:40:38.178 INFO:tasks.workunit.client.1.smithi136.stdout:2/602: mkdir d3/d4a/d9d/d18/d4b/dc0 0 2024-04-06T03:40:38.178 INFO:tasks.workunit.client.1.smithi136.stdout:2/603: read d3/d15/f2b [2013507,104818] 0 2024-04-06T03:40:38.178 INFO:tasks.workunit.client.1.smithi136.stdout:2/604: truncate d3/d15/d19/d43/d54/f84 1356736 0 2024-04-06T03:40:38.182 INFO:tasks.workunit.client.1.smithi136.stdout:1/486: link d12/d3b/d44/d5f/d5a/f62 d12/d1b/f9d 0 2024-04-06T03:40:38.183 INFO:tasks.workunit.client.0.smithi094.stdout:0/466: dwrite d1/d9/dd/d18/d1d/f8e [0,4194304] 0 2024-04-06T03:40:38.183 INFO:tasks.workunit.client.0.smithi094.stdout:0/467: fdatasync d1/d9/dd/f1c 0 2024-04-06T03:40:38.184 INFO:tasks.workunit.client.1.smithi136.stdout:0/495: link d1/d6/d2c/f3d d1/d6/d2c/d57/d43/faa 0 2024-04-06T03:40:38.185 INFO:tasks.workunit.client.1.smithi136.stdout:0/496: dread d1/d6/d2c/d57/d43/d6b/f80 [0,4194304] 0 2024-04-06T03:40:38.187 INFO:tasks.workunit.client.0.smithi094.stdout:7/489: dwrite d2/d6/dc/dd/f3b [0,4194304] 0 2024-04-06T03:40:38.187 INFO:tasks.workunit.client.0.smithi094.stdout:8/540: dread d18/d2c/d62/f72 [0,4194304] 0 2024-04-06T03:40:38.188 INFO:tasks.workunit.client.0.smithi094.stdout:3/541: mkdir d9/d9d/dc1 0 2024-04-06T03:40:38.190 INFO:tasks.workunit.client.0.smithi094.stdout:5/504: mknod db/d19/d30/d83/cb0 0 2024-04-06T03:40:38.190 INFO:tasks.workunit.client.0.smithi094.stdout:5/505: creat db/d40/d77/fb1 x:0 0 0 2024-04-06T03:40:38.193 INFO:tasks.workunit.client.0.smithi094.stdout:2/571: rename d0/dd/da4/l44 to d0/dd/d19/d31/d67/lbf 0 2024-04-06T03:40:38.194 INFO:tasks.workunit.client.1.smithi136.stdout:9/481: dread fd [0,4194304] 0 2024-04-06T03:40:38.196 INFO:tasks.workunit.client.0.smithi094.stdout:1/496: mkdir d0/d7/d2d/dab 0 2024-04-06T03:40:38.199 INFO:tasks.workunit.client.1.smithi136.stdout:3/477: link d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/f9b d0/d4/d12/d19/d55/d6f/f9e 0 2024-04-06T03:40:38.199 INFO:tasks.workunit.client.0.smithi094.stdout:1/497: dread d0/d7/f86 [0,4194304] 0 2024-04-06T03:40:38.200 INFO:tasks.workunit.client.1.smithi136.stdout:0/497: dwrite d1/d6/d2c/d1a/d36/d59/f48 [0,4194304] 0 2024-04-06T03:40:38.203 INFO:tasks.workunit.client.0.smithi094.stdout:4/568: sync 2024-04-06T03:40:38.203 INFO:tasks.workunit.client.0.smithi094.stdout:6/452: sync 2024-04-06T03:40:38.209 INFO:tasks.workunit.client.1.smithi136.stdout:6/439: mkdir d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/d94 0 2024-04-06T03:40:38.213 INFO:tasks.workunit.client.0.smithi094.stdout:1/498: dwrite d0/d4/d1b/d2f/d3c/d48/f88 [0,4194304] 0 2024-04-06T03:40:38.214 INFO:tasks.workunit.client.0.smithi094.stdout:1/499: fdatasync d0/d7/f52 0 2024-04-06T03:40:38.214 INFO:tasks.workunit.client.1.smithi136.stdout:7/526: symlink d5/df/d53/d8a/lb8 0 2024-04-06T03:40:38.216 INFO:tasks.workunit.client.0.smithi094.stdout:7/490: dwrite d2/d6/dc/f2b [0,4194304] 0 2024-04-06T03:40:38.216 INFO:tasks.workunit.client.1.smithi136.stdout:4/453: mkdir d1d/d2f/d6c/da5 0 2024-04-06T03:40:38.216 INFO:tasks.workunit.client.1.smithi136.stdout:4/454: write d1d/d2f/d6c/fa3 [898761,13631] 0 2024-04-06T03:40:38.216 INFO:tasks.workunit.client.1.smithi136.stdout:4/455: write d1d/d31/d3e/d56/f5e [1868748,130014] 0 2024-04-06T03:40:38.218 INFO:tasks.workunit.client.1.smithi136.stdout:0/498: write d1/d6/d2c/d1a/f1d [1652209,129423] 0 2024-04-06T03:40:38.218 INFO:tasks.workunit.client.1.smithi136.stdout:0/499: dread - d1/d6/d2c/d1a/fa0 zero size 2024-04-06T03:40:38.218 INFO:tasks.workunit.client.1.smithi136.stdout:0/500: chown d1/d6/fa1 22 1 2024-04-06T03:40:38.219 INFO:tasks.workunit.client.1.smithi136.stdout:3/478: dwrite d0/d4/d12/d19/f29 [0,4194304] 0 2024-04-06T03:40:38.219 INFO:tasks.workunit.client.1.smithi136.stdout:3/479: chown d0/d4/d12/d19/d55/d6f/d82/d5d/d61/f8f 806 1 2024-04-06T03:40:38.219 INFO:tasks.workunit.client.1.smithi136.stdout:3/480: fsync d0/d4/d12/d3a/f3e 0 2024-04-06T03:40:38.221 INFO:tasks.workunit.client.1.smithi136.stdout:6/440: dwrite d2/f3c [0,4194304] 0 2024-04-06T03:40:38.225 INFO:tasks.workunit.client.0.smithi094.stdout:7/491: dread d2/d6/dc/d12/f2a [0,4194304] 0 2024-04-06T03:40:38.228 INFO:tasks.workunit.client.0.smithi094.stdout:0/468: symlink d1/d9/dd/d18/d5d/d8d/l9d 0 2024-04-06T03:40:38.228 INFO:tasks.workunit.client.0.smithi094.stdout:0/469: creat d1/d10/d4e/f9e x:0 0 0 2024-04-06T03:40:38.233 INFO:tasks.workunit.client.1.smithi136.stdout:2/605: symlink d3/d4a/d9d/d18/d1d/d3c/d58/lc1 0 2024-04-06T03:40:38.234 INFO:tasks.workunit.client.0.smithi094.stdout:8/541: mknod d18/d30/cbe 0 2024-04-06T03:40:38.238 INFO:tasks.workunit.client.1.smithi136.stdout:1/487: unlink d12/d1b/c21 0 2024-04-06T03:40:38.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/506: mknod db/cb2 0 2024-04-06T03:40:38.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/507: dread - db/d7c/d98/f9f zero size 2024-04-06T03:40:38.240 INFO:tasks.workunit.client.1.smithi136.stdout:6/441: dwrite d2/d7/d26/d30/d4f/f5f [0,4194304] 0 2024-04-06T03:40:38.241 INFO:tasks.workunit.client.1.smithi136.stdout:9/482: mkdir d17/d20/da0 0 2024-04-06T03:40:38.241 INFO:tasks.workunit.client.1.smithi136.stdout:1/488: dread d12/d1b/d1e/d4d/f51 [0,4194304] 0 2024-04-06T03:40:38.241 INFO:tasks.workunit.client.1.smithi136.stdout:1/489: readlink d12/d1b/d1e/l36 0 2024-04-06T03:40:38.242 INFO:tasks.workunit.client.1.smithi136.stdout:9/483: dread d17/d21/d29/f99 [0,4194304] 0 2024-04-06T03:40:38.243 INFO:tasks.workunit.client.0.smithi094.stdout:3/542: rename d9/d20/d76/l6d to d9/d1e/d6e/d75/dba/d97/lc2 0 2024-04-06T03:40:38.243 INFO:tasks.workunit.client.0.smithi094.stdout:4/569: rename d0/d15/d3a/d3e to d0/d15/d3a/d3e/dbc 22 2024-04-06T03:40:38.243 INFO:tasks.workunit.client.0.smithi094.stdout:4/570: truncate d0/df/d13/f58 692668 0 2024-04-06T03:40:38.244 INFO:tasks.workunit.client.0.smithi094.stdout:0/470: dwrite d1/d9/fc [4194304,4194304] 0 2024-04-06T03:40:38.245 INFO:tasks.workunit.client.0.smithi094.stdout:0/471: truncate d1/d9/dd/d18/d1a/f54 675774 0 2024-04-06T03:40:38.245 INFO:tasks.workunit.client.0.smithi094.stdout:0/472: creat d1/d9/dd/d18/d5d/d60/d6b/f9f x:0 0 0 2024-04-06T03:40:38.245 INFO:tasks.workunit.client.0.smithi094.stdout:2/572: rmdir d0/d8/d4f/dab/dba 0 2024-04-06T03:40:38.245 INFO:tasks.workunit.client.0.smithi094.stdout:2/573: creat d0/d8/d1d/fc0 x:0 0 0 2024-04-06T03:40:38.245 INFO:tasks.workunit.client.1.smithi136.stdout:0/501: dwrite d1/d6/d2c/d57/d43/d6d/d85/d78/f7a [0,4194304] 0 2024-04-06T03:40:38.245 INFO:tasks.workunit.client.0.smithi094.stdout:0/473: write d1/d9/dd/d18/d32/d42/f45 [167083,127766] 0 2024-04-06T03:40:38.247 INFO:tasks.workunit.client.0.smithi094.stdout:6/453: mknod dc/d3a/d4a/ca4 0 2024-04-06T03:40:38.249 INFO:tasks.workunit.client.1.smithi136.stdout:7/527: creat d5/df/d53/fb9 x:0 0 0 2024-04-06T03:40:38.252 INFO:tasks.workunit.client.0.smithi094.stdout:8/542: dread d18/f64 [0,4194304] 0 2024-04-06T03:40:38.252 INFO:tasks.workunit.client.0.smithi094.stdout:8/543: write d18/d2c/d37/d60/d70/f84 [534935,72] 0 2024-04-06T03:40:38.253 INFO:tasks.workunit.client.1.smithi136.stdout:4/456: mkdir d1d/d2f/da6 0 2024-04-06T03:40:38.254 INFO:tasks.workunit.client.0.smithi094.stdout:2/574: dread d0/dd/f62 [0,4194304] 0 2024-04-06T03:40:38.254 INFO:tasks.workunit.client.0.smithi094.stdout:2/575: write d0/dd/d1b/d72/d8f/fbb [5221864,120692] 0 2024-04-06T03:40:38.256 INFO:tasks.workunit.client.0.smithi094.stdout:6/454: dread dc/d3a/f41 [0,4194304] 0 2024-04-06T03:40:38.256 INFO:tasks.workunit.client.0.smithi094.stdout:5/508: dwrite db/d40/d77/f82 [0,4194304] 0 2024-04-06T03:40:38.260 INFO:tasks.workunit.client.1.smithi136.stdout:3/481: getdents d0/d4/d12/d19/d55/d6f 0 2024-04-06T03:40:38.262 INFO:tasks.workunit.client.1.smithi136.stdout:7/528: dwrite d5/df/d1d/d54/fa4 [0,4194304] 0 2024-04-06T03:40:38.262 INFO:tasks.workunit.client.1.smithi136.stdout:7/529: chown d5/df/d11 527 1 2024-04-06T03:40:38.262 INFO:tasks.workunit.client.1.smithi136.stdout:7/530: chown d5/df/d12/l6a 33 1 2024-04-06T03:40:38.262 INFO:tasks.workunit.client.1.smithi136.stdout:5/559: rmdir d1/dc/d1c/d32/d8b/d20/d66 39 2024-04-06T03:40:38.266 INFO:tasks.workunit.client.1.smithi136.stdout:8/560: sync 2024-04-06T03:40:38.266 INFO:tasks.workunit.client.1.smithi136.stdout:8/561: dread - d3/da/d26/d35/d53/fa9 zero size 2024-04-06T03:40:38.266 INFO:tasks.workunit.client.0.smithi094.stdout:7/492: creat d2/d6/f8f x:0 0 0 2024-04-06T03:40:38.267 INFO:tasks.workunit.client.0.smithi094.stdout:7/493: write d2/d6/dc/d12/f84 [924795,34076] 0 2024-04-06T03:40:38.267 INFO:tasks.workunit.client.0.smithi094.stdout:7/494: fdatasync d2/d6/f8 0 2024-04-06T03:40:38.267 INFO:tasks.workunit.client.0.smithi094.stdout:7/495: readlink d2/d6/dc/d12/l50 0 2024-04-06T03:40:38.267 INFO:tasks.workunit.client.1.smithi136.stdout:2/606: symlink d3/d15/d19/d4e/lc2 0 2024-04-06T03:40:38.269 INFO:tasks.workunit.client.0.smithi094.stdout:3/543: creat d9/d1e/d6e/d75/dba/d3a/daf/fc3 x:0 0 0 2024-04-06T03:40:38.270 INFO:tasks.workunit.client.1.smithi136.stdout:6/442: symlink d2/l95 0 2024-04-06T03:40:38.271 INFO:tasks.workunit.client.0.smithi094.stdout:4/571: mkdir d0/df/dbd 0 2024-04-06T03:40:38.273 INFO:tasks.workunit.client.1.smithi136.stdout:1/490: rename d12/d3b/d60/d7e/d37/f46 to d12/d3b/d6b/f9e 0 2024-04-06T03:40:38.274 INFO:tasks.workunit.client.1.smithi136.stdout:9/484: mknod d17/d21/d29/d34/d66/ca1 0 2024-04-06T03:40:38.276 INFO:tasks.workunit.client.1.smithi136.stdout:5/560: read d1/dc/d1c/d32/f6a [3927369,85835] 0 2024-04-06T03:40:38.276 INFO:tasks.workunit.client.1.smithi136.stdout:8/562: write d3/f21 [901544,90770] 0 2024-04-06T03:40:38.276 INFO:tasks.workunit.client.1.smithi136.stdout:8/563: chown d3/d2d/f67 3 1 2024-04-06T03:40:38.277 INFO:tasks.workunit.client.0.smithi094.stdout:0/474: unlink d1/d9/dd/d18/l47 0 2024-04-06T03:40:38.279 INFO:tasks.workunit.client.1.smithi136.stdout:0/502: symlink d1/d6/d2c/d57/d43/d6d/d85/d86/lab 0 2024-04-06T03:40:38.281 INFO:tasks.workunit.client.0.smithi094.stdout:8/544: unlink d18/d2c/f41 0 2024-04-06T03:40:38.282 INFO:tasks.workunit.client.1.smithi136.stdout:4/457: symlink d1d/d2f/d6c/da5/la7 0 2024-04-06T03:40:38.284 INFO:tasks.workunit.client.0.smithi094.stdout:1/500: getdents d0/d4/d1b/d2f/d3c 0 2024-04-06T03:40:38.284 INFO:tasks.workunit.client.0.smithi094.stdout:1/501: read - d0/d4/d1b/d2f/d4b/f9e zero size 2024-04-06T03:40:38.285 INFO:tasks.workunit.client.1.smithi136.stdout:3/482: creat d0/d4/d12/d19/d55/f9f x:0 0 0 2024-04-06T03:40:38.290 INFO:tasks.workunit.client.1.smithi136.stdout:2/607: dread d3/d4a/f8d [0,4194304] 0 2024-04-06T03:40:38.290 INFO:tasks.workunit.client.1.smithi136.stdout:2/608: write d3/d4a/d9d/d8a/d93/fa9 [1031106,121751] 0 2024-04-06T03:40:38.291 INFO:tasks.workunit.client.1.smithi136.stdout:5/561: dread d1/d16/d4b/f95 [0,4194304] 0 2024-04-06T03:40:38.291 INFO:tasks.workunit.client.1.smithi136.stdout:7/531: symlink d5/df/d53/d8a/lba 0 2024-04-06T03:40:38.293 INFO:tasks.workunit.client.1.smithi136.stdout:7/532: dread d5/df/f62 [0,4194304] 0 2024-04-06T03:40:38.295 INFO:tasks.workunit.client.0.smithi094.stdout:5/509: creat db/d19/d24/d2e/fb3 x:0 0 0 2024-04-06T03:40:38.295 INFO:tasks.workunit.client.1.smithi136.stdout:6/443: unlink d2/d7/d26/d30/d6e/d3d/d47/d84/f8a 0 2024-04-06T03:40:38.297 INFO:tasks.workunit.client.0.smithi094.stdout:7/496: mkdir d2/d6/dc/dd/d90 0 2024-04-06T03:40:38.298 INFO:tasks.workunit.client.1.smithi136.stdout:9/485: rename fd to d17/d20/d25/d71/fa2 0 2024-04-06T03:40:38.298 INFO:tasks.workunit.client.1.smithi136.stdout:1/491: dread d12/d3b/d60/d7e/d37/f78 [0,4194304] 0 2024-04-06T03:40:38.298 INFO:tasks.workunit.client.1.smithi136.stdout:9/486: truncate f13 5152580 0 2024-04-06T03:40:38.298 INFO:tasks.workunit.client.1.smithi136.stdout:9/487: truncate d17/d20/d25/d3b/f4e 110388 0 2024-04-06T03:40:38.298 INFO:tasks.workunit.client.1.smithi136.stdout:9/488: stat d17/d20/d28/d49/d75/f9d 0 2024-04-06T03:40:38.298 INFO:tasks.workunit.client.1.smithi136.stdout:8/564: mkdir d3/d2d/dad 0 2024-04-06T03:40:38.299 INFO:tasks.workunit.client.0.smithi094.stdout:3/544: truncate d9/d20/d30/f69 24213 0 2024-04-06T03:40:38.301 INFO:tasks.workunit.client.0.smithi094.stdout:2/576: link d0/dd/d19/dbe/lb2 d0/dd/d1b/lc1 0 2024-04-06T03:40:38.302 INFO:tasks.workunit.client.1.smithi136.stdout:4/458: readlink d1d/l6d 0 2024-04-06T03:40:38.303 INFO:tasks.workunit.client.0.smithi094.stdout:6/455: getdents dc/d11/d22 0 2024-04-06T03:40:38.306 INFO:tasks.workunit.client.1.smithi136.stdout:2/609: creat d3/d4a/d9d/d18/d4b/dbd/fc3 x:0 0 0 2024-04-06T03:40:38.306 INFO:tasks.workunit.client.1.smithi136.stdout:2/610: stat d3/d4a/d9d/d18/d4b/f9c 0 2024-04-06T03:40:38.306 INFO:tasks.workunit.client.0.smithi094.stdout:7/497: read d2/d6/dc/f2b [2508315,83850] 0 2024-04-06T03:40:38.306 INFO:tasks.workunit.client.0.smithi094.stdout:7/498: chown d2/d6/dc/d12/d29/d3f/c7d 47615 1 2024-04-06T03:40:38.307 INFO:tasks.workunit.client.0.smithi094.stdout:4/572: dwrite d0/d4/f21 [0,4194304] 0 2024-04-06T03:40:38.307 INFO:tasks.workunit.client.0.smithi094.stdout:6/456: dread dc/d11/d22/d50/d61/f73 [0,4194304] 0 2024-04-06T03:40:38.308 INFO:tasks.workunit.client.1.smithi136.stdout:5/562: mkdir d1/d16/d89/daf 0 2024-04-06T03:40:38.312 INFO:tasks.workunit.client.0.smithi094.stdout:5/510: getdents db/d7c/d8b 0 2024-04-06T03:40:38.312 INFO:tasks.workunit.client.0.smithi094.stdout:5/511: chown db/d7c/d8b 15 1 2024-04-06T03:40:38.317 INFO:tasks.workunit.client.1.smithi136.stdout:3/483: rename d0/d4/d12/d19/d55/l26 to d0/d4/d12/d2d/d37/la0 0 2024-04-06T03:40:38.317 INFO:tasks.workunit.client.1.smithi136.stdout:3/484: chown d0/d4/d91 68621322 1 2024-04-06T03:40:38.317 INFO:tasks.workunit.client.0.smithi094.stdout:6/457: dread dc/d11/d1b/f52 [0,4194304] 0 2024-04-06T03:40:38.317 INFO:tasks.workunit.client.0.smithi094.stdout:6/458: write dc/d3a/f89 [314246,47896] 0 2024-04-06T03:40:38.317 INFO:tasks.workunit.client.0.smithi094.stdout:6/459: creat dc/d11/d1b/d24/d46/fa5 x:0 0 0 2024-04-06T03:40:38.317 INFO:tasks.workunit.client.0.smithi094.stdout:6/460: fdatasync dc/d3a/d4a/f6b 0 2024-04-06T03:40:38.318 INFO:tasks.workunit.client.1.smithi136.stdout:1/492: mkdir d12/d1b/d1e/d9f 0 2024-04-06T03:40:38.318 INFO:tasks.workunit.client.1.smithi136.stdout:1/493: read - d12/d3b/d60/d7e/d37/f8b zero size 2024-04-06T03:40:38.319 INFO:tasks.workunit.client.1.smithi136.stdout:9/489: mkdir d17/d19/da3 0 2024-04-06T03:40:38.320 INFO:tasks.workunit.client.0.smithi094.stdout:7/499: mknod d2/d6/c91 0 2024-04-06T03:40:38.320 INFO:tasks.workunit.client.0.smithi094.stdout:7/500: chown d2/d6/f8f 98 1 2024-04-06T03:40:38.321 INFO:tasks.workunit.client.0.smithi094.stdout:8/545: dwrite d18/d2c/f9b [0,4194304] 0 2024-04-06T03:40:38.321 INFO:tasks.workunit.client.1.smithi136.stdout:0/503: getdents d1 0 2024-04-06T03:40:38.324 INFO:tasks.workunit.client.0.smithi094.stdout:4/573: creat d0/df/d1e/d34/db6/fbe x:0 0 0 2024-04-06T03:40:38.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/475: dwrite d1/d9/f5e [0,4194304] 0 2024-04-06T03:40:38.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/476: write d1/f23 [1433842,5384] 0 2024-04-06T03:40:38.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/477: dread - d1/d9/dd/d18/d1e/d22/f87 zero size 2024-04-06T03:40:38.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/478: write d1/f2c [4740107,24058] 0 2024-04-06T03:40:38.324 INFO:tasks.workunit.client.0.smithi094.stdout:0/479: fsync d1/d9/dd/d18/d1e/d22/d35/f58 0 2024-04-06T03:40:38.324 INFO:tasks.workunit.client.0.smithi094.stdout:3/545: dwrite d9/d1e/d26/d55/d62/f68 [0,4194304] 0 2024-04-06T03:40:38.325 INFO:tasks.workunit.client.1.smithi136.stdout:6/444: dwrite d2/d7/d26/d68/f80 [4194304,4194304] 0 2024-04-06T03:40:38.327 INFO:tasks.workunit.client.1.smithi136.stdout:5/563: write d1/d16/d4b/f80 [3449941,92472] 0 2024-04-06T03:40:38.328 INFO:tasks.workunit.client.0.smithi094.stdout:6/461: mknod dc/d11/d1b/d72/d87/ca6 0 2024-04-06T03:40:38.330 INFO:tasks.workunit.client.0.smithi094.stdout:8/546: write f13 [1278330,50596] 0 2024-04-06T03:40:38.330 INFO:tasks.workunit.client.0.smithi094.stdout:8/547: fdatasync d18/d3c/d49/f7b 0 2024-04-06T03:40:38.331 INFO:tasks.workunit.client.1.smithi136.stdout:8/565: dwrite f1 [8388608,4194304] 0 2024-04-06T03:40:38.331 INFO:tasks.workunit.client.1.smithi136.stdout:9/490: dread fa [0,4194304] 0 2024-04-06T03:40:38.331 INFO:tasks.workunit.client.0.smithi094.stdout:2/577: rename d0/dd/d1b/l3c to d0/d8/da0/lc2 0 2024-04-06T03:40:38.332 INFO:tasks.workunit.client.1.smithi136.stdout:8/566: chown d3/da/d26/d35/d3d/d48/d6c/dab 159 1 2024-04-06T03:40:38.334 INFO:tasks.workunit.client.1.smithi136.stdout:3/485: link d0/d4/d12/d19/d2e/d3b/d49/l4c d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/la1 0 2024-04-06T03:40:38.334 INFO:tasks.workunit.client.1.smithi136.stdout:3/486: chown d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/f83 99575 1 2024-04-06T03:40:38.334 INFO:tasks.workunit.client.1.smithi136.stdout:3/487: write d0/f1 [4351004,60240] 0 2024-04-06T03:40:38.335 INFO:tasks.workunit.client.1.smithi136.stdout:3/488: dread - d0/d4/d12/d19/d55/d3c/d46/f78 zero size 2024-04-06T03:40:38.335 INFO:tasks.workunit.client.1.smithi136.stdout:3/489: read d0/d4/d12/d19/d55/d3c/d5a/f71 [224431,66147] 0 2024-04-06T03:40:38.345 INFO:tasks.workunit.client.0.smithi094.stdout:7/501: creat d2/d6/dc/dd/f92 x:0 0 0 2024-04-06T03:40:38.349 INFO:tasks.workunit.client.1.smithi136.stdout:4/459: getdents d1d/d31/d3e/d55 0 2024-04-06T03:40:38.354 INFO:tasks.workunit.client.0.smithi094.stdout:2/578: dwrite d0/dd/d19/d31/f8c [0,4194304] 0 2024-04-06T03:40:38.358 INFO:tasks.workunit.client.0.smithi094.stdout:0/480: mknod d1/d9/dd/d18/d1e/d22/d35/ca0 0 2024-04-06T03:40:38.359 INFO:tasks.workunit.client.0.smithi094.stdout:9/457: sync 2024-04-06T03:40:38.363 INFO:tasks.workunit.client.1.smithi136.stdout:5/564: dwrite d1/dc/d1c/f56 [0,4194304] 0 2024-04-06T03:40:38.363 INFO:tasks.workunit.client.1.smithi136.stdout:0/504: dwrite d1/f63 [0,4194304] 0 2024-04-06T03:40:38.363 INFO:tasks.workunit.client.1.smithi136.stdout:0/505: stat d1/d6/d2c/f3d 0 2024-04-06T03:40:38.364 INFO:tasks.workunit.client.1.smithi136.stdout:1/494: dwrite d12/d1b/d28/f3f [0,4194304] 0 2024-04-06T03:40:38.364 INFO:tasks.workunit.client.1.smithi136.stdout:1/495: creat d12/d3b/d44/d5f/fa0 x:0 0 0 2024-04-06T03:40:38.364 INFO:tasks.workunit.client.1.smithi136.stdout:6/445: dwrite d2/d25/f59 [4194304,4194304] 0 2024-04-06T03:40:38.364 INFO:tasks.workunit.client.1.smithi136.stdout:6/446: getdents d2/d7/d26/d30 0 2024-04-06T03:40:38.364 INFO:tasks.workunit.client.0.smithi094.stdout:7/502: dwrite d2/d6/dc/dd/f28 [4194304,4194304] 0 2024-04-06T03:40:38.365 INFO:tasks.workunit.client.1.smithi136.stdout:1/496: write d12/d3b/d44/f57 [1074347,22271] 0 2024-04-06T03:40:38.365 INFO:tasks.workunit.client.0.smithi094.stdout:3/546: creat d9/fc4 x:0 0 0 2024-04-06T03:40:38.365 INFO:tasks.workunit.client.0.smithi094.stdout:3/547: stat d9/d1e/c44 0 2024-04-06T03:40:38.366 INFO:tasks.workunit.client.1.smithi136.stdout:1/497: write d12/d1b/d1e/d4d/f51 [3661,83827] 0 2024-04-06T03:40:38.368 INFO:tasks.workunit.client.0.smithi094.stdout:6/462: truncate f8 2011945 0 2024-04-06T03:40:38.369 INFO:tasks.workunit.client.0.smithi094.stdout:2/579: dread d0/f64 [4194304,4194304] 0 2024-04-06T03:40:38.369 INFO:tasks.workunit.client.0.smithi094.stdout:6/463: write dc/d11/d97/fa1 [954273,117816] 0 2024-04-06T03:40:38.369 INFO:tasks.workunit.client.0.smithi094.stdout:6/464: truncate dc/d11/d1b/d72/d87/f8a 231401 0 2024-04-06T03:40:38.369 INFO:tasks.workunit.client.0.smithi094.stdout:6/465: readlink dc/d11/d1b/d72/d88/l9b 0 2024-04-06T03:40:38.370 INFO:tasks.workunit.client.0.smithi094.stdout:6/466: dread dc/d11/d1b/f74 [0,4194304] 0 2024-04-06T03:40:38.370 INFO:tasks.workunit.client.1.smithi136.stdout:2/611: rmdir d3/d9/d3b 39 2024-04-06T03:40:38.372 INFO:tasks.workunit.client.0.smithi094.stdout:8/548: truncate fd 3800245 0 2024-04-06T03:40:38.373 INFO:tasks.workunit.client.1.smithi136.stdout:9/491: rmdir d17/d21/d29/d34 39 2024-04-06T03:40:38.374 INFO:tasks.workunit.client.0.smithi094.stdout:5/512: rename db/d19/d24/c35 to db/cb4 0 2024-04-06T03:40:38.375 INFO:tasks.workunit.client.1.smithi136.stdout:8/567: mkdir d3/da/d26/d90/dae 0 2024-04-06T03:40:38.378 INFO:tasks.workunit.client.1.smithi136.stdout:3/490: link d0/d4/d12/d19/d2e/d3b/d49/l4c d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/la2 0 2024-04-06T03:40:38.380 INFO:tasks.workunit.client.1.smithi136.stdout:4/460: truncate d1d/f26 99343 0 2024-04-06T03:40:38.383 INFO:tasks.workunit.client.0.smithi094.stdout:0/481: truncate d1/d9/dd/f3b 3669108 0 2024-04-06T03:40:38.384 INFO:tasks.workunit.client.0.smithi094.stdout:9/458: creat d2/dc/d21/fad x:0 0 0 2024-04-06T03:40:38.385 INFO:tasks.workunit.client.1.smithi136.stdout:5/565: mkdir d1/dc/d1c/d32/d8b/db0 0 2024-04-06T03:40:38.385 INFO:tasks.workunit.client.1.smithi136.stdout:5/566: rename d1/d16/d17 to d1/d16/d17/db1 22 2024-04-06T03:40:38.386 INFO:tasks.workunit.client.0.smithi094.stdout:7/503: mkdir d2/d6/d93 0 2024-04-06T03:40:38.388 INFO:tasks.workunit.client.0.smithi094.stdout:3/548: mkdir d9/d1e/d6e/d75/dba/dc5 0 2024-04-06T03:40:38.390 INFO:tasks.workunit.client.1.smithi136.stdout:0/506: rmdir d1/d6/d2c/d57/d62/d98 0 2024-04-06T03:40:38.391 INFO:tasks.workunit.client.0.smithi094.stdout:2/580: mkdir d0/d8/d4f/dab/dc3 0 2024-04-06T03:40:38.394 INFO:tasks.workunit.client.0.smithi094.stdout:6/467: link dc/f17 dc/d3a/da2/fa7 0 2024-04-06T03:40:38.395 INFO:tasks.workunit.client.1.smithi136.stdout:0/507: dread d1/d6/d2c/d1a/f1d [0,4194304] 0 2024-04-06T03:40:38.397 INFO:tasks.workunit.client.0.smithi094.stdout:8/549: creat d18/d34/d96/fbf x:0 0 0 2024-04-06T03:40:38.398 INFO:tasks.workunit.client.1.smithi136.stdout:5/567: dread d1/d16/f1b [0,4194304] 0 2024-04-06T03:40:38.399 INFO:tasks.workunit.client.1.smithi136.stdout:5/568: write d1/dc/f5b [1839833,54510] 0 2024-04-06T03:40:38.399 INFO:tasks.workunit.client.1.smithi136.stdout:5/569: chown d1/dc/d1c/d32/d8b/c3a 1240329 1 2024-04-06T03:40:38.399 INFO:tasks.workunit.client.1.smithi136.stdout:6/447: mknod d2/d7/d26/d68/c96 0 2024-04-06T03:40:38.399 INFO:tasks.workunit.client.1.smithi136.stdout:6/448: chown d2/d7/d26/f54 0 1 2024-04-06T03:40:38.399 INFO:tasks.workunit.client.0.smithi094.stdout:5/513: mkdir db/d19/d24/d28/d85/db5 0 2024-04-06T03:40:38.401 INFO:tasks.workunit.client.1.smithi136.stdout:1/498: mknod d12/d53/ca1 0 2024-04-06T03:40:38.402 INFO:tasks.workunit.client.1.smithi136.stdout:1/499: write d12/d3b/d44/d5f/d5a/f62 [810407,86275] 0 2024-04-06T03:40:38.402 INFO:tasks.workunit.client.1.smithi136.stdout:1/500: write d12/d1b/d28/f5e [1092427,39590] 0 2024-04-06T03:40:38.402 INFO:tasks.workunit.client.1.smithi136.stdout:1/501: truncate d12/d1b/d1e/f99 105194 0 2024-04-06T03:40:38.402 INFO:tasks.workunit.client.1.smithi136.stdout:1/502: chown d12/d3b/d44/d5f/d5a/l9a 121 1 2024-04-06T03:40:38.403 INFO:tasks.workunit.client.0.smithi094.stdout:0/482: unlink d1/d9/dd/f1c 0 2024-04-06T03:40:38.405 INFO:tasks.workunit.client.0.smithi094.stdout:9/459: link d2/dc/d80/fac d2/dc/d21/d26/d78/fae 0 2024-04-06T03:40:38.406 INFO:tasks.workunit.client.0.smithi094.stdout:5/514: write db/d19/d24/d2e/f48 [4193878,27479] 0 2024-04-06T03:40:38.407 INFO:tasks.workunit.client.1.smithi136.stdout:9/492: unlink d17/d21/d29/d34/d46/l7e 0 2024-04-06T03:40:38.410 INFO:tasks.workunit.client.0.smithi094.stdout:6/468: truncate dc/f17 2270890 0 2024-04-06T03:40:38.411 INFO:tasks.workunit.client.0.smithi094.stdout:8/550: truncate d18/f64 4003602 0 2024-04-06T03:40:38.411 INFO:tasks.workunit.client.0.smithi094.stdout:8/551: readlink d18/d3c/d49/l5e 0 2024-04-06T03:40:38.412 INFO:tasks.workunit.client.0.smithi094.stdout:8/552: write d18/d34/d96/fbf [422429,90217] 0 2024-04-06T03:40:38.413 INFO:tasks.workunit.client.0.smithi094.stdout:0/483: mknod d1/d9/dd/d18/d5d/d60/d6b/d67/ca1 0 2024-04-06T03:40:38.415 INFO:tasks.workunit.client.1.smithi136.stdout:3/491: mknod d0/d4/d12/d19/d55/d3c/d5a/ca3 0 2024-04-06T03:40:38.415 INFO:tasks.workunit.client.0.smithi094.stdout:9/460: creat d2/d2c/d99/faf x:0 0 0 2024-04-06T03:40:38.416 INFO:tasks.workunit.client.1.smithi136.stdout:4/461: mknod d1d/d2f/ca8 0 2024-04-06T03:40:38.417 INFO:tasks.workunit.client.1.smithi136.stdout:0/508: creat d1/d6/d2c/d57/d43/d9e/fac x:0 0 0 2024-04-06T03:40:38.418 INFO:tasks.workunit.client.0.smithi094.stdout:6/469: dread fa [0,4194304] 0 2024-04-06T03:40:38.418 INFO:tasks.workunit.client.0.smithi094.stdout:5/515: symlink db/d19/d24/d28/d85/lb6 0 2024-04-06T03:40:38.420 INFO:tasks.workunit.client.0.smithi094.stdout:2/581: getdents d0/d8/d1d/d77 0 2024-04-06T03:40:38.420 INFO:tasks.workunit.client.0.smithi094.stdout:2/582: creat d0/dd/d1b/d72/d8f/fc4 x:0 0 0 2024-04-06T03:40:38.421 INFO:tasks.workunit.client.1.smithi136.stdout:5/570: getdents d1/d1f 0 2024-04-06T03:40:38.422 INFO:tasks.workunit.client.0.smithi094.stdout:8/553: rename d18/f4d to d18/d34/d96/db9/fc0 0 2024-04-06T03:40:38.423 INFO:tasks.workunit.client.1.smithi136.stdout:6/449: mkdir d2/d7/d26/d97 0 2024-04-06T03:40:38.423 INFO:tasks.workunit.client.0.smithi094.stdout:0/484: creat d1/d9/dd/fa2 x:0 0 0 2024-04-06T03:40:38.424 INFO:tasks.workunit.client.0.smithi094.stdout:9/461: mkdir d2/dc/d21/d23/db0 0 2024-04-06T03:40:38.426 INFO:tasks.workunit.client.0.smithi094.stdout:6/470: unlink dc/d3a/d4a/d54/d71/f90 0 2024-04-06T03:40:38.426 INFO:tasks.workunit.client.1.smithi136.stdout:2/612: rmdir d3/d15/d19/d43 39 2024-04-06T03:40:38.428 INFO:tasks.workunit.client.1.smithi136.stdout:3/492: rename d0/f1 to d0/d4/d12/d19/d55/d3c/d5a/d79/fa4 0 2024-04-06T03:40:38.430 INFO:tasks.workunit.client.0.smithi094.stdout:2/583: link d0/d8/d27/l36 d0/d8/d27/d2b/d76/lc5 0 2024-04-06T03:40:38.430 INFO:tasks.workunit.client.1.smithi136.stdout:5/571: mkdir d1/d16/d17/db2 0 2024-04-06T03:40:38.431 INFO:tasks.workunit.client.1.smithi136.stdout:6/450: link d2/d7/d26/d30/d6e/f35 d2/d7/d26/d68/f98 0 2024-04-06T03:40:38.432 INFO:tasks.workunit.client.1.smithi136.stdout:5/572: mknod d1/d16/d89/d8a/cb3 0 2024-04-06T03:40:38.433 INFO:tasks.workunit.client.0.smithi094.stdout:8/554: creat d18/d34/d96/dbc/fc1 x:0 0 0 2024-04-06T03:40:38.433 INFO:tasks.workunit.client.1.smithi136.stdout:6/451: creat d2/d92/f99 x:0 0 0 2024-04-06T03:40:38.433 INFO:tasks.workunit.client.1.smithi136.stdout:6/452: creat d2/d7/d13/f9a x:0 0 0 2024-04-06T03:40:38.434 INFO:tasks.workunit.client.0.smithi094.stdout:9/462: symlink d2/d2c/d33/lb1 0 2024-04-06T03:40:38.435 INFO:tasks.workunit.client.1.smithi136.stdout:5/573: mkdir d1/d9b/db4 0 2024-04-06T03:40:38.435 INFO:tasks.workunit.client.1.smithi136.stdout:5/574: stat d1/dc/d1c/d32/d8b/c5d 0 2024-04-06T03:40:38.436 INFO:tasks.workunit.client.1.smithi136.stdout:5/575: write d1/dc/d1c/d32/d8b/f73 [262,112961] 0 2024-04-06T03:40:38.438 INFO:tasks.workunit.client.0.smithi094.stdout:8/555: creat d18/d3c/d49/db5/fc2 x:0 0 0 2024-04-06T03:40:38.440 INFO:tasks.workunit.client.1.smithi136.stdout:6/453: mkdir d2/d7/d26/d68/d9b 0 2024-04-06T03:40:38.441 INFO:tasks.workunit.client.1.smithi136.stdout:5/576: link d1/dc/d1c/d32/d8b/l65 d1/dc/d1c/d32/d8b/d20/lb5 0 2024-04-06T03:40:38.442 INFO:tasks.workunit.client.1.smithi136.stdout:5/577: stat d1/dc/d10/d1e/f9d 0 2024-04-06T03:40:38.442 INFO:tasks.workunit.client.0.smithi094.stdout:9/463: dread d2/d2c/f36 [0,4194304] 0 2024-04-06T03:40:38.444 INFO:tasks.workunit.client.1.smithi136.stdout:1/503: dread d12/d1b/f9d [0,4194304] 0 2024-04-06T03:40:38.444 INFO:tasks.workunit.client.1.smithi136.stdout:1/504: getdents d12/d3b/d74/d94 0 2024-04-06T03:40:38.447 INFO:tasks.workunit.client.0.smithi094.stdout:7/504: dwrite d2/d6/dc/d12/f5f [0,4194304] 0 2024-04-06T03:40:38.447 INFO:tasks.workunit.client.0.smithi094.stdout:7/505: write d2/d6/dc/d12/d29/d35/f8d [73481,88821] 0 2024-04-06T03:40:38.448 INFO:tasks.workunit.client.0.smithi094.stdout:8/556: mkdir d18/d2c/d37/d60/da3/db8/dc3 0 2024-04-06T03:40:38.452 INFO:tasks.workunit.client.1.smithi136.stdout:6/454: dread d2/d7/f12 [0,4194304] 0 2024-04-06T03:40:38.455 INFO:tasks.workunit.client.0.smithi094.stdout:9/464: read d2/dc/d21/d23/d2f/f83 [128778,128314] 0 2024-04-06T03:40:38.457 INFO:tasks.workunit.client.0.smithi094.stdout:9/465: creat d2/dc/d10/fb2 x:0 0 0 2024-04-06T03:40:38.457 INFO:tasks.workunit.client.0.smithi094.stdout:9/466: truncate d2/dc/d46/f60 25739 0 2024-04-06T03:40:38.460 INFO:tasks.workunit.client.0.smithi094.stdout:9/467: unlink d2/dc/d21/fa4 0 2024-04-06T03:40:38.469 INFO:tasks.workunit.client.0.smithi094.stdout:6/471: fsync dc/d11/d97/fa1 0 2024-04-06T03:40:38.469 INFO:tasks.workunit.client.0.smithi094.stdout:6/472: fdatasync f5 0 2024-04-06T03:40:38.471 INFO:tasks.workunit.client.0.smithi094.stdout:6/473: symlink dc/d11/d1b/d72/d87/la8 0 2024-04-06T03:40:38.472 INFO:tasks.workunit.client.1.smithi136.stdout:9/493: dwrite d17/d21/d32/f4f [0,4194304] 0 2024-04-06T03:40:38.474 INFO:tasks.workunit.client.1.smithi136.stdout:9/494: mkdir d17/d20/d25/da4 0 2024-04-06T03:40:38.476 INFO:tasks.workunit.client.1.smithi136.stdout:9/495: mkdir d17/d20/d25/d3d/da5 0 2024-04-06T03:40:38.476 INFO:tasks.workunit.client.1.smithi136.stdout:9/496: creat d17/d21/d32/fa6 x:0 0 0 2024-04-06T03:40:38.476 INFO:tasks.workunit.client.1.smithi136.stdout:9/497: write d17/d21/d32/d43/f8c [4577186,117876] 0 2024-04-06T03:40:38.484 INFO:tasks.workunit.client.1.smithi136.stdout:9/498: dread d17/d20/d25/f26 [0,4194304] 0 2024-04-06T03:40:38.487 INFO:tasks.workunit.client.0.smithi094.stdout:3/549: dwrite d9/d1e/d6e/d75/dba/d3a/f40 [0,4194304] 0 2024-04-06T03:40:38.487 INFO:tasks.workunit.client.0.smithi094.stdout:0/485: dwrite d1/d9/f5e [0,4194304] 0 2024-04-06T03:40:38.488 INFO:tasks.workunit.client.1.smithi136.stdout:5/578: dwrite d1/dc/d1c/d32/d8b/d20/d66/f9e [0,4194304] 0 2024-04-06T03:40:38.488 INFO:tasks.workunit.client.1.smithi136.stdout:0/509: dwrite d1/d6/d2c/d57/f5f [0,4194304] 0 2024-04-06T03:40:38.488 INFO:tasks.workunit.client.1.smithi136.stdout:0/510: truncate d1/d6/d2c/d57/d43/d6d/d85/d78/f97 351245 0 2024-04-06T03:40:38.488 INFO:tasks.workunit.client.1.smithi136.stdout:0/511: truncate d1/d6/d2c/d1a/fa0 760241 0 2024-04-06T03:40:38.489 INFO:tasks.workunit.client.1.smithi136.stdout:6/455: dwrite d2/d7/d26/d30/d6e/f10 [4194304,4194304] 0 2024-04-06T03:40:38.489 INFO:tasks.workunit.client.1.smithi136.stdout:6/456: chown d2/d7/d26/d30/d4f/c72 1746 1 2024-04-06T03:40:38.489 INFO:tasks.workunit.client.1.smithi136.stdout:2/613: dwrite d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:38.489 INFO:tasks.workunit.client.1.smithi136.stdout:3/493: dwrite d0/d4/d12/d19/d2c/f31 [0,4194304] 0 2024-04-06T03:40:38.491 INFO:tasks.workunit.client.0.smithi094.stdout:3/550: truncate d9/fc 3946069 0 2024-04-06T03:40:38.492 INFO:tasks.workunit.client.0.smithi094.stdout:3/551: chown d9/d10/f47 589672450 1 2024-04-06T03:40:38.492 INFO:tasks.workunit.client.0.smithi094.stdout:3/552: chown d9/f15 496 1 2024-04-06T03:40:38.495 INFO:tasks.workunit.client.1.smithi136.stdout:9/499: rmdir d17/d20/da0 0 2024-04-06T03:40:38.495 INFO:tasks.workunit.client.1.smithi136.stdout:9/500: dread - d17/d20/d25/f84 zero size 2024-04-06T03:40:38.496 INFO:tasks.workunit.client.1.smithi136.stdout:9/501: creat d17/d21/fa7 x:0 0 0 2024-04-06T03:40:38.496 INFO:tasks.workunit.client.1.smithi136.stdout:9/502: fsync d17/d19/f35 0 2024-04-06T03:40:38.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/468: write d2/d2c/d33/f44 [483266,44033] 0 2024-04-06T03:40:38.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/469: creat d2/d2c/d99/fb3 x:0 0 0 2024-04-06T03:40:38.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/470: creat d2/d2c/d99/fb4 x:0 0 0 2024-04-06T03:40:38.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/471: chown d2/dc/d21/l5d 0 1 2024-04-06T03:40:38.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/472: write d2/dc/d46/f95 [580418,114486] 0 2024-04-06T03:40:38.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/473: dread - d2/dc/d21/d23/d2f/f90 zero size 2024-04-06T03:40:38.497 INFO:tasks.workunit.client.0.smithi094.stdout:0/486: creat d1/d9/dd/d18/d1e/d86/fa3 x:0 0 0 2024-04-06T03:40:38.497 INFO:tasks.workunit.client.0.smithi094.stdout:0/487: chown d1/d10/d4e 18850777 1 2024-04-06T03:40:38.498 INFO:tasks.workunit.client.1.smithi136.stdout:9/503: dread d17/d21/d32/f74 [0,4194304] 0 2024-04-06T03:40:38.501 INFO:tasks.workunit.client.0.smithi094.stdout:3/553: mknod d9/d1e/d26/d55/cc6 0 2024-04-06T03:40:38.501 INFO:tasks.workunit.client.0.smithi094.stdout:3/554: read - d9/d20/d76/fac zero size 2024-04-06T03:40:38.501 INFO:tasks.workunit.client.0.smithi094.stdout:3/555: fsync d9/d1d/d7d/fb0 0 2024-04-06T03:40:38.503 INFO:tasks.workunit.client.1.smithi136.stdout:5/579: truncate d1/dc/d1c/d32/d8b/f7f 3079113 0 2024-04-06T03:40:38.507 INFO:tasks.workunit.client.0.smithi094.stdout:9/474: dwrite d2/dc/d21/d26/d78/d69/da8/d96/fab [0,4194304] 0 2024-04-06T03:40:38.508 INFO:tasks.workunit.client.0.smithi094.stdout:9/475: write d2/dc/d80/fa1 [732016,48540] 0 2024-04-06T03:40:38.508 INFO:tasks.workunit.client.0.smithi094.stdout:0/488: unlink d1/d9/dd/d18/d5d/d60/d6b/f9f 0 2024-04-06T03:40:38.508 INFO:tasks.workunit.client.0.smithi094.stdout:0/489: fsync d1/f7a 0 2024-04-06T03:40:38.508 INFO:tasks.workunit.client.0.smithi094.stdout:0/490: creat d1/d9/dd/d18/d1e/d86/d96/fa4 x:0 0 0 2024-04-06T03:40:38.509 INFO:tasks.workunit.client.1.smithi136.stdout:6/457: mkdir d2/d7/d26/d30/d9c 0 2024-04-06T03:40:38.510 INFO:tasks.workunit.client.0.smithi094.stdout:3/556: mknod d9/d1e/d6e/d75/dba/d3a/cc7 0 2024-04-06T03:40:38.515 INFO:tasks.workunit.client.0.smithi094.stdout:2/584: rmdir d0/dd 39 2024-04-06T03:40:38.517 INFO:tasks.workunit.client.0.smithi094.stdout:3/557: write d9/d20/f34 [2080543,83196] 0 2024-04-06T03:40:38.521 INFO:tasks.workunit.client.0.smithi094.stdout:2/585: mknod d0/dd/d19/cc6 0 2024-04-06T03:40:38.526 INFO:tasks.workunit.client.0.smithi094.stdout:3/558: creat d9/d1e/d6e/fc8 x:0 0 0 2024-04-06T03:40:38.526 INFO:tasks.workunit.client.0.smithi094.stdout:3/559: write d9/d1e/d6e/d75/dba/d7c/f92 [412957,15476] 0 2024-04-06T03:40:38.526 INFO:tasks.workunit.client.0.smithi094.stdout:3/560: write d9/d1d/d7d/f9c [37398,103750] 0 2024-04-06T03:40:38.529 INFO:tasks.workunit.client.1.smithi136.stdout:3/494: rename d0/d4/d12/d2d/l2f to d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/la5 0 2024-04-06T03:40:38.529 INFO:tasks.workunit.client.1.smithi136.stdout:3/495: chown d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/la1 0 1 2024-04-06T03:40:38.531 INFO:tasks.workunit.client.1.smithi136.stdout:3/496: dread d0/f11 [0,4194304] 0 2024-04-06T03:40:38.535 INFO:tasks.workunit.client.0.smithi094.stdout:0/491: dwrite d1/d9/dd/fa2 [0,4194304] 0 2024-04-06T03:40:38.552 INFO:tasks.workunit.client.0.smithi094.stdout:3/561: creat d9/d1e/d6e/d75/dba/d7c/da2/fc9 x:0 0 0 2024-04-06T03:40:38.555 INFO:tasks.workunit.client.0.smithi094.stdout:8/557: rename d18/d34/d96 to d18/d3c/dc4 0 2024-04-06T03:40:38.556 INFO:tasks.workunit.client.1.smithi136.stdout:9/504: symlink d17/d21/d32/d3f/la8 0 2024-04-06T03:40:38.559 INFO:tasks.workunit.client.1.smithi136.stdout:5/580: symlink d1/dc/d10/d1e/lb6 0 2024-04-06T03:40:38.559 INFO:tasks.workunit.client.1.smithi136.stdout:5/581: truncate d1/dc/d1c/d32/d8b/d20/f42 1845362 0 2024-04-06T03:40:38.566 INFO:tasks.workunit.client.1.smithi136.stdout:3/497: mkdir d0/d4/d12/d3a/d51/da6 0 2024-04-06T03:40:38.566 INFO:tasks.workunit.client.0.smithi094.stdout:7/506: write d2/d6/dc/d12/f5f [4643169,8339] 0 2024-04-06T03:40:38.566 INFO:tasks.workunit.client.0.smithi094.stdout:7/507: read d2/fb [1237628,117646] 0 2024-04-06T03:40:38.583 INFO:tasks.workunit.client.0.smithi094.stdout:2/586: dwrite d0/dd/d1b/d72/d8f/fc4 [0,4194304] 0 2024-04-06T03:40:38.587 INFO:tasks.workunit.client.0.smithi094.stdout:9/476: dwrite d2/dc/d21/d26/d78/d69/da8/d96/f9f [0,4194304] 0 2024-04-06T03:40:38.587 INFO:tasks.workunit.client.1.smithi136.stdout:9/505: dwrite d17/d20/d25/d3c/d55/f64 [0,4194304] 0 2024-04-06T03:40:38.587 INFO:tasks.workunit.client.0.smithi094.stdout:0/492: dwrite d1/d9/dd/d18/f31 [0,4194304] 0 2024-04-06T03:40:38.588 INFO:tasks.workunit.client.0.smithi094.stdout:0/493: dread d1/d9/dd/d18/d1e/f71 [0,4194304] 0 2024-04-06T03:40:38.597 INFO:tasks.workunit.client.1.smithi136.stdout:9/506: dread d17/d20/d28/d49/f54 [0,4194304] 0 2024-04-06T03:40:38.597 INFO:tasks.workunit.client.1.smithi136.stdout:9/507: dread - d17/d20/d28/d49/d75/f9d zero size 2024-04-06T03:40:38.597 INFO:tasks.workunit.client.1.smithi136.stdout:9/508: write d17/d19/f8e [4604176,120501] 0 2024-04-06T03:40:38.606 INFO:tasks.workunit.client.1.smithi136.stdout:9/509: dread d17/d21/d32/f4f [0,4194304] 0 2024-04-06T03:40:38.606 INFO:tasks.workunit.client.1.smithi136.stdout:9/510: chown d17/d21/d32/d43/d51/d57 26752 1 2024-04-06T03:40:38.611 INFO:tasks.workunit.client.1.smithi136.stdout:7/533: sync 2024-04-06T03:40:38.641 INFO:tasks.workunit.client.0.smithi094.stdout:3/562: creat d9/d1e/d26/d55/d62/db8/fca x:0 0 0 2024-04-06T03:40:38.641 INFO:tasks.workunit.client.0.smithi094.stdout:3/563: fsync d9/f8d 0 2024-04-06T03:40:38.642 INFO:tasks.workunit.client.0.smithi094.stdout:8/558: symlink d18/d3c/dc4/dbc/lc5 0 2024-04-06T03:40:38.642 INFO:tasks.workunit.client.0.smithi094.stdout:3/564: dread d9/d1e/f59 [0,4194304] 0 2024-04-06T03:40:38.644 INFO:tasks.workunit.client.0.smithi094.stdout:7/508: mkdir d2/d6/d93/d94 0 2024-04-06T03:40:38.645 INFO:tasks.workunit.client.0.smithi094.stdout:7/509: truncate d2/d6/dc/dd/f3e 1016549 0 2024-04-06T03:40:38.645 INFO:tasks.workunit.client.0.smithi094.stdout:7/510: stat d2/d6/dc/d12/f2a 0 2024-04-06T03:40:38.646 INFO:tasks.workunit.client.1.smithi136.stdout:5/582: mknod d1/d9b/d9c/da9/cb7 0 2024-04-06T03:40:38.646 INFO:tasks.workunit.client.1.smithi136.stdout:5/583: fsync d1/dc/d1c/d32/d70/fa8 0 2024-04-06T03:40:38.650 INFO:tasks.workunit.client.0.smithi094.stdout:2/587: symlink d0/dd/d1b/d72/d8f/lc7 0 2024-04-06T03:40:38.650 INFO:tasks.workunit.client.0.smithi094.stdout:2/588: chown d0/dd/da4/c8a 5676824 1 2024-04-06T03:40:38.651 INFO:tasks.workunit.client.0.smithi094.stdout:2/589: chown d0/dd/d19/d31/d67/l9b 16380 1 2024-04-06T03:40:38.663 INFO:tasks.workunit.client.0.smithi094.stdout:7/511: dwrite d2/d6/dc/dd/f3e [0,4194304] 0 2024-04-06T03:40:38.665 INFO:tasks.workunit.client.1.smithi136.stdout:5/584: dwrite d1/d16/f1b [0,4194304] 0 2024-04-06T03:40:38.665 INFO:tasks.workunit.client.1.smithi136.stdout:5/585: fsync d1/f96 0 2024-04-06T03:40:38.666 INFO:tasks.workunit.client.1.smithi136.stdout:6/458: rmdir d2 39 2024-04-06T03:40:38.666 INFO:tasks.workunit.client.1.smithi136.stdout:4/462: sync 2024-04-06T03:40:38.667 INFO:tasks.workunit.client.1.smithi136.stdout:1/505: sync 2024-04-06T03:40:38.667 INFO:tasks.workunit.client.1.smithi136.stdout:0/512: sync 2024-04-06T03:40:38.667 INFO:tasks.workunit.client.1.smithi136.stdout:8/568: sync 2024-04-06T03:40:38.667 INFO:tasks.workunit.client.1.smithi136.stdout:0/513: chown d1/fc 205306084 1 2024-04-06T03:40:38.667 INFO:tasks.workunit.client.1.smithi136.stdout:8/569: readlink d3/da/d26/d35/d3d/d48/d6c/d7f/l94 0 2024-04-06T03:40:38.667 INFO:tasks.workunit.client.1.smithi136.stdout:8/570: chown d3/de/d18/f1e 28 1 2024-04-06T03:40:38.667 INFO:tasks.workunit.client.1.smithi136.stdout:8/571: chown d3/da/d26/d35/d39 67819 1 2024-04-06T03:40:38.667 INFO:tasks.workunit.client.1.smithi136.stdout:8/572: chown d3/de/f4b 73887 1 2024-04-06T03:40:38.667 INFO:tasks.workunit.client.1.smithi136.stdout:8/573: fsync d3/de/f4c 0 2024-04-06T03:40:38.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:38 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:38.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:38 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:38.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:38 smithi094 ceph-mon[29713]: Reconfiguring alertmanager.smithi094 (dependencies changed)... 2024-04-06T03:40:38.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:38 smithi094 ceph-mon[29713]: Reconfiguring daemon alertmanager.smithi094 on smithi094 2024-04-06T03:40:38.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:38 smithi094 ceph-mon[29713]: pgmap v19: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 248 MiB/s rd, 328 MiB/s wr, 447 op/s 2024-04-06T03:40:38.670 INFO:tasks.workunit.client.1.smithi136.stdout:3/498: rename d0/d4/d12/d2d/d37/la0 to d0/d4/d12/d19/d55/d3c/d5a/la7 0 2024-04-06T03:40:38.670 INFO:tasks.workunit.client.1.smithi136.stdout:3/499: write d0/d4/d12/d2d/d37/f62 [317441,15945] 0 2024-04-06T03:40:38.679 INFO:tasks.workunit.client.1.smithi136.stdout:9/511: creat d17/d20/d25/d71/fa9 x:0 0 0 2024-04-06T03:40:38.682 INFO:tasks.workunit.client.1.smithi136.stdout:6/459: dwrite d2/d7/d26/d30/d6e/d58/d5d/d62/f64 [0,4194304] 0 2024-04-06T03:40:38.682 INFO:tasks.workunit.client.1.smithi136.stdout:6/460: fsync d2/d7/d26/d30/d6e/f10 0 2024-04-06T03:40:38.683 INFO:tasks.workunit.client.1.smithi136.stdout:5/586: mkdir d1/dc/d1c/d32/d8b/db8 0 2024-04-06T03:40:38.685 INFO:tasks.workunit.client.1.smithi136.stdout:1/506: mkdir d12/d3b/d60/d6d/da2 0 2024-04-06T03:40:38.688 INFO:tasks.workunit.client.1.smithi136.stdout:3/500: dwrite d0/fb [0,4194304] 0 2024-04-06T03:40:38.690 INFO:tasks.workunit.client.1.smithi136.stdout:0/514: link d1/d6/d2c/d57/f5f d1/d6/d2c/d57/d43/d6d/d85/d78/fad 0 2024-04-06T03:40:38.694 INFO:tasks.workunit.client.1.smithi136.stdout:4/463: rename d1d/d2f/d6c/fa4 to d1d/d31/d78/fa9 0 2024-04-06T03:40:38.695 INFO:tasks.workunit.client.1.smithi136.stdout:4/464: read f7 [1121744,15435] 0 2024-04-06T03:40:38.696 INFO:tasks.workunit.client.0.smithi094.stdout:8/559: getdents d18/d3c 0 2024-04-06T03:40:38.698 INFO:tasks.workunit.client.1.smithi136.stdout:9/512: symlink d17/d21/d29/d34/d66/laa 0 2024-04-06T03:40:38.700 INFO:tasks.workunit.client.0.smithi094.stdout:3/565: symlink d9/d1e/d6e/d75/dba/d7c/da2/lcb 0 2024-04-06T03:40:38.700 INFO:tasks.workunit.client.0.smithi094.stdout:3/566: fdatasync d9/d1e/f8f 0 2024-04-06T03:40:38.700 INFO:tasks.workunit.client.0.smithi094.stdout:3/567: truncate d9/d20/d76/fac 213821 0 2024-04-06T03:40:38.700 INFO:tasks.workunit.client.0.smithi094.stdout:3/568: read - d9/d1e/d6e/d75/dba/f98 zero size 2024-04-06T03:40:38.703 INFO:tasks.workunit.client.0.smithi094.stdout:2/590: mknod d0/d8/d4f/dab/dc3/cc8 0 2024-04-06T03:40:38.705 INFO:tasks.workunit.client.0.smithi094.stdout:0/494: getdents d1/d9/dd/d18/d1e 0 2024-04-06T03:40:38.710 INFO:tasks.workunit.client.1.smithi136.stdout:1/507: unlink d12/d1b/c4a 0 2024-04-06T03:40:38.710 INFO:tasks.workunit.client.1.smithi136.stdout:1/508: creat d12/d1b/d1e/d23/fa3 x:0 0 0 2024-04-06T03:40:38.710 INFO:tasks.workunit.client.1.smithi136.stdout:1/509: dread - d12/d1b/d1e/d23/f92 zero size 2024-04-06T03:40:38.710 INFO:tasks.workunit.client.1.smithi136.stdout:6/461: dwrite d2/d7/d26/d30/d6e/d3d/f81 [0,4194304] 0 2024-04-06T03:40:38.711 INFO:tasks.workunit.client.1.smithi136.stdout:4/465: dwrite d1d/d31/d78/f7a [0,4194304] 0 2024-04-06T03:40:38.712 INFO:tasks.workunit.client.1.smithi136.stdout:3/501: dwrite d0/d4/fe [0,4194304] 0 2024-04-06T03:40:38.712 INFO:tasks.workunit.client.1.smithi136.stdout:4/466: dread d1d/d31/f4c [0,4194304] 0 2024-04-06T03:40:38.712 INFO:tasks.workunit.client.1.smithi136.stdout:4/467: write d1d/d31/d3e/d55/f99 [48471,86612] 0 2024-04-06T03:40:38.713 INFO:tasks.workunit.client.1.smithi136.stdout:0/515: creat d1/d6/d2c/d1a/d9c/fae x:0 0 0 2024-04-06T03:40:38.716 INFO:tasks.workunit.client.0.smithi094.stdout:8/560: rmdir d18/d5c 39 2024-04-06T03:40:38.719 INFO:tasks.workunit.client.0.smithi094.stdout:3/569: symlink d9/d1e/d26/d6c/lcc 0 2024-04-06T03:40:38.721 INFO:tasks.workunit.client.0.smithi094.stdout:2/591: mknod d0/dd/d19/d31/d67/cc9 0 2024-04-06T03:40:38.722 INFO:tasks.workunit.client.1.smithi136.stdout:9/513: creat d17/d20/d25/d3d/d4b/fab x:0 0 0 2024-04-06T03:40:38.723 INFO:tasks.workunit.client.0.smithi094.stdout:0/495: mkdir d1/d9/dd/d18/d1a/da5 0 2024-04-06T03:40:38.724 INFO:tasks.workunit.client.0.smithi094.stdout:0/496: chown d1/d9/dd/d18/d1d/l34 735841350 1 2024-04-06T03:40:38.724 INFO:tasks.workunit.client.0.smithi094.stdout:0/497: creat d1/d9/dd/d18/d1e/d86/fa6 x:0 0 0 2024-04-06T03:40:38.726 INFO:tasks.workunit.client.0.smithi094.stdout:7/512: rmdir d2/d6/dc/d12/d3a/d56 39 2024-04-06T03:40:38.727 INFO:tasks.workunit.client.0.smithi094.stdout:8/561: symlink d18/d2c/d37/d60/da3/db1/lc6 0 2024-04-06T03:40:38.727 INFO:tasks.workunit.client.0.smithi094.stdout:8/562: fdatasync d18/d3c/d49/d9f/fb3 0 2024-04-06T03:40:38.729 INFO:tasks.workunit.client.0.smithi094.stdout:2/592: creat d0/dd/d1b/d72/fca x:0 0 0 2024-04-06T03:40:38.731 INFO:tasks.workunit.client.1.smithi136.stdout:5/587: unlink d1/dc/d10/f6d 0 2024-04-06T03:40:38.733 INFO:tasks.workunit.client.0.smithi094.stdout:0/498: dread d1/f23 [0,4194304] 0 2024-04-06T03:40:38.734 INFO:tasks.workunit.client.1.smithi136.stdout:6/462: symlink d2/d7/d26/d30/d6e/d3d/d47/l9d 0 2024-04-06T03:40:38.735 INFO:tasks.workunit.client.0.smithi094.stdout:3/570: dwrite d9/d20/d76/fac [0,4194304] 0 2024-04-06T03:40:38.738 INFO:tasks.workunit.client.0.smithi094.stdout:7/513: mkdir d2/d6/dc/dd/df/d95 0 2024-04-06T03:40:38.739 INFO:tasks.workunit.client.0.smithi094.stdout:7/514: truncate d2/d6/dc/d12/d29/d35/f8d 1033245 0 2024-04-06T03:40:38.742 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:38 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:38.742 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:38 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:38.742 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:38 smithi136 ceph-mon[36436]: Reconfiguring alertmanager.smithi094 (dependencies changed)... 2024-04-06T03:40:38.742 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:38 smithi136 ceph-mon[36436]: Reconfiguring daemon alertmanager.smithi094 on smithi094 2024-04-06T03:40:38.742 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:38 smithi136 ceph-mon[36436]: pgmap v19: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 248 MiB/s rd, 328 MiB/s wr, 447 op/s 2024-04-06T03:40:38.748 INFO:tasks.workunit.client.0.smithi094.stdout:2/593: rename d0/dd/d19/d31/f86 to d0/dd/da4/fcb 0 2024-04-06T03:40:38.748 INFO:tasks.workunit.client.0.smithi094.stdout:2/594: readlink d0/dd/d3d/d49/lb1 0 2024-04-06T03:40:38.751 INFO:tasks.workunit.client.1.smithi136.stdout:4/468: creat d1d/d2f/d6c/da5/faa x:0 0 0 2024-04-06T03:40:38.752 INFO:tasks.workunit.client.1.smithi136.stdout:0/516: fsync d1/d6/d2c/f40 0 2024-04-06T03:40:38.754 INFO:tasks.workunit.client.1.smithi136.stdout:9/514: truncate d17/d21/d29/f2a 763571 0 2024-04-06T03:40:38.756 INFO:tasks.workunit.client.1.smithi136.stdout:7/534: mkdir d5/df/d12/d1b/dbb 0 2024-04-06T03:40:38.756 INFO:tasks.workunit.client.1.smithi136.stdout:7/535: chown d5/df/d1d/d6f/d2c/f81 14016 1 2024-04-06T03:40:38.757 INFO:tasks.workunit.client.1.smithi136.stdout:0/517: link d1/c5b d1/d6/d2c/d57/d43/d6d/d85/caf 0 2024-04-06T03:40:38.758 INFO:tasks.workunit.client.0.smithi094.stdout:7/515: getdents d2/d6/dc/d12/d3a/d56 0 2024-04-06T03:40:38.758 INFO:tasks.workunit.client.0.smithi094.stdout:7/516: write d2/d6/dc/dd/df/f2d [2325477,10136] 0 2024-04-06T03:40:38.758 INFO:tasks.workunit.client.0.smithi094.stdout:7/517: chown d2/d6/d93 29477 1 2024-04-06T03:40:38.759 INFO:tasks.workunit.client.1.smithi136.stdout:9/515: rmdir d17/d20 39 2024-04-06T03:40:38.759 INFO:tasks.workunit.client.0.smithi094.stdout:0/499: dwrite d1/d9/dd/d18/d32/f9c [0,4194304] 0 2024-04-06T03:40:38.760 INFO:tasks.workunit.client.0.smithi094.stdout:2/595: truncate d0/d8/d27/d2b/f5d 3147456 0 2024-04-06T03:40:38.762 INFO:tasks.workunit.client.1.smithi136.stdout:7/536: creat d5/df/d53/d8a/fbc x:0 0 0 2024-04-06T03:40:38.763 INFO:tasks.workunit.client.1.smithi136.stdout:6/463: dwrite d2/d7/d26/d30/d6e/f69 [0,4194304] 0 2024-04-06T03:40:38.763 INFO:tasks.workunit.client.1.smithi136.stdout:6/464: chown d2/d7/d26/d30/d6e 2010514 1 2024-04-06T03:40:38.763 INFO:tasks.workunit.client.1.smithi136.stdout:6/465: chown d2/d7/d13/f9a 1764 1 2024-04-06T03:40:38.763 INFO:tasks.workunit.client.1.smithi136.stdout:6/466: read d2/d7/d26/d30/d6e/f6b [796376,45435] 0 2024-04-06T03:40:38.763 INFO:tasks.workunit.client.1.smithi136.stdout:6/467: fdatasync d2/d7/d26/d30/d6e/f35 0 2024-04-06T03:40:38.763 INFO:tasks.workunit.client.0.smithi094.stdout:7/518: symlink d2/d6/d93/l96 0 2024-04-06T03:40:38.765 INFO:tasks.workunit.client.1.smithi136.stdout:9/516: truncate f16 1188671 0 2024-04-06T03:40:38.765 INFO:tasks.workunit.client.1.smithi136.stdout:9/517: write d17/d21/d29/d34/d66/f9b [16337,110632] 0 2024-04-06T03:40:38.766 INFO:tasks.workunit.client.1.smithi136.stdout:7/537: symlink d5/df/d12/d1b/d3a/d4a/lbd 0 2024-04-06T03:40:38.768 INFO:tasks.workunit.client.1.smithi136.stdout:5/588: dwrite d1/d16/d4b/f57 [0,4194304] 0 2024-04-06T03:40:38.768 INFO:tasks.workunit.client.1.smithi136.stdout:5/589: chown d1/dc/d10/d1e/l51 604260 1 2024-04-06T03:40:38.769 INFO:tasks.workunit.client.0.smithi094.stdout:8/563: dwrite d18/f67 [0,4194304] 0 2024-04-06T03:40:38.770 INFO:tasks.workunit.client.0.smithi094.stdout:2/596: mkdir d0/dd/d19/d31/dcc 0 2024-04-06T03:40:38.771 INFO:tasks.workunit.client.1.smithi136.stdout:0/518: dwrite d1/d6/d2c/d57/d43/d6d/d85/d78/f9a [0,4194304] 0 2024-04-06T03:40:38.772 INFO:tasks.workunit.client.1.smithi136.stdout:0/519: creat d1/d4e/fb0 x:0 0 0 2024-04-06T03:40:38.772 INFO:tasks.workunit.client.1.smithi136.stdout:0/520: creat d1/d6/d2c/d1a/d36/fb1 x:0 0 0 2024-04-06T03:40:38.772 INFO:tasks.workunit.client.0.smithi094.stdout:7/519: rmdir d2/d6/d93/d94 0 2024-04-06T03:40:38.774 INFO:tasks.workunit.client.0.smithi094.stdout:0/500: rmdir d1/d9/dd/d18/d32/d42 39 2024-04-06T03:40:38.777 INFO:tasks.workunit.client.1.smithi136.stdout:7/538: mkdir d5/df/d12/d1b/dbb/dbe 0 2024-04-06T03:40:38.779 INFO:tasks.workunit.client.1.smithi136.stdout:6/468: dwrite d2/d7/d26/d30/d4f/f6d [0,4194304] 0 2024-04-06T03:40:38.781 INFO:tasks.workunit.client.1.smithi136.stdout:5/590: creat d1/dc/d1c/d32/d8b/d20/d66/fb9 x:0 0 0 2024-04-06T03:40:38.791 INFO:tasks.workunit.client.0.smithi094.stdout:7/520: dread d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:38.791 INFO:tasks.workunit.client.0.smithi094.stdout:7/521: chown d2/d6/dc/f74 1708 1 2024-04-06T03:40:38.791 INFO:tasks.workunit.client.0.smithi094.stdout:7/522: chown d2/d6/fa 4623874 1 2024-04-06T03:40:38.791 INFO:tasks.workunit.client.1.smithi136.stdout:0/521: link d1/d6/d2c/d1a/l32 d1/d6/d2c/d57/d43/d6d/d9b/d5c/d96/lb2 0 2024-04-06T03:40:38.792 INFO:tasks.workunit.client.0.smithi094.stdout:2/597: creat d0/d8/d1d/d77/fcd x:0 0 0 2024-04-06T03:40:38.793 INFO:tasks.workunit.client.1.smithi136.stdout:7/539: mknod d5/df/d3d/cbf 0 2024-04-06T03:40:38.795 INFO:tasks.workunit.client.0.smithi094.stdout:7/523: rmdir d2/d6/dc/d12/d29 39 2024-04-06T03:40:38.798 INFO:tasks.workunit.client.0.smithi094.stdout:0/501: dwrite d1/d9/dd/d18/d1a/f46 [0,4194304] 0 2024-04-06T03:40:38.799 INFO:tasks.workunit.client.1.smithi136.stdout:6/469: dwrite d2/d7/d26/d30/d4f/f5f [4194304,4194304] 0 2024-04-06T03:40:38.799 INFO:tasks.workunit.client.1.smithi136.stdout:6/470: dread - d2/d7/d26/d30/d4f/d75/f82 zero size 2024-04-06T03:40:38.800 INFO:tasks.workunit.client.0.smithi094.stdout:2/598: symlink d0/dd/d1b/lce 0 2024-04-06T03:40:38.800 INFO:tasks.workunit.client.0.smithi094.stdout:2/599: write d0/dd/d19/d31/f7d [2927273,114938] 0 2024-04-06T03:40:38.800 INFO:tasks.workunit.client.0.smithi094.stdout:2/600: creat d0/d8/d1d/d77/fcf x:0 0 0 2024-04-06T03:40:38.800 INFO:tasks.workunit.client.0.smithi094.stdout:2/601: read - d0/d8/fb3 zero size 2024-04-06T03:40:38.800 INFO:tasks.workunit.client.0.smithi094.stdout:2/602: fsync d0/dd/d19/d31/f3a 0 2024-04-06T03:40:38.800 INFO:tasks.workunit.client.0.smithi094.stdout:9/477: unlink d2/dc/d10/f12 0 2024-04-06T03:40:38.801 INFO:tasks.workunit.client.0.smithi094.stdout:9/478: truncate d2/dc/d21/d23/fa3 544277 0 2024-04-06T03:40:38.801 INFO:tasks.workunit.client.0.smithi094.stdout:9/479: fsync d2/dc/d10/d2a/fa2 0 2024-04-06T03:40:38.801 INFO:tasks.workunit.client.0.smithi094.stdout:9/480: fdatasync d2/dc/d21/d23/d2f/f91 0 2024-04-06T03:40:38.801 INFO:tasks.workunit.client.0.smithi094.stdout:9/481: chown d2/dc/d21/d23/d47 136959 1 2024-04-06T03:40:38.801 INFO:tasks.workunit.client.0.smithi094.stdout:9/482: readlink d2/dc/d21/l5d 0 2024-04-06T03:40:38.801 INFO:tasks.workunit.client.1.smithi136.stdout:2/614: sync 2024-04-06T03:40:38.802 INFO:tasks.workunit.client.1.smithi136.stdout:0/522: mknod d1/d6/d2c/d57/d43/d6d/d9b/cb3 0 2024-04-06T03:40:38.803 INFO:tasks.workunit.client.1.smithi136.stdout:2/615: dread d3/d4a/d9d/d8a/d93/d5f/f82 [0,4194304] 0 2024-04-06T03:40:38.803 INFO:tasks.workunit.client.1.smithi136.stdout:2/616: stat d3/d4a/d9d/d18/d4b/f65 0 2024-04-06T03:40:38.803 INFO:tasks.workunit.client.1.smithi136.stdout:7/540: creat d5/df/d12/d1b/d3a/d4a/d70/fc0 x:0 0 0 2024-04-06T03:40:38.803 INFO:tasks.workunit.client.1.smithi136.stdout:7/541: chown d5/df/d53 116 1 2024-04-06T03:40:38.804 INFO:tasks.workunit.client.1.smithi136.stdout:7/542: stat d5/df/d11 0 2024-04-06T03:40:38.805 INFO:tasks.workunit.client.0.smithi094.stdout:9/483: creat d2/dc/d21/fb5 x:0 0 0 2024-04-06T03:40:38.805 INFO:tasks.workunit.client.0.smithi094.stdout:9/484: stat d2/dc/d21/d23/l4b 0 2024-04-06T03:40:38.808 INFO:tasks.workunit.client.0.smithi094.stdout:9/485: mknod d2/dc/d21/d26/d78/d69/cb6 0 2024-04-06T03:40:38.810 INFO:tasks.workunit.client.0.smithi094.stdout:9/486: mkdir d2/dc/d80/db7 0 2024-04-06T03:40:38.811 INFO:tasks.workunit.client.1.smithi136.stdout:6/471: mkdir d2/d7/d9e 0 2024-04-06T03:40:38.812 INFO:tasks.workunit.client.0.smithi094.stdout:8/564: rename d18/d3c/d76/c59 to d18/d2c/cc7 0 2024-04-06T03:40:38.812 INFO:tasks.workunit.client.0.smithi094.stdout:8/565: write d18/d30/f77 [1140556,55127] 0 2024-04-06T03:40:38.814 INFO:tasks.workunit.client.0.smithi094.stdout:9/487: symlink d2/dc/d21/d26/d78/d69/lb8 0 2024-04-06T03:40:38.814 INFO:tasks.workunit.client.0.smithi094.stdout:9/488: write d2/dc/d21/d26/d78/fae [319230,111083] 0 2024-04-06T03:40:38.814 INFO:tasks.workunit.client.0.smithi094.stdout:9/489: chown d2/dc/d21/d26 7455 1 2024-04-06T03:40:38.815 INFO:tasks.workunit.client.1.smithi136.stdout:0/523: truncate d1/d6/f1f 3560974 0 2024-04-06T03:40:38.815 INFO:tasks.workunit.client.1.smithi136.stdout:0/524: truncate d1/d6/d2c/d57/d43/d6d/d9b/d5c/f91 635498 0 2024-04-06T03:40:38.816 INFO:tasks.workunit.client.0.smithi094.stdout:7/524: rename d2/d6/dc/d12/d29/c52 to d2/d6/dc/d12/d29/d35/d57/c97 0 2024-04-06T03:40:38.816 INFO:tasks.workunit.client.0.smithi094.stdout:7/525: chown d2/d6/dc/d12/d29/d35/d57/c88 1059108978 1 2024-04-06T03:40:38.817 INFO:tasks.workunit.client.1.smithi136.stdout:2/617: symlink d3/d15/d19/d43/d54/lc4 0 2024-04-06T03:40:38.818 INFO:tasks.workunit.client.0.smithi094.stdout:8/566: mkdir d18/d3c/d49/d91/d94/daa/dc8 0 2024-04-06T03:40:38.819 INFO:tasks.workunit.client.1.smithi136.stdout:7/543: creat d5/df/d53/d68/d8f/fc1 x:0 0 0 2024-04-06T03:40:38.820 INFO:tasks.workunit.client.0.smithi094.stdout:8/567: write d18/f25 [376900,111236] 0 2024-04-06T03:40:38.820 INFO:tasks.workunit.client.0.smithi094.stdout:9/490: rmdir d2/dc 39 2024-04-06T03:40:38.820 INFO:tasks.workunit.client.0.smithi094.stdout:9/491: chown d2/dc/d21/f9c 2 1 2024-04-06T03:40:38.821 INFO:tasks.workunit.client.1.smithi136.stdout:6/472: getdents d2/d7/d26/d30 0 2024-04-06T03:40:38.824 INFO:tasks.workunit.client.1.smithi136.stdout:2/618: write d3/d4a/d9d/d18/d1d/d3c/f51 [434361,32662] 0 2024-04-06T03:40:38.824 INFO:tasks.workunit.client.0.smithi094.stdout:8/568: mknod d18/d3c/dc4/db9/cc9 0 2024-04-06T03:40:38.825 INFO:tasks.workunit.client.0.smithi094.stdout:8/569: chown d18/d2c/d62/f89 45 1 2024-04-06T03:40:38.826 INFO:tasks.workunit.client.0.smithi094.stdout:9/492: mknod d2/dc/d21/d26/cb9 0 2024-04-06T03:40:38.826 INFO:tasks.workunit.client.0.smithi094.stdout:9/493: write d2/dc/d21/d23/d47/f9b [315366,57922] 0 2024-04-06T03:40:38.827 INFO:tasks.workunit.client.1.smithi136.stdout:2/619: dread d3/d4a/d9d/d18/d4b/f5d [4194304,4194304] 0 2024-04-06T03:40:38.827 INFO:tasks.workunit.client.0.smithi094.stdout:7/526: dread d2/d6/dc/dd/f3e [0,4194304] 0 2024-04-06T03:40:38.827 INFO:tasks.workunit.client.0.smithi094.stdout:7/527: chown d2/d6/dc/d12/d29/d3f/c47 3 1 2024-04-06T03:40:38.828 INFO:tasks.workunit.client.1.smithi136.stdout:7/544: getdents d5/df/d1d/d54 0 2024-04-06T03:40:38.828 INFO:tasks.workunit.client.1.smithi136.stdout:7/545: dread - d5/df/d12/d1b/d58/f7a zero size 2024-04-06T03:40:38.830 INFO:tasks.workunit.client.0.smithi094.stdout:2/603: rename d0/dd/da4/c8a to d0/cd0 0 2024-04-06T03:40:38.830 INFO:tasks.workunit.client.0.smithi094.stdout:2/604: readlink d0/dd/d3d/d49/l68 0 2024-04-06T03:40:38.830 INFO:tasks.workunit.client.0.smithi094.stdout:2/605: chown d0/d8/d27/d2b/c43 80550 1 2024-04-06T03:40:38.832 INFO:tasks.workunit.client.0.smithi094.stdout:2/606: read d0/d8/f33 [516212,22542] 0 2024-04-06T03:40:38.832 INFO:tasks.workunit.client.0.smithi094.stdout:8/570: symlink d18/d34/lca 0 2024-04-06T03:40:38.832 INFO:tasks.workunit.client.1.smithi136.stdout:6/473: read d2/d7/d26/d30/d6e/f10 [7322167,82499] 0 2024-04-06T03:40:38.832 INFO:tasks.workunit.client.1.smithi136.stdout:2/620: truncate d3/d4a/d9d/d18/d1d/d3c/fa0 3973764 0 2024-04-06T03:40:38.833 INFO:tasks.workunit.client.0.smithi094.stdout:9/494: mkdir d2/dc/d21/d26/dba 0 2024-04-06T03:40:38.835 INFO:tasks.workunit.client.1.smithi136.stdout:0/525: dwrite d1/d6/d2c/d57/f66 [0,4194304] 0 2024-04-06T03:40:38.835 INFO:tasks.workunit.client.1.smithi136.stdout:7/546: creat d5/df/d1d/d6f/fc2 x:0 0 0 2024-04-06T03:40:38.836 INFO:tasks.workunit.client.0.smithi094.stdout:7/528: link d2/d6/dc/d12/d29/d35/d57/c87 d2/d6/dc/d12/d29/d35/c98 0 2024-04-06T03:40:38.836 INFO:tasks.workunit.client.0.smithi094.stdout:7/529: fdatasync d2/f11 0 2024-04-06T03:40:38.837 INFO:tasks.workunit.client.1.smithi136.stdout:6/474: mkdir d2/d7/d26/d9f 0 2024-04-06T03:40:38.837 INFO:tasks.workunit.client.0.smithi094.stdout:2/607: truncate d0/f57 4051671 0 2024-04-06T03:40:38.840 INFO:tasks.workunit.client.1.smithi136.stdout:0/526: link d1/d6/d2c/d57/d43/d6d/d9b/d5c/f71 d1/d6/d2c/d57/d43/d6d/d85/da7/fb4 0 2024-04-06T03:40:38.840 INFO:tasks.workunit.client.1.smithi136.stdout:0/527: fdatasync d1/f63 0 2024-04-06T03:40:38.842 INFO:tasks.workunit.client.0.smithi094.stdout:8/571: unlink d18/d3c/d49/l61 0 2024-04-06T03:40:38.842 INFO:tasks.workunit.client.0.smithi094.stdout:8/572: chown d18/d3c/d49/f7b 192775 1 2024-04-06T03:40:38.842 INFO:tasks.workunit.client.0.smithi094.stdout:8/573: chown d18/lad 11 1 2024-04-06T03:40:38.842 INFO:tasks.workunit.client.1.smithi136.stdout:7/547: truncate d5/df/d12/d1b/d3a/d4a/f64 458167 0 2024-04-06T03:40:38.843 INFO:tasks.workunit.client.1.smithi136.stdout:7/548: chown d5/df/d12/d1b/d3a/fa6 11008 1 2024-04-06T03:40:38.843 INFO:tasks.workunit.client.1.smithi136.stdout:7/549: write d5/df/d1d/d6f/d2c/f81 [359459,35905] 0 2024-04-06T03:40:38.843 INFO:tasks.workunit.client.1.smithi136.stdout:7/550: chown d5/df/d12/d1b/d3a/d8d/l95 27658 1 2024-04-06T03:40:38.843 INFO:tasks.workunit.client.0.smithi094.stdout:9/495: mknod d2/dc/d46/cbb 0 2024-04-06T03:40:38.843 INFO:tasks.workunit.client.0.smithi094.stdout:9/496: stat d2/dc/d21/d26/cb9 0 2024-04-06T03:40:38.849 INFO:tasks.workunit.client.1.smithi136.stdout:7/551: dread d5/df/d1d/d6f/d4d/d69/f7b [0,4194304] 0 2024-04-06T03:40:38.857 INFO:tasks.workunit.client.1.smithi136.stdout:7/552: dread d5/df/d53/f39 [0,4194304] 0 2024-04-06T03:40:38.858 INFO:tasks.workunit.client.1.smithi136.stdout:6/475: creat d2/d7/d26/d30/d6e/d3d/d47/d84/fa0 x:0 0 0 2024-04-06T03:40:38.860 INFO:tasks.workunit.client.0.smithi094.stdout:7/530: symlink d2/d6/dc/d12/d29/d3f/l99 0 2024-04-06T03:40:38.860 INFO:tasks.workunit.client.0.smithi094.stdout:7/531: readlink d2/d6/dc/dd/df/l6f 0 2024-04-06T03:40:38.862 INFO:tasks.workunit.client.1.smithi136.stdout:3/502: rename d0/d4/d12/d19/d55/c64 to d0/d4/d12/d2d/d37/ca8 0 2024-04-06T03:40:38.862 INFO:tasks.workunit.client.1.smithi136.stdout:3/503: write d0/d4/d12/d2d/d37/f6d [745160,125208] 0 2024-04-06T03:40:38.862 INFO:tasks.workunit.client.1.smithi136.stdout:3/504: fsync d0/d4/d12/d19/d55/f24 0 2024-04-06T03:40:38.865 INFO:tasks.workunit.client.1.smithi136.stdout:7/553: creat d5/df/d12/d1b/d3a/fc3 x:0 0 0 2024-04-06T03:40:38.868 INFO:tasks.workunit.client.1.smithi136.stdout:6/476: creat d2/d7/d26/d30/d6e/d58/d5d/d62/d53/fa1 x:0 0 0 2024-04-06T03:40:38.868 INFO:tasks.workunit.client.1.smithi136.stdout:6/477: fsync d2/d7/d26/d30/d6e/d58/f7d 0 2024-04-06T03:40:38.871 INFO:tasks.workunit.client.1.smithi136.stdout:4/469: rename d1d/f2b to d1d/d2f/fab 0 2024-04-06T03:40:38.873 INFO:tasks.workunit.client.1.smithi136.stdout:3/505: symlink d0/d4/d12/d19/d55/d3c/la9 0 2024-04-06T03:40:38.873 INFO:tasks.workunit.client.1.smithi136.stdout:3/506: fsync d0/d4/d12/d19/d55/d6f/d82/d5d/d61/f8f 0 2024-04-06T03:40:38.875 INFO:tasks.workunit.client.0.smithi094.stdout:8/574: link d18/d2c/d37/d60/lbd d18/d2c/d37/d60/da3/lcb 0 2024-04-06T03:40:38.875 INFO:tasks.workunit.client.0.smithi094.stdout:8/575: write d18/d3c/dc4/fbf [988798,57118] 0 2024-04-06T03:40:38.875 INFO:tasks.workunit.client.1.smithi136.stdout:3/507: dread d0/d4/f45 [0,4194304] 0 2024-04-06T03:40:38.876 INFO:tasks.workunit.client.1.smithi136.stdout:0/528: getdents d1/d6 0 2024-04-06T03:40:38.878 INFO:tasks.workunit.client.0.smithi094.stdout:9/497: rename d2/dc to d2/dc/d10/d6a/dbc 22 2024-04-06T03:40:38.879 INFO:tasks.workunit.client.1.smithi136.stdout:7/554: mkdir d5/df/d12/d1b/d3a/d4a/dc4 0 2024-04-06T03:40:38.880 INFO:tasks.workunit.client.0.smithi094.stdout:7/532: rmdir d2 39 2024-04-06T03:40:38.880 INFO:tasks.workunit.client.0.smithi094.stdout:7/533: fdatasync d2/d6/dc/dd/f48 0 2024-04-06T03:40:38.880 INFO:tasks.workunit.client.0.smithi094.stdout:7/534: fsync d2/d6/dc/f4a 0 2024-04-06T03:40:38.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/576: symlink d18/d3c/lcc 0 2024-04-06T03:40:38.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/577: fdatasync d18/d3c/d49/d9f/fb3 0 2024-04-06T03:40:38.884 INFO:tasks.workunit.client.0.smithi094.stdout:7/535: rmdir d2/d6/dc/dd/df/d36/d79 39 2024-04-06T03:40:38.886 INFO:tasks.workunit.client.0.smithi094.stdout:8/578: symlink d18/d34/lcd 0 2024-04-06T03:40:38.887 INFO:tasks.workunit.client.0.smithi094.stdout:7/536: mkdir d2/d6/dc/d12/d9a 0 2024-04-06T03:40:38.888 INFO:tasks.workunit.client.1.smithi136.stdout:1/510: mknod d12/d3b/ca4 0 2024-04-06T03:40:38.888 INFO:tasks.workunit.client.1.smithi136.stdout:1/511: fsync d12/d1b/d1e/d23/f92 0 2024-04-06T03:40:38.889 INFO:tasks.workunit.client.0.smithi094.stdout:3/571: mkdir d9/d20/d76/dcd 0 2024-04-06T03:40:38.889 INFO:tasks.workunit.client.0.smithi094.stdout:3/572: chown d9/d1e/d26/l5e 7 1 2024-04-06T03:40:38.889 INFO:tasks.workunit.client.0.smithi094.stdout:3/573: chown d9/d1d/c31 11 1 2024-04-06T03:40:38.890 INFO:tasks.workunit.client.0.smithi094.stdout:8/579: creat d18/d3c/d49/d91/d94/fce x:0 0 0 2024-04-06T03:40:38.890 INFO:tasks.workunit.client.0.smithi094.stdout:8/580: write d18/d2c/d62/f89 [523350,74974] 0 2024-04-06T03:40:38.894 INFO:tasks.workunit.client.1.smithi136.stdout:6/478: truncate d2/d7/f5a 717679 0 2024-04-06T03:40:38.895 INFO:tasks.workunit.client.0.smithi094.stdout:7/537: mknod d2/c9b 0 2024-04-06T03:40:38.898 INFO:tasks.workunit.client.1.smithi136.stdout:9/518: rename d17/d21/d29/f3a to d17/d20/d28/d81/d88/fac 0 2024-04-06T03:40:38.898 INFO:tasks.workunit.client.1.smithi136.stdout:9/519: dread - d17/d20/d28/d49/d75/f9d zero size 2024-04-06T03:40:38.900 INFO:tasks.workunit.client.0.smithi094.stdout:3/574: rename d9/d20/c8a to d9/d10/cce 0 2024-04-06T03:40:38.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/538: creat d2/d6/dc/d12/d3a/d56/f9c x:0 0 0 2024-04-06T03:40:38.902 INFO:tasks.workunit.client.0.smithi094.stdout:7/539: chown d2/d6/dc/d12 0 1 2024-04-06T03:40:38.904 INFO:tasks.workunit.client.1.smithi136.stdout:3/508: creat d0/d4/d12/d19/d55/d3c/faa x:0 0 0 2024-04-06T03:40:38.906 INFO:tasks.workunit.client.1.smithi136.stdout:0/529: symlink d1/d6/d2c/d57/d43/d6b/lb5 0 2024-04-06T03:40:38.906 INFO:tasks.workunit.client.1.smithi136.stdout:0/530: fdatasync d1/d6/d2c/f88 0 2024-04-06T03:40:38.909 INFO:tasks.workunit.client.0.smithi094.stdout:7/540: symlink d2/d6/dc/d12/d3a/d56/l9d 0 2024-04-06T03:40:38.910 INFO:tasks.workunit.client.0.smithi094.stdout:7/541: write d2/d6/dc/f2b [2949281,5343] 0 2024-04-06T03:40:38.913 INFO:tasks.workunit.client.1.smithi136.stdout:7/555: mknod d5/df/d53/cc5 0 2024-04-06T03:40:38.913 INFO:tasks.workunit.client.1.smithi136.stdout:7/556: fdatasync d5/df/d11/f2b 0 2024-04-06T03:40:38.913 INFO:tasks.workunit.client.1.smithi136.stdout:1/512: creat d12/d3b/d44/fa5 x:0 0 0 2024-04-06T03:40:38.913 INFO:tasks.workunit.client.1.smithi136.stdout:1/513: write d12/d3b/d6b/f70 [854984,107870] 0 2024-04-06T03:40:38.914 INFO:tasks.workunit.client.0.smithi094.stdout:7/542: dread d2/d6/f8 [0,4194304] 0 2024-04-06T03:40:38.914 INFO:tasks.workunit.client.0.smithi094.stdout:7/543: dread - d2/f63 zero size 2024-04-06T03:40:38.914 INFO:tasks.workunit.client.0.smithi094.stdout:7/544: getdents d2/d6/dc/d12/d9a 0 2024-04-06T03:40:38.914 INFO:tasks.workunit.client.0.smithi094.stdout:7/545: chown d2/d6/dc/dd/df/c6e 114 1 2024-04-06T03:40:38.916 INFO:tasks.workunit.client.1.smithi136.stdout:6/479: link d2/d7/d26/d30/d6e/d58/c5c d2/d7/d13/ca2 0 2024-04-06T03:40:38.917 INFO:tasks.workunit.client.0.smithi094.stdout:7/546: symlink d2/d6/dc/dd/d5a/l9e 0 2024-04-06T03:40:38.918 INFO:tasks.workunit.client.0.smithi094.stdout:9/498: dwrite d2/fe [0,4194304] 0 2024-04-06T03:40:38.918 INFO:tasks.workunit.client.0.smithi094.stdout:9/499: readlink d2/dc/d21/d23/l4b 0 2024-04-06T03:40:38.918 INFO:tasks.workunit.client.0.smithi094.stdout:9/500: creat d2/d2c/d99/fbd x:0 0 0 2024-04-06T03:40:38.919 INFO:tasks.workunit.client.1.smithi136.stdout:9/520: dwrite d17/d20/d25/d71/fa2 [0,4194304] 0 2024-04-06T03:40:38.920 INFO:tasks.workunit.client.0.smithi094.stdout:8/581: dwrite d18/f22 [0,4194304] 0 2024-04-06T03:40:38.920 INFO:tasks.workunit.client.0.smithi094.stdout:8/582: truncate d18/d3c/d76/f5b 9058308 0 2024-04-06T03:40:38.921 INFO:tasks.workunit.client.1.smithi136.stdout:2/621: rename d3/d4a/d9d/d18/f48 to d3/d4a/d9d/d8a/d93/fc5 0 2024-04-06T03:40:38.922 INFO:tasks.workunit.client.0.smithi094.stdout:7/547: symlink d2/d6/dc/dd/df/l9f 0 2024-04-06T03:40:38.924 INFO:tasks.workunit.client.1.smithi136.stdout:4/470: unlink d1d/f22 0 2024-04-06T03:40:38.930 INFO:tasks.workunit.client.0.smithi094.stdout:9/501: mkdir d2/dc/d21/d23/db0/dbe 0 2024-04-06T03:40:38.932 INFO:tasks.workunit.client.1.smithi136.stdout:7/557: creat d5/df/d1d/fc6 x:0 0 0 2024-04-06T03:40:38.932 INFO:tasks.workunit.client.0.smithi094.stdout:0/502: dwrite d1/d9/dd/d18/d32/f9c [4194304,4194304] 0 2024-04-06T03:40:38.935 INFO:tasks.workunit.client.1.smithi136.stdout:1/514: symlink d12/d3b/d44/d5f/d5a/la6 0 2024-04-06T03:40:38.936 INFO:tasks.workunit.client.0.smithi094.stdout:7/548: mkdir d2/d6/dc/d12/d9a/da0 0 2024-04-06T03:40:38.936 INFO:tasks.workunit.client.0.smithi094.stdout:7/549: chown d2/d6/dc/d12/l8b 0 1 2024-04-06T03:40:38.937 INFO:tasks.workunit.client.1.smithi136.stdout:6/480: mknod d2/d7/ca3 0 2024-04-06T03:40:38.941 INFO:tasks.workunit.client.0.smithi094.stdout:0/503: unlink d1/d9/dd/d18/d1e/d22/d35/f58 0 2024-04-06T03:40:38.943 INFO:tasks.workunit.client.0.smithi094.stdout:8/583: getdents d18/d34/d52 0 2024-04-06T03:40:38.943 INFO:tasks.workunit.client.0.smithi094.stdout:8/584: stat d18/d3c/dc4 0 2024-04-06T03:40:38.943 INFO:tasks.workunit.client.1.smithi136.stdout:1/515: write d12/d3b/d74/d7a/f50 [317579,37547] 0 2024-04-06T03:40:38.945 INFO:tasks.workunit.client.0.smithi094.stdout:0/504: truncate d1/d9/dd/d18/d1e/d22/d35/f5c 4087975 0 2024-04-06T03:40:38.945 INFO:tasks.workunit.client.0.smithi094.stdout:0/505: write d1/d9/dd/d18/d1a/f54 [1196546,91212] 0 2024-04-06T03:40:38.945 INFO:tasks.workunit.client.0.smithi094.stdout:0/506: fdatasync d1/d9/dd/d18/d1d/f8e 0 2024-04-06T03:40:38.946 INFO:tasks.workunit.client.0.smithi094.stdout:0/507: write d1/d9/f91 [24301,107515] 0 2024-04-06T03:40:38.946 INFO:tasks.workunit.client.1.smithi136.stdout:2/622: symlink d3/d9b/lc6 0 2024-04-06T03:40:38.947 INFO:tasks.workunit.client.0.smithi094.stdout:7/550: dread d2/f23 [0,4194304] 0 2024-04-06T03:40:38.947 INFO:tasks.workunit.client.0.smithi094.stdout:9/502: dwrite d2/d2c/fa5 [0,4194304] 0 2024-04-06T03:40:38.947 INFO:tasks.workunit.client.0.smithi094.stdout:9/503: fsync d2/d2c/f85 0 2024-04-06T03:40:38.947 INFO:tasks.workunit.client.0.smithi094.stdout:8/585: creat d18/d3c/d49/d91/fcf x:0 0 0 2024-04-06T03:40:38.949 INFO:tasks.workunit.client.1.smithi136.stdout:4/471: symlink d1d/d2f/d6c/d91/lac 0 2024-04-06T03:40:38.949 INFO:tasks.workunit.client.0.smithi094.stdout:0/508: symlink d1/d9/dd/d18/d1d/la7 0 2024-04-06T03:40:38.949 INFO:tasks.workunit.client.0.smithi094.stdout:0/509: chown d1/d9/dd/d18/d1d/la7 1740 1 2024-04-06T03:40:38.949 INFO:tasks.workunit.client.0.smithi094.stdout:0/510: chown d1/d9/dd/d63 0 1 2024-04-06T03:40:38.953 INFO:tasks.workunit.client.1.smithi136.stdout:7/558: rmdir d5/df/d12 39 2024-04-06T03:40:38.954 INFO:tasks.workunit.client.1.smithi136.stdout:2/623: dread d3/f64 [0,4194304] 0 2024-04-06T03:40:38.954 INFO:tasks.workunit.client.1.smithi136.stdout:2/624: read d3/d15/d19/f60 [292287,58815] 0 2024-04-06T03:40:38.956 INFO:tasks.workunit.client.1.smithi136.stdout:6/481: mkdir d2/d7/d26/d30/d6e/d58/d5d/d62/da4 0 2024-04-06T03:40:38.957 INFO:tasks.workunit.client.0.smithi094.stdout:7/551: link d2/d6/dc/c3c d2/d6/dc/d12/d29/d35/d57/ca1 0 2024-04-06T03:40:38.959 INFO:tasks.workunit.client.1.smithi136.stdout:9/521: mknod d17/d21/d29/d34/cad 0 2024-04-06T03:40:38.962 INFO:tasks.workunit.client.0.smithi094.stdout:9/504: link d2/dc/d21/d23/d2f/f57 d2/dc/d21/d23/db0/fbf 0 2024-04-06T03:40:38.969 INFO:tasks.workunit.client.1.smithi136.stdout:6/482: dread d2/d7/f31 [0,4194304] 0 2024-04-06T03:40:38.969 INFO:tasks.workunit.client.1.smithi136.stdout:1/516: mkdir d12/d1b/da7 0 2024-04-06T03:40:38.971 INFO:tasks.workunit.client.0.smithi094.stdout:9/505: link d2/dc/d21/d23/d2f/l38 d2/dc/d21/d23/d5b/lc0 0 2024-04-06T03:40:38.971 INFO:tasks.workunit.client.0.smithi094.stdout:9/506: chown d2/dc/d46 39 1 2024-04-06T03:40:38.972 INFO:tasks.workunit.client.1.smithi136.stdout:4/472: symlink d1d/d2f/lad 0 2024-04-06T03:40:38.973 INFO:tasks.workunit.client.1.smithi136.stdout:7/559: mkdir d5/df/d1d/d54/d5c/dc7 0 2024-04-06T03:40:38.975 INFO:tasks.workunit.client.1.smithi136.stdout:2/625: creat d3/d15/d19/fc7 x:0 0 0 2024-04-06T03:40:38.976 INFO:tasks.workunit.client.0.smithi094.stdout:8/586: dwrite d18/d2c/d37/d60/fa4 [0,4194304] 0 2024-04-06T03:40:38.976 INFO:tasks.workunit.client.0.smithi094.stdout:8/587: write d18/d2c/d37/d60/da3/f92 [550997,20900] 0 2024-04-06T03:40:38.976 INFO:tasks.workunit.client.0.smithi094.stdout:8/588: write d18/d3c/fb2 [311205,80963] 0 2024-04-06T03:40:38.976 INFO:tasks.workunit.client.0.smithi094.stdout:8/589: write d18/d3c/dc4/dbc/fc1 [1024770,65630] 0 2024-04-06T03:40:38.976 INFO:tasks.workunit.client.1.smithi136.stdout:9/522: rmdir d17/d20/d28/d81 39 2024-04-06T03:40:38.976 INFO:tasks.workunit.client.1.smithi136.stdout:9/523: truncate d17/d20/d25/d71/fa9 107163 0 2024-04-06T03:40:38.978 INFO:tasks.workunit.client.1.smithi136.stdout:6/483: link d2/d7/d26/d30/f4c d2/d7/d26/d30/d6e/fa5 0 2024-04-06T03:40:38.978 INFO:tasks.workunit.client.1.smithi136.stdout:6/484: fdatasync d2/d7/d26/d30/d4f/f6d 0 2024-04-06T03:40:38.979 INFO:tasks.workunit.client.1.smithi136.stdout:6/485: fsync d2/d7/f2c 0 2024-04-06T03:40:38.980 INFO:tasks.workunit.client.0.smithi094.stdout:7/552: dwrite d2/fb [0,4194304] 0 2024-04-06T03:40:38.981 INFO:tasks.workunit.client.0.smithi094.stdout:7/553: truncate d2/d6/dc/dd/df/d36/d79/f81 171050 0 2024-04-06T03:40:38.981 INFO:tasks.workunit.client.0.smithi094.stdout:0/511: dwrite d1/f2c [0,4194304] 0 2024-04-06T03:40:38.982 INFO:tasks.workunit.client.1.smithi136.stdout:1/517: creat d12/d1b/d1e/d9f/fa8 x:0 0 0 2024-04-06T03:40:38.982 INFO:tasks.workunit.client.1.smithi136.stdout:1/518: chown d12/d1b/d1e/f99 133064149 1 2024-04-06T03:40:38.982 INFO:tasks.workunit.client.1.smithi136.stdout:1/519: write d12/f2c [2269519,118263] 0 2024-04-06T03:40:38.986 INFO:tasks.workunit.client.0.smithi094.stdout:7/554: rename d2/d6/dc/d12/f84 to d2/d6/dc/d12/d3a/fa2 0 2024-04-06T03:40:38.987 INFO:tasks.workunit.client.0.smithi094.stdout:7/555: read d2/d6/f8 [37972,48746] 0 2024-04-06T03:40:38.987 INFO:tasks.workunit.client.0.smithi094.stdout:9/507: write d2/dc/d21/d23/f3f [2000470,105711] 0 2024-04-06T03:40:38.987 INFO:tasks.workunit.client.1.smithi136.stdout:6/486: write d2/d7/d26/d30/d6e/f5e [2154905,1424] 0 2024-04-06T03:40:38.988 INFO:tasks.workunit.client.1.smithi136.stdout:9/524: creat d17/d20/d28/fae x:0 0 0 2024-04-06T03:40:38.989 INFO:tasks.workunit.client.0.smithi094.stdout:7/556: dread d2/d6/dc/d12/d3a/fa2 [0,4194304] 0 2024-04-06T03:40:38.989 INFO:tasks.workunit.client.0.smithi094.stdout:0/512: mknod d1/d9/dd/d18/d5d/d60/ca8 0 2024-04-06T03:40:38.989 INFO:tasks.workunit.client.0.smithi094.stdout:0/513: write d1/d10/f4f [890602,110232] 0 2024-04-06T03:40:38.994 INFO:tasks.workunit.client.1.smithi136.stdout:4/473: write d1d/f25 [1096789,74855] 0 2024-04-06T03:40:38.995 INFO:tasks.workunit.client.1.smithi136.stdout:4/474: creat d1d/d2f/d6c/fae x:0 0 0 2024-04-06T03:40:38.995 INFO:tasks.workunit.client.1.smithi136.stdout:4/475: truncate d1d/d31/fa2 791758 0 2024-04-06T03:40:38.995 INFO:tasks.workunit.client.1.smithi136.stdout:4/476: chown d1d/d31/l82 46624136 1 2024-04-06T03:40:38.995 INFO:tasks.workunit.client.1.smithi136.stdout:4/477: write d1d/d31/f67 [1670711,8641] 0 2024-04-06T03:40:38.995 INFO:tasks.workunit.client.1.smithi136.stdout:4/478: chown d1d/f27 95 1 2024-04-06T03:40:38.995 INFO:tasks.workunit.client.1.smithi136.stdout:6/487: write d2/d25/f3b [1635028,58714] 0 2024-04-06T03:40:38.996 INFO:tasks.workunit.client.1.smithi136.stdout:6/488: truncate d2/d7/d26/d30/d6e/d58/d5d/d62/d53/fa1 315554 0 2024-04-06T03:40:38.997 INFO:tasks.workunit.client.1.smithi136.stdout:4/479: dread d1d/d31/f3b [0,4194304] 0 2024-04-06T03:40:38.997 INFO:tasks.workunit.client.1.smithi136.stdout:4/480: stat d1d/d31/d3e/f5a 0 2024-04-06T03:40:38.998 INFO:tasks.workunit.client.1.smithi136.stdout:1/520: unlink d12/f89 0 2024-04-06T03:40:38.999 INFO:tasks.workunit.client.1.smithi136.stdout:1/521: dread - d12/d3b/d44/fa5 zero size 2024-04-06T03:40:38.999 INFO:tasks.workunit.client.1.smithi136.stdout:1/522: dread - d12/d1b/d1e/d23/f92 zero size 2024-04-06T03:40:39.002 INFO:tasks.workunit.client.1.smithi136.stdout:1/523: dread d12/d3b/d44/f57 [0,4194304] 0 2024-04-06T03:40:39.002 INFO:tasks.workunit.client.1.smithi136.stdout:9/525: truncate d17/d20/d25/d3b/f4e 109053 0 2024-04-06T03:40:39.003 INFO:tasks.workunit.client.1.smithi136.stdout:9/526: stat f16 0 2024-04-06T03:40:39.011 INFO:tasks.workunit.client.1.smithi136.stdout:6/489: symlink d2/d7/d26/d30/d6e/d3d/la6 0 2024-04-06T03:40:39.011 INFO:tasks.workunit.client.0.smithi094.stdout:9/508: truncate d2/dc/d21/d26/f3c 7749619 0 2024-04-06T03:40:39.014 INFO:tasks.workunit.client.1.smithi136.stdout:1/524: getdents d12/d3b/d44/d5f/d5a/d72/d77 0 2024-04-06T03:40:39.014 INFO:tasks.workunit.client.0.smithi094.stdout:7/557: mknod d2/d6/dc/dd/df/d36/d79/ca3 0 2024-04-06T03:40:39.016 INFO:tasks.workunit.client.1.smithi136.stdout:4/481: dwrite d1d/d2f/f57 [0,4194304] 0 2024-04-06T03:40:39.016 INFO:tasks.workunit.client.1.smithi136.stdout:4/482: fsync d1d/d2f/d6c/f76 0 2024-04-06T03:40:39.017 INFO:tasks.workunit.client.1.smithi136.stdout:9/527: symlink d17/laf 0 2024-04-06T03:40:39.017 INFO:tasks.workunit.client.1.smithi136.stdout:9/528: chown d17/d21/d32/f74 31 1 2024-04-06T03:40:39.021 INFO:tasks.workunit.client.0.smithi094.stdout:0/514: mknod d1/d9/dd/d63/ca9 0 2024-04-06T03:40:39.024 INFO:tasks.workunit.client.0.smithi094.stdout:9/509: mknod d2/dc/d46/cc1 0 2024-04-06T03:40:39.024 INFO:tasks.workunit.client.1.smithi136.stdout:6/490: write d2/d7/f12 [3867340,64505] 0 2024-04-06T03:40:39.027 INFO:tasks.workunit.client.0.smithi094.stdout:7/558: mkdir d2/d6/dc/d12/d29/d35/da4 0 2024-04-06T03:40:39.029 INFO:tasks.workunit.client.0.smithi094.stdout:0/515: mkdir d1/d9/dd/d18/d1e/d86/d96/daa 0 2024-04-06T03:40:39.029 INFO:tasks.workunit.client.0.smithi094.stdout:0/516: truncate d1/d9/dd/d18/d32/f7d 874121 0 2024-04-06T03:40:39.029 INFO:tasks.workunit.client.0.smithi094.stdout:0/517: fdatasync d1/d9/dd/d18/d1e/d22/d35/f3c 0 2024-04-06T03:40:39.032 INFO:tasks.workunit.client.1.smithi136.stdout:3/509: truncate d0/d4/d12/d2d/d37/f6d 42577 0 2024-04-06T03:40:39.032 INFO:tasks.workunit.client.1.smithi136.stdout:3/510: truncate d0/d4/d12/d19/d55/d3c/d46/f78 255421 0 2024-04-06T03:40:39.033 INFO:tasks.workunit.client.0.smithi094.stdout:7/559: truncate d2/d6/dc/d12/d61/f83 2543308 0 2024-04-06T03:40:39.033 INFO:tasks.workunit.client.1.smithi136.stdout:1/525: dwrite d12/d1b/f29 [0,4194304] 0 2024-04-06T03:40:39.038 INFO:tasks.workunit.client.1.smithi136.stdout:9/529: rename d17/d20/l59 to d17/d20/d25/d3c/d55/lb0 0 2024-04-06T03:40:39.039 INFO:tasks.workunit.client.1.smithi136.stdout:8/574: sync 2024-04-06T03:40:39.040 INFO:tasks.workunit.client.1.smithi136.stdout:5/591: sync 2024-04-06T03:40:39.040 INFO:tasks.workunit.client.0.smithi094.stdout:0/518: rename d1/d10 to d1/d9/dd/d18/d5d/d60/d6b/dab 0 2024-04-06T03:40:39.040 INFO:tasks.workunit.client.0.smithi094.stdout:6/474: sync 2024-04-06T03:40:39.041 INFO:tasks.workunit.client.0.smithi094.stdout:1/502: sync 2024-04-06T03:40:39.041 INFO:tasks.workunit.client.0.smithi094.stdout:4/574: sync 2024-04-06T03:40:39.041 INFO:tasks.workunit.client.0.smithi094.stdout:5/516: sync 2024-04-06T03:40:39.041 INFO:tasks.workunit.client.0.smithi094.stdout:6/475: write dc/d3a/f41 [4762311,110026] 0 2024-04-06T03:40:39.041 INFO:tasks.workunit.client.1.smithi136.stdout:5/592: dread d1/dc/d1c/d32/d8b/d20/f42 [0,4194304] 0 2024-04-06T03:40:39.044 INFO:tasks.workunit.client.1.smithi136.stdout:1/526: write d12/d1b/d1e/d23/f3d [79033,19528] 0 2024-04-06T03:40:39.045 INFO:tasks.workunit.client.0.smithi094.stdout:7/560: write d2/d6/dc/d12/d29/f76 [1494482,28956] 0 2024-04-06T03:40:39.045 INFO:tasks.workunit.client.0.smithi094.stdout:7/561: readlink d2/d6/dc/d12/l50 0 2024-04-06T03:40:39.047 INFO:tasks.workunit.client.0.smithi094.stdout:0/519: mkdir d1/d9/dd/d18/d1e/d22/d35/dac 0 2024-04-06T03:40:39.047 INFO:tasks.workunit.client.0.smithi094.stdout:0/520: fsync d1/d9/dd/d18/d1a/f4d 0 2024-04-06T03:40:39.049 INFO:tasks.workunit.client.0.smithi094.stdout:1/503: getdents d0/d4/d1b/d2f/d76 0 2024-04-06T03:40:39.049 INFO:tasks.workunit.client.0.smithi094.stdout:1/504: readlink d0/le 0 2024-04-06T03:40:39.052 INFO:tasks.workunit.client.0.smithi094.stdout:5/517: mknod db/d19/d24/d28/cb7 0 2024-04-06T03:40:39.055 INFO:tasks.workunit.client.1.smithi136.stdout:4/483: dwrite d1d/d31/f40 [0,4194304] 0 2024-04-06T03:40:39.057 INFO:tasks.workunit.client.0.smithi094.stdout:6/476: rename dc/d11/d1b/l63 to dc/d11/la9 0 2024-04-06T03:40:39.057 INFO:tasks.workunit.client.0.smithi094.stdout:4/575: dwrite d0/d15/d3a/d61/d81/fba [0,4194304] 0 2024-04-06T03:40:39.059 INFO:tasks.workunit.client.0.smithi094.stdout:7/562: unlink d2/d6/dc/d12/c4b 0 2024-04-06T03:40:39.062 INFO:tasks.workunit.client.0.smithi094.stdout:5/518: unlink db/d7c/f7e 0 2024-04-06T03:40:39.065 INFO:tasks.workunit.client.0.smithi094.stdout:6/477: creat dc/d11/d1b/d72/d88/da3/faa x:0 0 0 2024-04-06T03:40:39.066 INFO:tasks.workunit.client.0.smithi094.stdout:6/478: chown dc/d11/d1b/d72/f8c 2430665 1 2024-04-06T03:40:39.067 INFO:tasks.workunit.client.1.smithi136.stdout:8/575: creat d3/de/d18/faf x:0 0 0 2024-04-06T03:40:39.068 INFO:tasks.workunit.client.1.smithi136.stdout:8/576: fdatasync d3/de/d18/f1e 0 2024-04-06T03:40:39.068 INFO:tasks.workunit.client.1.smithi136.stdout:8/577: write d3/da/d26/d88/fa2 [369481,89345] 0 2024-04-06T03:40:39.068 INFO:tasks.workunit.client.1.smithi136.stdout:8/578: stat d3/da/d26/d90/ca1 0 2024-04-06T03:40:39.068 INFO:tasks.workunit.client.1.smithi136.stdout:8/579: chown d3/da/d26/d35/f1d 486578497 1 2024-04-06T03:40:39.071 INFO:tasks.workunit.client.1.smithi136.stdout:5/593: unlink d1/dc/d10/l6c 0 2024-04-06T03:40:39.072 INFO:tasks.workunit.client.0.smithi094.stdout:7/563: read d2/d6/dc/d12/f5f [3640361,41903] 0 2024-04-06T03:40:39.072 INFO:tasks.workunit.client.0.smithi094.stdout:7/564: dread - d2/d6/dc/dd/df/f66 zero size 2024-04-06T03:40:39.072 INFO:tasks.workunit.client.0.smithi094.stdout:7/565: dread - d2/d6/dc/d12/d29/d35/d57/f82 zero size 2024-04-06T03:40:39.073 INFO:tasks.workunit.client.0.smithi094.stdout:9/510: getdents d2/d2c/d99 0 2024-04-06T03:40:39.073 INFO:tasks.workunit.client.0.smithi094.stdout:9/511: chown d2/dc/d10 0 1 2024-04-06T03:40:39.075 INFO:tasks.workunit.client.0.smithi094.stdout:4/576: dwrite d0/df/d13/f8c [0,4194304] 0 2024-04-06T03:40:39.076 INFO:tasks.workunit.client.1.smithi136.stdout:4/484: symlink d1d/d31/d3e/d55/d89/laf 0 2024-04-06T03:40:39.076 INFO:tasks.workunit.client.1.smithi136.stdout:4/485: fdatasync d1d/d2f/d6c/f98 0 2024-04-06T03:40:39.076 INFO:tasks.workunit.client.0.smithi094.stdout:5/519: dwrite db/d40/d77/fa9 [0,4194304] 0 2024-04-06T03:40:39.078 INFO:tasks.workunit.client.0.smithi094.stdout:6/479: mkdir dc/dab 0 2024-04-06T03:40:39.080 INFO:tasks.workunit.client.1.smithi136.stdout:3/511: rename d0/d4/l10 to d0/d4/d12/d19/d55/lab 0 2024-04-06T03:40:39.080 INFO:tasks.workunit.client.1.smithi136.stdout:3/512: chown d0/f2 765317264 1 2024-04-06T03:40:39.080 INFO:tasks.workunit.client.0.smithi094.stdout:9/512: dread d2/fe [0,4194304] 0 2024-04-06T03:40:39.081 INFO:tasks.workunit.client.0.smithi094.stdout:6/480: dread dc/d11/d97/fa1 [0,4194304] 0 2024-04-06T03:40:39.083 INFO:tasks.workunit.client.1.smithi136.stdout:8/580: unlink d3/de/d60/c68 0 2024-04-06T03:40:39.083 INFO:tasks.workunit.client.1.smithi136.stdout:8/581: dread - d3/da/d26/d35/d3d/d48/f92 zero size 2024-04-06T03:40:39.086 INFO:tasks.workunit.client.1.smithi136.stdout:5/594: stat d1/d16/d4b/l8d 0 2024-04-06T03:40:39.092 INFO:tasks.workunit.client.1.smithi136.stdout:9/530: rename d17/d21/d29/d34/d66/f6c to d17/d20/d25/d3b/fb1 0 2024-04-06T03:40:39.093 INFO:tasks.workunit.client.1.smithi136.stdout:8/582: write d3/da/d26/d35/d53/fa5 [2760291,85249] 0 2024-04-06T03:40:39.095 INFO:tasks.workunit.client.1.smithi136.stdout:5/595: mknod d1/dc/d1c/d32/d8b/db8/cba 0 2024-04-06T03:40:39.099 INFO:tasks.workunit.client.1.smithi136.stdout:4/486: creat d1d/d31/d78/fb0 x:0 0 0 2024-04-06T03:40:39.101 INFO:tasks.workunit.client.1.smithi136.stdout:9/531: write d17/d21/f7c [117884,60164] 0 2024-04-06T03:40:39.102 INFO:tasks.workunit.client.0.smithi094.stdout:7/566: link d2/d6/dc/dd/df/c1d d2/d6/dc/d12/d29/ca5 0 2024-04-06T03:40:39.103 INFO:tasks.workunit.client.1.smithi136.stdout:8/583: truncate d3/f5e 285292 0 2024-04-06T03:40:39.103 INFO:tasks.workunit.client.1.smithi136.stdout:8/584: chown d3/da/d26/d35/d53/d86/l9c 1 1 2024-04-06T03:40:39.104 INFO:tasks.workunit.client.0.smithi094.stdout:5/520: dwrite db/d80/f97 [0,4194304] 0 2024-04-06T03:40:39.104 INFO:tasks.workunit.client.0.smithi094.stdout:5/521: truncate db/d40/f43 31200 0 2024-04-06T03:40:39.104 INFO:tasks.workunit.client.0.smithi094.stdout:9/513: dwrite d2/f1d [0,4194304] 0 2024-04-06T03:40:39.107 INFO:tasks.workunit.client.1.smithi136.stdout:5/596: unlink d1/dc/d1c/d32/d8b/db8/cba 0 2024-04-06T03:40:39.110 INFO:tasks.workunit.client.0.smithi094.stdout:7/567: read d2/d6/dc/d12/f2a [322374,45517] 0 2024-04-06T03:40:39.110 INFO:tasks.workunit.client.1.smithi136.stdout:9/532: dread d17/d19/f8e [0,4194304] 0 2024-04-06T03:40:39.111 INFO:tasks.workunit.client.1.smithi136.stdout:8/585: mkdir d3/da/d26/d35/d53/d86/db0 0 2024-04-06T03:40:39.111 INFO:tasks.workunit.client.0.smithi094.stdout:6/481: mkdir dc/d3a/d4a/d94/dac 0 2024-04-06T03:40:39.111 INFO:tasks.workunit.client.0.smithi094.stdout:6/482: getdents dc/d3a/d4a/d94/dac 0 2024-04-06T03:40:39.113 INFO:tasks.workunit.client.1.smithi136.stdout:5/597: creat d1/d1f/fbb x:0 0 0 2024-04-06T03:40:39.114 INFO:tasks.workunit.client.0.smithi094.stdout:5/522: truncate db/d19/d24/d28/f3f 177457 0 2024-04-06T03:40:39.117 INFO:tasks.workunit.client.1.smithi136.stdout:4/487: symlink d1d/d31/d3e/d55/lb1 0 2024-04-06T03:40:39.117 INFO:tasks.workunit.client.1.smithi136.stdout:4/488: write d1d/d2f/f60 [989167,112937] 0 2024-04-06T03:40:39.119 INFO:tasks.workunit.client.1.smithi136.stdout:8/586: creat d3/de/d60/fb1 x:0 0 0 2024-04-06T03:40:39.120 INFO:tasks.workunit.client.0.smithi094.stdout:9/514: symlink d2/dc/d21/d23/d47/lc2 0 2024-04-06T03:40:39.122 INFO:tasks.workunit.client.0.smithi094.stdout:6/483: creat dc/d11/d1b/d72/d88/fad x:0 0 0 2024-04-06T03:40:39.124 INFO:tasks.workunit.client.0.smithi094.stdout:6/484: dread dc/d11/d22/d50/d61/f68 [0,4194304] 0 2024-04-06T03:40:39.124 INFO:tasks.workunit.client.0.smithi094.stdout:6/485: truncate dc/d11/d1b/d24/d46/fa5 684205 0 2024-04-06T03:40:39.126 INFO:tasks.workunit.client.1.smithi136.stdout:8/587: creat d3/da/d26/d35/d3d/d48/d6c/d79/fb2 x:0 0 0 2024-04-06T03:40:39.127 INFO:tasks.workunit.client.0.smithi094.stdout:5/523: creat db/d19/d30/fb8 x:0 0 0 2024-04-06T03:40:39.127 INFO:tasks.workunit.client.0.smithi094.stdout:5/524: readlink db/d19/d21/la7 0 2024-04-06T03:40:39.128 INFO:tasks.workunit.client.0.smithi094.stdout:9/515: symlink d2/d2c/d33/lc3 0 2024-04-06T03:40:39.128 INFO:tasks.workunit.client.1.smithi136.stdout:9/533: dwrite d17/d20/d28/d49/d75/d60/f7d [0,4194304] 0 2024-04-06T03:40:39.129 INFO:tasks.workunit.client.1.smithi136.stdout:8/588: mknod d3/de/d18/d6d/cb3 0 2024-04-06T03:40:39.129 INFO:tasks.workunit.client.1.smithi136.stdout:8/589: chown d3/d2d/d5f/c9b 91 1 2024-04-06T03:40:39.129 INFO:tasks.workunit.client.1.smithi136.stdout:8/590: fdatasync d3/da/fd 0 2024-04-06T03:40:39.130 INFO:tasks.workunit.client.0.smithi094.stdout:6/486: truncate dc/d11/d22/f39 3594604 0 2024-04-06T03:40:39.130 INFO:tasks.workunit.client.0.smithi094.stdout:6/487: read - dc/d11/d1b/d72/d88/fad zero size 2024-04-06T03:40:39.132 INFO:tasks.workunit.client.1.smithi136.stdout:9/534: mknod d17/d21/d29/d34/d46/cb2 0 2024-04-06T03:40:39.134 INFO:tasks.workunit.client.0.smithi094.stdout:5/525: rmdir db/d19/d21/d9b 0 2024-04-06T03:40:39.135 INFO:tasks.workunit.client.1.smithi136.stdout:4/489: dwrite d1d/d70/f68 [0,4194304] 0 2024-04-06T03:40:39.135 INFO:tasks.workunit.client.1.smithi136.stdout:4/490: write d1d/f9d [164653,10867] 0 2024-04-06T03:40:39.136 INFO:tasks.workunit.client.0.smithi094.stdout:9/516: symlink d2/dc/d46/lc4 0 2024-04-06T03:40:39.137 INFO:tasks.workunit.client.1.smithi136.stdout:6/491: getdents d2/d7/d26/d30/d6e 0 2024-04-06T03:40:39.141 INFO:tasks.workunit.client.0.smithi094.stdout:5/526: unlink db/f69 0 2024-04-06T03:40:39.143 INFO:tasks.workunit.client.1.smithi136.stdout:9/535: truncate d17/d20/d28/d81/f83 1887801 0 2024-04-06T03:40:39.143 INFO:tasks.workunit.client.1.smithi136.stdout:9/536: creat d17/d19/fb3 x:0 0 0 2024-04-06T03:40:39.144 INFO:tasks.workunit.client.0.smithi094.stdout:5/527: mknod db/d19/d2c/d6d/cb9 0 2024-04-06T03:40:39.146 INFO:tasks.workunit.client.1.smithi136.stdout:6/492: write d2/f3c [4125946,76929] 0 2024-04-06T03:40:39.147 INFO:tasks.workunit.client.1.smithi136.stdout:4/491: rename d1d/d2f/ca8 to d1d/d2f/d6c/cb2 0 2024-04-06T03:40:39.150 INFO:tasks.workunit.client.1.smithi136.stdout:6/493: mknod d2/d7/d26/d30/d4f/ca7 0 2024-04-06T03:40:39.152 INFO:tasks.workunit.client.1.smithi136.stdout:9/537: rename fe to d17/d21/d32/d3f/fb4 0 2024-04-06T03:40:39.156 INFO:tasks.workunit.client.0.smithi094.stdout:6/488: dwrite dc/d11/d1b/d72/f81 [0,4194304] 0 2024-04-06T03:40:39.158 INFO:tasks.workunit.client.0.smithi094.stdout:9/517: dwrite d2/fe [4194304,4194304] 0 2024-04-06T03:40:39.158 INFO:tasks.workunit.client.0.smithi094.stdout:9/518: dread - d2/dc/d21/fad zero size 2024-04-06T03:40:39.160 INFO:tasks.workunit.client.0.smithi094.stdout:6/489: mkdir dc/d11/d22/d50/d61/dae 0 2024-04-06T03:40:39.163 INFO:tasks.workunit.client.0.smithi094.stdout:9/519: link d2/dc/d21/l5d d2/d2c/d33/d64/lc5 0 2024-04-06T03:40:39.163 INFO:tasks.workunit.client.0.smithi094.stdout:9/520: chown d2/d2c/d99/fbd 30384976 1 2024-04-06T03:40:39.163 INFO:tasks.workunit.client.0.smithi094.stdout:5/528: write db/d80/f97 [3483263,52055] 0 2024-04-06T03:40:39.165 INFO:tasks.workunit.client.0.smithi094.stdout:6/490: mkdir dc/d11/d22/d50/daf 0 2024-04-06T03:40:39.165 INFO:tasks.workunit.client.0.smithi094.stdout:9/521: read d2/dc/d21/d23/d47/f77 [559074,43251] 0 2024-04-06T03:40:39.165 INFO:tasks.workunit.client.0.smithi094.stdout:9/522: dread - d2/dc/d21/d26/d78/d69/d6d/fa7 zero size 2024-04-06T03:40:39.165 INFO:tasks.workunit.client.0.smithi094.stdout:9/523: dread - d2/dc/d46/f65 zero size 2024-04-06T03:40:39.165 INFO:tasks.workunit.client.0.smithi094.stdout:9/524: readlink d2/dc/d21/d23/d2f/l38 0 2024-04-06T03:40:39.165 INFO:tasks.workunit.client.0.smithi094.stdout:6/491: rmdir dc/d11/d1b/d24/d46/d7b 39 2024-04-06T03:40:39.167 INFO:tasks.workunit.client.0.smithi094.stdout:9/525: mkdir d2/d2c/d33/d64/dc6 0 2024-04-06T03:40:39.170 INFO:tasks.workunit.client.1.smithi136.stdout:9/538: dwrite d17/d21/d29/f92 [0,4194304] 0 2024-04-06T03:40:39.170 INFO:tasks.workunit.client.1.smithi136.stdout:6/494: dwrite d2/f21 [0,4194304] 0 2024-04-06T03:40:39.172 INFO:tasks.workunit.client.0.smithi094.stdout:6/492: write dc/d3a/f41 [3288982,31282] 0 2024-04-06T03:40:39.173 INFO:tasks.workunit.client.0.smithi094.stdout:0/521: rename d1/d9/dd/d63 to d1/d9/dd/d18/dad 0 2024-04-06T03:40:39.173 INFO:tasks.workunit.client.0.smithi094.stdout:0/522: dread - d1/d9/dd/d18/d32/d42/f68 zero size 2024-04-06T03:40:39.173 INFO:tasks.workunit.client.0.smithi094.stdout:0/523: readlink d1/d9/dd/d18/d5d/l6f 0 2024-04-06T03:40:39.175 INFO:tasks.workunit.client.0.smithi094.stdout:6/493: mknod dc/d11/d1b/d4e/cb0 0 2024-04-06T03:40:39.176 INFO:tasks.workunit.client.1.smithi136.stdout:6/495: link d2/d7/l1f d2/d7/d26/d97/la8 0 2024-04-06T03:40:39.178 INFO:tasks.workunit.client.1.smithi136.stdout:3/513: rmdir d0/d4/d12/d2d/d37 39 2024-04-06T03:40:39.178 INFO:tasks.workunit.client.1.smithi136.stdout:3/514: read - d0/d4/d12/d19/d55/d3c/faa zero size 2024-04-06T03:40:39.178 INFO:tasks.workunit.client.1.smithi136.stdout:3/515: write d0/d4/d12/d19/d2c/f9c [549409,78546] 0 2024-04-06T03:40:39.178 INFO:tasks.workunit.client.1.smithi136.stdout:3/516: dread - d0/d4/d12/d19/d55/f9f zero size 2024-04-06T03:40:39.178 INFO:tasks.workunit.client.1.smithi136.stdout:3/517: write d0/d4/d12/d19/d55/d3c/faa [409202,23184] 0 2024-04-06T03:40:39.179 INFO:tasks.workunit.client.0.smithi094.stdout:4/577: rename d0/df/d1e/d34/f8a to d0/df/d57/fbf 0 2024-04-06T03:40:39.179 INFO:tasks.workunit.client.0.smithi094.stdout:4/578: chown d0/d4/d67/cae 4047217 1 2024-04-06T03:40:39.179 INFO:tasks.workunit.client.0.smithi094.stdout:4/579: creat d0/d15/d3a/d61/d81/fc0 x:0 0 0 2024-04-06T03:40:39.180 INFO:tasks.workunit.client.1.smithi136.stdout:9/539: write d17/d20/d28/d49/d75/d60/f7d [3802394,630] 0 2024-04-06T03:40:39.180 INFO:tasks.workunit.client.0.smithi094.stdout:0/524: write d1/d9/f20 [2773212,108456] 0 2024-04-06T03:40:39.183 INFO:tasks.workunit.client.1.smithi136.stdout:3/518: truncate d0/d4/d12/d19/d55/d3c/d5a/d79/fa4 1669674 0 2024-04-06T03:40:39.183 INFO:tasks.workunit.client.1.smithi136.stdout:3/519: creat d0/d4/d12/d19/d55/d3c/d5a/fac x:0 0 0 2024-04-06T03:40:39.184 INFO:tasks.workunit.client.1.smithi136.stdout:6/496: write d2/d7/d26/d30/d6e/d58/d5d/d62/d53/f74 [1440936,124515] 0 2024-04-06T03:40:39.186 INFO:tasks.workunit.client.0.smithi094.stdout:4/580: read d0/df/d13/d4a/f8b [3495580,28041] 0 2024-04-06T03:40:39.188 INFO:tasks.workunit.client.0.smithi094.stdout:0/525: write d1/d9/dd/d18/d5d/d60/d6b/f59 [2859273,86025] 0 2024-04-06T03:40:39.188 INFO:tasks.workunit.client.0.smithi094.stdout:0/526: chown d1/d9/l3f 0 1 2024-04-06T03:40:39.189 INFO:tasks.workunit.client.0.smithi094.stdout:7/568: rename d2/f63 to d2/d6/dc/dd/d90/fa6 0 2024-04-06T03:40:39.189 INFO:tasks.workunit.client.0.smithi094.stdout:0/527: dread d1/d9/f91 [0,4194304] 0 2024-04-06T03:40:39.191 INFO:tasks.workunit.client.1.smithi136.stdout:9/540: rename d17/d19/l7f to d17/d20/d25/d3b/d9e/lb5 0 2024-04-06T03:40:39.191 INFO:tasks.workunit.client.1.smithi136.stdout:9/541: chown d17/d20/d28/d49 51 1 2024-04-06T03:40:39.193 INFO:tasks.workunit.client.1.smithi136.stdout:3/520: write d0/d4/fe [577105,44012] 0 2024-04-06T03:40:39.194 INFO:tasks.workunit.client.1.smithi136.stdout:0/531: sync 2024-04-06T03:40:39.194 INFO:tasks.workunit.client.1.smithi136.stdout:0/532: write d1/d6/d2c/d1a/f94 [357111,5924] 0 2024-04-06T03:40:39.195 INFO:tasks.workunit.client.0.smithi094.stdout:4/581: truncate d0/d23/f32 1236535 0 2024-04-06T03:40:39.196 INFO:tasks.workunit.client.1.smithi136.stdout:7/560: sync 2024-04-06T03:40:39.196 INFO:tasks.workunit.client.1.smithi136.stdout:2/626: sync 2024-04-06T03:40:39.196 INFO:tasks.workunit.client.1.smithi136.stdout:7/561: creat d5/df/d1d/d6f/d4d/d69/fc8 x:0 0 0 2024-04-06T03:40:39.197 INFO:tasks.workunit.client.1.smithi136.stdout:7/562: read - d5/df/d1d/d6f/fc2 zero size 2024-04-06T03:40:39.197 INFO:tasks.workunit.client.1.smithi136.stdout:7/563: write d5/df/d11/f79 [424294,112928] 0 2024-04-06T03:40:39.201 INFO:tasks.workunit.client.0.smithi094.stdout:9/526: rename d2/d2c/d99/fb3 to d2/dc/d80/db7/fc7 0 2024-04-06T03:40:39.201 INFO:tasks.workunit.client.0.smithi094.stdout:9/527: write d2/dc/d21/d23/d2f/f90 [886717,97942] 0 2024-04-06T03:40:39.204 INFO:tasks.workunit.client.0.smithi094.stdout:7/569: creat d2/d6/dc/d12/d29/fa7 x:0 0 0 2024-04-06T03:40:39.205 INFO:tasks.workunit.client.0.smithi094.stdout:6/494: getdents dc/d11/d1b/d72/d88/da3 0 2024-04-06T03:40:39.206 INFO:tasks.workunit.client.0.smithi094.stdout:4/582: mkdir d0/d23/dc1 0 2024-04-06T03:40:39.207 INFO:tasks.workunit.client.1.smithi136.stdout:8/591: getdents d3/de/d18 0 2024-04-06T03:40:39.207 INFO:tasks.workunit.client.1.smithi136.stdout:2/627: write d3/f68 [2971633,61547] 0 2024-04-06T03:40:39.207 INFO:tasks.workunit.client.1.smithi136.stdout:2/628: stat d3/d15/f90 0 2024-04-06T03:40:39.209 INFO:tasks.workunit.client.1.smithi136.stdout:9/542: link d17/d21/d32/d3f/l7b d17/d20/d25/d3d/da5/lb6 0 2024-04-06T03:40:39.210 INFO:tasks.workunit.client.1.smithi136.stdout:3/521: truncate d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 3201875 0 2024-04-06T03:40:39.210 INFO:tasks.workunit.client.0.smithi094.stdout:9/528: mkdir d2/dc/d21/dc8 0 2024-04-06T03:40:39.212 INFO:tasks.workunit.client.0.smithi094.stdout:7/570: creat d2/fa8 x:0 0 0 2024-04-06T03:40:39.213 INFO:tasks.workunit.client.1.smithi136.stdout:0/533: getdents d1/d6/d2c/d57/d43/d6d/d9b/d5c 0 2024-04-06T03:40:39.213 INFO:tasks.workunit.client.1.smithi136.stdout:0/534: truncate d1/d6/d2c/d1a/f81 696670 0 2024-04-06T03:40:39.213 INFO:tasks.workunit.client.1.smithi136.stdout:0/535: chown d1/d6/d2c/d1a/d36/f49 6286683 1 2024-04-06T03:40:39.213 INFO:tasks.workunit.client.1.smithi136.stdout:0/536: chown d1/d4e/f55 1 1 2024-04-06T03:40:39.213 INFO:tasks.workunit.client.1.smithi136.stdout:0/537: chown d1/d6/l61 29676756 1 2024-04-06T03:40:39.215 INFO:tasks.workunit.client.0.smithi094.stdout:9/529: mknod d2/dc/d21/d26/d78/cc9 0 2024-04-06T03:40:39.215 INFO:tasks.workunit.client.0.smithi094.stdout:9/530: write d2/dc/d21/d23/f7c [423226,106518] 0 2024-04-06T03:40:39.215 INFO:tasks.workunit.client.0.smithi094.stdout:9/531: truncate d2/dc/d21/fad 640380 0 2024-04-06T03:40:39.216 INFO:tasks.workunit.client.0.smithi094.stdout:0/528: dwrite d1/d9/dd/d18/d5d/d60/d6b/dab/f73 [0,4194304] 0 2024-04-06T03:40:39.218 INFO:tasks.workunit.client.0.smithi094.stdout:6/495: dread f5 [0,4194304] 0 2024-04-06T03:40:39.218 INFO:tasks.workunit.client.0.smithi094.stdout:6/496: write dc/d11/d1b/d24/f32 [1321402,5774] 0 2024-04-06T03:40:39.221 INFO:tasks.workunit.client.0.smithi094.stdout:9/532: rename d2/dc/d21/d26/d78/cc9 to d2/dc/d21/d26/dba/cca 0 2024-04-06T03:40:39.223 INFO:tasks.workunit.client.1.smithi136.stdout:7/564: link d5/df/d1d/d54/lb3 d5/df/d1d/lc9 0 2024-04-06T03:40:39.224 INFO:tasks.workunit.client.1.smithi136.stdout:8/592: dread d3/da/d26/d35/d3d/d48/f72 [0,4194304] 0 2024-04-06T03:40:39.224 INFO:tasks.workunit.client.1.smithi136.stdout:8/593: chown d3/da/d26/d35/d53/fa9 457 1 2024-04-06T03:40:39.224 INFO:tasks.workunit.client.1.smithi136.stdout:7/565: read d5/df/d1d/d6f/d2c/f72 [98154,22224] 0 2024-04-06T03:40:39.226 INFO:tasks.workunit.client.0.smithi094.stdout:4/583: dread d0/d15/d3a/d3e/d46/fa3 [0,4194304] 0 2024-04-06T03:40:39.227 INFO:tasks.workunit.client.0.smithi094.stdout:0/529: symlink d1/d9/dd/d18/d5d/d60/lae 0 2024-04-06T03:40:39.229 INFO:tasks.workunit.client.1.smithi136.stdout:2/629: mkdir d3/d9/d3b/d70/dc8 0 2024-04-06T03:40:39.236 INFO:tasks.workunit.client.1.smithi136.stdout:9/543: symlink d17/d20/d28/d49/d75/d60/lb7 0 2024-04-06T03:40:39.238 INFO:tasks.workunit.client.0.smithi094.stdout:9/533: readlink d2/dc/d10/l13 0 2024-04-06T03:40:39.238 INFO:tasks.workunit.client.0.smithi094.stdout:9/534: write d2/dc/d80/fac [856333,88225] 0 2024-04-06T03:40:39.238 INFO:tasks.workunit.client.0.smithi094.stdout:9/535: dread - d2/dc/d10/fb2 zero size 2024-04-06T03:40:39.238 INFO:tasks.workunit.client.0.smithi094.stdout:9/536: chown d2/d2c/d33/f5c 2252609 1 2024-04-06T03:40:39.238 INFO:tasks.workunit.client.0.smithi094.stdout:9/537: write d2/d2c/d99/faf [367012,66323] 0 2024-04-06T03:40:39.238 INFO:tasks.workunit.client.0.smithi094.stdout:9/538: truncate d2/d2c/d99/f9e 1258612 0 2024-04-06T03:40:39.239 INFO:tasks.workunit.client.0.smithi094.stdout:6/497: dwrite dc/d11/d1b/f1f [0,4194304] 0 2024-04-06T03:40:39.240 INFO:tasks.workunit.client.1.smithi136.stdout:3/522: rename d0/d4/d12/d19/d55/d6f/d82/d5d/f73 to d0/d4/d12/d19/d55/d6f/fad 0 2024-04-06T03:40:39.241 INFO:tasks.workunit.client.0.smithi094.stdout:0/530: rmdir d1/d9/dd/d18/d32/d42 39 2024-04-06T03:40:39.241 INFO:tasks.workunit.client.1.smithi136.stdout:7/566: dwrite d5/df/d1d/d6f/d4d/d69/f7b [4194304,4194304] 0 2024-04-06T03:40:39.242 INFO:tasks.workunit.client.0.smithi094.stdout:0/531: dread d1/d9/dd/d18/d5d/d60/d6b/f81 [0,4194304] 0 2024-04-06T03:40:39.244 INFO:tasks.workunit.client.1.smithi136.stdout:0/538: creat d1/d6/d2c/d57/d43/d6d/d85/d78/da4/fb6 x:0 0 0 2024-04-06T03:40:39.245 INFO:tasks.workunit.client.0.smithi094.stdout:9/539: mknod d2/dc/d21/d23/d47/ccb 0 2024-04-06T03:40:39.247 INFO:tasks.workunit.client.1.smithi136.stdout:8/594: creat d3/d2d/d5f/fb4 x:0 0 0 2024-04-06T03:40:39.247 INFO:tasks.workunit.client.1.smithi136.stdout:8/595: chown d3/da/d26/d35/d3d/d48/d6c/d79/fb2 23640 1 2024-04-06T03:40:39.249 INFO:tasks.workunit.client.1.smithi136.stdout:2/630: mkdir d3/d15/d19/d4e/dc9 0 2024-04-06T03:40:39.251 INFO:tasks.workunit.client.0.smithi094.stdout:6/498: rename dc/d3a/l6d to dc/d3a/lb1 0 2024-04-06T03:40:39.251 INFO:tasks.workunit.client.0.smithi094.stdout:6/499: creat dc/d3a/d4a/d54/fb2 x:0 0 0 2024-04-06T03:40:39.253 INFO:tasks.workunit.client.1.smithi136.stdout:0/539: dread d1/f7 [0,4194304] 0 2024-04-06T03:40:39.253 INFO:tasks.workunit.client.1.smithi136.stdout:0/540: stat d1/d6/d2c/d57/d43/d9e/fac 0 2024-04-06T03:40:39.253 INFO:tasks.workunit.client.1.smithi136.stdout:0/541: truncate d1/d6/fa1 778228 0 2024-04-06T03:40:39.253 INFO:tasks.workunit.client.1.smithi136.stdout:0/542: chown d1/la6 19750 1 2024-04-06T03:40:39.253 INFO:tasks.workunit.client.1.smithi136.stdout:0/543: chown d1/d6/d2c/d57/d43/d6d/d85/d78/fad 534 1 2024-04-06T03:40:39.253 INFO:tasks.workunit.client.1.smithi136.stdout:0/544: truncate d1/d6/d2c/d57/d43/d6d/d9b/d5c/f6a 912109 0 2024-04-06T03:40:39.254 INFO:tasks.workunit.client.0.smithi094.stdout:0/532: creat d1/d9/dd/d18/d1e/d22/d35/dac/faf x:0 0 0 2024-04-06T03:40:39.258 INFO:tasks.workunit.client.0.smithi094.stdout:6/500: mknod dc/d3a/d4a/d54/cb3 0 2024-04-06T03:40:39.259 INFO:tasks.workunit.client.0.smithi094.stdout:0/533: symlink d1/d9/dd/d18/d78/lb0 0 2024-04-06T03:40:39.263 INFO:tasks.workunit.client.1.smithi136.stdout:7/567: symlink d5/df/d12/d1b/d3a/d4a/dc4/lca 0 2024-04-06T03:40:39.263 INFO:tasks.workunit.client.1.smithi136.stdout:7/568: creat d5/fcb x:0 0 0 2024-04-06T03:40:39.263 INFO:tasks.workunit.client.0.smithi094.stdout:0/534: creat d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/fb1 x:0 0 0 2024-04-06T03:40:39.265 INFO:tasks.workunit.client.1.smithi136.stdout:8/596: mkdir d3/da/d26/d90/dae/db5 0 2024-04-06T03:40:39.265 INFO:tasks.workunit.client.1.smithi136.stdout:8/597: chown d3/de/f61 5760 1 2024-04-06T03:40:39.267 INFO:tasks.workunit.client.0.smithi094.stdout:0/535: mknod d1/d9/dd/d18/d5d/d60/d6b/cb2 0 2024-04-06T03:40:39.267 INFO:tasks.workunit.client.0.smithi094.stdout:0/536: chown d1/d9/dd/d18/d1e/d86/d96/fa4 46904289 1 2024-04-06T03:40:39.267 INFO:tasks.workunit.client.0.smithi094.stdout:0/537: write d1/d9/dd/d18/d1e/d86/d96/f9a [657008,121858] 0 2024-04-06T03:40:39.269 INFO:tasks.workunit.client.1.smithi136.stdout:2/631: symlink d3/d4a/d9d/d18/d1d/d3c/d58/lca 0 2024-04-06T03:40:39.271 INFO:tasks.workunit.client.1.smithi136.stdout:7/569: dread d5/df/d12/d1b/f2d [0,4194304] 0 2024-04-06T03:40:39.272 INFO:tasks.workunit.client.0.smithi094.stdout:0/538: rename d1/d9/dd/d18/d1e/d86/d96/f9a to d1/d9/dd/fb3 0 2024-04-06T03:40:39.274 INFO:tasks.workunit.client.1.smithi136.stdout:7/570: unlink d5/df/d1d/lc9 0 2024-04-06T03:40:39.275 INFO:tasks.workunit.client.0.smithi094.stdout:0/539: creat d1/d9/dd/d18/d1a/d44/fb4 x:0 0 0 2024-04-06T03:40:39.275 INFO:tasks.workunit.client.0.smithi094.stdout:0/540: fdatasync d1/d9/dd/d18/d1a/f89 0 2024-04-06T03:40:39.276 INFO:tasks.workunit.client.1.smithi136.stdout:7/571: creat d5/df/d1d/d6f/d4d/fcc x:0 0 0 2024-04-06T03:40:39.276 INFO:tasks.workunit.client.1.smithi136.stdout:7/572: chown d5/df/d53/f39 4 1 2024-04-06T03:40:39.277 INFO:tasks.workunit.client.1.smithi136.stdout:1/527: sync 2024-04-06T03:40:39.277 INFO:tasks.workunit.client.0.smithi094.stdout:0/541: mknod d1/d9/dd/d18/cb5 0 2024-04-06T03:40:39.277 INFO:tasks.workunit.client.1.smithi136.stdout:2/632: dread d3/d4a/d9d/d18/d1d/d3c/fa0 [0,4194304] 0 2024-04-06T03:40:39.279 INFO:tasks.workunit.client.0.smithi094.stdout:0/542: read d1/d9/dd/d18/d1d/f57 [853882,24018] 0 2024-04-06T03:40:39.279 INFO:tasks.workunit.client.1.smithi136.stdout:1/528: dread d12/d3b/d44/f67 [0,4194304] 0 2024-04-06T03:40:39.279 INFO:tasks.workunit.client.1.smithi136.stdout:1/529: chown d12/d1b/d1e/d9f/fa8 0 1 2024-04-06T03:40:39.279 INFO:tasks.workunit.client.1.smithi136.stdout:1/530: chown d12/d3b/d6b/f70 14934 1 2024-04-06T03:40:39.281 INFO:tasks.workunit.client.0.smithi094.stdout:0/543: unlink d1/d9/dd/d18/d1e/d86/fa6 0 2024-04-06T03:40:39.281 INFO:tasks.workunit.client.0.smithi094.stdout:0/544: creat d1/d9/dd/d18/d5d/d60/d6b/fb6 x:0 0 0 2024-04-06T03:40:39.284 INFO:tasks.workunit.client.1.smithi136.stdout:1/531: unlink f3 0 2024-04-06T03:40:39.298 INFO:tasks.workunit.client.0.smithi094.stdout:9/540: dwrite d2/dc/d21/d26/f72 [0,4194304] 0 2024-04-06T03:40:39.301 INFO:tasks.workunit.client.0.smithi094.stdout:9/541: creat d2/dc/d10/d2a/fcc x:0 0 0 2024-04-06T03:40:39.302 INFO:tasks.workunit.client.1.smithi136.stdout:8/598: dread d3/de/f29 [0,4194304] 0 2024-04-06T03:40:39.302 INFO:tasks.workunit.client.1.smithi136.stdout:8/599: fdatasync d3/da/d26/d35/d53/f5b 0 2024-04-06T03:40:39.302 INFO:tasks.workunit.client.1.smithi136.stdout:8/600: read d3/da/d26/d35/d39/d70/f93 [201,56128] 0 2024-04-06T03:40:39.304 INFO:tasks.workunit.client.0.smithi094.stdout:0/545: write d1/d9/dd/d18/d1e/d22/d35/f5c [1023486,9041] 0 2024-04-06T03:40:39.309 INFO:tasks.workunit.client.1.smithi136.stdout:7/573: dwrite d5/df/d11/fa2 [0,4194304] 0 2024-04-06T03:40:39.311 INFO:tasks.workunit.client.1.smithi136.stdout:5/598: sync 2024-04-06T03:40:39.311 INFO:tasks.workunit.client.1.smithi136.stdout:5/599: stat d1/dc/d1c/d32/d8b/d20/d7c 0 2024-04-06T03:40:39.313 INFO:tasks.workunit.client.1.smithi136.stdout:7/574: getdents d5/df/d11 0 2024-04-06T03:40:39.313 INFO:tasks.workunit.client.1.smithi136.stdout:7/575: chown d5/df/d12/d1b/d3a/lae 220 1 2024-04-06T03:40:39.314 INFO:tasks.workunit.client.1.smithi136.stdout:5/600: mkdir d1/d98/dbc 0 2024-04-06T03:40:39.314 INFO:tasks.workunit.client.1.smithi136.stdout:5/601: write d1/dc/f91 [882504,18825] 0 2024-04-06T03:40:39.314 INFO:tasks.workunit.client.1.smithi136.stdout:5/602: fsync d1/d16/d4b/f57 0 2024-04-06T03:40:39.315 INFO:tasks.workunit.client.1.smithi136.stdout:7/576: rename d5/df/d1d/d6f/d2c to d5/df/d53/d8a/dcd 0 2024-04-06T03:40:39.318 INFO:tasks.workunit.client.1.smithi136.stdout:7/577: truncate d5/df/d12/d1b/f30 603409 0 2024-04-06T03:40:39.318 INFO:tasks.workunit.client.1.smithi136.stdout:7/578: dread - d5/df/d12/d1b/d3a/d4a/d70/fc0 zero size 2024-04-06T03:40:39.318 INFO:tasks.workunit.client.1.smithi136.stdout:7/579: fdatasync d5/df/d1d/d6f/d4d/d69/f94 0 2024-04-06T03:40:39.318 INFO:tasks.workunit.client.1.smithi136.stdout:7/580: fsync d5/df/d1d/d6f/d2a/f5d 0 2024-04-06T03:40:39.318 INFO:tasks.workunit.client.1.smithi136.stdout:7/581: stat d5/df/d12/d59/d71 0 2024-04-06T03:40:39.318 INFO:tasks.workunit.client.1.smithi136.stdout:7/582: truncate d5/df/d53/d8a/fbc 878996 0 2024-04-06T03:40:39.320 INFO:tasks.workunit.client.1.smithi136.stdout:3/523: dwrite d0/d4/d12/d19/d55/d3c/d5a/fac [0,4194304] 0 2024-04-06T03:40:39.320 INFO:tasks.workunit.client.1.smithi136.stdout:3/524: write d0/d4/d12/d19/d55/d6f/f9e [88757,125317] 0 2024-04-06T03:40:39.320 INFO:tasks.workunit.client.1.smithi136.stdout:3/525: readlink d0/d4/d12/d19/d55/d3c/d46/l66 0 2024-04-06T03:40:39.320 INFO:tasks.workunit.client.1.smithi136.stdout:0/545: dwrite d1/d6/d2c/d57/d43/d6d/d85/d78/f97 [0,4194304] 0 2024-04-06T03:40:39.322 INFO:tasks.workunit.client.1.smithi136.stdout:1/532: dwrite d12/f24 [0,4194304] 0 2024-04-06T03:40:39.322 INFO:tasks.workunit.client.1.smithi136.stdout:2/633: dwrite d3/d15/d19/f60 [0,4194304] 0 2024-04-06T03:40:39.325 INFO:tasks.workunit.client.0.smithi094.stdout:4/584: dread d0/d23/f32 [0,4194304] 0 2024-04-06T03:40:39.327 INFO:tasks.workunit.client.1.smithi136.stdout:7/583: dread d5/f52 [0,4194304] 0 2024-04-06T03:40:39.327 INFO:tasks.workunit.client.0.smithi094.stdout:9/542: read d2/dc/d21/d26/d78/d69/da8/f7f [2237416,476] 0 2024-04-06T03:40:39.329 INFO:tasks.workunit.client.0.smithi094.stdout:6/501: dread dc/d11/d22/f39 [0,4194304] 0 2024-04-06T03:40:39.330 INFO:tasks.workunit.client.0.smithi094.stdout:9/543: symlink d2/dc/d21/d26/lcd 0 2024-04-06T03:40:39.331 INFO:tasks.workunit.client.1.smithi136.stdout:0/546: getdents d1/d6/d2c/d57/d43/d6b 0 2024-04-06T03:40:39.333 INFO:tasks.workunit.client.0.smithi094.stdout:6/502: unlink dc/d11/d1b/d24/d46/c91 0 2024-04-06T03:40:39.335 INFO:tasks.workunit.client.1.smithi136.stdout:1/533: mknod d12/d1b/da7/ca9 0 2024-04-06T03:40:39.339 INFO:tasks.workunit.client.0.smithi094.stdout:9/544: write d2/f30 [3681991,89158] 0 2024-04-06T03:40:39.339 INFO:tasks.workunit.client.0.smithi094.stdout:9/545: fsync d2/dc/d21/d23/d47/f77 0 2024-04-06T03:40:39.344 INFO:tasks.workunit.client.1.smithi136.stdout:2/634: rmdir d3/d9/d3b 39 2024-04-06T03:40:39.344 INFO:tasks.workunit.client.1.smithi136.stdout:2/635: fdatasync d3/d4a/d9d/d8a/d93/fc5 0 2024-04-06T03:40:39.344 INFO:tasks.workunit.client.1.smithi136.stdout:2/636: fsync d3/d15/d19/d43/f94 0 2024-04-06T03:40:39.345 INFO:tasks.workunit.client.0.smithi094.stdout:9/546: rmdir d2/dc/d21/d23/d2f 39 2024-04-06T03:40:39.346 INFO:tasks.workunit.client.1.smithi136.stdout:3/526: getdents d0/d4/d12/d2d 0 2024-04-06T03:40:39.348 INFO:tasks.workunit.client.0.smithi094.stdout:9/547: creat d2/dc/d21/d23/db0/fce x:0 0 0 2024-04-06T03:40:39.349 INFO:tasks.workunit.client.0.smithi094.stdout:9/548: dread d2/dc/d21/d23/d47/f77 [0,4194304] 0 2024-04-06T03:40:39.350 INFO:tasks.workunit.client.1.smithi136.stdout:0/547: symlink d1/lb7 0 2024-04-06T03:40:39.350 INFO:tasks.workunit.client.1.smithi136.stdout:0/548: write d1/fc [732651,69157] 0 2024-04-06T03:40:39.350 INFO:tasks.workunit.client.1.smithi136.stdout:1/534: mkdir d12/d1b/da7/daa 0 2024-04-06T03:40:39.353 INFO:tasks.workunit.client.0.smithi094.stdout:9/549: creat d2/dc/d21/d26/d78/d69/fcf x:0 0 0 2024-04-06T03:40:39.353 INFO:tasks.workunit.client.0.smithi094.stdout:9/550: fsync d2/dc/d21/d23/d47/d51/f8c 0 2024-04-06T03:40:39.353 INFO:tasks.workunit.client.0.smithi094.stdout:9/551: chown d2/fe 22125 1 2024-04-06T03:40:39.354 INFO:tasks.workunit.client.1.smithi136.stdout:2/637: rename d3/d4a/d9d/d18/d1d/d3c/f40 to d3/d9b/fcb 0 2024-04-06T03:40:39.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/552: mkdir d2/dc/d80/db7/dd0 0 2024-04-06T03:40:39.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/553: readlink d2/l40 0 2024-04-06T03:40:39.356 INFO:tasks.workunit.client.1.smithi136.stdout:3/527: rename d0/d4/d12/d19/d55/d3c/c41 to d0/d4/d12/d2d/d37/cae 0 2024-04-06T03:40:39.357 INFO:tasks.workunit.client.1.smithi136.stdout:2/638: link d3/d4a/d9d/d18/c75 d3/d9/d3b/d70/dc8/ccc 0 2024-04-06T03:40:39.358 INFO:tasks.workunit.client.1.smithi136.stdout:2/639: write d3/d4a/d9d/d18/d1d/f35 [8575480,122470] 0 2024-04-06T03:40:39.360 INFO:tasks.workunit.client.1.smithi136.stdout:0/549: rename d1/d6/d2c/d1a/d36/f44 to d1/d6/d2c/d1a/d36/d59/fb8 0 2024-04-06T03:40:39.360 INFO:tasks.workunit.client.1.smithi136.stdout:0/550: fsync d1/d6/d2c/d57/d43/d6d/d85/d78/f97 0 2024-04-06T03:40:39.361 INFO:tasks.workunit.client.1.smithi136.stdout:3/528: symlink d0/d4/d12/d19/d55/d6f/d82/d5d/d61/laf 0 2024-04-06T03:40:39.361 INFO:tasks.workunit.client.1.smithi136.stdout:3/529: readlink d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/la2 0 2024-04-06T03:40:39.363 INFO:tasks.workunit.client.1.smithi136.stdout:4/492: sync 2024-04-06T03:40:39.363 INFO:tasks.workunit.client.0.smithi094.stdout:9/554: read d2/dc/d21/d23/db0/fbf [2235957,45033] 0 2024-04-06T03:40:39.363 INFO:tasks.workunit.client.0.smithi094.stdout:9/555: write d2/dc/d80/db7/fc7 [781197,22938] 0 2024-04-06T03:40:39.364 INFO:tasks.workunit.client.1.smithi136.stdout:7/584: dwrite d5/df/d1d/d54/fa4 [0,4194304] 0 2024-04-06T03:40:39.365 INFO:tasks.workunit.client.0.smithi094.stdout:4/585: dwrite d0/d15/d3a/d3e/d50/f65 [4194304,4194304] 0 2024-04-06T03:40:39.365 INFO:tasks.workunit.client.0.smithi094.stdout:4/586: chown d0/df/d1e/c96 3856 1 2024-04-06T03:40:39.365 INFO:tasks.workunit.client.0.smithi094.stdout:4/587: write d0/df/d13/f36 [1208539,91475] 0 2024-04-06T03:40:39.365 INFO:tasks.workunit.client.0.smithi094.stdout:6/503: dwrite dc/d3a/d4a/d54/fb2 [0,4194304] 0 2024-04-06T03:40:39.366 INFO:tasks.workunit.client.1.smithi136.stdout:0/551: unlink d1/d4e/f77 0 2024-04-06T03:40:39.366 INFO:tasks.workunit.client.1.smithi136.stdout:0/552: read d1/d4e/f55 [1002622,15694] 0 2024-04-06T03:40:39.366 INFO:tasks.workunit.client.1.smithi136.stdout:2/640: dread d3/d15/d19/f39 [0,4194304] 0 2024-04-06T03:40:39.368 INFO:tasks.workunit.client.1.smithi136.stdout:3/530: rename d0/d4/d12/d19/d55/l1d to d0/d4/d12/d2d/d37/lb0 0 2024-04-06T03:40:39.371 INFO:tasks.workunit.client.1.smithi136.stdout:4/493: mkdir d1d/d31/d3e/d55/d89/db3 0 2024-04-06T03:40:39.376 INFO:tasks.workunit.client.0.smithi094.stdout:4/588: creat d0/d15/d3a/d3e/d49/db2/fc2 x:0 0 0 2024-04-06T03:40:39.383 INFO:tasks.workunit.client.1.smithi136.stdout:0/553: unlink d1/d6/l22 0 2024-04-06T03:40:39.383 INFO:tasks.workunit.client.1.smithi136.stdout:0/554: stat d1/f29 0 2024-04-06T03:40:39.384 INFO:tasks.workunit.client.0.smithi094.stdout:6/504: dwrite dc/d11/d22/f9e [0,4194304] 0 2024-04-06T03:40:39.384 INFO:tasks.workunit.client.0.smithi094.stdout:6/505: creat dc/d11/d1b/fb4 x:0 0 0 2024-04-06T03:40:39.384 INFO:tasks.workunit.client.0.smithi094.stdout:0/546: rmdir d1/d9/dd/d18/d1e/d86/d96 39 2024-04-06T03:40:39.385 INFO:tasks.workunit.client.0.smithi094.stdout:0/547: dread - d1/d9/dd/d18/d1a/d44/f80 zero size 2024-04-06T03:40:39.385 INFO:tasks.workunit.client.0.smithi094.stdout:0/548: write d1/d9/dd/d18/dad/f93 [327650,114828] 0 2024-04-06T03:40:39.389 INFO:tasks.workunit.client.1.smithi136.stdout:4/494: creat d1d/d2f/d6c/d91/fb4 x:0 0 0 2024-04-06T03:40:39.392 INFO:tasks.workunit.client.1.smithi136.stdout:0/555: creat d1/d6/d2c/d1a/d36/d59/fb9 x:0 0 0 2024-04-06T03:40:39.392 INFO:tasks.workunit.client.1.smithi136.stdout:0/556: fsync d1/d6/d2c/d57/d43/d6d/d85/da7/fb4 0 2024-04-06T03:40:39.394 INFO:tasks.workunit.client.0.smithi094.stdout:4/589: unlink d0/d15/d3a/d3e/d46/f85 0 2024-04-06T03:40:39.395 INFO:tasks.workunit.client.1.smithi136.stdout:4/495: write d1d/d31/f41 [4948952,41141] 0 2024-04-06T03:40:39.398 INFO:tasks.workunit.client.1.smithi136.stdout:0/557: creat d1/d6/d2c/d57/d43/d6d/d85/d86/fba x:0 0 0 2024-04-06T03:40:39.398 INFO:tasks.workunit.client.1.smithi136.stdout:0/558: chown d1/d6/d2c/d57/d43/d6d/d9b/c64 323 1 2024-04-06T03:40:39.399 INFO:tasks.workunit.client.0.smithi094.stdout:6/506: getdents dc/d11/d22/d50 0 2024-04-06T03:40:39.399 INFO:tasks.workunit.client.0.smithi094.stdout:6/507: chown dc/d3a/d4a/f92 3 1 2024-04-06T03:40:39.399 INFO:tasks.workunit.client.0.smithi094.stdout:6/508: chown fa 79 1 2024-04-06T03:40:39.400 INFO:tasks.workunit.client.1.smithi136.stdout:0/559: mknod d1/d6/d2c/d57/d62/cbb 0 2024-04-06T03:40:39.401 INFO:tasks.workunit.client.0.smithi094.stdout:4/590: mknod d0/df/d57/cc3 0 2024-04-06T03:40:39.403 INFO:tasks.workunit.client.1.smithi136.stdout:4/496: symlink d1d/lb5 0 2024-04-06T03:40:39.403 INFO:tasks.workunit.client.0.smithi094.stdout:4/591: mkdir d0/d77/dc4 0 2024-04-06T03:40:39.406 INFO:tasks.workunit.client.0.smithi094.stdout:4/592: write d0/d15/d3a/d3e/d46/f6a [300599,117910] 0 2024-04-06T03:40:39.407 INFO:tasks.workunit.client.0.smithi094.stdout:4/593: creat d0/d15/d3a/d61/d81/da0/fc5 x:0 0 0 2024-04-06T03:40:39.409 INFO:tasks.workunit.client.0.smithi094.stdout:4/594: truncate d0/d15/d3a/d61/d81/fba 2727078 0 2024-04-06T03:40:39.409 INFO:tasks.workunit.client.0.smithi094.stdout:4/595: write d0/d15/f1a [5354566,91830] 0 2024-04-06T03:40:39.410 INFO:tasks.workunit.client.1.smithi136.stdout:4/497: dread f1c [0,4194304] 0 2024-04-06T03:40:39.410 INFO:tasks.workunit.client.0.smithi094.stdout:4/596: symlink d0/d4/d4b/lc6 0 2024-04-06T03:40:39.410 INFO:tasks.workunit.client.0.smithi094.stdout:4/597: rmdir d0/df/d13 39 2024-04-06T03:40:39.412 INFO:tasks.workunit.client.0.smithi094.stdout:4/598: mkdir d0/d15/d3a/d3e/d49/db2/dc7 0 2024-04-06T03:40:39.412 INFO:tasks.workunit.client.0.smithi094.stdout:4/599: fsync d0/d4/fb 0 2024-04-06T03:40:39.413 INFO:tasks.workunit.client.0.smithi094.stdout:4/600: symlink d0/d15/d3a/d3e/d46/lc8 0 2024-04-06T03:40:39.416 INFO:tasks.workunit.client.0.smithi094.stdout:4/601: unlink d0/d15/d3a/d61/d81/fab 0 2024-04-06T03:40:39.417 INFO:tasks.workunit.client.0.smithi094.stdout:4/602: creat d0/df/db3/fc9 x:0 0 0 2024-04-06T03:40:39.418 INFO:tasks.workunit.client.1.smithi136.stdout:4/498: dread f7 [0,4194304] 0 2024-04-06T03:40:39.418 INFO:tasks.workunit.client.1.smithi136.stdout:4/499: write d1d/d2f/f7d [4965878,1288] 0 2024-04-06T03:40:39.419 INFO:tasks.workunit.client.0.smithi094.stdout:4/603: creat d0/d4/d4b/fca x:0 0 0 2024-04-06T03:40:39.420 INFO:tasks.workunit.client.0.smithi094.stdout:4/604: mkdir d0/d4/d67/dcb 0 2024-04-06T03:40:39.420 INFO:tasks.workunit.client.0.smithi094.stdout:4/605: stat d0/l28 0 2024-04-06T03:40:39.421 INFO:tasks.workunit.client.1.smithi136.stdout:4/500: getdents d1d/d2f 0 2024-04-06T03:40:39.422 INFO:tasks.workunit.client.0.smithi094.stdout:4/606: mknod d0/df/d91/dbb/ccc 0 2024-04-06T03:40:39.423 INFO:tasks.workunit.client.1.smithi136.stdout:4/501: creat d1d/d31/d3e/d55/d7f/fb6 x:0 0 0 2024-04-06T03:40:39.424 INFO:tasks.workunit.client.0.smithi094.stdout:4/607: creat d0/d23/dc1/fcd x:0 0 0 2024-04-06T03:40:39.425 INFO:tasks.workunit.client.1.smithi136.stdout:4/502: creat d1d/d31/d3e/d55/d89/db3/fb7 x:0 0 0 2024-04-06T03:40:39.426 INFO:tasks.workunit.client.1.smithi136.stdout:4/503: write d1d/d31/d3e/d56/f6f [714738,40233] 0 2024-04-06T03:40:39.426 INFO:tasks.workunit.client.1.smithi136.stdout:4/504: dread - d1d/d31/d3e/f5a zero size 2024-04-06T03:40:39.426 INFO:tasks.workunit.client.0.smithi094.stdout:4/608: creat d0/d15/d3a/d3e/d46/fce x:0 0 0 2024-04-06T03:40:39.427 INFO:tasks.workunit.client.1.smithi136.stdout:4/505: creat d1d/d31/d3e/d55/fb8 x:0 0 0 2024-04-06T03:40:39.427 INFO:tasks.workunit.client.1.smithi136.stdout:4/506: chown d1d/lb5 8 1 2024-04-06T03:40:39.436 INFO:tasks.workunit.client.1.smithi136.stdout:6/497: sync 2024-04-06T03:40:39.436 INFO:tasks.workunit.client.1.smithi136.stdout:9/544: sync 2024-04-06T03:40:39.437 INFO:tasks.workunit.client.1.smithi136.stdout:6/498: rmdir d2/d7/d26/d30/d6e/d3d 39 2024-04-06T03:40:39.439 INFO:tasks.workunit.client.1.smithi136.stdout:9/545: rename d17/d20/d28/d81/d88/d8d to d17/d20/d25/d3d/da5/db8 0 2024-04-06T03:40:39.439 INFO:tasks.workunit.client.1.smithi136.stdout:9/546: chown d17/d20/d28/d49/d75/d60/f7a 956 1 2024-04-06T03:40:39.440 INFO:tasks.workunit.client.1.smithi136.stdout:9/547: fsync d17/d20/d28/d81/d88/f90 0 2024-04-06T03:40:39.447 INFO:tasks.workunit.client.1.smithi136.stdout:6/499: write d2/d7/d26/d30/d6e/f2f [2934830,14940] 0 2024-04-06T03:40:39.449 INFO:tasks.workunit.client.1.smithi136.stdout:6/500: unlink d2/f9 0 2024-04-06T03:40:39.450 INFO:tasks.workunit.client.1.smithi136.stdout:6/501: creat d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/fa9 x:0 0 0 2024-04-06T03:40:39.450 INFO:tasks.workunit.client.1.smithi136.stdout:6/502: rmdir d2/d7 39 2024-04-06T03:40:39.451 INFO:tasks.workunit.client.1.smithi136.stdout:6/503: creat d2/d7/d26/faa x:0 0 0 2024-04-06T03:40:39.453 INFO:tasks.workunit.client.1.smithi136.stdout:6/504: mkdir d2/d93/dab 0 2024-04-06T03:40:39.453 INFO:tasks.workunit.client.1.smithi136.stdout:6/505: readlink d2/d25/l7a 0 2024-04-06T03:40:39.453 INFO:tasks.workunit.client.1.smithi136.stdout:6/506: write d2/d7/d26/d30/d6e/d58/d5d/d62/d53/fa1 [419751,28410] 0 2024-04-06T03:40:39.455 INFO:tasks.workunit.client.1.smithi136.stdout:6/507: unlink d2/d7/d26/d30/d4f/f6d 0 2024-04-06T03:40:39.455 INFO:tasks.workunit.client.1.smithi136.stdout:6/508: write d2/d7/d26/d30/d6e/d3d/d47/d84/fa0 [897855,129264] 0 2024-04-06T03:40:39.457 INFO:tasks.workunit.client.1.smithi136.stdout:6/509: rename d2/d25/l87 to d2/d7/d26/lac 0 2024-04-06T03:40:39.458 INFO:tasks.workunit.client.1.smithi136.stdout:2/641: dwrite d3/d15/d19/f60 [0,4194304] 0 2024-04-06T03:40:39.458 INFO:tasks.workunit.client.1.smithi136.stdout:7/585: dwrite d5/df/d1d/d6f/d4d/d69/f7b [0,4194304] 0 2024-04-06T03:40:39.459 INFO:tasks.workunit.client.1.smithi136.stdout:6/510: mkdir d2/d7/d26/d30/d6e/d3d/d47/d84/dad 0 2024-04-06T03:40:39.459 INFO:tasks.workunit.client.0.smithi094.stdout:0/549: dwrite d1/f7a [0,4194304] 0 2024-04-06T03:40:39.460 INFO:tasks.workunit.client.1.smithi136.stdout:2/642: creat d3/d4a/d77/fcd x:0 0 0 2024-04-06T03:40:39.461 INFO:tasks.workunit.client.1.smithi136.stdout:6/511: unlink d2/c6 0 2024-04-06T03:40:39.462 INFO:tasks.workunit.client.1.smithi136.stdout:7/586: symlink d5/df/d1d/d6f/d4d/d69/lce 0 2024-04-06T03:40:39.462 INFO:tasks.workunit.client.1.smithi136.stdout:7/587: creat d5/fcf x:0 0 0 2024-04-06T03:40:39.463 INFO:tasks.workunit.client.1.smithi136.stdout:7/588: chown d5/f80 198616 1 2024-04-06T03:40:39.463 INFO:tasks.workunit.client.0.smithi094.stdout:0/550: truncate d1/d9/fc 7999825 0 2024-04-06T03:40:39.463 INFO:tasks.workunit.client.0.smithi094.stdout:0/551: fdatasync d1/d9/dd/d18/d1a/f54 0 2024-04-06T03:40:39.464 INFO:tasks.workunit.client.1.smithi136.stdout:6/512: creat d2/d7/d26/d68/d9b/fae x:0 0 0 2024-04-06T03:40:39.465 INFO:tasks.workunit.client.1.smithi136.stdout:0/560: dwrite d1/d6/d2c/d1a/d36/f8f [0,4194304] 0 2024-04-06T03:40:39.466 INFO:tasks.workunit.client.0.smithi094.stdout:0/552: mknod d1/d9/dd/d18/dad/cb7 0 2024-04-06T03:40:39.466 INFO:tasks.workunit.client.1.smithi136.stdout:3/531: dwrite d0/d4/d12/d19/d2e/d3b/d49/f5e [4194304,4194304] 0 2024-04-06T03:40:39.467 INFO:tasks.workunit.client.1.smithi136.stdout:7/589: mkdir d5/df/d53/d68/da3/dd0 0 2024-04-06T03:40:39.467 INFO:tasks.workunit.client.1.smithi136.stdout:7/590: fsync d5/df/f62 0 2024-04-06T03:40:39.467 INFO:tasks.workunit.client.0.smithi094.stdout:4/609: dread d0/df/d1e/f24 [4194304,4194304] 0 2024-04-06T03:40:39.468 INFO:tasks.workunit.client.0.smithi094.stdout:6/509: dwrite dc/d11/d1b/d72/d88/f9f [0,4194304] 0 2024-04-06T03:40:39.468 INFO:tasks.workunit.client.0.smithi094.stdout:6/510: creat dc/d3a/d4a/d54/fb5 x:0 0 0 2024-04-06T03:40:39.468 INFO:tasks.workunit.client.0.smithi094.stdout:6/511: readlink dc/d11/l2a 0 2024-04-06T03:40:39.468 INFO:tasks.workunit.client.0.smithi094.stdout:6/512: readlink dc/d11/l5a 0 2024-04-06T03:40:39.468 INFO:tasks.workunit.client.1.smithi136.stdout:9/548: dwrite d17/d20/d25/d3d/d4b/f80 [0,4194304] 0 2024-04-06T03:40:39.469 INFO:tasks.workunit.client.1.smithi136.stdout:4/507: rmdir d1d/d2f/d6c 39 2024-04-06T03:40:39.471 INFO:tasks.workunit.client.0.smithi094.stdout:0/553: unlink d1/d9/dd/d18/d5d/d60/d6b/dab/l74 0 2024-04-06T03:40:39.473 INFO:tasks.workunit.client.1.smithi136.stdout:9/549: write d17/d20/d28/d81/f83 [1435786,88252] 0 2024-04-06T03:40:39.473 INFO:tasks.workunit.client.0.smithi094.stdout:4/610: symlink d0/df/lcf 0 2024-04-06T03:40:39.474 INFO:tasks.workunit.client.0.smithi094.stdout:4/611: chown d0/d77/f9e 339316 1 2024-04-06T03:40:39.474 INFO:tasks.workunit.client.0.smithi094.stdout:4/612: truncate d0/d15/d3a/d3e/d46/fce 154588 0 2024-04-06T03:40:39.474 INFO:tasks.workunit.client.0.smithi094.stdout:4/613: write d0/d15/fa2 [258299,125231] 0 2024-04-06T03:40:39.477 INFO:tasks.workunit.client.1.smithi136.stdout:6/513: write d2/d25/f38 [4743511,82836] 0 2024-04-06T03:40:39.480 INFO:tasks.workunit.client.0.smithi094.stdout:6/513: rename dc/d11/d22/d50/d61/dae to dc/db6 0 2024-04-06T03:40:39.482 INFO:tasks.workunit.client.0.smithi094.stdout:0/554: mknod d1/d9/dd/d18/d1d/cb8 0 2024-04-06T03:40:39.484 INFO:tasks.workunit.client.0.smithi094.stdout:4/614: symlink d0/df/db3/ld0 0 2024-04-06T03:40:39.484 INFO:tasks.workunit.client.0.smithi094.stdout:4/615: fsync d0/df/f1b 0 2024-04-06T03:40:39.485 INFO:tasks.workunit.client.0.smithi094.stdout:4/616: fdatasync d0/d23/f32 0 2024-04-06T03:40:39.488 INFO:tasks.workunit.client.0.smithi094.stdout:6/514: rmdir dc/d11/d1b/d72/d88/da3 39 2024-04-06T03:40:39.491 INFO:tasks.workunit.client.1.smithi136.stdout:3/532: creat d0/d4/d12/d19/d55/d6f/fb1 x:0 0 0 2024-04-06T03:40:39.492 INFO:tasks.workunit.client.0.smithi094.stdout:6/515: mknod dc/d3a/d4a/d54/cb7 0 2024-04-06T03:40:39.494 INFO:tasks.workunit.client.1.smithi136.stdout:7/591: rename d5/df/d12/c18 to d5/df/d3d/cd1 0 2024-04-06T03:40:39.500 INFO:tasks.workunit.client.1.smithi136.stdout:9/550: mknod d17/d19/cb9 0 2024-04-06T03:40:39.501 INFO:tasks.workunit.client.1.smithi136.stdout:2/643: dwrite d3/d15/d19/f9a [4194304,4194304] 0 2024-04-06T03:40:39.502 INFO:tasks.workunit.client.0.smithi094.stdout:2/608: sync 2024-04-06T03:40:39.503 INFO:tasks.workunit.client.0.smithi094.stdout:4/617: fdatasync d0/d15/d3a/d3e/d46/f6a 0 2024-04-06T03:40:39.504 INFO:tasks.workunit.client.1.smithi136.stdout:0/561: truncate d1/d6/d2c/d57/d43/d6d/d85/d78/f7a 2941302 0 2024-04-06T03:40:39.505 INFO:tasks.workunit.client.0.smithi094.stdout:0/555: dwrite d1/d9/dd/d18/d1e/d22/d35/f3c [0,4194304] 0 2024-04-06T03:40:39.510 INFO:tasks.workunit.client.1.smithi136.stdout:6/514: truncate d2/d25/f17 2583091 0 2024-04-06T03:40:39.514 INFO:tasks.workunit.client.0.smithi094.stdout:4/618: write d0/d4/f6 [3639216,74308] 0 2024-04-06T03:40:39.514 INFO:tasks.workunit.client.1.smithi136.stdout:3/533: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/fb2 x:0 0 0 2024-04-06T03:40:39.514 INFO:tasks.workunit.client.1.smithi136.stdout:3/534: fdatasync d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/f9b 0 2024-04-06T03:40:39.516 INFO:tasks.workunit.client.0.smithi094.stdout:6/516: dwrite dc/d11/d1b/d72/d88/f96 [0,4194304] 0 2024-04-06T03:40:39.516 INFO:tasks.workunit.client.0.smithi094.stdout:6/517: fdatasync dc/d11/d1b/d24/d46/fa5 0 2024-04-06T03:40:39.516 INFO:tasks.workunit.client.0.smithi094.stdout:6/518: chown dc/d11/l1e 11490650 1 2024-04-06T03:40:39.518 INFO:tasks.workunit.client.0.smithi094.stdout:0/556: read d1/d9/dd/d18/d32/f9c [1553265,56978] 0 2024-04-06T03:40:39.520 INFO:tasks.workunit.client.1.smithi136.stdout:7/592: unlink d5/df/d12/d1b/d58/ca5 0 2024-04-06T03:40:39.520 INFO:tasks.workunit.client.1.smithi136.stdout:7/593: dread - d5/df/d12/d1b/d58/f7a zero size 2024-04-06T03:40:39.523 INFO:tasks.workunit.client.0.smithi094.stdout:2/609: dwrite d0/d8/d27/d2b/d8e/fac [0,4194304] 0 2024-04-06T03:40:39.524 INFO:tasks.workunit.client.0.smithi094.stdout:4/619: creat d0/d23/fd1 x:0 0 0 2024-04-06T03:40:39.524 INFO:tasks.workunit.client.0.smithi094.stdout:6/519: read dc/d11/d22/f9e [3012095,2215] 0 2024-04-06T03:40:39.525 INFO:tasks.workunit.client.1.smithi136.stdout:2/644: rmdir d3/d4a/d9d/d8a/d93/d5f 39 2024-04-06T03:40:39.527 INFO:tasks.workunit.client.0.smithi094.stdout:4/620: dread d0/d4/f7c [0,4194304] 0 2024-04-06T03:40:39.530 INFO:tasks.workunit.client.1.smithi136.stdout:6/515: getdents d2/d7 0 2024-04-06T03:40:39.530 INFO:tasks.workunit.client.1.smithi136.stdout:9/551: dwrite d17/d21/d32/d43/f8c [0,4194304] 0 2024-04-06T03:40:39.534 INFO:tasks.workunit.client.1.smithi136.stdout:3/535: symlink d0/d4/d12/d19/d55/d6f/d82/d5d/lb3 0 2024-04-06T03:40:39.538 INFO:tasks.workunit.client.1.smithi136.stdout:4/508: rename d1d/c1e to d1d/d2f/d6c/d91/cb9 0 2024-04-06T03:40:39.540 INFO:tasks.workunit.client.0.smithi094.stdout:0/557: symlink d1/d9/dd/d18/d5d/d88/lb9 0 2024-04-06T03:40:39.546 INFO:tasks.workunit.client.1.smithi136.stdout:2/645: mkdir d3/d15/d19/d43/daa/dce 0 2024-04-06T03:40:39.550 INFO:tasks.workunit.client.1.smithi136.stdout:9/552: rename d17/d21/d29/d34/d66/f9b to d17/d20/fba 0 2024-04-06T03:40:39.552 INFO:tasks.workunit.client.0.smithi094.stdout:0/558: write d1/d9/dd/d18/d1e/d22/f37 [414778,1752] 0 2024-04-06T03:40:39.554 INFO:tasks.workunit.client.0.smithi094.stdout:0/559: creat d1/d9/dd/d18/d5d/d60/d6b/d67/d76/fba x:0 0 0 2024-04-06T03:40:39.555 INFO:tasks.workunit.client.0.smithi094.stdout:2/610: dwrite d0/dd/d1b/d72/d8f/fbb [0,4194304] 0 2024-04-06T03:40:39.556 INFO:tasks.workunit.client.0.smithi094.stdout:2/611: fsync d0/dd/d19/f4c 0 2024-04-06T03:40:39.556 INFO:tasks.workunit.client.0.smithi094.stdout:2/612: readlink d0/d8/d27/d2b/d76/lc5 0 2024-04-06T03:40:39.556 INFO:tasks.workunit.client.0.smithi094.stdout:0/560: creat d1/d9/dd/d18/d1e/fbb x:0 0 0 2024-04-06T03:40:39.558 INFO:tasks.workunit.client.0.smithi094.stdout:2/613: symlink d0/d8/d4f/ld1 0 2024-04-06T03:40:39.559 INFO:tasks.workunit.client.0.smithi094.stdout:2/614: mkdir d0/dd/d1b/d72/da8/dd2 0 2024-04-06T03:40:39.560 INFO:tasks.workunit.client.0.smithi094.stdout:2/615: creat d0/dd/d1b/fd3 x:0 0 0 2024-04-06T03:40:39.563 INFO:tasks.workunit.client.1.smithi136.stdout:1/535: sync 2024-04-06T03:40:39.563 INFO:tasks.workunit.client.1.smithi136.stdout:5/603: sync 2024-04-06T03:40:39.563 INFO:tasks.workunit.client.1.smithi136.stdout:8/601: sync 2024-04-06T03:40:39.563 INFO:tasks.workunit.client.1.smithi136.stdout:8/602: fdatasync d3/de/f8d 0 2024-04-06T03:40:39.564 INFO:tasks.workunit.client.1.smithi136.stdout:5/604: rmdir d1/dc/d1c/d32/d70 39 2024-04-06T03:40:39.564 INFO:tasks.workunit.client.1.smithi136.stdout:5/605: write d1/dc/d1c/d32/d8b/d20/d66/fb9 [312933,119987] 0 2024-04-06T03:40:39.564 INFO:tasks.workunit.client.1.smithi136.stdout:9/553: dread d17/d20/d28/d81/f83 [0,4194304] 0 2024-04-06T03:40:39.564 INFO:tasks.workunit.client.0.smithi094.stdout:0/561: dread d1/f7a [0,4194304] 0 2024-04-06T03:40:39.565 INFO:tasks.workunit.client.1.smithi136.stdout:1/536: mknod d12/d3b/d44/d5f/cab 0 2024-04-06T03:40:39.566 INFO:tasks.workunit.client.1.smithi136.stdout:8/603: mknod d3/d2d/d5f/cb6 0 2024-04-06T03:40:39.567 INFO:tasks.workunit.client.0.smithi094.stdout:0/562: mkdir d1/d9/dd/d18/d5d/d60/dbc 0 2024-04-06T03:40:39.569 INFO:tasks.workunit.client.1.smithi136.stdout:9/554: mkdir d17/d20/d25/da4/dbb 0 2024-04-06T03:40:39.570 INFO:tasks.workunit.client.1.smithi136.stdout:1/537: symlink d12/d3b/d74/d94/lac 0 2024-04-06T03:40:39.570 INFO:tasks.workunit.client.1.smithi136.stdout:1/538: chown d12/d1b/d1e/d23/fa3 4981 1 2024-04-06T03:40:39.571 INFO:tasks.workunit.client.1.smithi136.stdout:8/604: mknod d3/da/d26/d35/d53/d86/cb7 0 2024-04-06T03:40:39.571 INFO:tasks.workunit.client.1.smithi136.stdout:8/605: stat d3/de/f4b 0 2024-04-06T03:40:39.573 INFO:tasks.workunit.client.1.smithi136.stdout:9/555: truncate d17/d20/d25/d3d/d4b/f80 3561269 0 2024-04-06T03:40:39.574 INFO:tasks.workunit.client.1.smithi136.stdout:1/539: truncate d12/d1b/f29 2467689 0 2024-04-06T03:40:39.574 INFO:tasks.workunit.client.1.smithi136.stdout:1/540: creat d12/d3b/d74/d7a/fad x:0 0 0 2024-04-06T03:40:39.574 INFO:tasks.workunit.client.1.smithi136.stdout:1/541: write d12/f2c [2822542,27082] 0 2024-04-06T03:40:39.574 INFO:tasks.workunit.client.1.smithi136.stdout:6/516: dwrite d2/f3c [4194304,4194304] 0 2024-04-06T03:40:39.574 INFO:tasks.workunit.client.1.smithi136.stdout:6/517: chown d2/d7/d26 26676 1 2024-04-06T03:40:39.574 INFO:tasks.workunit.client.1.smithi136.stdout:6/518: read d2/d7/d26/d30/d6e/f6b [108203,8053] 0 2024-04-06T03:40:39.575 INFO:tasks.workunit.client.0.smithi094.stdout:0/563: dread d1/d9/dd/d18/d5d/d60/d6b/dab/f73 [0,4194304] 0 2024-04-06T03:40:39.575 INFO:tasks.workunit.client.1.smithi136.stdout:3/536: dwrite d0/f23 [0,4194304] 0 2024-04-06T03:40:39.576 INFO:tasks.workunit.client.1.smithi136.stdout:2/646: dwrite d3/d4a/d9d/d18/f2a [0,4194304] 0 2024-04-06T03:40:39.576 INFO:tasks.workunit.client.1.smithi136.stdout:8/606: rename d3/de/d18/c30 to d3/da/d26/d35/d39/d70/d7e/cb8 0 2024-04-06T03:40:39.576 INFO:tasks.workunit.client.1.smithi136.stdout:7/594: dwrite d5/df/d12/d1b/f30 [0,4194304] 0 2024-04-06T03:40:39.576 INFO:tasks.workunit.client.1.smithi136.stdout:7/595: write d5/df/d11/f26 [34846,45711] 0 2024-04-06T03:40:39.576 INFO:tasks.workunit.client.1.smithi136.stdout:7/596: write d5/fcb [780372,76339] 0 2024-04-06T03:40:39.576 INFO:tasks.workunit.client.1.smithi136.stdout:2/647: dread d3/d4a/d9d/d8a/fb2 [0,4194304] 0 2024-04-06T03:40:39.579 INFO:tasks.workunit.client.0.smithi094.stdout:6/520: dwrite dc/d11/d1b/d72/f81 [4194304,4194304] 0 2024-04-06T03:40:39.579 INFO:tasks.workunit.client.0.smithi094.stdout:0/564: rename d1/d9/dd/d18/d32/d42/l56 to d1/d9/dd/d18/d1e/d22/d3e/lbd 0 2024-04-06T03:40:39.579 INFO:tasks.workunit.client.0.smithi094.stdout:0/565: write d1/d9/dd/d18/d5d/d60/d6b/fb6 [995589,83243] 0 2024-04-06T03:40:39.580 INFO:tasks.workunit.client.0.smithi094.stdout:0/566: dread d1/d9/dd/d18/d1d/f57 [0,4194304] 0 2024-04-06T03:40:39.580 INFO:tasks.workunit.client.0.smithi094.stdout:0/567: creat d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/fbe x:0 0 0 2024-04-06T03:40:39.580 INFO:tasks.workunit.client.0.smithi094.stdout:0/568: chown d1/d9/dd/d18/d1e/d22 0 1 2024-04-06T03:40:39.581 INFO:tasks.workunit.client.1.smithi136.stdout:1/542: getdents d12/d3b/d44/d5f/d5a 0 2024-04-06T03:40:39.582 INFO:tasks.workunit.client.1.smithi136.stdout:6/519: read d2/fa [2885220,51282] 0 2024-04-06T03:40:39.582 INFO:tasks.workunit.client.1.smithi136.stdout:3/537: truncate d0/d4/d12/d19/d55/d3c/d46/f78 164022 0 2024-04-06T03:40:39.583 INFO:tasks.workunit.client.1.smithi136.stdout:4/509: getdents d1d 0 2024-04-06T03:40:39.585 INFO:tasks.workunit.client.1.smithi136.stdout:9/556: rename d17/d21/d32/f4f to d17/d21/d29/d34/d46/d9a/fbc 0 2024-04-06T03:40:39.586 INFO:tasks.workunit.client.1.smithi136.stdout:8/607: creat d3/de/d18/d7d/fb9 x:0 0 0 2024-04-06T03:40:39.588 INFO:tasks.workunit.client.1.smithi136.stdout:7/597: symlink d5/df/d1d/d6f/d2a/ld2 0 2024-04-06T03:40:39.588 INFO:tasks.workunit.client.1.smithi136.stdout:7/598: dread - d5/df/d1d/fc6 zero size 2024-04-06T03:40:39.590 INFO:tasks.workunit.client.1.smithi136.stdout:1/543: link d12/d1b/d1e/d23/fa3 d12/d3b/d44/d5f/d5a/fae 0 2024-04-06T03:40:39.590 INFO:tasks.workunit.client.1.smithi136.stdout:1/544: chown d12/d53/f95 59 1 2024-04-06T03:40:39.591 INFO:tasks.workunit.client.1.smithi136.stdout:6/520: mknod d2/d7/d9e/caf 0 2024-04-06T03:40:39.592 INFO:tasks.workunit.client.1.smithi136.stdout:4/510: mkdir d1d/d31/d3e/d55/dba 0 2024-04-06T03:40:39.593 INFO:tasks.workunit.client.1.smithi136.stdout:4/511: chown d1d/c2a 0 1 2024-04-06T03:40:39.593 INFO:tasks.workunit.client.1.smithi136.stdout:4/512: chown d1d/d2f/d6c/d91/c93 382545735 1 2024-04-06T03:40:39.593 INFO:tasks.workunit.client.0.smithi094.stdout:6/521: creat dc/d3a/d4a/d94/dac/fb8 x:0 0 0 2024-04-06T03:40:39.595 INFO:tasks.workunit.client.1.smithi136.stdout:3/538: write d0/fb [3815595,96916] 0 2024-04-06T03:40:39.597 INFO:tasks.workunit.client.1.smithi136.stdout:2/648: rename d3/d9/d3b/d70/l92 to d3/d4a/d9d/d18/d1d/d3c/d58/lcf 0 2024-04-06T03:40:39.598 INFO:tasks.workunit.client.1.smithi136.stdout:8/608: write d3/da/d26/f28 [5433866,87484] 0 2024-04-06T03:40:39.598 INFO:tasks.workunit.client.1.smithi136.stdout:2/649: write d3/d9b/f86 [67382,84214] 0 2024-04-06T03:40:39.598 INFO:tasks.workunit.client.0.smithi094.stdout:0/569: truncate d1/d9/dd/d18/f31 3285246 0 2024-04-06T03:40:39.600 INFO:tasks.workunit.client.0.smithi094.stdout:6/522: dread dc/f28 [0,4194304] 0 2024-04-06T03:40:39.602 INFO:tasks.workunit.client.1.smithi136.stdout:7/599: creat d5/df/d12/d1b/dbb/fd3 x:0 0 0 2024-04-06T03:40:39.603 INFO:tasks.workunit.client.1.smithi136.stdout:5/606: truncate d1/dc/d1c/d32/d8b/d20/d66/fb9 413029 0 2024-04-06T03:40:39.604 INFO:tasks.workunit.client.1.smithi136.stdout:1/545: mkdir d12/d3b/d44/d5f/d5a/daf 0 2024-04-06T03:40:39.605 INFO:tasks.workunit.client.1.smithi136.stdout:1/546: write d12/d3b/d44/f57 [309219,111047] 0 2024-04-06T03:40:39.606 INFO:tasks.workunit.client.0.smithi094.stdout:0/570: rename d1/d9/dd/d18/d1e/d22/f87 to d1/d9/dd/d18/d5d/d8d/fbf 0 2024-04-06T03:40:39.607 INFO:tasks.workunit.client.0.smithi094.stdout:0/571: write d1/d9/dd/d18/d5d/d60/d6b/dab/f4f [1305970,103039] 0 2024-04-06T03:40:39.608 INFO:tasks.workunit.client.1.smithi136.stdout:6/521: creat d2/d7/d26/d30/d6e/d58/d5d/d62/fb0 x:0 0 0 2024-04-06T03:40:39.608 INFO:tasks.workunit.client.1.smithi136.stdout:6/522: fsync d2/f21 0 2024-04-06T03:40:39.609 INFO:tasks.workunit.client.0.smithi094.stdout:6/523: mkdir dc/d11/d1b/d72/d88/da3/db9 0 2024-04-06T03:40:39.610 INFO:tasks.workunit.client.1.smithi136.stdout:4/513: write d1d/d31/f44 [1500139,25260] 0 2024-04-06T03:40:39.613 INFO:tasks.workunit.client.0.smithi094.stdout:0/572: mkdir d1/d9/dd/d18/d1e/d86/d96/dc0 0 2024-04-06T03:40:39.613 INFO:tasks.workunit.client.0.smithi094.stdout:0/573: dread - d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/fb1 zero size 2024-04-06T03:40:39.613 INFO:tasks.workunit.client.0.smithi094.stdout:0/574: stat d1/d9/dd/d18/dad/ca9 0 2024-04-06T03:40:39.614 INFO:tasks.workunit.client.1.smithi136.stdout:3/539: creat d0/d4/d12/d19/d55/d6f/d82/fb4 x:0 0 0 2024-04-06T03:40:39.616 INFO:tasks.workunit.client.0.smithi094.stdout:6/524: creat dc/d11/d22/d50/daf/fba x:0 0 0 2024-04-06T03:40:39.616 INFO:tasks.workunit.client.0.smithi094.stdout:6/525: chown dc/d3a/d4a/d54 177 1 2024-04-06T03:40:39.616 INFO:tasks.workunit.client.1.smithi136.stdout:8/609: dwrite d3/da/d26/f75 [0,4194304] 0 2024-04-06T03:40:39.616 INFO:tasks.workunit.client.1.smithi136.stdout:8/610: write d3/f5e [1032266,3239] 0 2024-04-06T03:40:39.618 INFO:tasks.workunit.client.1.smithi136.stdout:2/650: symlink d3/d9/d3b/d87/ld0 0 2024-04-06T03:40:39.621 INFO:tasks.workunit.client.0.smithi094.stdout:0/575: link d1/d9/dd/d18/d1e/d22/d35/f5c d1/d9/dd/d18/d5d/d60/fc1 0 2024-04-06T03:40:39.622 INFO:tasks.workunit.client.1.smithi136.stdout:2/651: write d3/f64 [523855,57765] 0 2024-04-06T03:40:39.622 INFO:tasks.workunit.client.1.smithi136.stdout:9/557: mkdir d17/d21/d29/d34/dbd 0 2024-04-06T03:40:39.624 INFO:tasks.workunit.client.1.smithi136.stdout:7/600: getdents d5/df/d12/d59 0 2024-04-06T03:40:39.624 INFO:tasks.workunit.client.1.smithi136.stdout:7/601: creat d5/df/d1d/d6f/d4d/fd4 x:0 0 0 2024-04-06T03:40:39.625 INFO:tasks.workunit.client.1.smithi136.stdout:7/602: stat d5/df/d3d/f3f 0 2024-04-06T03:40:39.627 INFO:tasks.workunit.client.0.smithi094.stdout:1/505: sync 2024-04-06T03:40:39.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/575: sync 2024-04-06T03:40:39.627 INFO:tasks.workunit.client.0.smithi094.stdout:3/576: fsync d9/d1d/f4a 0 2024-04-06T03:40:39.627 INFO:tasks.workunit.client.0.smithi094.stdout:8/590: sync 2024-04-06T03:40:39.627 INFO:tasks.workunit.client.0.smithi094.stdout:5/529: sync 2024-04-06T03:40:39.627 INFO:tasks.workunit.client.0.smithi094.stdout:7/571: sync 2024-04-06T03:40:39.628 INFO:tasks.workunit.client.0.smithi094.stdout:0/576: unlink d1/d9/dd/d18/d5d/d60/d6b/l75 0 2024-04-06T03:40:39.629 INFO:tasks.workunit.client.1.smithi136.stdout:5/607: symlink d1/d9b/db4/lbd 0 2024-04-06T03:40:39.633 INFO:tasks.workunit.client.1.smithi136.stdout:1/547: link d12/d3b/d60/d7e/f42 d12/d1b/d22/fb0 0 2024-04-06T03:40:39.633 INFO:tasks.workunit.client.1.smithi136.stdout:1/548: stat d12/d1b/l87 0 2024-04-06T03:40:39.635 INFO:tasks.workunit.client.0.smithi094.stdout:3/577: mknod d9/d1e/d6e/d75/dba/d3a/ccf 0 2024-04-06T03:40:39.636 INFO:tasks.workunit.client.0.smithi094.stdout:0/577: write d1/d9/dd/d18/d5d/d60/d6b/dab/f73 [4095447,27380] 0 2024-04-06T03:40:39.636 INFO:tasks.workunit.client.0.smithi094.stdout:0/578: chown d1/d9/dd/d18/d1e/d86/d96/daa 4 1 2024-04-06T03:40:39.636 INFO:tasks.workunit.client.0.smithi094.stdout:0/579: write d1/d9/dd/d18/d1e/d22/d35/f5c [1664186,14855] 0 2024-04-06T03:40:39.636 INFO:tasks.workunit.client.0.smithi094.stdout:0/580: write d1/d9/dd/d18/d32/d42/f45 [1643049,42121] 0 2024-04-06T03:40:39.636 INFO:tasks.workunit.client.0.smithi094.stdout:0/581: write d1/d9/dd/d18/d1e/d22/d35/dac/faf [74677,76860] 0 2024-04-06T03:40:39.637 INFO:tasks.workunit.client.1.smithi136.stdout:6/523: mknod d2/d7/d9e/cb1 0 2024-04-06T03:40:39.637 INFO:tasks.workunit.client.1.smithi136.stdout:6/524: chown d2/d7/d26/d30/d6e/d3d/d47/f8c 216338801 1 2024-04-06T03:40:39.637 INFO:tasks.workunit.client.1.smithi136.stdout:6/525: chown d2/d7/d26/d30/d6e/d58/d5d/d62/fb0 226 1 2024-04-06T03:40:39.638 INFO:tasks.workunit.client.1.smithi136.stdout:6/526: write d2/d7/d26/d30/d6e/d58/d5d/d62/d53/fa1 [143490,19070] 0 2024-04-06T03:40:39.638 INFO:tasks.workunit.client.0.smithi094.stdout:8/591: mknod d18/d2c/d37/d60/d70/cd0 0 2024-04-06T03:40:39.639 INFO:tasks.workunit.client.1.smithi136.stdout:4/514: creat d1d/fbb x:0 0 0 2024-04-06T03:40:39.639 INFO:tasks.workunit.client.1.smithi136.stdout:4/515: dread - d1d/d2f/d6c/d91/fb4 zero size 2024-04-06T03:40:39.639 INFO:tasks.workunit.client.1.smithi136.stdout:4/516: stat d1d/d31/f41 0 2024-04-06T03:40:39.640 INFO:tasks.workunit.client.0.smithi094.stdout:5/530: mknod db/d19/d2c/d57/cba 0 2024-04-06T03:40:39.640 INFO:tasks.workunit.client.0.smithi094.stdout:5/531: creat db/d40/d77/fbb x:0 0 0 2024-04-06T03:40:39.642 INFO:tasks.workunit.client.0.smithi094.stdout:7/572: unlink d2/d6/dc/dd/df/d36/d79/f81 0 2024-04-06T03:40:39.649 INFO:tasks.workunit.client.0.smithi094.stdout:1/506: getdents d0/d1 0 2024-04-06T03:40:39.652 INFO:tasks.workunit.client.0.smithi094.stdout:3/578: creat d9/d1e/d6e/d75/dba/d97/fd0 x:0 0 0 2024-04-06T03:40:39.652 INFO:tasks.workunit.client.0.smithi094.stdout:3/579: truncate d9/d1e/d6e/d75/dba/d97/fd0 249850 0 2024-04-06T03:40:39.652 INFO:tasks.workunit.client.0.smithi094.stdout:3/580: chown d9/d1e/d6e/d75/dba/cb1 102625043 1 2024-04-06T03:40:39.658 INFO:tasks.workunit.client.0.smithi094.stdout:6/526: dwrite dc/f40 [0,4194304] 0 2024-04-06T03:40:39.660 INFO:tasks.workunit.client.0.smithi094.stdout:8/592: rename d18/d3c/d49/f7b to d18/d3c/d49/d91/d94/daa/fd1 0 2024-04-06T03:40:39.660 INFO:tasks.workunit.client.0.smithi094.stdout:8/593: fdatasync f1 0 2024-04-06T03:40:39.660 INFO:tasks.workunit.client.1.smithi136.stdout:1/549: write d12/d3b/d6b/f9e [5091545,1291] 0 2024-04-06T03:40:39.666 INFO:tasks.workunit.client.0.smithi094.stdout:1/507: mknod d0/d7/d1a/da8/cac 0 2024-04-06T03:40:39.668 INFO:tasks.workunit.client.0.smithi094.stdout:3/581: symlink d9/d1e/d6e/d75/dba/d7c/ld1 0 2024-04-06T03:40:39.675 INFO:tasks.workunit.client.0.smithi094.stdout:3/582: rename d9/d1e/d26/d55/d62/cae to d9/d1e/d26/d6c/d8e/cd2 0 2024-04-06T03:40:39.676 INFO:tasks.workunit.client.1.smithi136.stdout:2/652: mkdir d3/d4a/d9d/d18/d4b/dc0/dd1 0 2024-04-06T03:40:39.676 INFO:tasks.workunit.client.1.smithi136.stdout:2/653: stat d3/d4a/d9d/d18/l96 0 2024-04-06T03:40:39.677 INFO:tasks.workunit.client.0.smithi094.stdout:3/583: mknod d9/d1e/d6e/d75/cd3 0 2024-04-06T03:40:39.679 INFO:tasks.workunit.client.0.smithi094.stdout:1/508: dread d0/d4/d1b/d2f/d76/d30/d38/f4d [0,4194304] 0 2024-04-06T03:40:39.679 INFO:tasks.workunit.client.0.smithi094.stdout:1/509: fsync d0/f19 0 2024-04-06T03:40:39.679 INFO:tasks.workunit.client.0.smithi094.stdout:1/510: fdatasync d0/d4/d1b/d2f/d4b/f91 0 2024-04-06T03:40:39.679 INFO:tasks.workunit.client.0.smithi094.stdout:1/511: creat d0/d7/fad x:0 0 0 2024-04-06T03:40:39.679 INFO:tasks.workunit.client.1.smithi136.stdout:7/603: mknod d5/df/d11/cd5 0 2024-04-06T03:40:39.680 INFO:tasks.workunit.client.1.smithi136.stdout:5/608: unlink d1/d16/c25 0 2024-04-06T03:40:39.683 INFO:tasks.workunit.client.1.smithi136.stdout:1/550: creat d12/d3b/d60/d6d/da2/fb1 x:0 0 0 2024-04-06T03:40:39.683 INFO:tasks.workunit.client.1.smithi136.stdout:5/609: unlink d1/d16/c54 0 2024-04-06T03:40:39.685 INFO:tasks.workunit.client.1.smithi136.stdout:5/610: creat d1/d9b/d9c/fbe x:0 0 0 2024-04-06T03:40:39.685 INFO:tasks.workunit.client.1.smithi136.stdout:5/611: truncate d1/dc/d10/f94 4525885 0 2024-04-06T03:40:39.694 INFO:tasks.workunit.client.1.smithi136.stdout:8/611: dwrite d3/d2d/f67 [0,4194304] 0 2024-04-06T03:40:39.694 INFO:tasks.workunit.client.1.smithi136.stdout:8/612: stat d3/d2d/c4a 0 2024-04-06T03:40:39.697 INFO:tasks.workunit.client.1.smithi136.stdout:8/613: mkdir d3/da/d26/d35/d3d/d48/d6c/dab/dba 0 2024-04-06T03:40:39.704 INFO:tasks.workunit.client.1.smithi136.stdout:7/604: dread d5/fd [4194304,4194304] 0 2024-04-06T03:40:39.704 INFO:tasks.workunit.client.1.smithi136.stdout:7/605: truncate d5/df/d53/d8a/dcd/fb1 872937 0 2024-04-06T03:40:39.704 INFO:tasks.workunit.client.1.smithi136.stdout:7/606: chown d5/df/d53/d8a/dcd/l85 41 1 2024-04-06T03:40:39.705 INFO:tasks.workunit.client.1.smithi136.stdout:7/607: rmdir d5/df/d12/d1b 39 2024-04-06T03:40:39.705 INFO:tasks.workunit.client.1.smithi136.stdout:7/608: write d5/df/d1d/f1f [3067053,54340] 0 2024-04-06T03:40:39.705 INFO:tasks.workunit.client.1.smithi136.stdout:7/609: chown d5/df/d53/f9d 216 1 2024-04-06T03:40:39.724 INFO:tasks.workunit.client.1.smithi136.stdout:6/527: dwrite d2/d7/d26/d68/f98 [0,4194304] 0 2024-04-06T03:40:39.724 INFO:tasks.workunit.client.1.smithi136.stdout:6/528: chown d2/d7/d26/d30/d6e/d3d/f81 0 1 2024-04-06T03:40:39.726 INFO:tasks.workunit.client.1.smithi136.stdout:9/558: dwrite d17/d19/f1f [0,4194304] 0 2024-04-06T03:40:39.726 INFO:tasks.workunit.client.1.smithi136.stdout:4/517: dwrite d1d/d31/d3e/d56/d7b/f77 [0,4194304] 0 2024-04-06T03:40:39.729 INFO:tasks.workunit.client.1.smithi136.stdout:4/518: rmdir d1d/d31 39 2024-04-06T03:40:39.731 INFO:tasks.workunit.client.1.smithi136.stdout:4/519: creat d1d/d2f/d6c/da5/fbc x:0 0 0 2024-04-06T03:40:39.731 INFO:tasks.workunit.client.1.smithi136.stdout:4/520: creat d1d/d31/d78/fbd x:0 0 0 2024-04-06T03:40:39.732 INFO:tasks.workunit.client.1.smithi136.stdout:4/521: read - d1d/d31/d3e/d55/d7f/fb6 zero size 2024-04-06T03:40:39.732 INFO:tasks.workunit.client.1.smithi136.stdout:4/522: creat d1d/d31/d3e/d55/d89/db3/fbe x:0 0 0 2024-04-06T03:40:39.732 INFO:tasks.workunit.client.1.smithi136.stdout:4/523: fsync d1d/d2f/d6c/f76 0 2024-04-06T03:40:39.735 INFO:tasks.workunit.client.1.smithi136.stdout:9/559: dread d17/d19/f1f [0,4194304] 0 2024-04-06T03:40:39.735 INFO:tasks.workunit.client.1.smithi136.stdout:9/560: truncate d17/d21/fa7 637269 0 2024-04-06T03:40:39.735 INFO:tasks.workunit.client.1.smithi136.stdout:9/561: readlink d17/laf 0 2024-04-06T03:40:39.736 INFO:tasks.workunit.client.1.smithi136.stdout:9/562: creat d17/d21/d32/fbe x:0 0 0 2024-04-06T03:40:39.737 INFO:tasks.workunit.client.1.smithi136.stdout:9/563: chown d17/d20/d28/d49/d75/f65 55 1 2024-04-06T03:40:39.760 INFO:tasks.workunit.client.0.smithi094.stdout:8/594: dwrite d18/d3c/dc4/dbc/f93 [0,4194304] 0 2024-04-06T03:40:39.766 INFO:tasks.workunit.client.0.smithi094.stdout:8/595: dread f13 [0,4194304] 0 2024-04-06T03:40:39.767 INFO:tasks.workunit.client.0.smithi094.stdout:8/596: write d18/d3c/d49/db5/fc2 [1011093,14278] 0 2024-04-06T03:40:39.767 INFO:tasks.workunit.client.0.smithi094.stdout:8/597: fsync d18/d2c/f9b 0 2024-04-06T03:40:39.767 INFO:tasks.workunit.client.0.smithi094.stdout:8/598: mkdir d18/d2c/dd2 0 2024-04-06T03:40:39.768 INFO:tasks.workunit.client.0.smithi094.stdout:8/599: write d18/d3c/d49/d9f/fb3 [414493,26298] 0 2024-04-06T03:40:39.768 INFO:tasks.workunit.client.0.smithi094.stdout:8/600: dread - d18/d3c/d49/d91/d94/daa/fd1 zero size 2024-04-06T03:40:39.768 INFO:tasks.workunit.client.1.smithi136.stdout:4/524: dread d1d/d2f/f60 [0,4194304] 0 2024-04-06T03:40:39.768 INFO:tasks.workunit.client.0.smithi094.stdout:8/601: creat d18/d2c/fd3 x:0 0 0 2024-04-06T03:40:39.769 INFO:tasks.workunit.client.1.smithi136.stdout:4/525: mkdir d1d/d2f/da6/dbf 0 2024-04-06T03:40:39.769 INFO:tasks.workunit.client.1.smithi136.stdout:4/526: chown d1d/d2f/d6c/d91 26 1 2024-04-06T03:40:39.769 INFO:tasks.workunit.client.1.smithi136.stdout:4/527: read - d1d/d70/f74 zero size 2024-04-06T03:40:39.771 INFO:tasks.workunit.client.0.smithi094.stdout:8/602: dread d18/d30/f77 [0,4194304] 0 2024-04-06T03:40:39.772 INFO:tasks.workunit.client.1.smithi136.stdout:8/614: dread d3/de/f56 [0,4194304] 0 2024-04-06T03:40:39.773 INFO:tasks.workunit.client.0.smithi094.stdout:8/603: rename d18/f20 to d18/d2c/d37/d60/da3/db8/dc3/fd4 0 2024-04-06T03:40:39.774 INFO:tasks.workunit.client.0.smithi094.stdout:8/604: unlink d18/d3c/dc4/db9/cc9 0 2024-04-06T03:40:39.775 INFO:tasks.workunit.client.0.smithi094.stdout:8/605: readlink d18/d34/l5f 0 2024-04-06T03:40:39.777 INFO:tasks.workunit.client.0.smithi094.stdout:7/573: dwrite d2/d6/dc/d12/d3a/fa2 [0,4194304] 0 2024-04-06T03:40:39.777 INFO:tasks.workunit.client.0.smithi094.stdout:6/527: dwrite dc/d11/d1b/f31 [0,4194304] 0 2024-04-06T03:40:39.777 INFO:tasks.workunit.client.0.smithi094.stdout:5/532: dwrite db/d19/d24/d2e/fb3 [0,4194304] 0 2024-04-06T03:40:39.777 INFO:tasks.workunit.client.0.smithi094.stdout:8/606: link d18/d2c/d37/d60/da3/f92 d18/d3c/d49/fd5 0 2024-04-06T03:40:39.777 INFO:tasks.workunit.client.0.smithi094.stdout:8/607: readlink d18/l26 0 2024-04-06T03:40:39.777 INFO:tasks.workunit.client.0.smithi094.stdout:8/608: fdatasync d18/d3c/dc4/db9/fc0 0 2024-04-06T03:40:39.777 INFO:tasks.workunit.client.0.smithi094.stdout:8/609: write d18/d3c/dc4/db9/fc0 [1891864,1824] 0 2024-04-06T03:40:39.782 INFO:tasks.workunit.client.0.smithi094.stdout:1/512: dwrite d0/d7/d1a/f9b [0,4194304] 0 2024-04-06T03:40:39.782 INFO:tasks.workunit.client.0.smithi094.stdout:1/513: truncate d0/d4/d1b/d2f/d76/d30/f32 4556188 0 2024-04-06T03:40:39.782 INFO:tasks.workunit.client.1.smithi136.stdout:7/610: dwrite d5/df/d53/d8a/dcd/fb1 [0,4194304] 0 2024-04-06T03:40:39.783 INFO:tasks.workunit.client.1.smithi136.stdout:9/564: dread d17/d20/d28/d49/f54 [0,4194304] 0 2024-04-06T03:40:39.783 INFO:tasks.workunit.client.1.smithi136.stdout:2/654: dwrite d3/d15/f90 [0,4194304] 0 2024-04-06T03:40:39.784 INFO:tasks.workunit.client.1.smithi136.stdout:1/551: dwrite d12/d3b/f84 [0,4194304] 0 2024-04-06T03:40:39.784 INFO:tasks.workunit.client.1.smithi136.stdout:1/552: chown d12/d3b/d6b/f9e 5120615 1 2024-04-06T03:40:39.784 INFO:tasks.workunit.client.1.smithi136.stdout:5/612: dwrite d1/dc/d1c/d32/d8b/f9f [0,4194304] 0 2024-04-06T03:40:39.785 INFO:tasks.workunit.client.1.smithi136.stdout:5/613: truncate d1/dc/d1c/d32/d8b/d20/d66/fb9 623898 0 2024-04-06T03:40:39.785 INFO:tasks.workunit.client.1.smithi136.stdout:5/614: chown d1/l5c 35 1 2024-04-06T03:40:39.785 INFO:tasks.workunit.client.1.smithi136.stdout:5/615: chown d1/dc/d1c/d32/f38 6 1 2024-04-06T03:40:39.785 INFO:tasks.workunit.client.0.smithi094.stdout:5/533: rename db/d19/d2c/d57/c64 to db/d80/cbc 0 2024-04-06T03:40:39.785 INFO:tasks.workunit.client.0.smithi094.stdout:5/534: fsync db/d19/d24/f34 0 2024-04-06T03:40:39.786 INFO:tasks.workunit.client.1.smithi136.stdout:8/615: write d3/da/d26/d35/d3d/f64 [2563588,29452] 0 2024-04-06T03:40:39.787 INFO:tasks.workunit.client.0.smithi094.stdout:7/574: creat d2/d6/dc/dd/df/fa9 x:0 0 0 2024-04-06T03:40:39.789 INFO:tasks.workunit.client.1.smithi136.stdout:6/529: dwrite d2/f3c [0,4194304] 0 2024-04-06T03:40:39.789 INFO:tasks.workunit.client.1.smithi136.stdout:6/530: read d2/d7/d26/d30/f50 [2995147,3331] 0 2024-04-06T03:40:39.789 INFO:tasks.workunit.client.1.smithi136.stdout:6/531: fdatasync d2/d7/d26/d30/d6e/d58/d5d/d62/f64 0 2024-04-06T03:40:39.790 INFO:tasks.workunit.client.0.smithi094.stdout:1/514: read d0/d7/f46 [490639,106221] 0 2024-04-06T03:40:39.791 INFO:tasks.workunit.client.0.smithi094.stdout:8/610: creat d18/d3c/d49/d91/d94/daa/dc8/fd6 x:0 0 0 2024-04-06T03:40:39.793 INFO:tasks.workunit.client.0.smithi094.stdout:5/535: dread db/fa8 [4194304,4194304] 0 2024-04-06T03:40:39.798 INFO:tasks.workunit.client.1.smithi136.stdout:2/655: mknod d3/d9b/cd2 0 2024-04-06T03:40:39.801 INFO:tasks.workunit.client.1.smithi136.stdout:9/565: creat d17/d20/d25/d3b/d9e/fbf x:0 0 0 2024-04-06T03:40:39.810 INFO:tasks.workunit.client.1.smithi136.stdout:5/616: truncate d1/dc/d1c/d32/f38 2044067 0 2024-04-06T03:40:39.810 INFO:tasks.workunit.client.1.smithi136.stdout:5/617: creat d1/dc/d1c/d32/d8b/d20/d66/fbf x:0 0 0 2024-04-06T03:40:39.811 INFO:tasks.workunit.client.0.smithi094.stdout:7/575: unlink d2/d6/dc/c21 0 2024-04-06T03:40:39.812 INFO:tasks.workunit.client.1.smithi136.stdout:0/562: sync 2024-04-06T03:40:39.813 INFO:tasks.workunit.client.1.smithi136.stdout:8/616: symlink d3/de/d60/lbb 0 2024-04-06T03:40:39.814 INFO:tasks.workunit.client.0.smithi094.stdout:5/536: truncate db/d19/d2c/f2d 5625846 0 2024-04-06T03:40:39.814 INFO:tasks.workunit.client.0.smithi094.stdout:5/537: write db/d19/d30/f8e [694193,36888] 0 2024-04-06T03:40:39.816 INFO:tasks.workunit.client.1.smithi136.stdout:6/532: mknod d2/d93/dab/cb2 0 2024-04-06T03:40:39.817 INFO:tasks.workunit.client.0.smithi094.stdout:7/576: rmdir d2/d6/dc/d12/d29/d35/d57 39 2024-04-06T03:40:39.817 INFO:tasks.workunit.client.0.smithi094.stdout:7/577: fdatasync d2/d6/dc/d12/d29/d35/f59 0 2024-04-06T03:40:39.818 INFO:tasks.workunit.client.0.smithi094.stdout:7/578: chown d2/d6/f73 1406633 1 2024-04-06T03:40:39.821 INFO:tasks.workunit.client.1.smithi136.stdout:2/656: mkdir d3/d4a/d9d/d18/d1d/d3c/d58/dd3 0 2024-04-06T03:40:39.821 INFO:tasks.workunit.client.0.smithi094.stdout:1/515: write d0/d7/f36 [1390377,46883] 0 2024-04-06T03:40:39.822 INFO:tasks.workunit.client.1.smithi136.stdout:4/528: dwrite d1d/f4f [0,4194304] 0 2024-04-06T03:40:39.823 INFO:tasks.workunit.client.1.smithi136.stdout:9/566: chown d17/d20/d28/l2d 2064 1 2024-04-06T03:40:39.824 INFO:tasks.workunit.client.0.smithi094.stdout:1/516: truncate d0/d4/f9 5216495 0 2024-04-06T03:40:39.824 INFO:tasks.workunit.client.0.smithi094.stdout:1/517: fdatasync d0/d7/f86 0 2024-04-06T03:40:39.828 INFO:tasks.workunit.client.1.smithi136.stdout:5/618: mkdir d1/d98/dc0 0 2024-04-06T03:40:39.828 INFO:tasks.workunit.client.1.smithi136.stdout:5/619: chown d1/f7a 438075 1 2024-04-06T03:40:39.828 INFO:tasks.workunit.client.1.smithi136.stdout:5/620: fsync d1/d16/d4b/f7e 0 2024-04-06T03:40:39.829 INFO:tasks.workunit.client.0.smithi094.stdout:1/518: unlink d0/d7/c13 0 2024-04-06T03:40:39.829 INFO:tasks.workunit.client.0.smithi094.stdout:1/519: write d0/d1/d27/fa9 [211484,70770] 0 2024-04-06T03:40:39.832 INFO:tasks.workunit.client.0.smithi094.stdout:7/579: write d2/d6/dc/dd/f48 [1422552,67239] 0 2024-04-06T03:40:39.834 INFO:tasks.workunit.client.1.smithi136.stdout:6/533: rename d2/d93/dab/cb2 to d2/d7/d26/d30/d6e/d58/d5d/cb3 0 2024-04-06T03:40:39.834 INFO:tasks.workunit.client.1.smithi136.stdout:2/657: symlink d3/d9/ld4 0 2024-04-06T03:40:39.834 INFO:tasks.workunit.client.0.smithi094.stdout:7/580: mkdir d2/d6/dc/d12/d29/daa 0 2024-04-06T03:40:39.838 INFO:tasks.workunit.client.1.smithi136.stdout:1/553: dwrite d12/d1b/f35 [0,4194304] 0 2024-04-06T03:40:39.838 INFO:tasks.workunit.client.1.smithi136.stdout:7/611: dwrite d5/df/d53/fb9 [0,4194304] 0 2024-04-06T03:40:39.838 INFO:tasks.workunit.client.1.smithi136.stdout:1/554: write d12/d3b/d44/d5f/f7f [318406,38273] 0 2024-04-06T03:40:39.840 INFO:tasks.workunit.client.1.smithi136.stdout:9/567: unlink d17/c85 0 2024-04-06T03:40:39.840 INFO:tasks.workunit.client.1.smithi136.stdout:9/568: stat d17/d21/d32/d43 0 2024-04-06T03:40:39.842 INFO:tasks.workunit.client.0.smithi094.stdout:6/528: dwrite dc/d3a/d4a/d54/f76 [4194304,4194304] 0 2024-04-06T03:40:39.842 INFO:tasks.workunit.client.0.smithi094.stdout:6/529: chown dc/d3a/d4a/ca4 7749 1 2024-04-06T03:40:39.842 INFO:tasks.workunit.client.0.smithi094.stdout:6/530: fsync dc/d11/d1b/f52 0 2024-04-06T03:40:39.842 INFO:tasks.workunit.client.0.smithi094.stdout:1/520: dread d0/d1/d27/da5/f3b [0,4194304] 0 2024-04-06T03:40:39.843 INFO:tasks.workunit.client.0.smithi094.stdout:6/531: dread dc/d11/d1b/d72/f77 [0,4194304] 0 2024-04-06T03:40:39.853 INFO:tasks.workunit.client.1.smithi136.stdout:4/529: rename d1d/d31/d3e/d55/c59 to d1d/cc0 0 2024-04-06T03:40:39.853 INFO:tasks.workunit.client.1.smithi136.stdout:4/530: truncate d1d/d2f/d6c/da5/fbc 979617 0 2024-04-06T03:40:39.854 INFO:tasks.workunit.client.1.smithi136.stdout:2/658: getdents d3/d4a/d9d/d18/d4b 0 2024-04-06T03:40:39.857 INFO:tasks.workunit.client.1.smithi136.stdout:6/534: read d2/f21 [3762345,70731] 0 2024-04-06T03:40:39.857 INFO:tasks.workunit.client.0.smithi094.stdout:1/521: creat d0/d4/d1b/d2f/d76/d7d/fae x:0 0 0 2024-04-06T03:40:39.861 INFO:tasks.workunit.client.0.smithi094.stdout:6/532: link dc/d3a/d4a/d54/d71/l83 dc/d11/d22/d50/daf/lbb 0 2024-04-06T03:40:39.865 INFO:tasks.workunit.client.1.smithi136.stdout:7/612: mkdir d5/df/d12/dd6 0 2024-04-06T03:40:39.866 INFO:tasks.workunit.client.1.smithi136.stdout:9/569: truncate d17/d21/d29/d34/d46/d9a/fbc 247245 0 2024-04-06T03:40:39.867 INFO:tasks.workunit.client.0.smithi094.stdout:5/538: dwrite db/d19/d2c/f73 [4194304,4194304] 0 2024-04-06T03:40:39.868 INFO:tasks.workunit.client.0.smithi094.stdout:5/539: chown db/l20 0 1 2024-04-06T03:40:39.869 INFO:tasks.workunit.client.1.smithi136.stdout:4/531: unlink d1d/d31/d78/f7a 0 2024-04-06T03:40:39.870 INFO:tasks.workunit.client.0.smithi094.stdout:6/533: write dc/d11/d1b/f1f [352312,41452] 0 2024-04-06T03:40:39.872 INFO:tasks.workunit.client.1.smithi136.stdout:7/613: creat d5/df/d12/d59/fd7 x:0 0 0 2024-04-06T03:40:39.873 INFO:tasks.workunit.client.0.smithi094.stdout:5/540: truncate db/f11 7086865 0 2024-04-06T03:40:39.873 INFO:tasks.workunit.client.0.smithi094.stdout:5/541: write db/d19/d24/f37 [4136217,27481] 0 2024-04-06T03:40:39.874 INFO:tasks.workunit.client.1.smithi136.stdout:9/570: mknod d17/d20/d28/d81/d88/cc0 0 2024-04-06T03:40:39.875 INFO:tasks.workunit.client.1.smithi136.stdout:6/535: dread d2/d7/d26/d30/d6e/f5e [0,4194304] 0 2024-04-06T03:40:39.877 INFO:tasks.workunit.client.1.smithi136.stdout:4/532: truncate d1d/d2f/f53 2119390 0 2024-04-06T03:40:39.881 INFO:tasks.workunit.client.1.smithi136.stdout:6/536: rmdir d2/d7 39 2024-04-06T03:40:39.881 INFO:tasks.workunit.client.0.smithi094.stdout:8/611: getdents d18/d3c/d49 0 2024-04-06T03:40:39.882 INFO:tasks.workunit.client.1.smithi136.stdout:7/614: dread d5/df/d53/f39 [0,4194304] 0 2024-04-06T03:40:39.883 INFO:tasks.workunit.client.0.smithi094.stdout:5/542: rmdir db/d40/d77 39 2024-04-06T03:40:39.883 INFO:tasks.workunit.client.0.smithi094.stdout:5/543: creat db/d7c/d98/fbd x:0 0 0 2024-04-06T03:40:39.886 INFO:tasks.workunit.client.1.smithi136.stdout:4/533: unlink l13 0 2024-04-06T03:40:39.887 INFO:tasks.workunit.client.1.smithi136.stdout:6/537: symlink d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/d94/lb4 0 2024-04-06T03:40:39.890 INFO:tasks.workunit.client.1.smithi136.stdout:7/615: creat d5/df/d1d/d54/d5c/dc7/fd8 x:0 0 0 2024-04-06T03:40:39.893 INFO:tasks.workunit.client.1.smithi136.stdout:6/538: creat d2/d7/d26/d30/d6e/d85/fb5 x:0 0 0 2024-04-06T03:40:39.894 INFO:tasks.workunit.client.1.smithi136.stdout:7/616: symlink d5/df/d12/ld9 0 2024-04-06T03:40:39.895 INFO:tasks.workunit.client.1.smithi136.stdout:4/534: unlink d1d/d31/f3a 0 2024-04-06T03:40:39.896 INFO:tasks.workunit.client.1.smithi136.stdout:9/571: dread d17/d20/d28/d49/d75/d60/f7d [0,4194304] 0 2024-04-06T03:40:39.896 INFO:tasks.workunit.client.1.smithi136.stdout:9/572: readlink d17/d19/l33 0 2024-04-06T03:40:39.896 INFO:tasks.workunit.client.1.smithi136.stdout:6/539: symlink d2/d93/lb6 0 2024-04-06T03:40:39.897 INFO:tasks.workunit.client.1.smithi136.stdout:6/540: fsync d2/f21 0 2024-04-06T03:40:39.903 INFO:tasks.workunit.client.1.smithi136.stdout:7/617: getdents d5/df/d53/d68 0 2024-04-06T03:40:39.903 INFO:tasks.workunit.client.1.smithi136.stdout:7/618: truncate d5/df/d1d/d6f/d4d/d69/fc8 548452 0 2024-04-06T03:40:39.903 INFO:tasks.workunit.client.1.smithi136.stdout:4/535: rmdir d1d/d2f/d6c/da5 39 2024-04-06T03:40:39.903 INFO:tasks.workunit.client.1.smithi136.stdout:6/541: getdents d2/d7/d26/d30/d6e/d58/d5d/d62/d53 0 2024-04-06T03:40:39.903 INFO:tasks.workunit.client.1.smithi136.stdout:6/542: write d2/d7/d26/faa [305799,66401] 0 2024-04-06T03:40:39.903 INFO:tasks.workunit.client.0.smithi094.stdout:9/556: sync 2024-04-06T03:40:39.904 INFO:tasks.workunit.client.0.smithi094.stdout:2/616: sync 2024-04-06T03:40:39.904 INFO:tasks.workunit.client.0.smithi094.stdout:0/582: sync 2024-04-06T03:40:39.904 INFO:tasks.workunit.client.0.smithi094.stdout:4/621: sync 2024-04-06T03:40:39.904 INFO:tasks.workunit.client.0.smithi094.stdout:0/583: dread d1/d9/dd/d18/d5d/d60/d6b/f81 [0,4194304] 0 2024-04-06T03:40:39.905 INFO:tasks.workunit.client.1.smithi136.stdout:6/543: mkdir d2/d7/d26/d30/d6e/db7 0 2024-04-06T03:40:39.905 INFO:tasks.workunit.client.1.smithi136.stdout:9/573: read d17/d20/f48 [2365838,57878] 0 2024-04-06T03:40:39.908 INFO:tasks.workunit.client.0.smithi094.stdout:0/584: rename d1/d9/dd/d18/d1a/da5 to d1/d9/dd/d18/d1a/d44/dc2 0 2024-04-06T03:40:39.908 INFO:tasks.workunit.client.0.smithi094.stdout:0/585: creat d1/d9/dd/d18/d1d/fc3 x:0 0 0 2024-04-06T03:40:39.910 INFO:tasks.workunit.client.0.smithi094.stdout:3/584: dwrite d9/d20/d30/f69 [0,4194304] 0 2024-04-06T03:40:39.911 INFO:tasks.workunit.client.1.smithi136.stdout:6/544: link d2/d7/d26/d30/d6e/d3d/l3e d2/d7/d26/d30/d4f/d75/lb8 0 2024-04-06T03:40:39.912 INFO:tasks.workunit.client.0.smithi094.stdout:2/617: write d0/d8/d1d/f73 [2797825,39422] 0 2024-04-06T03:40:39.912 INFO:tasks.workunit.client.0.smithi094.stdout:2/618: write d0/d8/d1d/d77/fcd [928985,76023] 0 2024-04-06T03:40:39.913 INFO:tasks.workunit.client.1.smithi136.stdout:9/574: unlink d17/l4a 0 2024-04-06T03:40:39.913 INFO:tasks.workunit.client.1.smithi136.stdout:4/536: read d1d/d70/f6a [1730644,19090] 0 2024-04-06T03:40:39.914 INFO:tasks.workunit.client.0.smithi094.stdout:4/622: dread d0/d15/f40 [0,4194304] 0 2024-04-06T03:40:39.916 INFO:tasks.workunit.client.0.smithi094.stdout:0/586: mknod d1/d9/dd/d18/d5d/d60/cc4 0 2024-04-06T03:40:39.921 INFO:tasks.workunit.client.0.smithi094.stdout:3/585: truncate d9/d1e/d26/d55/f65 1348492 0 2024-04-06T03:40:39.925 INFO:tasks.workunit.client.1.smithi136.stdout:9/575: dread d17/d20/d25/d3c/d55/f64 [0,4194304] 0 2024-04-06T03:40:39.925 INFO:tasks.workunit.client.1.smithi136.stdout:9/576: chown d17/d21/d29/c86 0 1 2024-04-06T03:40:39.925 INFO:tasks.workunit.client.0.smithi094.stdout:4/623: symlink d0/d15/d3a/d3e/d50/ld2 0 2024-04-06T03:40:39.927 INFO:tasks.workunit.client.0.smithi094.stdout:0/587: link d1/d9/dd/d18/d5d/d60/d6b/dab/f2e d1/d9/dd/d18/d1e/d86/fc5 0 2024-04-06T03:40:39.927 INFO:tasks.workunit.client.0.smithi094.stdout:4/624: dread d0/d23/f32 [0,4194304] 0 2024-04-06T03:40:39.928 INFO:tasks.workunit.client.1.smithi136.stdout:9/577: mknod d17/d20/d28/d49/d75/cc1 0 2024-04-06T03:40:39.928 INFO:tasks.workunit.client.1.smithi136.stdout:9/578: write d17/d20/d25/d3b/d9e/fbf [769766,45041] 0 2024-04-06T03:40:39.929 INFO:tasks.workunit.client.0.smithi094.stdout:3/586: mkdir d9/d10/dd4 0 2024-04-06T03:40:39.929 INFO:tasks.workunit.client.0.smithi094.stdout:3/587: creat d9/fd5 x:0 0 0 2024-04-06T03:40:39.929 INFO:tasks.workunit.client.0.smithi094.stdout:3/588: write d9/d1d/d7d/fb0 [415081,119404] 0 2024-04-06T03:40:39.931 INFO:tasks.workunit.client.1.smithi136.stdout:1/555: dwrite d12/d1b/d22/f43 [0,4194304] 0 2024-04-06T03:40:39.931 INFO:tasks.workunit.client.1.smithi136.stdout:1/556: truncate d12/d3b/d44/fa5 851504 0 2024-04-06T03:40:39.932 INFO:tasks.workunit.client.0.smithi094.stdout:0/588: mknod d1/d9/dd/d18/d5d/d60/d6b/d67/d76/cc6 0 2024-04-06T03:40:39.936 INFO:tasks.workunit.client.0.smithi094.stdout:4/625: creat d0/d15/d3a/d3e/d49/db2/dc7/fd3 x:0 0 0 2024-04-06T03:40:39.938 INFO:tasks.workunit.client.1.smithi136.stdout:1/557: mknod d12/d1b/d1e/d4d/cb2 0 2024-04-06T03:40:39.941 INFO:tasks.workunit.client.0.smithi094.stdout:3/589: mknod d9/d1e/d6e/d75/d90/cd6 0 2024-04-06T03:40:39.943 INFO:tasks.workunit.client.0.smithi094.stdout:4/626: creat d0/df/d91/fd4 x:0 0 0 2024-04-06T03:40:39.944 INFO:tasks.workunit.client.0.smithi094.stdout:3/590: mkdir d9/d1d/d7d/dd7 0 2024-04-06T03:40:39.944 INFO:tasks.workunit.client.1.smithi136.stdout:9/579: read d17/d20/d28/f89 [2004518,52845] 0 2024-04-06T03:40:39.945 INFO:tasks.workunit.client.1.smithi136.stdout:1/558: symlink d12/d3b/d60/d7e/lb3 0 2024-04-06T03:40:39.946 INFO:tasks.workunit.client.0.smithi094.stdout:9/557: read d2/dc/d21/d26/d78/d69/da8/d96/fa0 [4028623,129596] 0 2024-04-06T03:40:39.947 INFO:tasks.workunit.client.1.smithi136.stdout:9/580: symlink d17/d21/d32/d43/lc2 0 2024-04-06T03:40:39.947 INFO:tasks.workunit.client.1.smithi136.stdout:9/581: chown d17/d20/d25/d3c/d55/d8a 4 1 2024-04-06T03:40:39.947 INFO:tasks.workunit.client.1.smithi136.stdout:9/582: creat d17/d21/d32/fc3 x:0 0 0 2024-04-06T03:40:39.947 INFO:tasks.workunit.client.1.smithi136.stdout:9/583: chown d17/d20/d25/d3d/l5e 21168 1 2024-04-06T03:40:39.948 INFO:tasks.workunit.client.0.smithi094.stdout:3/591: getdents d9/d1e/d26 0 2024-04-06T03:40:39.948 INFO:tasks.workunit.client.0.smithi094.stdout:4/627: rmdir d0/d23/dc1 39 2024-04-06T03:40:39.950 INFO:tasks.workunit.client.0.smithi094.stdout:9/558: rename d2/dc/l28 to d2/dc/d21/ld1 0 2024-04-06T03:40:39.951 INFO:tasks.workunit.client.1.smithi136.stdout:9/584: creat d17/d20/d25/da4/dbb/fc4 x:0 0 0 2024-04-06T03:40:39.951 INFO:tasks.workunit.client.1.smithi136.stdout:9/585: write d17/d21/d32/f74 [849695,30382] 0 2024-04-06T03:40:39.951 INFO:tasks.workunit.client.1.smithi136.stdout:9/586: dread - d17/d21/d32/fc3 zero size 2024-04-06T03:40:39.951 INFO:tasks.workunit.client.1.smithi136.stdout:9/587: fdatasync d17/d20/f95 0 2024-04-06T03:40:39.952 INFO:tasks.workunit.client.0.smithi094.stdout:6/534: dwrite dc/d3a/d4a/d54/f58 [0,4194304] 0 2024-04-06T03:40:39.952 INFO:tasks.workunit.client.0.smithi094.stdout:6/535: write dc/d11/d1b/d72/d87/f8a [878340,60811] 0 2024-04-06T03:40:39.952 INFO:tasks.workunit.client.0.smithi094.stdout:4/628: mknod d0/d15/d3a/d61/d81/da0/cd5 0 2024-04-06T03:40:39.954 INFO:tasks.workunit.client.1.smithi136.stdout:9/588: unlink f11 0 2024-04-06T03:40:39.954 INFO:tasks.workunit.client.1.smithi136.stdout:0/563: dwrite d1/d6/d2c/d1a/f4b [4194304,4194304] 0 2024-04-06T03:40:39.954 INFO:tasks.workunit.client.1.smithi136.stdout:0/564: chown d1/d6/d2c/d57/d43/d6d/d85/la5 27856064 1 2024-04-06T03:40:39.955 INFO:tasks.workunit.client.0.smithi094.stdout:3/592: dread d9/d1e/f9f [0,4194304] 0 2024-04-06T03:40:39.956 INFO:tasks.workunit.client.0.smithi094.stdout:0/589: dwrite d1/d9/dd/d18/d32/d42/f4a [0,4194304] 0 2024-04-06T03:40:39.956 INFO:tasks.workunit.client.0.smithi094.stdout:0/590: truncate d1/d9/dd/d18/d5d/d60/d6b/f81 731961 0 2024-04-06T03:40:39.956 INFO:tasks.workunit.client.0.smithi094.stdout:1/522: dwrite d0/d4/d1b/d2f/d4b/fa1 [0,4194304] 0 2024-04-06T03:40:39.956 INFO:tasks.workunit.client.0.smithi094.stdout:1/523: chown d0/d4/c75 2677 1 2024-04-06T03:40:39.956 INFO:tasks.workunit.client.1.smithi136.stdout:8/617: dwrite d3/d2d/f67 [0,4194304] 0 2024-04-06T03:40:39.957 INFO:tasks.workunit.client.0.smithi094.stdout:6/536: link dc/d11/d1b/d24/d46/f48 dc/d11/d1b/d72/d87/fbc 0 2024-04-06T03:40:39.959 INFO:tasks.workunit.client.1.smithi136.stdout:9/589: mkdir d17/d19/da3/dc5 0 2024-04-06T03:40:39.962 INFO:tasks.workunit.client.0.smithi094.stdout:9/559: getdents d2/dc/d21/d26/d78/d69/d6d 0 2024-04-06T03:40:39.962 INFO:tasks.workunit.client.0.smithi094.stdout:9/560: dread - d2/dc/d21/fb5 zero size 2024-04-06T03:40:39.962 INFO:tasks.workunit.client.0.smithi094.stdout:9/561: chown d2/f1d 137 1 2024-04-06T03:40:39.962 INFO:tasks.workunit.client.0.smithi094.stdout:9/562: fsync d2/dc/d46/f65 0 2024-04-06T03:40:39.962 INFO:tasks.workunit.client.0.smithi094.stdout:9/563: dread - d2/dc/d21/d23/d5b/f89 zero size 2024-04-06T03:40:39.963 INFO:tasks.workunit.client.0.smithi094.stdout:9/564: fsync d2/f14 0 2024-04-06T03:40:39.963 INFO:tasks.workunit.client.0.smithi094.stdout:9/565: write d2/dc/d21/d23/db0/fce [757705,75501] 0 2024-04-06T03:40:39.965 INFO:tasks.workunit.client.0.smithi094.stdout:9/566: dread d2/dc/d21/d23/f7c [0,4194304] 0 2024-04-06T03:40:39.965 INFO:tasks.workunit.client.0.smithi094.stdout:9/567: read d2/dc/d21/d23/d2f/f5f [2550023,115372] 0 2024-04-06T03:40:39.968 INFO:tasks.workunit.client.0.smithi094.stdout:0/591: creat d1/d9/dd/d18/d1e/d86/d96/daa/fc7 x:0 0 0 2024-04-06T03:40:39.968 INFO:tasks.workunit.client.0.smithi094.stdout:0/592: write d1/d9/dd/fb3 [1753550,60999] 0 2024-04-06T03:40:39.968 INFO:tasks.workunit.client.0.smithi094.stdout:0/593: dread - d1/d9/dd/d18/d32/d42/f77 zero size 2024-04-06T03:40:39.968 INFO:tasks.workunit.client.0.smithi094.stdout:0/594: write d1/d9/dd/d18/d1e/fbb [532399,79665] 0 2024-04-06T03:40:39.970 INFO:tasks.workunit.client.1.smithi136.stdout:3/540: sync 2024-04-06T03:40:39.972 INFO:tasks.workunit.client.1.smithi136.stdout:8/618: symlink d3/da/d26/d35/d39/lbc 0 2024-04-06T03:40:39.975 INFO:tasks.workunit.client.1.smithi136.stdout:9/590: mkdir d17/d21/d32/dc6 0 2024-04-06T03:40:39.977 INFO:tasks.workunit.client.0.smithi094.stdout:1/524: mkdir d0/d4/d1b/d2f/d76/daf 0 2024-04-06T03:40:39.977 INFO:tasks.workunit.client.0.smithi094.stdout:1/525: fsync d0/d1/d27/da5/d40/f7a 0 2024-04-06T03:40:39.978 INFO:tasks.workunit.client.1.smithi136.stdout:0/565: getdents d1/d6 0 2024-04-06T03:40:39.979 INFO:tasks.workunit.client.1.smithi136.stdout:0/566: write d1/d6/d2c/d1a/f81 [412537,77048] 0 2024-04-06T03:40:39.979 INFO:tasks.workunit.client.1.smithi136.stdout:0/567: chown d1/d6/d2c/d1a/d36/d59/f21 102530689 1 2024-04-06T03:40:39.979 INFO:tasks.workunit.client.1.smithi136.stdout:0/568: creat d1/d6/d2c/d57/d43/d6d/d85/d86/fbc x:0 0 0 2024-04-06T03:40:39.980 INFO:tasks.workunit.client.0.smithi094.stdout:9/568: unlink d2/dc/d21/d23/d2f/l82 0 2024-04-06T03:40:39.982 INFO:tasks.workunit.client.1.smithi136.stdout:8/619: creat d3/de/d18/d6d/fbd x:0 0 0 2024-04-06T03:40:39.982 INFO:tasks.workunit.client.1.smithi136.stdout:8/620: chown d3/de/d18/l91 33162431 1 2024-04-06T03:40:39.985 INFO:tasks.workunit.client.1.smithi136.stdout:2/659: dwrite d3/d4a/d9d/d18/d1d/d3c/fa0 [0,4194304] 0 2024-04-06T03:40:39.985 INFO:tasks.workunit.client.1.smithi136.stdout:9/591: rename d17/d21/d29/f2a to d17/d21/d32/fc7 0 2024-04-06T03:40:39.985 INFO:tasks.workunit.client.1.smithi136.stdout:9/592: write d17/d20/d28/d81/f83 [2264577,109999] 0 2024-04-06T03:40:39.985 INFO:tasks.workunit.client.1.smithi136.stdout:9/593: creat d17/d19/fc8 x:0 0 0 2024-04-06T03:40:39.985 INFO:tasks.workunit.client.0.smithi094.stdout:1/526: dread d0/d1/d27/da5/f3a [0,4194304] 0 2024-04-06T03:40:39.986 INFO:tasks.workunit.client.0.smithi094.stdout:9/569: mknod d2/dc/d21/d26/d78/d69/cd2 0 2024-04-06T03:40:39.989 INFO:tasks.workunit.client.0.smithi094.stdout:1/527: getdents d0/d7/d1a/da8 0 2024-04-06T03:40:39.991 INFO:tasks.workunit.client.1.smithi136.stdout:0/569: link d1/d6/d2c/d1a/f81 d1/d6/d2c/d57/d43/d6d/d85/da7/fbd 0 2024-04-06T03:40:39.991 INFO:tasks.workunit.client.1.smithi136.stdout:0/570: chown d1/d6/f45 2474107 1 2024-04-06T03:40:39.993 INFO:tasks.workunit.client.0.smithi094.stdout:3/593: write d9/d1e/d26/f2c [2844558,14457] 0 2024-04-06T03:40:39.994 INFO:tasks.workunit.client.0.smithi094.stdout:3/594: fsync d9/f5f 0 2024-04-06T03:40:39.994 INFO:tasks.workunit.client.0.smithi094.stdout:1/528: mkdir d0/d1/d27/da5/d40/db0 0 2024-04-06T03:40:39.995 INFO:tasks.workunit.client.0.smithi094.stdout:1/529: dread d0/d1/d27/f2e [0,4194304] 0 2024-04-06T03:40:39.996 INFO:tasks.workunit.client.0.smithi094.stdout:9/570: getdents d2/dc/d80 0 2024-04-06T03:40:40.001 INFO:tasks.workunit.client.0.smithi094.stdout:3/595: link d9/d1e/d26/f28 d9/d10/fd8 0 2024-04-06T03:40:40.004 INFO:tasks.workunit.client.0.smithi094.stdout:8/612: dwrite d18/f21 [0,4194304] 0 2024-04-06T03:40:40.004 INFO:tasks.workunit.client.0.smithi094.stdout:5/544: dwrite db/f1e [0,4194304] 0 2024-04-06T03:40:40.005 INFO:tasks.workunit.client.0.smithi094.stdout:5/545: stat db/d40/d77/f82 0 2024-04-06T03:40:40.005 INFO:tasks.workunit.client.0.smithi094.stdout:5/546: chown db/d19/d2c/d57/d8f/f90 288208 1 2024-04-06T03:40:40.005 INFO:tasks.workunit.client.0.smithi094.stdout:5/547: chown db/d40/d77/f6a 19189184 1 2024-04-06T03:40:40.005 INFO:tasks.workunit.client.0.smithi094.stdout:5/548: readlink db/d19/d30/l4f 0 2024-04-06T03:40:40.005 INFO:tasks.workunit.client.1.smithi136.stdout:6/545: dwrite d2/d7/d26/d30/f7e [0,4194304] 0 2024-04-06T03:40:40.005 INFO:tasks.workunit.client.1.smithi136.stdout:1/559: dwrite d12/d1b/d28/f3f [0,4194304] 0 2024-04-06T03:40:40.006 INFO:tasks.workunit.client.1.smithi136.stdout:1/560: dread d12/d1b/d1e/f99 [0,4194304] 0 2024-04-06T03:40:40.007 INFO:tasks.workunit.client.0.smithi094.stdout:3/596: stat d9/d1e/d6e/d75/lab 0 2024-04-06T03:40:40.008 INFO:tasks.workunit.client.0.smithi094.stdout:3/597: truncate d9/d1e/d6e/fc8 190648 0 2024-04-06T03:40:40.008 INFO:tasks.workunit.client.0.smithi094.stdout:5/549: write db/d19/d24/d2e/f75 [507123,76191] 0 2024-04-06T03:40:40.008 INFO:tasks.workunit.client.0.smithi094.stdout:2/619: dwrite d0/d8/d1d/fc0 [0,4194304] 0 2024-04-06T03:40:40.008 INFO:tasks.workunit.client.0.smithi094.stdout:5/550: chown db/d7c/d8b/c99 10423 1 2024-04-06T03:40:40.008 INFO:tasks.workunit.client.0.smithi094.stdout:2/620: creat d0/d8/d1d/d77/fd4 x:0 0 0 2024-04-06T03:40:40.009 INFO:tasks.workunit.client.0.smithi094.stdout:5/551: dread db/d40/f43 [0,4194304] 0 2024-04-06T03:40:40.009 INFO:tasks.workunit.client.1.smithi136.stdout:7/619: dwrite d5/df/d12/d1b/f30 [0,4194304] 0 2024-04-06T03:40:40.009 INFO:tasks.workunit.client.1.smithi136.stdout:7/620: write d5/df/d1d/d6f/d2a/f48 [970107,114243] 0 2024-04-06T03:40:40.009 INFO:tasks.workunit.client.1.smithi136.stdout:4/537: dwrite d1d/d31/f4c [0,4194304] 0 2024-04-06T03:40:40.017 INFO:tasks.workunit.client.0.smithi094.stdout:4/629: dwrite d0/d15/d3a/d3e/d46/f95 [0,4194304] 0 2024-04-06T03:40:40.017 INFO:tasks.workunit.client.1.smithi136.stdout:2/660: rmdir d3/d15/d19/d4e/dc9 0 2024-04-06T03:40:40.017 INFO:tasks.workunit.client.0.smithi094.stdout:6/537: dwrite dc/d11/d1b/f2f [0,4194304] 0 2024-04-06T03:40:40.023 INFO:tasks.workunit.client.1.smithi136.stdout:9/594: rmdir d17/d20/d25/d3d 39 2024-04-06T03:40:40.024 INFO:tasks.workunit.client.0.smithi094.stdout:8/613: mknod d18/d3c/d49/d91/d94/da8/cd7 0 2024-04-06T03:40:40.024 INFO:tasks.workunit.client.1.smithi136.stdout:3/541: dwrite d0/d4/d12/d19/d55/d3c/f58 [0,4194304] 0 2024-04-06T03:40:40.034 INFO:tasks.workunit.client.1.smithi136.stdout:6/546: creat d2/d7/d13/d89/fb9 x:0 0 0 2024-04-06T03:40:40.035 INFO:tasks.workunit.client.1.smithi136.stdout:6/547: truncate d2/d7/d26/d30/d6e/d85/fb5 976754 0 2024-04-06T03:40:40.035 INFO:tasks.workunit.client.1.smithi136.stdout:6/548: write d2/d7/d26/d30/d6e/d58/d5d/d62/d53/fa1 [1436121,87396] 0 2024-04-06T03:40:40.035 INFO:tasks.workunit.client.0.smithi094.stdout:9/571: dwrite d2/dc/d21/d23/d2f/f91 [0,4194304] 0 2024-04-06T03:40:40.035 INFO:tasks.workunit.client.0.smithi094.stdout:9/572: write d2/dc/d21/d23/db0/fbf [745094,13826] 0 2024-04-06T03:40:40.038 INFO:tasks.workunit.client.1.smithi136.stdout:1/561: link fb d12/d3b/d74/fb4 0 2024-04-06T03:40:40.041 INFO:tasks.workunit.client.0.smithi094.stdout:3/598: rename d9/d1e/d26/d55/l82 to d9/d1e/d26/d55/ld9 0 2024-04-06T03:40:40.044 INFO:tasks.workunit.client.1.smithi136.stdout:7/621: rename d5/df/fb7 to d5/df/d1d/d6f/d2a/fda 0 2024-04-06T03:40:40.045 INFO:tasks.workunit.client.1.smithi136.stdout:7/622: getdents d5/df/d12/d1b/dbb/dbe 0 2024-04-06T03:40:40.045 INFO:tasks.workunit.client.1.smithi136.stdout:7/623: fdatasync d5/df/d1d/d54/d5c/dc7/fd8 0 2024-04-06T03:40:40.045 INFO:tasks.workunit.client.1.smithi136.stdout:7/624: write d5/df/d1d/d6f/d2a/f48 [2019547,28705] 0 2024-04-06T03:40:40.045 INFO:tasks.workunit.client.0.smithi094.stdout:2/621: truncate d0/dd/d3d/f88 351764 0 2024-04-06T03:40:40.046 INFO:tasks.workunit.client.0.smithi094.stdout:5/552: getdents db/d19/d21 0 2024-04-06T03:40:40.048 INFO:tasks.workunit.client.1.smithi136.stdout:4/538: mkdir d1d/d2f/d6c/da5/dc1 0 2024-04-06T03:40:40.050 INFO:tasks.workunit.client.1.smithi136.stdout:2/661: creat d3/d15/fd5 x:0 0 0 2024-04-06T03:40:40.051 INFO:tasks.workunit.client.1.smithi136.stdout:9/595: creat d17/d21/fc9 x:0 0 0 2024-04-06T03:40:40.051 INFO:tasks.workunit.client.1.smithi136.stdout:9/596: write d17/d21/fa7 [1130848,100044] 0 2024-04-06T03:40:40.054 INFO:tasks.workunit.client.0.smithi094.stdout:6/538: mknod dc/d11/d22/d50/cbd 0 2024-04-06T03:40:40.056 INFO:tasks.workunit.client.1.smithi136.stdout:3/542: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/fb5 x:0 0 0 2024-04-06T03:40:40.056 INFO:tasks.workunit.client.1.smithi136.stdout:3/543: read d0/d4/d12/d2d/f39 [94618,102029] 0 2024-04-06T03:40:40.058 INFO:tasks.workunit.client.1.smithi136.stdout:3/544: read d0/d4/d12/d2d/f33 [387929,78832] 0 2024-04-06T03:40:40.058 INFO:tasks.workunit.client.1.smithi136.stdout:3/545: dread - d0/d4/f76 zero size 2024-04-06T03:40:40.064 INFO:tasks.workunit.client.1.smithi136.stdout:6/549: symlink d2/d7/d26/d30/d6e/d3d/d47/d84/lba 0 2024-04-06T03:40:40.069 INFO:tasks.workunit.client.1.smithi136.stdout:7/625: rename d5/df/d53/d8a/dcd to d5/df/d12/d1b/d3a/d4a/d70/ddb 0 2024-04-06T03:40:40.071 INFO:tasks.workunit.client.1.smithi136.stdout:4/539: symlink d1d/d31/d3e/d56/lc2 0 2024-04-06T03:40:40.073 INFO:tasks.workunit.client.1.smithi136.stdout:2/662: mknod d3/d4a/d9d/d18/d1d/d3c/d58/cd6 0 2024-04-06T03:40:40.073 INFO:tasks.workunit.client.0.smithi094.stdout:6/539: write dc/d11/d1b/d24/d46/f4d [3457084,9477] 0 2024-04-06T03:40:40.074 INFO:tasks.workunit.client.1.smithi136.stdout:6/550: dread d2/fa [0,4194304] 0 2024-04-06T03:40:40.077 INFO:tasks.workunit.client.1.smithi136.stdout:1/562: mknod d12/d3b/d60/d7e/cb5 0 2024-04-06T03:40:40.079 INFO:tasks.workunit.client.1.smithi136.stdout:7/626: truncate d5/df/d12/d1b/d3a/f6c 584595 0 2024-04-06T03:40:40.080 INFO:tasks.workunit.client.1.smithi136.stdout:7/627: truncate d5/df/d1d/d6f/d2a/f5d 565637 0 2024-04-06T03:40:40.080 INFO:tasks.workunit.client.1.smithi136.stdout:7/628: write d5/df/d3d/f97 [303000,43959] 0 2024-04-06T03:40:40.083 INFO:tasks.workunit.client.1.smithi136.stdout:2/663: link d3/d15/d19/d43/d54/f8c d3/d9/d3b/fd7 0 2024-04-06T03:40:40.083 INFO:tasks.workunit.client.1.smithi136.stdout:2/664: stat d3 0 2024-04-06T03:40:40.086 INFO:tasks.workunit.client.1.smithi136.stdout:6/551: creat d2/d7/d26/d30/d6e/d3d/d47/d84/dad/fbb x:0 0 0 2024-04-06T03:40:40.086 INFO:tasks.workunit.client.1.smithi136.stdout:6/552: write d2/d7/d26/d30/d6e/fa5 [1037444,37404] 0 2024-04-06T03:40:40.086 INFO:tasks.workunit.client.1.smithi136.stdout:6/553: chown d2/d7/d26/d30/l2b 4772 1 2024-04-06T03:40:40.089 INFO:tasks.workunit.client.0.smithi094.stdout:4/630: dwrite d0/d15/d3a/d3e/d50/f5e [0,4194304] 0 2024-04-06T03:40:40.089 INFO:tasks.workunit.client.1.smithi136.stdout:7/629: creat d5/df/d1d/d54/d5c/fdc x:0 0 0 2024-04-06T03:40:40.089 INFO:tasks.workunit.client.1.smithi136.stdout:7/630: fsync d5/df/d12/d1b/d58/f7a 0 2024-04-06T03:40:40.090 INFO:tasks.workunit.client.0.smithi094.stdout:8/614: dwrite d18/f1b [0,4194304] 0 2024-04-06T03:40:40.090 INFO:tasks.workunit.client.0.smithi094.stdout:8/615: fsync d18/d3c/dc4/fbf 0 2024-04-06T03:40:40.090 INFO:tasks.workunit.client.0.smithi094.stdout:8/616: chown d18/d2c/d37/d60/da3/f97 304 1 2024-04-06T03:40:40.090 INFO:tasks.workunit.client.1.smithi136.stdout:0/571: dwrite d1/d6/d2c/d1a/f81 [0,4194304] 0 2024-04-06T03:40:40.090 INFO:tasks.workunit.client.1.smithi136.stdout:0/572: creat d1/d6/d2c/d1a/d36/d59/fbe x:0 0 0 2024-04-06T03:40:40.091 INFO:tasks.workunit.client.1.smithi136.stdout:0/573: fdatasync d1/d6/d2c/d1a/d36/d59/f48 0 2024-04-06T03:40:40.091 INFO:tasks.workunit.client.0.smithi094.stdout:9/573: creat d2/dc/d21/d23/fd3 x:0 0 0 2024-04-06T03:40:40.094 INFO:tasks.workunit.client.1.smithi136.stdout:6/554: unlink d2/l39 0 2024-04-06T03:40:40.095 INFO:tasks.workunit.client.1.smithi136.stdout:9/597: dwrite d17/d20/d28/d49/d75/f65 [0,4194304] 0 2024-04-06T03:40:40.097 INFO:tasks.workunit.client.0.smithi094.stdout:3/599: symlink d9/d10/d9b/lda 0 2024-04-06T03:40:40.098 INFO:tasks.workunit.client.0.smithi094.stdout:3/600: write d9/d1e/d6e/fc8 [118718,14150] 0 2024-04-06T03:40:40.098 INFO:tasks.workunit.client.0.smithi094.stdout:3/601: stat d9/d1d 0 2024-04-06T03:40:40.099 INFO:tasks.workunit.client.1.smithi136.stdout:7/631: dread d5/df/d12/d1b/f2d [0,4194304] 0 2024-04-06T03:40:40.100 INFO:tasks.workunit.client.1.smithi136.stdout:3/546: dwrite d0/d4/d12/f1a [0,4194304] 0 2024-04-06T03:40:40.101 INFO:tasks.workunit.client.1.smithi136.stdout:6/555: creat d2/d93/fbc x:0 0 0 2024-04-06T03:40:40.102 INFO:tasks.workunit.client.1.smithi136.stdout:4/540: dwrite f1b [0,4194304] 0 2024-04-06T03:40:40.106 INFO:tasks.workunit.client.0.smithi094.stdout:6/540: creat dc/d11/d1b/d24/fbe x:0 0 0 2024-04-06T03:40:40.109 INFO:tasks.workunit.client.1.smithi136.stdout:1/563: dwrite d12/d1b/f9d [0,4194304] 0 2024-04-06T03:40:40.110 INFO:tasks.workunit.client.1.smithi136.stdout:1/564: chown d12/d1b/d28/f91 29 1 2024-04-06T03:40:40.110 INFO:tasks.workunit.client.1.smithi136.stdout:2/665: dwrite d3/d15/fd5 [0,4194304] 0 2024-04-06T03:40:40.110 INFO:tasks.workunit.client.1.smithi136.stdout:2/666: chown d3/d9/c5a 1359 1 2024-04-06T03:40:40.110 INFO:tasks.workunit.client.1.smithi136.stdout:2/667: readlink d3/d4a/d77/l9f 0 2024-04-06T03:40:40.110 INFO:tasks.workunit.client.0.smithi094.stdout:1/530: rename d0/d4/d1b to d0/d1/d27/da5/d40/db1 0 2024-04-06T03:40:40.114 INFO:tasks.workunit.client.1.smithi136.stdout:1/565: dread d12/d3b/f4c [0,4194304] 0 2024-04-06T03:40:40.116 INFO:tasks.workunit.client.1.smithi136.stdout:0/574: dwrite d1/d6/d2c/d1a/d36/fb1 [0,4194304] 0 2024-04-06T03:40:40.117 INFO:tasks.workunit.client.0.smithi094.stdout:4/631: rmdir d0/df/d91/dbb 39 2024-04-06T03:40:40.119 INFO:tasks.workunit.client.0.smithi094.stdout:4/632: write d0/d15/d3a/f43 [412733,115009] 0 2024-04-06T03:40:40.120 INFO:tasks.workunit.client.0.smithi094.stdout:8/617: mkdir d18/d2c/d37/d60/da3/db8/dd8 0 2024-04-06T03:40:40.126 INFO:tasks.workunit.client.0.smithi094.stdout:9/574: creat d2/dc/d21/d23/d47/d51/fd4 x:0 0 0 2024-04-06T03:40:40.128 INFO:tasks.workunit.client.0.smithi094.stdout:3/602: creat d9/d1e/d6e/d75/dba/fdb x:0 0 0 2024-04-06T03:40:40.128 INFO:tasks.workunit.client.0.smithi094.stdout:3/603: chown d9/d10/d9b/db7 6471065 1 2024-04-06T03:40:40.128 INFO:tasks.workunit.client.0.smithi094.stdout:3/604: chown d9/l13 281 1 2024-04-06T03:40:40.128 INFO:tasks.workunit.client.0.smithi094.stdout:3/605: getdents d9/d1e/d6e/d75/dba/dc5 0 2024-04-06T03:40:40.129 INFO:tasks.workunit.client.0.smithi094.stdout:3/606: dread - d9/d1e/d26/d55/d62/db8/fca zero size 2024-04-06T03:40:40.129 INFO:tasks.workunit.client.0.smithi094.stdout:3/607: write d9/d1e/d6e/d75/dba/d7c/f92 [270482,37020] 0 2024-04-06T03:40:40.130 INFO:tasks.workunit.client.0.smithi094.stdout:2/622: link d0/c7 d0/d8/d4f/cd5 0 2024-04-06T03:40:40.131 INFO:tasks.workunit.client.0.smithi094.stdout:3/608: write d9/d1e/d26/f28 [359179,39451] 0 2024-04-06T03:40:40.133 INFO:tasks.workunit.client.0.smithi094.stdout:6/541: creat dc/d11/d22/d50/fbf x:0 0 0 2024-04-06T03:40:40.135 INFO:tasks.workunit.client.0.smithi094.stdout:5/553: rename db/d19/d21/f5e to db/d19/d2c/d57/fbe 0 2024-04-06T03:40:40.135 INFO:tasks.workunit.client.0.smithi094.stdout:5/554: fdatasync db/f11 0 2024-04-06T03:40:40.137 INFO:tasks.workunit.client.0.smithi094.stdout:4/633: mknod d0/d15/d3a/d61/cd6 0 2024-04-06T03:40:40.137 INFO:tasks.workunit.client.0.smithi094.stdout:4/634: chown d0/l28 120505 1 2024-04-06T03:40:40.137 INFO:tasks.workunit.client.0.smithi094.stdout:4/635: creat d0/d15/fd7 x:0 0 0 2024-04-06T03:40:40.138 INFO:tasks.workunit.client.1.smithi136.stdout:9/598: symlink d17/d20/d25/d3d/da5/lca 0 2024-04-06T03:40:40.139 INFO:tasks.workunit.client.1.smithi136.stdout:9/599: dread d17/d20/d25/d71/fa9 [0,4194304] 0 2024-04-06T03:40:40.139 INFO:tasks.workunit.client.0.smithi094.stdout:9/575: link d2/l3 d2/dc/d21/d23/d5b/ld5 0 2024-04-06T03:40:40.139 INFO:tasks.workunit.client.0.smithi094.stdout:9/576: stat d2/dc/d21/d23/d47/lc2 0 2024-04-06T03:40:40.140 INFO:tasks.workunit.client.1.smithi136.stdout:7/632: symlink d5/df/d3d/ldd 0 2024-04-06T03:40:40.142 INFO:tasks.workunit.client.0.smithi094.stdout:1/531: dwrite d0/d1/d27/da5/d40/db1/d2f/d76/d7d/fae [0,4194304] 0 2024-04-06T03:40:40.143 INFO:tasks.workunit.client.0.smithi094.stdout:3/609: read d9/fc [2694910,91244] 0 2024-04-06T03:40:40.143 INFO:tasks.workunit.client.0.smithi094.stdout:3/610: fdatasync d9/d1e/d26/d55/d62/db8/fca 0 2024-04-06T03:40:40.147 INFO:tasks.workunit.client.1.smithi136.stdout:3/547: rmdir d0/d4/d91 39 2024-04-06T03:40:40.147 INFO:tasks.workunit.client.0.smithi094.stdout:6/542: creat dc/d3a/d7f/fc0 x:0 0 0 2024-04-06T03:40:40.149 INFO:tasks.workunit.client.1.smithi136.stdout:6/556: rmdir d2/d7/d26/d68 39 2024-04-06T03:40:40.153 INFO:tasks.workunit.client.0.smithi094.stdout:5/555: creat db/d19/d24/d28/fbf x:0 0 0 2024-04-06T03:40:40.155 INFO:tasks.workunit.client.1.smithi136.stdout:4/541: rename d1d/d2f/d6c/fa3 to d1d/d31/d3e/d55/d7f/fc3 0 2024-04-06T03:40:40.156 INFO:tasks.workunit.client.0.smithi094.stdout:4/636: rmdir d0/d4/d67 39 2024-04-06T03:40:40.157 INFO:tasks.workunit.client.0.smithi094.stdout:4/637: dread d0/d15/d3a/fa6 [0,4194304] 0 2024-04-06T03:40:40.162 INFO:tasks.workunit.client.1.smithi136.stdout:1/566: symlink d12/d3b/lb6 0 2024-04-06T03:40:40.162 INFO:tasks.workunit.client.1.smithi136.stdout:5/621: sync 2024-04-06T03:40:40.166 INFO:tasks.workunit.client.0.smithi094.stdout:2/623: link d0/dd/f62 d0/d18/fd6 0 2024-04-06T03:40:40.166 INFO:tasks.workunit.client.0.smithi094.stdout:2/624: readlink d0/dd/d3d/d49/lb1 0 2024-04-06T03:40:40.166 INFO:tasks.workunit.client.0.smithi094.stdout:2/625: write d0/dd/d19/d31/f7d [2456975,38387] 0 2024-04-06T03:40:40.166 INFO:tasks.workunit.client.0.smithi094.stdout:2/626: write d0/d8/d27/d2b/f2d [202620,2462] 0 2024-04-06T03:40:40.166 INFO:tasks.workunit.client.1.smithi136.stdout:0/575: truncate d1/d6/d2c/d57/f3f 2808137 0 2024-04-06T03:40:40.171 INFO:tasks.workunit.client.1.smithi136.stdout:7/633: rename d5/df/d11/cd5 to d5/df/d1d/d6f/d4d/d69/cde 0 2024-04-06T03:40:40.171 INFO:tasks.workunit.client.1.smithi136.stdout:7/634: chown d5/df/d12/d1b/d3a/d4a/l60 411991304 1 2024-04-06T03:40:40.172 INFO:tasks.workunit.client.0.smithi094.stdout:2/627: dread d0/d18/fd6 [0,4194304] 0 2024-04-06T03:40:40.172 INFO:tasks.workunit.client.0.smithi094.stdout:5/556: link db/d19/d2c/c94 db/d40/d77/cc0 0 2024-04-06T03:40:40.172 INFO:tasks.workunit.client.0.smithi094.stdout:2/628: chown d0/dd/da4 1107 1 2024-04-06T03:40:40.173 INFO:tasks.workunit.client.0.smithi094.stdout:5/557: chown db/d19/d24/l54 572 1 2024-04-06T03:40:40.174 INFO:tasks.workunit.client.1.smithi136.stdout:4/542: creat d1d/d2f/d6c/d91/fc4 x:0 0 0 2024-04-06T03:40:40.174 INFO:tasks.workunit.client.1.smithi136.stdout:4/543: fdatasync d1d/d31/d3e/f5a 0 2024-04-06T03:40:40.174 INFO:tasks.workunit.client.1.smithi136.stdout:4/544: creat d1d/d31/d3e/d55/d89/db3/fc5 x:0 0 0 2024-04-06T03:40:40.174 INFO:tasks.workunit.client.1.smithi136.stdout:1/567: write d12/d1b/f29 [701322,47813] 0 2024-04-06T03:40:40.174 INFO:tasks.workunit.client.1.smithi136.stdout:1/568: write d12/d3b/d60/d6d/da2/fb1 [520659,69570] 0 2024-04-06T03:40:40.175 INFO:tasks.workunit.client.0.smithi094.stdout:4/638: mkdir d0/df/db3/dd8 0 2024-04-06T03:40:40.176 INFO:tasks.workunit.client.1.smithi136.stdout:5/622: truncate d1/dc/d1c/d32/d70/fa8 388985 0 2024-04-06T03:40:40.178 INFO:tasks.workunit.client.0.smithi094.stdout:8/618: rename d18/d2c/d62 to d18/d3c/d76/dd9 0 2024-04-06T03:40:40.179 INFO:tasks.workunit.client.1.smithi136.stdout:0/576: symlink d1/d6/d2c/d57/d43/d6d/d85/d86/lbf 0 2024-04-06T03:40:40.179 INFO:tasks.workunit.client.1.smithi136.stdout:0/577: write d1/d6/d2c/d1a/d36/f76 [644057,29569] 0 2024-04-06T03:40:40.179 INFO:tasks.workunit.client.1.smithi136.stdout:0/578: fsync d1/d6/d2c/d57/d43/d6d/d9b/d5c/f6a 0 2024-04-06T03:40:40.184 INFO:tasks.workunit.client.1.smithi136.stdout:3/548: getdents d0/d4/d12/d19/d55/d3c 0 2024-04-06T03:40:40.186 INFO:tasks.workunit.client.0.smithi094.stdout:2/629: mknod d0/d8/d1d/d77/cd7 0 2024-04-06T03:40:40.187 INFO:tasks.workunit.client.0.smithi094.stdout:6/543: dwrite dc/d11/d22/d50/d61/f6c [0,4194304] 0 2024-04-06T03:40:40.187 INFO:tasks.workunit.client.1.smithi136.stdout:7/635: truncate d5/df/d1d/f57 3518785 0 2024-04-06T03:40:40.189 INFO:tasks.workunit.client.0.smithi094.stdout:4/639: mknod d0/d23/cd9 0 2024-04-06T03:40:40.192 INFO:tasks.workunit.client.0.smithi094.stdout:9/577: rename d2/dc/d21/d23/d47/d51/c94 to d2/dc/d21/d26/dba/cd6 0 2024-04-06T03:40:40.193 INFO:tasks.workunit.client.0.smithi094.stdout:9/578: write d2/dc/d46/f4e [4277121,57122] 0 2024-04-06T03:40:40.195 INFO:tasks.workunit.client.0.smithi094.stdout:1/532: dwrite d0/f93 [0,4194304] 0 2024-04-06T03:40:40.195 INFO:tasks.workunit.client.0.smithi094.stdout:5/558: rmdir db/d19/d30 39 2024-04-06T03:40:40.198 INFO:tasks.workunit.client.0.smithi094.stdout:2/630: truncate d0/d18/fd6 2493888 0 2024-04-06T03:40:40.198 INFO:tasks.workunit.client.1.smithi136.stdout:7/636: dread d5/df/d11/fa2 [0,4194304] 0 2024-04-06T03:40:40.199 INFO:tasks.workunit.client.1.smithi136.stdout:1/569: truncate d12/d3b/d44/d5f/f63 548806 0 2024-04-06T03:40:40.199 INFO:tasks.workunit.client.1.smithi136.stdout:1/570: write d12/d1b/d1e/d9f/fa8 [171883,86236] 0 2024-04-06T03:40:40.199 INFO:tasks.workunit.client.1.smithi136.stdout:2/668: getdents d3/d15 0 2024-04-06T03:40:40.201 INFO:tasks.workunit.client.0.smithi094.stdout:4/640: dwrite d0/d15/f1a [4194304,4194304] 0 2024-04-06T03:40:40.201 INFO:tasks.workunit.client.0.smithi094.stdout:4/641: chown d0/df/d1e/d34/db6/fbe 15 1 2024-04-06T03:40:40.201 INFO:tasks.workunit.client.0.smithi094.stdout:4/642: fdatasync d0/d23/fd1 0 2024-04-06T03:40:40.201 INFO:tasks.workunit.client.0.smithi094.stdout:4/643: write d0/d23/fd1 [84697,95224] 0 2024-04-06T03:40:40.201 INFO:tasks.workunit.client.0.smithi094.stdout:4/644: dread - d0/df/db3/fc9 zero size 2024-04-06T03:40:40.203 INFO:tasks.workunit.client.0.smithi094.stdout:8/619: dwrite d18/d34/f58 [4194304,4194304] 0 2024-04-06T03:40:40.203 INFO:tasks.workunit.client.1.smithi136.stdout:3/549: link d0/d4/f76 d0/d4/d12/d3a/d51/da6/fb6 0 2024-04-06T03:40:40.205 INFO:tasks.workunit.client.0.smithi094.stdout:6/544: symlink dc/d11/d22/d50/d61/lc1 0 2024-04-06T03:40:40.205 INFO:tasks.workunit.client.1.smithi136.stdout:5/623: dwrite d1/dc/d1c/f56 [0,4194304] 0 2024-04-06T03:40:40.208 INFO:tasks.workunit.client.0.smithi094.stdout:3/611: rename d9/d1e/d26/d55/f93 to d9/d10/d7e/da3/fdc 0 2024-04-06T03:40:40.208 INFO:tasks.workunit.client.0.smithi094.stdout:3/612: chown d9/d1e/d6e/d75/dba/c43 14157 1 2024-04-06T03:40:40.210 INFO:tasks.workunit.client.0.smithi094.stdout:8/620: dread d18/d30/f3e [0,4194304] 0 2024-04-06T03:40:40.211 INFO:tasks.workunit.client.1.smithi136.stdout:4/545: truncate d1d/d31/fa2 393117 0 2024-04-06T03:40:40.212 INFO:tasks.workunit.client.0.smithi094.stdout:9/579: symlink d2/d2c/ld7 0 2024-04-06T03:40:40.213 INFO:tasks.workunit.client.1.smithi136.stdout:1/571: mkdir d12/d3b/d44/d5f/db7 0 2024-04-06T03:40:40.215 INFO:tasks.workunit.client.1.smithi136.stdout:2/669: truncate d3/d9/fe 1836921 0 2024-04-06T03:40:40.216 INFO:tasks.workunit.client.0.smithi094.stdout:1/533: mkdir d0/d1/d27/da5/db2 0 2024-04-06T03:40:40.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/559: mkdir db/d19/dc1 0 2024-04-06T03:40:40.218 INFO:tasks.workunit.client.1.smithi136.stdout:3/550: symlink d0/d4/d12/d19/d55/d3c/d5a/d79/d98/lb7 0 2024-04-06T03:40:40.219 INFO:tasks.workunit.client.0.smithi094.stdout:6/545: read dc/d11/d1b/d24/f43 [2717808,8628] 0 2024-04-06T03:40:40.219 INFO:tasks.workunit.client.0.smithi094.stdout:6/546: creat dc/d11/d1b/d24/d46/fc2 x:0 0 0 2024-04-06T03:40:40.219 INFO:tasks.workunit.client.0.smithi094.stdout:6/547: fsync dc/d11/d22/d50/d61/f73 0 2024-04-06T03:40:40.220 INFO:tasks.workunit.client.0.smithi094.stdout:4/645: mknod d0/df/d91/cda 0 2024-04-06T03:40:40.220 INFO:tasks.workunit.client.0.smithi094.stdout:4/646: chown d0/d15/d3a/l4f 1754009 1 2024-04-06T03:40:40.225 INFO:tasks.workunit.client.1.smithi136.stdout:4/546: rename d1d/d2f/d6c/f76 to d1d/d31/d3e/d55/d89/db3/fc6 0 2024-04-06T03:40:40.227 INFO:tasks.workunit.client.1.smithi136.stdout:3/551: read d0/d4/d12/d19/d55/d3c/f58 [1018383,122607] 0 2024-04-06T03:40:40.227 INFO:tasks.workunit.client.1.smithi136.stdout:1/572: mkdir d12/d1b/da7/db8 0 2024-04-06T03:40:40.230 INFO:tasks.workunit.client.1.smithi136.stdout:2/670: creat d3/d4a/d9d/d18/d1d/fd8 x:0 0 0 2024-04-06T03:40:40.230 INFO:tasks.workunit.client.1.smithi136.stdout:2/671: truncate d3/d4a/d77/fcd 414810 0 2024-04-06T03:40:40.234 INFO:tasks.workunit.client.1.smithi136.stdout:1/573: rename d12/d3b/d44/d5f/d5a/d72/d77/c93 to d12/d3b/d74/d7a/cb9 0 2024-04-06T03:40:40.235 INFO:tasks.workunit.client.1.smithi136.stdout:3/552: dread d0/d4/d12/d19/d55/d6f/fad [0,4194304] 0 2024-04-06T03:40:40.235 INFO:tasks.workunit.client.1.smithi136.stdout:7/637: dwrite d5/df/f62 [0,4194304] 0 2024-04-06T03:40:40.235 INFO:tasks.workunit.client.1.smithi136.stdout:7/638: write d5/df/d1d/d6f/d2a/f48 [2567295,114353] 0 2024-04-06T03:40:40.235 INFO:tasks.workunit.client.1.smithi136.stdout:7/639: chown d5/df/d12/d59/l5a 772 1 2024-04-06T03:40:40.236 INFO:tasks.workunit.client.1.smithi136.stdout:2/672: dread d3/d9/fad [0,4194304] 0 2024-04-06T03:40:40.238 INFO:tasks.workunit.client.0.smithi094.stdout:3/613: mkdir d9/d1e/d26/d6c/ddd 0 2024-04-06T03:40:40.239 INFO:tasks.workunit.client.1.smithi136.stdout:5/624: dwrite d1/dc/d1c/d32/d8b/f7f [0,4194304] 0 2024-04-06T03:40:40.239 INFO:tasks.workunit.client.1.smithi136.stdout:5/625: write d1/dc/d1c/d32/d8b/d20/f42 [2492732,110719] 0 2024-04-06T03:40:40.240 INFO:tasks.workunit.client.1.smithi136.stdout:5/626: chown d1/dc/l50 832 1 2024-04-06T03:40:40.240 INFO:tasks.workunit.client.1.smithi136.stdout:5/627: fdatasync d1/d16/f1b 0 2024-04-06T03:40:40.243 INFO:tasks.workunit.client.0.smithi094.stdout:9/580: mkdir d2/dc/d21/d23/d47/d51/dd8 0 2024-04-06T03:40:40.246 INFO:tasks.workunit.client.0.smithi094.stdout:1/534: getdents d0/d1/d27/da5/d40/db1 0 2024-04-06T03:40:40.248 INFO:tasks.workunit.client.1.smithi136.stdout:3/553: dread d0/d4/fe [0,4194304] 0 2024-04-06T03:40:40.254 INFO:tasks.workunit.client.1.smithi136.stdout:5/628: dread d1/d16/d4b/f57 [0,4194304] 0 2024-04-06T03:40:40.254 INFO:tasks.workunit.client.0.smithi094.stdout:6/548: symlink dc/d11/d1b/lc3 0 2024-04-06T03:40:40.256 INFO:tasks.workunit.client.1.smithi136.stdout:1/574: creat d12/d3b/d74/fba x:0 0 0 2024-04-06T03:40:40.257 INFO:tasks.workunit.client.0.smithi094.stdout:4/647: truncate d0/d4/f76 4334157 0 2024-04-06T03:40:40.259 INFO:tasks.workunit.client.1.smithi136.stdout:3/554: dread d0/d4/d12/d19/d2e/f87 [0,4194304] 0 2024-04-06T03:40:40.259 INFO:tasks.workunit.client.1.smithi136.stdout:3/555: creat d0/d4/d12/d19/d55/d3c/d5a/fb8 x:0 0 0 2024-04-06T03:40:40.260 INFO:tasks.workunit.client.1.smithi136.stdout:3/556: creat d0/d4/d12/d19/d2c/fb9 x:0 0 0 2024-04-06T03:40:40.260 INFO:tasks.workunit.client.1.smithi136.stdout:3/557: fsync d0/d4/d12/d19/d55/d3c/f4e 0 2024-04-06T03:40:40.260 INFO:tasks.workunit.client.0.smithi094.stdout:5/560: rmdir db/d19/d24/d2e 39 2024-04-06T03:40:40.260 INFO:tasks.workunit.client.0.smithi094.stdout:5/561: truncate db/d7c/d98/fa3 297889 0 2024-04-06T03:40:40.260 INFO:tasks.workunit.client.0.smithi094.stdout:5/562: readlink db/d19/l22 0 2024-04-06T03:40:40.261 INFO:tasks.workunit.client.0.smithi094.stdout:5/563: fsync db/d19/d24/d28/f8c 0 2024-04-06T03:40:40.262 INFO:tasks.workunit.client.0.smithi094.stdout:6/549: mkdir dc/d11/d1b/d72/d88/da3/dc4 0 2024-04-06T03:40:40.262 INFO:tasks.workunit.client.0.smithi094.stdout:6/550: fsync dc/d3a/d4a/f92 0 2024-04-06T03:40:40.263 INFO:tasks.workunit.client.0.smithi094.stdout:3/614: dwrite d9/d20/f3d [0,4194304] 0 2024-04-06T03:40:40.263 INFO:tasks.workunit.client.1.smithi136.stdout:7/640: mkdir d5/df/d12/d1b/d3a/d4a/ddf 0 2024-04-06T03:40:40.264 INFO:tasks.workunit.client.1.smithi136.stdout:8/621: sync 2024-04-06T03:40:40.264 INFO:tasks.workunit.client.1.smithi136.stdout:8/622: dread - d3/da/d26/d35/d3d/d48/f92 zero size 2024-04-06T03:40:40.265 INFO:tasks.workunit.client.0.smithi094.stdout:8/621: rename d18/d3c/d49/d91/d94/daa/dc8 to d18/d2c/dda 0 2024-04-06T03:40:40.265 INFO:tasks.workunit.client.0.smithi094.stdout:8/622: write f13 [5136709,85329] 0 2024-04-06T03:40:40.265 INFO:tasks.workunit.client.0.smithi094.stdout:8/623: chown d18/d2c/f6e 0 1 2024-04-06T03:40:40.265 INFO:tasks.workunit.client.0.smithi094.stdout:8/624: stat d18/d30 0 2024-04-06T03:40:40.267 INFO:tasks.workunit.client.0.smithi094.stdout:1/535: dwrite d0/d7/f26 [4194304,4194304] 0 2024-04-06T03:40:40.267 INFO:tasks.workunit.client.0.smithi094.stdout:1/536: truncate d0/d1/f73 319828 0 2024-04-06T03:40:40.267 INFO:tasks.workunit.client.0.smithi094.stdout:1/537: truncate d0/d7/d1a/d64/f72 1056522 0 2024-04-06T03:40:40.289 INFO:tasks.workunit.client.0.smithi094.stdout:5/564: dwrite db/d19/d30/d83/f87 [0,4194304] 0 2024-04-06T03:40:40.289 INFO:tasks.workunit.client.0.smithi094.stdout:5/565: read db/f12 [1122254,123] 0 2024-04-06T03:40:40.350 INFO:tasks.workunit.client.1.smithi136.stdout:2/673: link d3/d4a/d9d/f6b d3/d4a/d9d/d8a/fd9 0 2024-04-06T03:40:40.352 INFO:tasks.workunit.client.0.smithi094.stdout:6/551: mknod dc/d11/d1b/d4e/cc5 0 2024-04-06T03:40:40.357 INFO:tasks.workunit.client.0.smithi094.stdout:3/615: link d9/d20/l51 d9/d20/d76/lde 0 2024-04-06T03:40:40.359 INFO:tasks.workunit.client.1.smithi136.stdout:1/575: mkdir d12/d3b/d60/d7e/d37/dbb 0 2024-04-06T03:40:40.359 INFO:tasks.workunit.client.1.smithi136.stdout:1/576: fdatasync d12/d3b/d44/d5f/f63 0 2024-04-06T03:40:40.361 INFO:tasks.workunit.client.0.smithi094.stdout:4/648: rename d0/df/db3/fc9 to d0/d4/d67/d9f/fdb 0 2024-04-06T03:40:40.364 INFO:tasks.workunit.client.0.smithi094.stdout:8/625: mknod d18/d5c/cdb 0 2024-04-06T03:40:40.367 INFO:tasks.workunit.client.1.smithi136.stdout:3/558: mknod d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/cba 0 2024-04-06T03:40:40.367 INFO:tasks.workunit.client.1.smithi136.stdout:3/559: truncate d0/d4/d12/d19/d55/d6f/fb1 224393 0 2024-04-06T03:40:40.370 INFO:tasks.workunit.client.1.smithi136.stdout:8/623: rename d3/de/f29 to d3/da/d26/d35/d3d/d48/d6c/dab/fbe 0 2024-04-06T03:40:40.375 INFO:tasks.workunit.client.0.smithi094.stdout:5/566: creat db/d19/d2c/fc2 x:0 0 0 2024-04-06T03:40:40.377 INFO:tasks.workunit.client.0.smithi094.stdout:6/552: mkdir dc/d11/d1b/d24/d46/d7b/dc6 0 2024-04-06T03:40:40.378 INFO:tasks.workunit.client.0.smithi094.stdout:3/616: mknod d9/d1e/d26/d55/cdf 0 2024-04-06T03:40:40.381 INFO:tasks.workunit.client.0.smithi094.stdout:6/553: read dc/f17 [653120,120907] 0 2024-04-06T03:40:40.381 INFO:tasks.workunit.client.0.smithi094.stdout:6/554: chown dc/d3a/d4a/f56 1 1 2024-04-06T03:40:40.381 INFO:tasks.workunit.client.0.smithi094.stdout:6/555: creat dc/d3a/d4a/d54/fc7 x:0 0 0 2024-04-06T03:40:40.381 INFO:tasks.workunit.client.0.smithi094.stdout:4/649: rename d0/d4/d67/c9b to d0/d15/d3a/d61/d81/da0/cdc 0 2024-04-06T03:40:40.381 INFO:tasks.workunit.client.0.smithi094.stdout:4/650: truncate d0/df/d57/fbf 550982 0 2024-04-06T03:40:40.381 INFO:tasks.workunit.client.0.smithi094.stdout:4/651: creat d0/d15/d3a/d61/d81/da0/fdd x:0 0 0 2024-04-06T03:40:40.381 INFO:tasks.workunit.client.0.smithi094.stdout:5/567: dread db/d40/d77/fa9 [0,4194304] 0 2024-04-06T03:40:40.384 INFO:tasks.workunit.client.0.smithi094.stdout:3/617: creat d9/d20/d2d/da9/fe0 x:0 0 0 2024-04-06T03:40:40.385 INFO:tasks.workunit.client.1.smithi136.stdout:2/674: truncate d3/d4a/d9d/d18/d1d/f35 1715879 0 2024-04-06T03:40:40.386 INFO:tasks.workunit.client.0.smithi094.stdout:4/652: dread d0/d15/d3a/d3e/d46/f62 [0,4194304] 0 2024-04-06T03:40:40.386 INFO:tasks.workunit.client.0.smithi094.stdout:4/653: fsync d0/d15/f1a 0 2024-04-06T03:40:40.387 INFO:tasks.workunit.client.1.smithi136.stdout:1/577: creat d12/d3b/d8e/fbc x:0 0 0 2024-04-06T03:40:40.388 INFO:tasks.workunit.client.0.smithi094.stdout:6/556: creat dc/d3a/fc8 x:0 0 0 2024-04-06T03:40:40.390 INFO:tasks.workunit.client.1.smithi136.stdout:3/560: mkdir d0/d4/d12/d3a/d51/da6/dbb 0 2024-04-06T03:40:40.390 INFO:tasks.workunit.client.1.smithi136.stdout:3/561: readlink d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/la2 0 2024-04-06T03:40:40.390 INFO:tasks.workunit.client.1.smithi136.stdout:3/562: chown d0/d4/d12/d2d/d37/ca8 39 1 2024-04-06T03:40:40.394 INFO:tasks.workunit.client.1.smithi136.stdout:8/624: creat d3/da/d26/d35/d3d/d48/d6c/d7f/fbf x:0 0 0 2024-04-06T03:40:40.396 INFO:tasks.workunit.client.1.smithi136.stdout:1/578: rename d12/d53/f61 to d12/d3b/d44/d5f/d5a/d72/fbd 0 2024-04-06T03:40:40.397 INFO:tasks.workunit.client.0.smithi094.stdout:3/618: mkdir d9/d10/dbb/de1 0 2024-04-06T03:40:40.398 INFO:tasks.workunit.client.0.smithi094.stdout:8/626: rmdir d18/d3c/d76 39 2024-04-06T03:40:40.400 INFO:tasks.workunit.client.0.smithi094.stdout:6/557: truncate dc/d3a/f5e 3085539 0 2024-04-06T03:40:40.402 INFO:tasks.workunit.client.0.smithi094.stdout:3/619: mkdir d9/d10/d9b/db7/dc0/de2 0 2024-04-06T03:40:40.402 INFO:tasks.workunit.client.0.smithi094.stdout:3/620: fdatasync d9/d1e/f8f 0 2024-04-06T03:40:40.402 INFO:tasks.workunit.client.1.smithi136.stdout:1/579: mkdir d12/d3b/d60/d7e/dbe 0 2024-04-06T03:40:40.404 INFO:tasks.workunit.client.0.smithi094.stdout:8/627: rmdir d18/d34/d52 39 2024-04-06T03:40:40.405 INFO:tasks.workunit.client.1.smithi136.stdout:1/580: symlink d12/d3b/d44/lbf 0 2024-04-06T03:40:40.405 INFO:tasks.workunit.client.1.smithi136.stdout:1/581: fsync d12/d3b/d74/d7a/fad 0 2024-04-06T03:40:40.406 INFO:tasks.workunit.client.0.smithi094.stdout:4/654: mkdir d0/d4/dde 0 2024-04-06T03:40:40.407 INFO:tasks.workunit.client.0.smithi094.stdout:6/558: link dc/l15 dc/dab/lc9 0 2024-04-06T03:40:40.408 INFO:tasks.workunit.client.1.smithi136.stdout:1/582: mknod d12/d1b/d22/cc0 0 2024-04-06T03:40:40.408 INFO:tasks.workunit.client.1.smithi136.stdout:1/583: read - d12/d3b/d44/d5f/fa0 zero size 2024-04-06T03:40:40.409 INFO:tasks.workunit.client.0.smithi094.stdout:3/621: creat d9/d9d/fe3 x:0 0 0 2024-04-06T03:40:40.410 INFO:tasks.workunit.client.0.smithi094.stdout:4/655: unlink d0/df/d13/f35 0 2024-04-06T03:40:40.410 INFO:tasks.workunit.client.0.smithi094.stdout:4/656: getdents d0/d4/d67/dcb 0 2024-04-06T03:40:40.410 INFO:tasks.workunit.client.0.smithi094.stdout:4/657: truncate d0/d15/d3a/d61/d81/f89 4739548 0 2024-04-06T03:40:40.411 INFO:tasks.workunit.client.1.smithi136.stdout:1/584: creat d12/d3b/d44/d5f/d5a/d72/d77/fc1 x:0 0 0 2024-04-06T03:40:40.412 INFO:tasks.workunit.client.0.smithi094.stdout:8/628: dread f1 [4194304,4194304] 0 2024-04-06T03:40:40.412 INFO:tasks.workunit.client.0.smithi094.stdout:3/622: mknod d9/d1e/ce4 0 2024-04-06T03:40:40.413 INFO:tasks.workunit.client.0.smithi094.stdout:6/559: dread dc/d11/d1b/d24/d46/f48 [0,4194304] 0 2024-04-06T03:40:40.413 INFO:tasks.workunit.client.0.smithi094.stdout:6/560: fsync dc/d3a/d4a/d94/dac/fb8 0 2024-04-06T03:40:40.414 INFO:tasks.workunit.client.0.smithi094.stdout:4/658: mkdir d0/d15/d3a/d61/ddf 0 2024-04-06T03:40:40.414 INFO:tasks.workunit.client.0.smithi094.stdout:4/659: readlink d0/df/db3/ld0 0 2024-04-06T03:40:40.414 INFO:tasks.workunit.client.0.smithi094.stdout:8/629: mkdir d18/d3c/d49/d91/d94/daa/ddc 0 2024-04-06T03:40:40.422 INFO:tasks.workunit.client.1.smithi136.stdout:3/563: dread d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/f9b [0,4194304] 0 2024-04-06T03:40:40.422 INFO:tasks.workunit.client.1.smithi136.stdout:3/564: readlink d0/d4/d12/d19/d55/l21 0 2024-04-06T03:40:40.422 INFO:tasks.workunit.client.1.smithi136.stdout:3/565: dread - d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/fb2 zero size 2024-04-06T03:40:40.425 INFO:tasks.workunit.client.0.smithi094.stdout:4/660: creat d0/d4/d67/fe0 x:0 0 0 2024-04-06T03:40:40.425 INFO:tasks.workunit.client.0.smithi094.stdout:4/661: creat d0/d15/fe1 x:0 0 0 2024-04-06T03:40:40.428 INFO:tasks.workunit.client.0.smithi094.stdout:2/631: dread d0/dd/f62 [0,4194304] 0 2024-04-06T03:40:40.429 INFO:tasks.workunit.client.0.smithi094.stdout:5/568: dwrite db/d19/d24/f34 [4194304,4194304] 0 2024-04-06T03:40:40.433 INFO:tasks.workunit.client.0.smithi094.stdout:4/662: unlink d0/df/d13/d4a/l4c 0 2024-04-06T03:40:40.434 INFO:tasks.workunit.client.0.smithi094.stdout:3/623: read d9/d20/f33 [7970753,17228] 0 2024-04-06T03:40:40.434 INFO:tasks.workunit.client.0.smithi094.stdout:3/624: chown d9/d1e/f8f 72 1 2024-04-06T03:40:40.438 INFO:tasks.workunit.client.0.smithi094.stdout:8/630: dread d18/d3c/dc4/dbc/f93 [0,4194304] 0 2024-04-06T03:40:40.439 INFO:tasks.workunit.client.1.smithi136.stdout:3/566: rename d0/d4/d12/d19/d55/d3c/d5a/la7 to d0/d4/lbc 0 2024-04-06T03:40:40.441 INFO:tasks.workunit.client.0.smithi094.stdout:2/632: truncate d0/dd/d1b/d72/d8f/fbb 1476798 0 2024-04-06T03:40:40.441 INFO:tasks.workunit.client.0.smithi094.stdout:6/561: dread dc/d11/d22/f9e [0,4194304] 0 2024-04-06T03:40:40.443 INFO:tasks.workunit.client.0.smithi094.stdout:5/569: rename db/d19/d24/d28/fbf to db/d7c/d8b/fc3 0 2024-04-06T03:40:40.443 INFO:tasks.workunit.client.1.smithi136.stdout:2/675: dwrite d3/d9/fad [0,4194304] 0 2024-04-06T03:40:40.444 INFO:tasks.workunit.client.0.smithi094.stdout:3/625: mkdir d9/de5 0 2024-04-06T03:40:40.445 INFO:tasks.workunit.client.0.smithi094.stdout:2/633: readlink d0/lb 0 2024-04-06T03:40:40.445 INFO:tasks.workunit.client.1.smithi136.stdout:1/585: dwrite fb [0,4194304] 0 2024-04-06T03:40:40.448 INFO:tasks.workunit.client.1.smithi136.stdout:8/625: dwrite d3/da/d26/d35/d53/d86/f87 [0,4194304] 0 2024-04-06T03:40:40.449 INFO:tasks.workunit.client.1.smithi136.stdout:8/626: chown d3/da/d26/d35/d39/d70/d7e 0 1 2024-04-06T03:40:40.449 INFO:tasks.workunit.client.1.smithi136.stdout:8/627: chown d3/da/d26/d35/d39 122167669 1 2024-04-06T03:40:40.450 INFO:tasks.workunit.client.0.smithi094.stdout:8/631: rename d18/d2c/c81 to d18/d2c/d37/d60/da3/db8/cdd 0 2024-04-06T03:40:40.453 INFO:tasks.workunit.client.0.smithi094.stdout:2/634: mknod d0/d8/d1d/cd8 0 2024-04-06T03:40:40.453 INFO:tasks.workunit.client.1.smithi136.stdout:4/547: rmdir d1d/d31 39 2024-04-06T03:40:40.453 INFO:tasks.workunit.client.1.smithi136.stdout:4/548: chown d1d/d2f/c69 15 1 2024-04-06T03:40:40.454 INFO:tasks.workunit.client.1.smithi136.stdout:2/676: symlink d3/d4a/d77/lda 0 2024-04-06T03:40:40.457 INFO:tasks.workunit.client.0.smithi094.stdout:6/562: rename dc/d11/d22/l2d to dc/d3a/d4a/d94/dac/lca 0 2024-04-06T03:40:40.459 INFO:tasks.workunit.client.1.smithi136.stdout:4/549: creat d1d/d31/d3e/d55/fc7 x:0 0 0 2024-04-06T03:40:40.459 INFO:tasks.workunit.client.1.smithi136.stdout:4/550: creat d1d/d2f/d6c/d91/fc8 x:0 0 0 2024-04-06T03:40:40.460 INFO:tasks.workunit.client.0.smithi094.stdout:7/581: sync 2024-04-06T03:40:40.460 INFO:tasks.workunit.client.0.smithi094.stdout:7/582: chown d2/d6/f9 1903589 1 2024-04-06T03:40:40.460 INFO:tasks.workunit.client.0.smithi094.stdout:0/595: sync 2024-04-06T03:40:40.460 INFO:tasks.workunit.client.0.smithi094.stdout:5/570: write db/fc [1871709,93738] 0 2024-04-06T03:40:40.461 INFO:tasks.workunit.client.0.smithi094.stdout:8/632: creat d18/d2c/d37/d60/fde x:0 0 0 2024-04-06T03:40:40.463 INFO:tasks.workunit.client.0.smithi094.stdout:2/635: creat d0/d8/da0/fd9 x:0 0 0 2024-04-06T03:40:40.464 INFO:tasks.workunit.client.1.smithi136.stdout:8/628: read d3/d2d/f67 [1012560,46245] 0 2024-04-06T03:40:40.464 INFO:tasks.workunit.client.1.smithi136.stdout:8/629: chown d3/da/d26/d35/f57 702819 1 2024-04-06T03:40:40.464 INFO:tasks.workunit.client.1.smithi136.stdout:8/630: readlink d3/da/d26/l3b 0 2024-04-06T03:40:40.464 INFO:tasks.workunit.client.1.smithi136.stdout:2/677: mknod d3/d9b/cdb 0 2024-04-06T03:40:40.464 INFO:tasks.workunit.client.1.smithi136.stdout:2/678: chown d3/d9/l52 44 1 2024-04-06T03:40:40.465 INFO:tasks.workunit.client.0.smithi094.stdout:6/563: truncate dc/d3a/d4a/d54/fb2 8665 0 2024-04-06T03:40:40.465 INFO:tasks.workunit.client.0.smithi094.stdout:6/564: creat dc/d3a/d4a/d54/d71/fcb x:0 0 0 2024-04-06T03:40:40.467 INFO:tasks.workunit.client.0.smithi094.stdout:7/583: symlink d2/d6/dc/dd/df/d36/d79/lab 0 2024-04-06T03:40:40.469 INFO:tasks.workunit.client.1.smithi136.stdout:8/631: rmdir d3/da/d26/d90/dae 39 2024-04-06T03:40:40.470 INFO:tasks.workunit.client.0.smithi094.stdout:0/596: mknod d1/d9/dd/d18/d1e/d22/d35/dac/cc8 0 2024-04-06T03:40:40.470 INFO:tasks.workunit.client.0.smithi094.stdout:0/597: fsync d1/d9/dd/d18/d1e/f8c 0 2024-04-06T03:40:40.472 INFO:tasks.workunit.client.1.smithi136.stdout:2/679: mkdir d3/ddc 0 2024-04-06T03:40:40.472 INFO:tasks.workunit.client.1.smithi136.stdout:2/680: fsync d3/d9/fe 0 2024-04-06T03:40:40.472 INFO:tasks.workunit.client.1.smithi136.stdout:0/579: dwrite d1/d6/d2c/d57/d43/d6d/d9b/d5c/f6a [0,4194304] 0 2024-04-06T03:40:40.473 INFO:tasks.workunit.client.0.smithi094.stdout:6/565: dread dc/d3a/f41 [0,4194304] 0 2024-04-06T03:40:40.473 INFO:tasks.workunit.client.0.smithi094.stdout:5/571: creat db/d19/d2c/d57/d8f/fc4 x:0 0 0 2024-04-06T03:40:40.476 INFO:tasks.workunit.client.0.smithi094.stdout:8/633: creat d18/d3c/dc4/db9/fdf x:0 0 0 2024-04-06T03:40:40.477 INFO:tasks.workunit.client.0.smithi094.stdout:8/634: stat d18/d34 0 2024-04-06T03:40:40.478 INFO:tasks.workunit.client.1.smithi136.stdout:8/632: mkdir d3/da/d26/d35/d3d/dc0 0 2024-04-06T03:40:40.478 INFO:tasks.workunit.client.1.smithi136.stdout:8/633: write d3/f21 [543766,111188] 0 2024-04-06T03:40:40.482 INFO:tasks.workunit.client.0.smithi094.stdout:7/584: rename d2/d6/dc/dd/d5a to d2/d6/dac 0 2024-04-06T03:40:40.484 INFO:tasks.workunit.client.0.smithi094.stdout:0/598: mknod d1/d9/dd/d18/d1a/d44/cc9 0 2024-04-06T03:40:40.484 INFO:tasks.workunit.client.1.smithi136.stdout:2/681: dread d3/d4a/d9d/d18/f2a [0,4194304] 0 2024-04-06T03:40:40.484 INFO:tasks.workunit.client.1.smithi136.stdout:2/682: write d3/d9b/fcb [893287,52630] 0 2024-04-06T03:40:40.485 INFO:tasks.workunit.client.1.smithi136.stdout:0/580: dread - d1/d6/d2c/f88 zero size 2024-04-06T03:40:40.485 INFO:tasks.workunit.client.1.smithi136.stdout:0/581: chown d1/d6/d2c/d57/d43/d6d/d9b/d5c/d96 23 1 2024-04-06T03:40:40.485 INFO:tasks.workunit.client.1.smithi136.stdout:0/582: fdatasync d1/d6/d2c/d57/d43/fa2 0 2024-04-06T03:40:40.487 INFO:tasks.workunit.client.0.smithi094.stdout:7/585: truncate d2/d6/dc/f4a 17279 0 2024-04-06T03:40:40.490 INFO:tasks.workunit.client.1.smithi136.stdout:4/551: link d1d/d31/f3f d1d/d31/d8e/fc9 0 2024-04-06T03:40:40.490 INFO:tasks.workunit.client.1.smithi136.stdout:4/552: getdents d1d/d31/d3e/d55/dba 0 2024-04-06T03:40:40.492 INFO:tasks.workunit.client.0.smithi094.stdout:0/599: rename d1/d9/dd/d18/d32/d42/f77 to d1/d9/dd/d18/d1e/d86/d96/daa/fca 0 2024-04-06T03:40:40.492 INFO:tasks.workunit.client.0.smithi094.stdout:0/600: write d1/d9/dd/d18/d5d/d60/d6b/d67/d76/fba [124570,84830] 0 2024-04-06T03:40:40.493 INFO:tasks.workunit.client.0.smithi094.stdout:0/601: dread - d1/d9/dd/d18/d1e/f8c zero size 2024-04-06T03:40:40.493 INFO:tasks.workunit.client.0.smithi094.stdout:1/538: dwrite d0/f93 [4194304,4194304] 0 2024-04-06T03:40:40.495 INFO:tasks.workunit.client.0.smithi094.stdout:7/586: symlink d2/d6/dc/dd/d90/lad 0 2024-04-06T03:40:40.497 INFO:tasks.workunit.client.1.smithi136.stdout:0/583: creat d1/d6/d2c/fc0 x:0 0 0 2024-04-06T03:40:40.499 INFO:tasks.workunit.client.0.smithi094.stdout:0/602: mknod d1/d9/dd/d18/d1e/d86/ccb 0 2024-04-06T03:40:40.500 INFO:tasks.workunit.client.1.smithi136.stdout:4/553: getdents d1d/d2f/d6c/da5 0 2024-04-06T03:40:40.503 INFO:tasks.workunit.client.1.smithi136.stdout:0/584: mknod d1/d6/d2c/d57/d43/d6d/d85/d78/cc1 0 2024-04-06T03:40:40.504 INFO:tasks.workunit.client.0.smithi094.stdout:5/572: dread db/d19/d2c/f31 [0,4194304] 0 2024-04-06T03:40:40.505 INFO:tasks.workunit.client.0.smithi094.stdout:1/539: symlink d0/d1/d27/da5/d40/db1/lb3 0 2024-04-06T03:40:40.507 INFO:tasks.workunit.client.1.smithi136.stdout:4/554: creat d1d/d31/d3e/d55/d89/db3/fca x:0 0 0 2024-04-06T03:40:40.507 INFO:tasks.workunit.client.0.smithi094.stdout:1/540: write d0/d1/f65 [496710,89937] 0 2024-04-06T03:40:40.507 INFO:tasks.workunit.client.0.smithi094.stdout:1/541: write d0/d1/d27/da5/d40/db1/d2f/d4b/f8a [696505,42378] 0 2024-04-06T03:40:40.508 INFO:tasks.workunit.client.0.smithi094.stdout:7/587: rename d2/d6/dc/dd/df/d36/f68 to d2/d6/dc/dd/fae 0 2024-04-06T03:40:40.509 INFO:tasks.workunit.client.0.smithi094.stdout:0/603: truncate d1/d9/dd/d18/d32/d42/f4a 1709046 0 2024-04-06T03:40:40.509 INFO:tasks.workunit.client.0.smithi094.stdout:0/604: write d1/d9/dd/d18/d1a/f54 [2112535,99154] 0 2024-04-06T03:40:40.510 INFO:tasks.workunit.client.1.smithi136.stdout:4/555: mkdir d1d/d31/d78/dcb 0 2024-04-06T03:40:40.510 INFO:tasks.workunit.client.1.smithi136.stdout:4/556: stat d1d/d31/d3e/d55/d89/c8f 0 2024-04-06T03:40:40.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/573: stat db/d40/d77/cc0 0 2024-04-06T03:40:40.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/574: fdatasync db/f2a 0 2024-04-06T03:40:40.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/575: dread - db/d7c/d8b/fc3 zero size 2024-04-06T03:40:40.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/576: chown db/d19/d2c/d6d/c74 46 1 2024-04-06T03:40:40.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/577: chown db/d19/d30/f81 479681528 1 2024-04-06T03:40:40.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/542: symlink d0/d1/d27/da5/db2/lb4 0 2024-04-06T03:40:40.513 INFO:tasks.workunit.client.0.smithi094.stdout:0/605: read d1/d9/dd/d18/d5d/d60/d6b/f81 [548929,86575] 0 2024-04-06T03:40:40.514 INFO:tasks.workunit.client.1.smithi136.stdout:4/557: rename d1d/d2f/d6c/da5/fbc to d1d/d2f/d6c/da5/dc1/fcc 0 2024-04-06T03:40:40.515 INFO:tasks.workunit.client.1.smithi136.stdout:1/586: dwrite d12/d1b/f86 [0,4194304] 0 2024-04-06T03:40:40.516 INFO:tasks.workunit.client.0.smithi094.stdout:7/588: creat d2/d6/dc/d12/d9a/da0/faf x:0 0 0 2024-04-06T03:40:40.516 INFO:tasks.workunit.client.0.smithi094.stdout:7/589: creat d2/d6/dc/d12/d29/d35/fb0 x:0 0 0 2024-04-06T03:40:40.517 INFO:tasks.workunit.client.0.smithi094.stdout:3/626: dwrite d9/d9d/fe3 [0,4194304] 0 2024-04-06T03:40:40.517 INFO:tasks.workunit.client.0.smithi094.stdout:3/627: write d9/d1e/f59 [329507,52874] 0 2024-04-06T03:40:40.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/587: dread d12/d1b/d28/f91 [0,4194304] 0 2024-04-06T03:40:40.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/588: dread - d12/d3b/d74/d7a/fad zero size 2024-04-06T03:40:40.517 INFO:tasks.workunit.client.1.smithi136.stdout:0/585: dread d1/d6/d2c/d1a/d36/d59/f21 [0,4194304] 0 2024-04-06T03:40:40.519 INFO:tasks.workunit.client.0.smithi094.stdout:1/543: mknod d0/d1/d27/da5/d40/db1/d2f/d3c/da0/cb5 0 2024-04-06T03:40:40.519 INFO:tasks.workunit.client.0.smithi094.stdout:8/635: dwrite d18/f27 [0,4194304] 0 2024-04-06T03:40:40.521 INFO:tasks.workunit.client.1.smithi136.stdout:7/641: sync 2024-04-06T03:40:40.521 INFO:tasks.workunit.client.1.smithi136.stdout:5/629: sync 2024-04-06T03:40:40.521 INFO:tasks.workunit.client.1.smithi136.stdout:9/600: sync 2024-04-06T03:40:40.521 INFO:tasks.workunit.client.1.smithi136.stdout:9/601: stat d17/d20/d25/d3d/d4b/fab 0 2024-04-06T03:40:40.521 INFO:tasks.workunit.client.1.smithi136.stdout:6/557: sync 2024-04-06T03:40:40.521 INFO:tasks.workunit.client.1.smithi136.stdout:6/558: getdents d2/d7/d26/d30/d6e/d58/d5d/d62/d53 0 2024-04-06T03:40:40.523 INFO:tasks.workunit.client.1.smithi136.stdout:2/683: dwrite d3/d4a/d9d/d18/f23 [0,4194304] 0 2024-04-06T03:40:40.525 INFO:tasks.workunit.client.1.smithi136.stdout:3/567: dwrite d0/d4/d12/d19/d55/d3c/d5a/f89 [0,4194304] 0 2024-04-06T03:40:40.527 INFO:tasks.workunit.client.1.smithi136.stdout:1/589: getdents d12/d3b/d60 0 2024-04-06T03:40:40.527 INFO:tasks.workunit.client.1.smithi136.stdout:1/590: fsync d12/d1b/d22/fb0 0 2024-04-06T03:40:40.527 INFO:tasks.workunit.client.1.smithi136.stdout:1/591: creat d12/d3b/d60/d6d/da2/fc2 x:0 0 0 2024-04-06T03:40:40.528 INFO:tasks.workunit.client.1.smithi136.stdout:8/634: dwrite d3/de/f8d [0,4194304] 0 2024-04-06T03:40:40.530 INFO:tasks.workunit.client.1.smithi136.stdout:0/586: link d1/f18 d1/d6/d2c/d1a/d36/fc2 0 2024-04-06T03:40:40.532 INFO:tasks.workunit.client.0.smithi094.stdout:7/590: mkdir d2/d6/dc/d12/d3a/d56/db1 0 2024-04-06T03:40:40.535 INFO:tasks.workunit.client.1.smithi136.stdout:6/559: write d2/d7/d26/d30/f50 [4459965,10142] 0 2024-04-06T03:40:40.540 INFO:tasks.workunit.client.1.smithi136.stdout:9/602: getdents d17/d21/d32/d43 0 2024-04-06T03:40:40.541 INFO:tasks.workunit.client.0.smithi094.stdout:3/628: creat d9/d1d/fe6 x:0 0 0 2024-04-06T03:40:40.541 INFO:tasks.workunit.client.0.smithi094.stdout:3/629: fsync d9/d1e/d6e/d75/dba/d3a/fa8 0 2024-04-06T03:40:40.541 INFO:tasks.workunit.client.0.smithi094.stdout:3/630: chown d9/d20/f3d 29582 1 2024-04-06T03:40:40.542 INFO:tasks.workunit.client.1.smithi136.stdout:2/684: link d3/d15/d19/d43/daa/lbf d3/d4a/d9d/d18/d1d/d3c/ldd 0 2024-04-06T03:40:40.543 INFO:tasks.workunit.client.1.smithi136.stdout:2/685: write d3/d4a/d9d/d8a/d93/d5f/f78 [195289,114071] 0 2024-04-06T03:40:40.543 INFO:tasks.workunit.client.0.smithi094.stdout:1/544: mknod d0/d1/cb6 0 2024-04-06T03:40:40.548 INFO:tasks.workunit.client.1.smithi136.stdout:3/568: mkdir d0/d4/d12/d19/d55/d6f/d82/d5d/dbd 0 2024-04-06T03:40:40.548 INFO:tasks.workunit.client.1.smithi136.stdout:3/569: write d0/d4/f76 [948368,116438] 0 2024-04-06T03:40:40.549 INFO:tasks.workunit.client.1.smithi136.stdout:4/558: dwrite d1d/d2f/f97 [0,4194304] 0 2024-04-06T03:40:40.553 INFO:tasks.workunit.client.1.smithi136.stdout:8/635: creat d3/da/da0/fc1 x:0 0 0 2024-04-06T03:40:40.558 INFO:tasks.workunit.client.0.smithi094.stdout:3/631: creat d9/d20/da6/fe7 x:0 0 0 2024-04-06T03:40:40.559 INFO:tasks.workunit.client.1.smithi136.stdout:0/587: link d1/d6/d2c/d57/d43/d6d/d85/d78/da4/fb6 d1/d6/fc3 0 2024-04-06T03:40:40.562 INFO:tasks.workunit.client.0.smithi094.stdout:8/636: dwrite d18/f82 [0,4194304] 0 2024-04-06T03:40:40.562 INFO:tasks.workunit.client.1.smithi136.stdout:7/642: getdents d5/df/d1d 0 2024-04-06T03:40:40.562 INFO:tasks.workunit.client.1.smithi136.stdout:1/592: dwrite d12/d3b/d44/d5f/d5a/fae [0,4194304] 0 2024-04-06T03:40:40.562 INFO:tasks.workunit.client.1.smithi136.stdout:1/593: stat d12/d3b/ca4 0 2024-04-06T03:40:40.565 INFO:tasks.workunit.client.0.smithi094.stdout:3/632: read d9/f15 [3287575,102349] 0 2024-04-06T03:40:40.566 INFO:tasks.workunit.client.1.smithi136.stdout:6/560: unlink d2/d7/d26/d68/f80 0 2024-04-06T03:40:40.567 INFO:tasks.workunit.client.0.smithi094.stdout:1/545: dread d0/d1/d27/da5/d40/db1/d2f/f54 [4194304,4194304] 0 2024-04-06T03:40:40.567 INFO:tasks.workunit.client.0.smithi094.stdout:1/546: chown d0/d7/d1a/la7 15924588 1 2024-04-06T03:40:40.569 INFO:tasks.workunit.client.1.smithi136.stdout:9/603: mkdir d17/d20/d28/d81/d88/dcb 0 2024-04-06T03:40:40.571 INFO:tasks.workunit.client.1.smithi136.stdout:0/588: write d1/d6/d2c/d1a/d36/d59/fb8 [3556746,19906] 0 2024-04-06T03:40:40.571 INFO:tasks.workunit.client.1.smithi136.stdout:0/589: chown d1/d6/d2c/d57/d43/f50 2619228 1 2024-04-06T03:40:40.574 INFO:tasks.workunit.client.1.smithi136.stdout:3/570: creat d0/d4/d12/d2d/fbe x:0 0 0 2024-04-06T03:40:40.577 INFO:tasks.workunit.client.1.smithi136.stdout:4/559: rmdir d1d/d31/d3e/d56 39 2024-04-06T03:40:40.577 INFO:tasks.workunit.client.0.smithi094.stdout:8/637: dread d18/d2c/d37/fa1 [0,4194304] 0 2024-04-06T03:40:40.577 INFO:tasks.workunit.client.0.smithi094.stdout:8/638: chown d18/d3c/d8b 56 1 2024-04-06T03:40:40.578 INFO:tasks.workunit.client.1.smithi136.stdout:9/604: write d17/d19/f8e [4081346,27009] 0 2024-04-06T03:40:40.578 INFO:tasks.workunit.client.1.smithi136.stdout:9/605: creat d17/d19/fcc x:0 0 0 2024-04-06T03:40:40.579 INFO:tasks.workunit.client.1.smithi136.stdout:9/606: truncate d17/d20/d25/f31 65053 0 2024-04-06T03:40:40.579 INFO:tasks.workunit.client.1.smithi136.stdout:8/636: getdents d3 0 2024-04-06T03:40:40.581 INFO:tasks.workunit.client.1.smithi136.stdout:7/643: link d5/df/d12/c84 d5/df/d53/d68/da3/ce0 0 2024-04-06T03:40:40.581 INFO:tasks.workunit.client.1.smithi136.stdout:0/590: write d1/d6/d2c/d57/d43/d6d/d85/d78/f97 [1821117,46374] 0 2024-04-06T03:40:40.581 INFO:tasks.workunit.client.1.smithi136.stdout:0/591: write d1/d6/d2c/d1a/d36/fc2 [738230,14491] 0 2024-04-06T03:40:40.581 INFO:tasks.workunit.client.1.smithi136.stdout:0/592: read d1/d6/d2c/d1a/d36/fc2 [601160,61690] 0 2024-04-06T03:40:40.582 INFO:tasks.workunit.client.1.smithi136.stdout:0/593: read d1/d6/d2c/d1a/f94 [89294,5662] 0 2024-04-06T03:40:40.583 INFO:tasks.workunit.client.1.smithi136.stdout:0/594: dread d1/d6/d2c/d57/d43/d6d/d85/da7/fb4 [0,4194304] 0 2024-04-06T03:40:40.586 INFO:tasks.workunit.client.1.smithi136.stdout:1/594: symlink d12/d3b/d6b/lc3 0 2024-04-06T03:40:40.588 INFO:tasks.workunit.client.0.smithi094.stdout:3/633: creat d9/d1e/d6e/d75/dba/d7c/fe8 x:0 0 0 2024-04-06T03:40:40.590 INFO:tasks.workunit.client.0.smithi094.stdout:1/547: truncate d0/d7/f36 4059019 0 2024-04-06T03:40:40.592 INFO:tasks.workunit.client.1.smithi136.stdout:5/630: rename d1/d75 to d1/dc/d1c/d32/d70/dc1 0 2024-04-06T03:40:40.592 INFO:tasks.workunit.client.1.smithi136.stdout:5/631: chown d1/cb 0 1 2024-04-06T03:40:40.592 INFO:tasks.workunit.client.1.smithi136.stdout:5/632: write d1/dc/d1c/d32/d70/fa8 [930900,40602] 0 2024-04-06T03:40:40.596 INFO:tasks.workunit.client.0.smithi094.stdout:8/639: creat d18/d3c/d8b/fe0 x:0 0 0 2024-04-06T03:40:40.596 INFO:tasks.workunit.client.0.smithi094.stdout:8/640: write d18/d2c/f2e [4221523,95989] 0 2024-04-06T03:40:40.596 INFO:tasks.workunit.client.0.smithi094.stdout:8/641: creat d18/d3c/d49/d91/fe1 x:0 0 0 2024-04-06T03:40:40.596 INFO:tasks.workunit.client.0.smithi094.stdout:8/642: read d18/f6b [477842,88360] 0 2024-04-06T03:40:40.598 INFO:tasks.workunit.client.0.smithi094.stdout:3/634: mkdir d9/d20/da6/de9 0 2024-04-06T03:40:40.598 INFO:tasks.workunit.client.0.smithi094.stdout:3/635: fsync d9/d20/f50 0 2024-04-06T03:40:40.598 INFO:tasks.workunit.client.0.smithi094.stdout:3/636: dread - d9/d1d/fe6 zero size 2024-04-06T03:40:40.598 INFO:tasks.workunit.client.0.smithi094.stdout:3/637: creat d9/d10/d7e/fea x:0 0 0 2024-04-06T03:40:40.598 INFO:tasks.workunit.client.0.smithi094.stdout:3/638: write d9/d1d/f4a [593535,126173] 0 2024-04-06T03:40:40.600 INFO:tasks.workunit.client.1.smithi136.stdout:5/633: write d1/dc/d10/f94 [469402,31162] 0 2024-04-06T03:40:40.601 INFO:tasks.workunit.client.1.smithi136.stdout:5/634: read d1/d16/d4b/f8e [302486,39207] 0 2024-04-06T03:40:40.603 INFO:tasks.workunit.client.0.smithi094.stdout:8/643: creat d18/d2c/d37/d60/d70/fe2 x:0 0 0 2024-04-06T03:40:40.603 INFO:tasks.workunit.client.0.smithi094.stdout:8/644: truncate d18/d30/f99 898514 0 2024-04-06T03:40:40.608 INFO:tasks.workunit.client.0.smithi094.stdout:3/639: link d9/d1d/l53 d9/d10/dbb/de1/leb 0 2024-04-06T03:40:40.609 INFO:tasks.workunit.client.1.smithi136.stdout:0/595: dwrite d1/d6/f1f [0,4194304] 0 2024-04-06T03:40:40.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/640: read d9/d1d/d7d/fb0 [339012,130660] 0 2024-04-06T03:40:40.610 INFO:tasks.workunit.client.0.smithi094.stdout:0/606: dread d1/d9/dd/d18/d32/d42/f4a [0,4194304] 0 2024-04-06T03:40:40.610 INFO:tasks.workunit.client.0.smithi094.stdout:1/548: dread d0/d1/d27/da5/d40/db1/faa [0,4194304] 0 2024-04-06T03:40:40.610 INFO:tasks.workunit.client.0.smithi094.stdout:1/549: write d0/d1/d27/da5/f3b [1005573,74541] 0 2024-04-06T03:40:40.611 INFO:tasks.workunit.client.1.smithi136.stdout:3/571: symlink d0/d4/d12/d3a/d51/da6/dbb/lbf 0 2024-04-06T03:40:40.611 INFO:tasks.workunit.client.1.smithi136.stdout:3/572: fsync d0/f23 0 2024-04-06T03:40:40.611 INFO:tasks.workunit.client.0.smithi094.stdout:8/645: mkdir d18/d5c/de3 0 2024-04-06T03:40:40.612 INFO:tasks.workunit.client.1.smithi136.stdout:4/560: creat d1d/d2f/d6c/da5/fcd x:0 0 0 2024-04-06T03:40:40.612 INFO:tasks.workunit.client.1.smithi136.stdout:4/561: dread - d1d/fbb zero size 2024-04-06T03:40:40.614 INFO:tasks.workunit.client.0.smithi094.stdout:3/641: symlink d9/d1e/d6e/d75/dba/lec 0 2024-04-06T03:40:40.615 INFO:tasks.workunit.client.1.smithi136.stdout:4/562: dread d1d/d2f/f43 [0,4194304] 0 2024-04-06T03:40:40.618 INFO:tasks.workunit.client.0.smithi094.stdout:0/607: unlink d1/d9/dd/d18/d1d/cb8 0 2024-04-06T03:40:40.618 INFO:tasks.workunit.client.0.smithi094.stdout:0/608: read - d1/d9/dd/d18/d1e/d86/d96/fa4 zero size 2024-04-06T03:40:40.618 INFO:tasks.workunit.client.0.smithi094.stdout:0/609: chown d1/d9/dd/d18/d1e/c43 247132 1 2024-04-06T03:40:40.618 INFO:tasks.workunit.client.0.smithi094.stdout:0/610: stat d1/d9/dd/d18/d32/d42 0 2024-04-06T03:40:40.618 INFO:tasks.workunit.client.0.smithi094.stdout:0/611: write d1/d9/dd/d18/d1e/d86/fa3 [748387,97118] 0 2024-04-06T03:40:40.620 INFO:tasks.workunit.client.1.smithi136.stdout:9/607: link d17/d19/l38 d17/d20/d28/d49/d75/d60/lcd 0 2024-04-06T03:40:40.620 INFO:tasks.workunit.client.1.smithi136.stdout:9/608: read - d17/d21/d29/f78 zero size 2024-04-06T03:40:40.624 INFO:tasks.workunit.client.1.smithi136.stdout:8/637: symlink d3/da/d26/d35/d3d/d48/d6c/lc2 0 2024-04-06T03:40:40.626 INFO:tasks.workunit.client.0.smithi094.stdout:1/550: creat d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/fb7 x:0 0 0 2024-04-06T03:40:40.627 INFO:tasks.workunit.client.0.smithi094.stdout:1/551: read d0/d1/d27/da5/f3b [865530,30071] 0 2024-04-06T03:40:40.627 INFO:tasks.workunit.client.0.smithi094.stdout:0/612: write d1/f83 [3352252,69120] 0 2024-04-06T03:40:40.628 INFO:tasks.workunit.client.0.smithi094.stdout:8/646: creat d18/d5c/fe4 x:0 0 0 2024-04-06T03:40:40.630 INFO:tasks.workunit.client.1.smithi136.stdout:7/644: link d5/df/d1d/d6f/f56 d5/df/d11/fe1 0 2024-04-06T03:40:40.632 INFO:tasks.workunit.client.0.smithi094.stdout:1/552: mknod d0/d7/d2d/cb8 0 2024-04-06T03:40:40.634 INFO:tasks.workunit.client.0.smithi094.stdout:0/613: mknod d1/d9/dd/d18/d5d/d60/d6b/dab/ccc 0 2024-04-06T03:40:40.634 INFO:tasks.workunit.client.0.smithi094.stdout:0/614: creat d1/d9/fcd x:0 0 0 2024-04-06T03:40:40.634 INFO:tasks.workunit.client.0.smithi094.stdout:0/615: chown d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/fbe 121207 1 2024-04-06T03:40:40.636 INFO:tasks.workunit.client.1.smithi136.stdout:1/595: link d12/d3b/d44/d5f/cab d12/d1b/da7/daa/cc4 0 2024-04-06T03:40:40.636 INFO:tasks.workunit.client.1.smithi136.stdout:1/596: truncate d12/d3b/d74/fb4 5709399 0 2024-04-06T03:40:40.636 INFO:tasks.workunit.client.1.smithi136.stdout:1/597: dread - d12/d3b/d60/d6d/da2/fc2 zero size 2024-04-06T03:40:40.636 INFO:tasks.workunit.client.1.smithi136.stdout:1/598: dread - d12/d3b/d8e/fbc zero size 2024-04-06T03:40:40.637 INFO:tasks.workunit.client.1.smithi136.stdout:4/563: dwrite d1d/d31/d78/fa9 [0,4194304] 0 2024-04-06T03:40:40.638 INFO:tasks.workunit.client.1.smithi136.stdout:4/564: write d1d/d31/d3e/d55/fc7 [497520,67171] 0 2024-04-06T03:40:40.638 INFO:tasks.workunit.client.1.smithi136.stdout:4/565: fdatasync f7 0 2024-04-06T03:40:40.638 INFO:tasks.workunit.client.1.smithi136.stdout:4/566: dread - d1d/d31/d3e/d55/fb8 zero size 2024-04-06T03:40:40.638 INFO:tasks.workunit.client.0.smithi094.stdout:8/647: rename d18/d3c/dc4/dbc/l86 to d18/d3c/dc4/db9/le5 0 2024-04-06T03:40:40.638 INFO:tasks.workunit.client.0.smithi094.stdout:8/648: fdatasync d18/d2c/f9b 0 2024-04-06T03:40:40.639 INFO:tasks.workunit.client.0.smithi094.stdout:3/642: dwrite d9/d20/f34 [0,4194304] 0 2024-04-06T03:40:40.639 INFO:tasks.workunit.client.1.smithi136.stdout:0/596: dwrite d1/d6/d2c/d57/d43/d6d/d9b/d5c/f71 [0,4194304] 0 2024-04-06T03:40:40.639 INFO:tasks.workunit.client.1.smithi136.stdout:0/597: fsync d1/d6/d2c/d1a/f81 0 2024-04-06T03:40:40.644 INFO:tasks.workunit.client.1.smithi136.stdout:5/635: symlink d1/d1f/lc2 0 2024-04-06T03:40:40.644 INFO:tasks.workunit.client.0.smithi094.stdout:0/616: mknod d1/d9/dd/d18/d1e/d22/d35/dac/cce 0 2024-04-06T03:40:40.646 INFO:tasks.workunit.client.1.smithi136.stdout:3/573: symlink d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/lc0 0 2024-04-06T03:40:40.646 INFO:tasks.workunit.client.1.smithi136.stdout:3/574: chown d0/d4/d12/d19/d55/d6f/d82/d5d/c99 8 1 2024-04-06T03:40:40.646 INFO:tasks.workunit.client.0.smithi094.stdout:8/649: symlink d18/d5c/de3/le6 0 2024-04-06T03:40:40.648 INFO:tasks.workunit.client.1.smithi136.stdout:9/609: fsync d17/d20/d28/f82 0 2024-04-06T03:40:40.650 INFO:tasks.workunit.client.0.smithi094.stdout:0/617: symlink d1/d9/dd/d18/d1a/lcf 0 2024-04-06T03:40:40.650 INFO:tasks.workunit.client.0.smithi094.stdout:0/618: chown d1/d9/dd/d18/d1e/d86/d96 6 1 2024-04-06T03:40:40.652 INFO:tasks.workunit.client.1.smithi136.stdout:8/638: unlink d3/de/f8e 0 2024-04-06T03:40:40.659 INFO:tasks.workunit.client.1.smithi136.stdout:1/599: mkdir d12/d53/dc5 0 2024-04-06T03:40:40.664 INFO:tasks.workunit.client.1.smithi136.stdout:4/567: creat d1d/d31/d3e/d56/d7b/fce x:0 0 0 2024-04-06T03:40:40.672 INFO:tasks.workunit.client.0.smithi094.stdout:8/650: dread d18/f21 [0,4194304] 0 2024-04-06T03:40:40.672 INFO:tasks.workunit.client.0.smithi094.stdout:8/651: write d18/d3c/dc4/db9/fc0 [939950,27914] 0 2024-04-06T03:40:40.680 INFO:tasks.workunit.client.1.smithi136.stdout:0/598: dwrite d1/d6/fc3 [0,4194304] 0 2024-04-06T03:40:40.680 INFO:tasks.workunit.client.0.smithi094.stdout:1/553: dwrite d0/d1/d27/da5/d40/f7a [0,4194304] 0 2024-04-06T03:40:40.680 INFO:tasks.workunit.client.0.smithi094.stdout:8/652: symlink d18/d3c/d49/d9f/le7 0 2024-04-06T03:40:40.680 INFO:tasks.workunit.client.0.smithi094.stdout:8/653: write d18/d3c/d49/d91/d94/daa/fd1 [484756,42286] 0 2024-04-06T03:40:40.681 INFO:tasks.workunit.client.0.smithi094.stdout:8/654: fsync d18/d3c/d49/d80/faf 0 2024-04-06T03:40:40.681 INFO:tasks.workunit.client.1.smithi136.stdout:5/636: dwrite d1/dc/d1c/d32/f38 [0,4194304] 0 2024-04-06T03:40:40.681 INFO:tasks.workunit.client.1.smithi136.stdout:3/575: dwrite d0/d4/f76 [0,4194304] 0 2024-04-06T03:40:40.681 INFO:tasks.workunit.client.1.smithi136.stdout:3/576: dread - d0/d4/d12/d2d/f96 zero size 2024-04-06T03:40:40.684 INFO:tasks.workunit.client.0.smithi094.stdout:8/655: symlink d18/d2c/dd2/le8 0 2024-04-06T03:40:40.686 INFO:tasks.workunit.client.1.smithi136.stdout:9/610: mkdir d17/d20/d25/d3d/da5/dce 0 2024-04-06T03:40:40.686 INFO:tasks.workunit.client.1.smithi136.stdout:9/611: write d17/d20/d28/f89 [5366216,75618] 0 2024-04-06T03:40:40.686 INFO:tasks.workunit.client.1.smithi136.stdout:9/612: write d17/d20/fba [868434,106653] 0 2024-04-06T03:40:40.686 INFO:tasks.workunit.client.0.smithi094.stdout:1/554: mkdir d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/db9 0 2024-04-06T03:40:40.688 INFO:tasks.workunit.client.1.smithi136.stdout:7/645: getdents d5/df/d12/d1b/d3a/d4a 0 2024-04-06T03:40:40.690 INFO:tasks.workunit.client.0.smithi094.stdout:8/656: creat d18/d3c/d49/d91/d94/daa/fe9 x:0 0 0 2024-04-06T03:40:40.690 INFO:tasks.workunit.client.0.smithi094.stdout:8/657: creat d18/d3c/d49/d80/fea x:0 0 0 2024-04-06T03:40:40.690 INFO:tasks.workunit.client.1.smithi136.stdout:0/599: dread d1/d6/d2c/d57/d43/d6d/d85/da7/fbd [0,4194304] 0 2024-04-06T03:40:40.695 INFO:tasks.workunit.client.0.smithi094.stdout:8/658: truncate d18/d30/f3e 217754 0 2024-04-06T03:40:40.697 INFO:tasks.workunit.client.0.smithi094.stdout:1/555: symlink d0/d7/d1a/lba 0 2024-04-06T03:40:40.697 INFO:tasks.workunit.client.0.smithi094.stdout:1/556: write d0/d7/fad [282127,100152] 0 2024-04-06T03:40:40.697 INFO:tasks.workunit.client.0.smithi094.stdout:1/557: write d0/d7/f78 [817703,70824] 0 2024-04-06T03:40:40.697 INFO:tasks.workunit.client.1.smithi136.stdout:3/577: getdents d0/d4/d12/d19/d55/d3c/d5a/d79/d98 0 2024-04-06T03:40:40.697 INFO:tasks.workunit.client.1.smithi136.stdout:5/637: dwrite d1/d9b/d9c/fbe [0,4194304] 0 2024-04-06T03:40:40.723 INFO:tasks.workunit.client.1.smithi136.stdout:4/568: dwrite d1d/d31/f67 [0,4194304] 0 2024-04-06T03:40:40.723 INFO:tasks.workunit.client.1.smithi136.stdout:4/569: stat d1d/d2f/d6c/d91/fb4 0 2024-04-06T03:40:40.723 INFO:tasks.workunit.client.1.smithi136.stdout:7/646: dwrite d5/df/d12/d1b/d3a/d4a/d70/fc0 [0,4194304] 0 2024-04-06T03:40:40.723 INFO:tasks.workunit.client.1.smithi136.stdout:0/600: dwrite d1/d53/f6e [0,4194304] 0 2024-04-06T03:40:40.734 INFO:tasks.workunit.client.1.smithi136.stdout:7/647: dread d5/df/d1d/d6f/f56 [0,4194304] 0 2024-04-06T03:40:40.737 INFO:tasks.workunit.client.1.smithi136.stdout:0/601: dwrite d1/d6/fa1 [0,4194304] 0 2024-04-06T03:40:40.739 INFO:tasks.workunit.client.0.smithi094.stdout:8/659: truncate d18/d2c/d37/d60/fa4 2132225 0 2024-04-06T03:40:40.739 INFO:tasks.workunit.client.0.smithi094.stdout:8/660: write d18/d2c/dda/fd6 [350979,27285] 0 2024-04-06T03:40:40.741 INFO:tasks.workunit.client.0.smithi094.stdout:1/558: link d0/d1/d27/da5/d40/db1/d2f/d3c/d48/f88 d0/d7/d1a/da8/fbb 0 2024-04-06T03:40:40.741 INFO:tasks.workunit.client.1.smithi136.stdout:9/613: symlink d17/d20/d25/d3d/d4b/lcf 0 2024-04-06T03:40:40.742 INFO:tasks.workunit.client.1.smithi136.stdout:9/614: write d17/d21/d32/d3f/f44 [1006557,103389] 0 2024-04-06T03:40:40.744 INFO:tasks.workunit.client.1.smithi136.stdout:9/615: dread d17/d21/d32/fc7 [0,4194304] 0 2024-04-06T03:40:40.744 INFO:tasks.workunit.client.1.smithi136.stdout:9/616: truncate f16 2236735 0 2024-04-06T03:40:40.751 INFO:tasks.workunit.client.0.smithi094.stdout:1/559: dread d0/d1/d27/da5/f3b [0,4194304] 0 2024-04-06T03:40:40.755 INFO:tasks.workunit.client.0.smithi094.stdout:8/661: dread d18/d34/f58 [0,4194304] 0 2024-04-06T03:40:40.756 INFO:tasks.workunit.client.0.smithi094.stdout:1/560: dread d0/d1/d27/fa9 [0,4194304] 0 2024-04-06T03:40:40.767 INFO:tasks.workunit.client.1.smithi136.stdout:9/617: dwrite d17/d20/d25/f84 [0,4194304] 0 2024-04-06T03:40:40.768 INFO:tasks.workunit.client.1.smithi136.stdout:9/618: write d17/d19/fcc [154870,43513] 0 2024-04-06T03:40:40.777 INFO:tasks.workunit.client.1.smithi136.stdout:5/638: unlink d1/dc/d1c/d32/d8b/c3d 0 2024-04-06T03:40:40.777 INFO:tasks.workunit.client.1.smithi136.stdout:5/639: dread - d1/dc/d10/d1e/f97 zero size 2024-04-06T03:40:40.782 INFO:tasks.workunit.client.0.smithi094.stdout:8/662: unlink d18/f67 0 2024-04-06T03:40:40.784 INFO:tasks.workunit.client.1.smithi136.stdout:4/570: creat d1d/d2f/da6/fcf x:0 0 0 2024-04-06T03:40:40.784 INFO:tasks.workunit.client.0.smithi094.stdout:1/561: rmdir d0/d1/d27/da5/d40/d50 39 2024-04-06T03:40:40.785 INFO:tasks.workunit.client.0.smithi094.stdout:0/619: rmdir d1/d9/dd/d18/d5d 39 2024-04-06T03:40:40.787 INFO:tasks.workunit.client.0.smithi094.stdout:8/663: mkdir d18/d2c/d37/d60/da3/deb 0 2024-04-06T03:40:40.787 INFO:tasks.workunit.client.0.smithi094.stdout:8/664: write d18/d3c/d49/d91/d94/daa/fe9 [572926,29433] 0 2024-04-06T03:40:40.790 INFO:tasks.workunit.client.0.smithi094.stdout:1/562: creat d0/d1/fbc x:0 0 0 2024-04-06T03:40:40.790 INFO:tasks.workunit.client.0.smithi094.stdout:1/563: readlink d0/d1/l8 0 2024-04-06T03:40:40.793 INFO:tasks.workunit.client.0.smithi094.stdout:0/620: dread d1/f7a [0,4194304] 0 2024-04-06T03:40:40.793 INFO:tasks.workunit.client.0.smithi094.stdout:0/621: chown d1/d9/f20 2011221 1 2024-04-06T03:40:40.793 INFO:tasks.workunit.client.0.smithi094.stdout:0/622: read d1/d9/dd/d18/d1e/d22/d35/f5c [1829083,102049] 0 2024-04-06T03:40:40.795 INFO:tasks.workunit.client.0.smithi094.stdout:1/564: write d0/d1/d27/da5/d40/db1/faa [685785,63032] 0 2024-04-06T03:40:40.796 INFO:tasks.workunit.client.0.smithi094.stdout:8/665: dread d18/f82 [0,4194304] 0 2024-04-06T03:40:40.797 INFO:tasks.workunit.client.0.smithi094.stdout:0/623: dread d1/d9/dd/d18/d5d/d60/d6b/f81 [0,4194304] 0 2024-04-06T03:40:40.797 INFO:tasks.workunit.client.1.smithi136.stdout:0/602: mknod d1/d6/cc4 0 2024-04-06T03:40:40.797 INFO:tasks.workunit.client.1.smithi136.stdout:0/603: chown d1/d6/d2c/d1a/f4b 89832133 1 2024-04-06T03:40:40.797 INFO:tasks.workunit.client.1.smithi136.stdout:0/604: truncate d1/d6/d2c/d57/d43/d6d/d85/d86/fba 861174 0 2024-04-06T03:40:40.797 INFO:tasks.workunit.client.1.smithi136.stdout:0/605: creat d1/d6/d2c/d57/d43/d9e/fc5 x:0 0 0 2024-04-06T03:40:40.798 INFO:tasks.workunit.client.0.smithi094.stdout:8/666: creat d18/d34/d52/fec x:0 0 0 2024-04-06T03:40:40.799 INFO:tasks.workunit.client.0.smithi094.stdout:8/667: truncate d18/d3c/d76/dd9/f72 1853454 0 2024-04-06T03:40:40.799 INFO:tasks.workunit.client.0.smithi094.stdout:8/668: write d18/d2c/d37/d60/fde [553790,97155] 0 2024-04-06T03:40:40.799 INFO:tasks.workunit.client.1.smithi136.stdout:0/606: dread d1/d6/d2c/d1a/d36/fc2 [0,4194304] 0 2024-04-06T03:40:40.799 INFO:tasks.workunit.client.0.smithi094.stdout:0/624: symlink d1/d9/dd/d18/d1a/ld0 0 2024-04-06T03:40:40.801 INFO:tasks.workunit.client.0.smithi094.stdout:8/669: mkdir d18/d3c/d49/db5/ded 0 2024-04-06T03:40:40.804 INFO:tasks.workunit.client.1.smithi136.stdout:9/619: creat d17/d20/d25/d3d/da5/db8/fd0 x:0 0 0 2024-04-06T03:40:40.805 INFO:tasks.workunit.client.0.smithi094.stdout:1/565: read d0/d1/d27/da5/d40/db1/d2f/d76/d7d/fae [619626,75128] 0 2024-04-06T03:40:40.805 INFO:tasks.workunit.client.0.smithi094.stdout:1/566: truncate d0/d1/d27/da5/f9f 187609 0 2024-04-06T03:40:40.806 INFO:tasks.workunit.client.1.smithi136.stdout:2/686: sync 2024-04-06T03:40:40.806 INFO:tasks.workunit.client.1.smithi136.stdout:6/561: sync 2024-04-06T03:40:40.806 INFO:tasks.workunit.client.1.smithi136.stdout:6/562: stat d2/d7/d13/c86 0 2024-04-06T03:40:40.807 INFO:tasks.workunit.client.0.smithi094.stdout:0/625: dread d1/d9/dd/d18/d1e/d22/d35/f3c [0,4194304] 0 2024-04-06T03:40:40.807 INFO:tasks.workunit.client.0.smithi094.stdout:1/567: link d0/d7/d2d/l63 d0/d1/d27/da5/db2/lbd 0 2024-04-06T03:40:40.807 INFO:tasks.workunit.client.0.smithi094.stdout:0/626: write d1/d9/fcd [70408,7864] 0 2024-04-06T03:40:40.809 INFO:tasks.workunit.client.1.smithi136.stdout:4/571: mkdir d1d/dd0 0 2024-04-06T03:40:40.809 INFO:tasks.workunit.client.1.smithi136.stdout:4/572: chown d1d/d31/d3e/d55/da0 397966 1 2024-04-06T03:40:40.809 INFO:tasks.workunit.client.0.smithi094.stdout:8/670: getdents d18/d3c/d49/d9f 0 2024-04-06T03:40:40.810 INFO:tasks.workunit.client.0.smithi094.stdout:0/627: creat d1/d9/dd/d18/d5d/d60/d6b/fd1 x:0 0 0 2024-04-06T03:40:40.813 INFO:tasks.workunit.client.1.smithi136.stdout:0/607: mknod d1/d6/cc6 0 2024-04-06T03:40:40.815 INFO:tasks.workunit.client.1.smithi136.stdout:2/687: mkdir d3/d9/dde 0 2024-04-06T03:40:40.815 INFO:tasks.workunit.client.1.smithi136.stdout:2/688: fsync d3/d4a/d9d/d18/d4b/f65 0 2024-04-06T03:40:40.817 INFO:tasks.workunit.client.1.smithi136.stdout:6/563: creat d2/d7/d26/d30/d6e/db7/fbd x:0 0 0 2024-04-06T03:40:40.820 INFO:tasks.workunit.client.0.smithi094.stdout:8/671: dread d18/d3c/d76/f5b [0,4194304] 0 2024-04-06T03:40:40.820 INFO:tasks.workunit.client.0.smithi094.stdout:8/672: creat d18/d2c/d37/d60/d70/fee x:0 0 0 2024-04-06T03:40:40.820 INFO:tasks.workunit.client.0.smithi094.stdout:8/673: write d18/d2c/d37/d60/fde [1185038,122236] 0 2024-04-06T03:40:40.820 INFO:tasks.workunit.client.1.smithi136.stdout:4/573: creat d1d/d31/d3e/d55/da0/fd1 x:0 0 0 2024-04-06T03:40:40.822 INFO:tasks.workunit.client.1.smithi136.stdout:9/620: dwrite d17/d20/d28/d49/f54 [4194304,4194304] 0 2024-04-06T03:40:40.823 INFO:tasks.workunit.client.1.smithi136.stdout:0/608: creat d1/d6/d2c/d57/d43/d6d/d9b/d5c/fc7 x:0 0 0 2024-04-06T03:40:40.824 INFO:tasks.workunit.client.1.smithi136.stdout:2/689: truncate d3/d4a/f8d 558507 0 2024-04-06T03:40:40.824 INFO:tasks.workunit.client.1.smithi136.stdout:2/690: write d3/f61 [1556482,45566] 0 2024-04-06T03:40:40.829 INFO:tasks.workunit.client.1.smithi136.stdout:6/564: rename d2/d7/d26/d30/d6e/fa5 to d2/d7/d26/d30/d6e/d58/fbe 0 2024-04-06T03:40:40.830 INFO:tasks.workunit.client.0.smithi094.stdout:8/674: mknod d18/d3c/d49/db5/ded/cef 0 2024-04-06T03:40:40.830 INFO:tasks.workunit.client.0.smithi094.stdout:8/675: chown d18/d30/c4c 635 1 2024-04-06T03:40:40.833 INFO:tasks.workunit.client.0.smithi094.stdout:0/628: dwrite d1/d9/dd/d18/d5d/d60/d6b/d67/d76/fba [0,4194304] 0 2024-04-06T03:40:40.833 INFO:tasks.workunit.client.0.smithi094.stdout:0/629: truncate d1/d9/dd/d18/d1e/d86/d96/daa/fc7 8671 0 2024-04-06T03:40:40.833 INFO:tasks.workunit.client.0.smithi094.stdout:0/630: write d1/d9/fcd [1090800,40611] 0 2024-04-06T03:40:40.835 INFO:tasks.workunit.client.1.smithi136.stdout:0/609: link d1/d6/d2c/d1a/d36/d59/l8e d1/d4e/lc8 0 2024-04-06T03:40:40.837 INFO:tasks.workunit.client.1.smithi136.stdout:9/621: dread d17/d20/d25/d3c/f73 [0,4194304] 0 2024-04-06T03:40:40.840 INFO:tasks.workunit.client.1.smithi136.stdout:9/622: mkdir d17/d20/d25/d3d/da5/dce/dd1 0 2024-04-06T03:40:40.841 INFO:tasks.workunit.client.1.smithi136.stdout:9/623: chown d17/c18 34 1 2024-04-06T03:40:40.841 INFO:tasks.workunit.client.1.smithi136.stdout:9/624: readlink d17/d20/d25/d3c/l8b 0 2024-04-06T03:40:40.841 INFO:tasks.workunit.client.0.smithi094.stdout:1/568: dwrite d0/d1/d27/da5/f53 [0,4194304] 0 2024-04-06T03:40:40.841 INFO:tasks.workunit.client.0.smithi094.stdout:8/676: fdatasync d18/d3c/dc4/db9/fc0 0 2024-04-06T03:40:40.841 INFO:tasks.workunit.client.0.smithi094.stdout:1/569: read d0/d1/d27/f2e [161950,20746] 0 2024-04-06T03:40:40.845 INFO:tasks.workunit.client.1.smithi136.stdout:6/565: dread d2/d25/f38 [4194304,4194304] 0 2024-04-06T03:40:40.845 INFO:tasks.workunit.client.1.smithi136.stdout:4/574: dwrite d1d/d31/d8e/f9a [0,4194304] 0 2024-04-06T03:40:40.847 INFO:tasks.workunit.client.1.smithi136.stdout:9/625: rename d17/d20/d25/d3b/l76 to d17/d20/d25/ld2 0 2024-04-06T03:40:40.847 INFO:tasks.workunit.client.1.smithi136.stdout:9/626: fsync d17/d20/f48 0 2024-04-06T03:40:40.849 INFO:tasks.workunit.client.1.smithi136.stdout:2/691: dwrite d3/d4a/d9d/d18/d4b/f85 [0,4194304] 0 2024-04-06T03:40:40.854 INFO:tasks.workunit.client.0.smithi094.stdout:1/570: write d0/d1/d27/da5/d40/db1/d2f/d4b/fa1 [1220669,125991] 0 2024-04-06T03:40:40.855 INFO:tasks.workunit.client.1.smithi136.stdout:4/575: unlink d1d/f49 0 2024-04-06T03:40:40.857 INFO:tasks.workunit.client.0.smithi094.stdout:9/581: sync 2024-04-06T03:40:40.857 INFO:tasks.workunit.client.1.smithi136.stdout:9/627: unlink d17/d20/d28/d49/l58 0 2024-04-06T03:40:40.857 INFO:tasks.workunit.client.0.smithi094.stdout:0/631: dwrite d1/d9/dd/d18/d1a/d44/fb4 [0,4194304] 0 2024-04-06T03:40:40.861 INFO:tasks.workunit.client.1.smithi136.stdout:4/576: mkdir d1d/d31/d3e/dd2 0 2024-04-06T03:40:40.861 INFO:tasks.workunit.client.1.smithi136.stdout:2/692: read d3/d15/f50 [821098,117688] 0 2024-04-06T03:40:40.865 INFO:tasks.workunit.client.1.smithi136.stdout:9/628: creat d17/d19/da3/dc5/fd3 x:0 0 0 2024-04-06T03:40:40.865 INFO:tasks.workunit.client.1.smithi136.stdout:8/639: sync 2024-04-06T03:40:40.865 INFO:tasks.workunit.client.1.smithi136.stdout:1/600: sync 2024-04-06T03:40:40.865 INFO:tasks.workunit.client.1.smithi136.stdout:1/601: creat d12/d1b/d1e/d9f/fc6 x:0 0 0 2024-04-06T03:40:40.865 INFO:tasks.workunit.client.1.smithi136.stdout:1/602: chown d12/d3b/d74 9936 1 2024-04-06T03:40:40.867 INFO:tasks.workunit.client.1.smithi136.stdout:2/693: mknod d3/d4a/d9d/d8a/d93/d5f/cdf 0 2024-04-06T03:40:40.869 INFO:tasks.workunit.client.1.smithi136.stdout:9/629: rename d17/d20/d25/d3c/d55/d8a/c97 to d17/d20/d25/da4/dbb/cd4 0 2024-04-06T03:40:40.871 INFO:tasks.workunit.client.0.smithi094.stdout:9/582: link d2/dc/d21/d23/d2f/f91 d2/d2c/d99/fd9 0 2024-04-06T03:40:40.874 INFO:tasks.workunit.client.1.smithi136.stdout:9/630: mkdir d17/d20/d25/d71/dd5 0 2024-04-06T03:40:40.876 INFO:tasks.workunit.client.1.smithi136.stdout:4/577: write d1d/d31/d78/fa9 [2493032,113610] 0 2024-04-06T03:40:40.878 INFO:tasks.workunit.client.0.smithi094.stdout:0/632: mknod d1/d9/dd/d18/d1e/d22/d35/dac/cd2 0 2024-04-06T03:40:40.878 INFO:tasks.workunit.client.0.smithi094.stdout:0/633: read d1/d9/dd/d18/d1e/d86/fc5 [3481291,22755] 0 2024-04-06T03:40:40.879 INFO:tasks.workunit.client.0.smithi094.stdout:1/571: mkdir d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/dbe 0 2024-04-06T03:40:40.880 INFO:tasks.workunit.client.1.smithi136.stdout:3/578: sync 2024-04-06T03:40:40.883 INFO:tasks.workunit.client.1.smithi136.stdout:2/694: write d3/d15/fd5 [2697840,73424] 0 2024-04-06T03:40:40.884 INFO:tasks.workunit.client.0.smithi094.stdout:9/583: rmdir d2/dc/d10/d6a 39 2024-04-06T03:40:40.885 INFO:tasks.workunit.client.0.smithi094.stdout:9/584: chown d2/dc/d46/cbb 723 1 2024-04-06T03:40:40.885 INFO:tasks.workunit.client.1.smithi136.stdout:6/566: dwrite d2/d7/d13/f9a [0,4194304] 0 2024-04-06T03:40:40.887 INFO:tasks.workunit.client.1.smithi136.stdout:4/578: symlink d1d/d2f/da6/dbf/ld3 0 2024-04-06T03:40:40.888 INFO:tasks.workunit.client.1.smithi136.stdout:8/640: dwrite d3/da/f41 [0,4194304] 0 2024-04-06T03:40:40.890 INFO:tasks.workunit.client.1.smithi136.stdout:6/567: dread d2/d7/d26/d30/d6e/d58/fbe [0,4194304] 0 2024-04-06T03:40:40.890 INFO:tasks.workunit.client.1.smithi136.stdout:3/579: rename d0/d4/lbc to d0/d4/d12/d19/d2e/lc1 0 2024-04-06T03:40:40.891 INFO:tasks.workunit.client.1.smithi136.stdout:9/631: write d17/d20/d25/d3b/fb1 [2553541,112731] 0 2024-04-06T03:40:40.891 INFO:tasks.workunit.client.1.smithi136.stdout:9/632: readlink d17/d20/d25/ld2 0 2024-04-06T03:40:40.894 INFO:tasks.workunit.client.1.smithi136.stdout:4/579: write f1c [41356,1176] 0 2024-04-06T03:40:40.894 INFO:tasks.workunit.client.1.smithi136.stdout:1/603: dwrite d12/d3b/d60/d7e/d37/f4e [0,4194304] 0 2024-04-06T03:40:40.897 INFO:tasks.workunit.client.1.smithi136.stdout:8/641: creat d3/fc3 x:0 0 0 2024-04-06T03:40:40.898 INFO:tasks.workunit.client.0.smithi094.stdout:0/634: dwrite d1/d9/dd/d18/d32/d42/f45 [0,4194304] 0 2024-04-06T03:40:40.903 INFO:tasks.workunit.client.1.smithi136.stdout:2/695: dread d3/d4a/d9d/d18/f81 [0,4194304] 0 2024-04-06T03:40:40.903 INFO:tasks.workunit.client.1.smithi136.stdout:2/696: chown d3/d9/d3b/d87 646 1 2024-04-06T03:40:40.903 INFO:tasks.workunit.client.1.smithi136.stdout:2/697: readlink d3/d9/ld 0 2024-04-06T03:40:40.903 INFO:tasks.workunit.client.1.smithi136.stdout:2/698: fsync d3/d15/d19/d4e/fba 0 2024-04-06T03:40:40.903 INFO:tasks.workunit.client.1.smithi136.stdout:2/699: chown d3/d4a/d9d/d18/d1d/fd8 19891291 1 2024-04-06T03:40:40.908 INFO:tasks.workunit.client.1.smithi136.stdout:6/568: dread d2/d25/f3b [0,4194304] 0 2024-04-06T03:40:40.908 INFO:tasks.workunit.client.1.smithi136.stdout:1/604: creat d12/d1b/da7/db8/fc7 x:0 0 0 2024-04-06T03:40:40.908 INFO:tasks.workunit.client.1.smithi136.stdout:8/642: mkdir d3/de/d18/d7d/dc4 0 2024-04-06T03:40:40.908 INFO:tasks.workunit.client.1.smithi136.stdout:8/643: chown d3/da/d26/d35/d53/l9f 1 1 2024-04-06T03:40:40.909 INFO:tasks.workunit.client.0.smithi094.stdout:9/585: dwrite d2/dc/d21/d23/d5b/f89 [0,4194304] 0 2024-04-06T03:40:40.912 INFO:tasks.workunit.client.1.smithi136.stdout:3/580: rmdir d0/d4/d12/d19/d55/d6f/d82/d5d/d61 39 2024-04-06T03:40:40.912 INFO:tasks.workunit.client.1.smithi136.stdout:3/581: readlink d0/d4/d12/d19/d55/lab 0 2024-04-06T03:40:40.912 INFO:tasks.workunit.client.1.smithi136.stdout:3/582: dread - d0/d4/d12/d19/d55/d3c/d5a/f9a zero size 2024-04-06T03:40:40.915 INFO:tasks.workunit.client.1.smithi136.stdout:1/605: symlink d12/d3b/d60/lc8 0 2024-04-06T03:40:40.916 INFO:tasks.workunit.client.1.smithi136.stdout:1/606: dread - d12/d3b/d74/d7a/fad zero size 2024-04-06T03:40:40.916 INFO:tasks.workunit.client.1.smithi136.stdout:1/607: fsync d12/d1b/f35 0 2024-04-06T03:40:40.917 INFO:tasks.workunit.client.0.smithi094.stdout:1/572: dwrite d0/d1/d27/da5/d40/db1/d2f/d4b/f8a [0,4194304] 0 2024-04-06T03:40:40.917 INFO:tasks.workunit.client.0.smithi094.stdout:0/635: dwrite d1/d9/dd/d18/d5d/d8d/fbf [0,4194304] 0 2024-04-06T03:40:40.918 INFO:tasks.workunit.client.0.smithi094.stdout:9/586: rename d2/c45 to d2/dc/d21/d26/d78/d69/cda 0 2024-04-06T03:40:40.921 INFO:tasks.workunit.client.1.smithi136.stdout:8/644: link d3/da/d26/d88/fa2 d3/de/fc5 0 2024-04-06T03:40:40.921 INFO:tasks.workunit.client.1.smithi136.stdout:8/645: write d3/da/d26/d35/d53/d86/f8f [5077421,8378] 0 2024-04-06T03:40:40.921 INFO:tasks.workunit.client.1.smithi136.stdout:8/646: readlink d3/de/d60/la4 0 2024-04-06T03:40:40.921 INFO:tasks.workunit.client.1.smithi136.stdout:8/647: creat d3/da/d26/d35/d53/fc6 x:0 0 0 2024-04-06T03:40:40.921 INFO:tasks.workunit.client.1.smithi136.stdout:4/580: rename d1d/d31/d3e/d55/da0 to d1d/d31/d3e/dd4 0 2024-04-06T03:40:40.922 INFO:tasks.workunit.client.1.smithi136.stdout:4/581: fsync d1d/d2f/da6/fcf 0 2024-04-06T03:40:40.922 INFO:tasks.workunit.client.1.smithi136.stdout:7/648: sync 2024-04-06T03:40:40.922 INFO:tasks.workunit.client.1.smithi136.stdout:7/649: write d5/df/d1d/fc6 [286583,87432] 0 2024-04-06T03:40:40.922 INFO:tasks.workunit.client.1.smithi136.stdout:7/650: truncate d5/df/d53/d68/fb0 566144 0 2024-04-06T03:40:40.924 INFO:tasks.workunit.client.1.smithi136.stdout:7/651: dread d5/df/d12/d1b/d3a/d4a/d70/ddb/f81 [0,4194304] 0 2024-04-06T03:40:40.928 INFO:tasks.workunit.client.1.smithi136.stdout:8/648: truncate d3/da/d26/d35/d3d/d48/f72 1323784 0 2024-04-06T03:40:40.928 INFO:tasks.workunit.client.1.smithi136.stdout:8/649: write d3/f5e [1706801,29959] 0 2024-04-06T03:40:40.928 INFO:tasks.workunit.client.1.smithi136.stdout:8/650: chown d3/f10 68150727 1 2024-04-06T03:40:40.928 INFO:tasks.workunit.client.1.smithi136.stdout:3/583: dwrite d0/d4/d12/d3a/f4b [0,4194304] 0 2024-04-06T03:40:40.928 INFO:tasks.workunit.client.1.smithi136.stdout:6/569: dwrite d2/d7/d26/d68/f98 [0,4194304] 0 2024-04-06T03:40:40.928 INFO:tasks.workunit.client.1.smithi136.stdout:3/584: write d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/f9b [663084,129191] 0 2024-04-06T03:40:40.930 INFO:tasks.workunit.client.1.smithi136.stdout:3/585: dread d0/d4/d12/d19/d2c/f9c [0,4194304] 0 2024-04-06T03:40:40.930 INFO:tasks.workunit.client.0.smithi094.stdout:1/573: read d0/d1/d27/da5/d40/db1/d2f/d76/d30/f32 [2779604,98938] 0 2024-04-06T03:40:40.931 INFO:tasks.workunit.client.1.smithi136.stdout:3/586: dread d0/d4/d12/d2d/d37/f6d [0,4194304] 0 2024-04-06T03:40:40.933 INFO:tasks.workunit.client.0.smithi094.stdout:1/574: dread d0/d1/f65 [0,4194304] 0 2024-04-06T03:40:40.933 INFO:tasks.workunit.client.0.smithi094.stdout:1/575: dread d0/d1/d27/da5/d40/db1/f55 [0,4194304] 0 2024-04-06T03:40:40.933 INFO:tasks.workunit.client.0.smithi094.stdout:1/576: dread - d0/d1/fbc zero size 2024-04-06T03:40:40.934 INFO:tasks.workunit.client.0.smithi094.stdout:0/636: truncate d1/d9/dd/d18/d1e/d86/fc5 34309 0 2024-04-06T03:40:40.936 INFO:tasks.workunit.client.1.smithi136.stdout:4/582: rename f1b to d1d/d31/d3e/d56/d7b/fd5 0 2024-04-06T03:40:40.937 INFO:tasks.workunit.client.0.smithi094.stdout:9/587: stat d2/dc/d10/c53 0 2024-04-06T03:40:40.937 INFO:tasks.workunit.client.0.smithi094.stdout:9/588: stat d2/dc/d21/d23/d47/f9b 0 2024-04-06T03:40:40.938 INFO:tasks.workunit.client.1.smithi136.stdout:1/608: mknod d12/d3b/d60/d7e/cc9 0 2024-04-06T03:40:40.938 INFO:tasks.workunit.client.1.smithi136.stdout:6/570: write d2/d7/d26/d30/d6e/d3d/f46 [8156640,52470] 0 2024-04-06T03:40:40.939 INFO:tasks.workunit.client.1.smithi136.stdout:3/587: read d0/d4/d12/d19/d2e/d3b/d49/f5e [2097465,3305] 0 2024-04-06T03:40:40.940 INFO:tasks.workunit.client.0.smithi094.stdout:1/577: getdents d0/d1/d27/da5/d40/db1/d2f/d76 0 2024-04-06T03:40:40.941 INFO:tasks.workunit.client.1.smithi136.stdout:7/652: symlink d5/df/d12/d1b/d3a/d4a/le2 0 2024-04-06T03:40:40.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/589: unlink d2/dc/d21/d26/d78/d69/d6d/f3b 0 2024-04-06T03:40:40.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/590: dread - d2/dc/d10/fb2 zero size 2024-04-06T03:40:40.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/591: dread - d2/dc/d10/fb2 zero size 2024-04-06T03:40:40.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/592: read d2/d2c/d99/f9e [925830,47718] 0 2024-04-06T03:40:40.945 INFO:tasks.workunit.client.1.smithi136.stdout:7/653: write d5/df/d11/fe1 [816303,125817] 0 2024-04-06T03:40:40.945 INFO:tasks.workunit.client.1.smithi136.stdout:8/651: mkdir d3/da/d26/d35/d3d/dc7 0 2024-04-06T03:40:40.948 INFO:tasks.workunit.client.0.smithi094.stdout:9/593: creat d2/d2c/d33/d64/dc6/fdb x:0 0 0 2024-04-06T03:40:40.949 INFO:tasks.workunit.client.1.smithi136.stdout:6/571: write d2/d7/d26/d30/d6e/f69 [1843157,13510] 0 2024-04-06T03:40:40.950 INFO:tasks.workunit.client.1.smithi136.stdout:6/572: creat d2/d7/d26/d30/d6e/db7/fbf x:0 0 0 2024-04-06T03:40:40.954 INFO:tasks.workunit.client.1.smithi136.stdout:4/583: symlink d1d/d31/d3e/d55/d89/db3/ld6 0 2024-04-06T03:40:40.956 INFO:tasks.workunit.client.0.smithi094.stdout:0/637: dwrite d1/d9/dd/d18/d1a/f7e [0,4194304] 0 2024-04-06T03:40:40.956 INFO:tasks.workunit.client.1.smithi136.stdout:4/584: write d1d/d2f/f43 [372630,36453] 0 2024-04-06T03:40:40.956 INFO:tasks.workunit.client.1.smithi136.stdout:1/609: link d12/d53/f95 d12/d1b/d1e/d6f/fca 0 2024-04-06T03:40:40.959 INFO:tasks.workunit.client.1.smithi136.stdout:3/588: symlink d0/d4/d12/d3a/d69/lc2 0 2024-04-06T03:40:40.959 INFO:tasks.workunit.client.1.smithi136.stdout:3/589: write d0/d4/d12/d19/d2c/f90 [425504,88480] 0 2024-04-06T03:40:40.960 INFO:tasks.workunit.client.1.smithi136.stdout:7/654: symlink d5/df/d12/d1b/dbb/dbe/le3 0 2024-04-06T03:40:40.967 INFO:tasks.workunit.client.1.smithi136.stdout:7/655: getdents d5/df/d1d/d6f/d4d 0 2024-04-06T03:40:40.970 INFO:tasks.workunit.client.1.smithi136.stdout:0/610: sync 2024-04-06T03:40:40.970 INFO:tasks.workunit.client.1.smithi136.stdout:5/640: sync 2024-04-06T03:40:40.974 INFO:tasks.workunit.client.1.smithi136.stdout:4/585: write d1d/d2f/f97 [4725622,15975] 0 2024-04-06T03:40:40.974 INFO:tasks.workunit.client.1.smithi136.stdout:4/586: write d1d/d31/f67 [4829730,47758] 0 2024-04-06T03:40:40.974 INFO:tasks.workunit.client.1.smithi136.stdout:4/587: fsync d1d/d31/d78/fa9 0 2024-04-06T03:40:40.974 INFO:tasks.workunit.client.1.smithi136.stdout:4/588: dread - d1d/d31/d3e/d56/f81 zero size 2024-04-06T03:40:40.974 INFO:tasks.workunit.client.1.smithi136.stdout:6/573: dwrite d2/d7/d26/d30/d6e/d58/d5d/d62/d53/fa1 [0,4194304] 0 2024-04-06T03:40:40.975 INFO:tasks.workunit.client.0.smithi094.stdout:0/638: dread d1/d9/fc [0,4194304] 0 2024-04-06T03:40:40.975 INFO:tasks.workunit.client.0.smithi094.stdout:0/639: truncate d1/d9/dd/d18/d1e/d22/d35/dac/faf 202869 0 2024-04-06T03:40:40.975 INFO:tasks.workunit.client.1.smithi136.stdout:6/574: read d2/d7/d26/d30/d6e/f6b [1026633,93866] 0 2024-04-06T03:40:40.976 INFO:tasks.workunit.client.1.smithi136.stdout:5/641: rmdir d1/dc/d1c/d32/d8b/d20 39 2024-04-06T03:40:40.976 INFO:tasks.workunit.client.1.smithi136.stdout:3/590: dread d0/d4/f22 [4194304,4194304] 0 2024-04-06T03:40:40.976 INFO:tasks.workunit.client.1.smithi136.stdout:3/591: stat d0/d4/d12/d2d/d37 0 2024-04-06T03:40:40.977 INFO:tasks.workunit.client.0.smithi094.stdout:0/640: creat d1/d9/dd/d18/d32/d42/fd3 x:0 0 0 2024-04-06T03:40:40.978 INFO:tasks.workunit.client.0.smithi094.stdout:1/578: dwrite d0/f93 [4194304,4194304] 0 2024-04-06T03:40:40.980 INFO:tasks.workunit.client.0.smithi094.stdout:0/641: truncate d1/d9/dd/d18/d5d/d60/fc1 2698519 0 2024-04-06T03:40:40.982 INFO:tasks.workunit.client.1.smithi136.stdout:1/610: dwrite d12/d53/f66 [0,4194304] 0 2024-04-06T03:40:40.983 INFO:tasks.workunit.client.0.smithi094.stdout:1/579: creat d0/d1/d27/da5/d40/d50/fbf x:0 0 0 2024-04-06T03:40:40.985 INFO:tasks.workunit.client.1.smithi136.stdout:8/652: dwrite d3/da/d26/d35/f89 [8388608,4194304] 0 2024-04-06T03:40:40.985 INFO:tasks.workunit.client.1.smithi136.stdout:8/653: fsync d3/da/d26/d35/d53/d86/f87 0 2024-04-06T03:40:40.985 INFO:tasks.workunit.client.1.smithi136.stdout:7/656: dwrite d5/df/d1d/d6f/d2a/f5d [0,4194304] 0 2024-04-06T03:40:40.985 INFO:tasks.workunit.client.0.smithi094.stdout:9/594: dwrite d2/d2c/d33/f5c [0,4194304] 0 2024-04-06T03:40:40.989 INFO:tasks.workunit.client.0.smithi094.stdout:0/642: rename d1/d9/dd/d18/d1e/d86/d96/fa4 to d1/d9/fd4 0 2024-04-06T03:40:40.994 INFO:tasks.workunit.client.1.smithi136.stdout:6/575: symlink d2/d7/d26/d30/d4f/lc0 0 2024-04-06T03:40:40.994 INFO:tasks.workunit.client.1.smithi136.stdout:6/576: write d2/d7/d26/d30/d6e/d85/fb5 [1033032,79695] 0 2024-04-06T03:40:40.994 INFO:tasks.workunit.client.1.smithi136.stdout:6/577: read d2/d7/d26/d30/d6e/f10 [1532023,115849] 0 2024-04-06T03:40:40.994 INFO:tasks.workunit.client.0.smithi094.stdout:0/643: rename d1/d9/dd/d18/d5d/d60/d6b/fd1 to d1/d9/dd/d18/d1e/d22/fd5 0 2024-04-06T03:40:40.996 INFO:tasks.workunit.client.1.smithi136.stdout:1/611: read d12/d1b/d1e/d23/fa3 [694960,76796] 0 2024-04-06T03:40:40.996 INFO:tasks.workunit.client.1.smithi136.stdout:1/612: fdatasync d12/d1b/d28/f3f 0 2024-04-06T03:40:40.996 INFO:tasks.workunit.client.1.smithi136.stdout:1/613: creat d12/d3b/d74/fcb x:0 0 0 2024-04-06T03:40:40.996 INFO:tasks.workunit.client.1.smithi136.stdout:5/642: link d1/d1f/l8c d1/d16/d89/lc3 0 2024-04-06T03:40:40.998 INFO:tasks.workunit.client.1.smithi136.stdout:3/592: mkdir d0/d4/d12/dc3 0 2024-04-06T03:40:41.000 INFO:tasks.workunit.client.1.smithi136.stdout:4/589: getdents d1d/d2f/d6c/da5 0 2024-04-06T03:40:41.000 INFO:tasks.workunit.client.1.smithi136.stdout:4/590: truncate d1d/d70/f94 173807 0 2024-04-06T03:40:41.002 INFO:tasks.workunit.client.0.smithi094.stdout:0/644: write d1/d9/dd/d18/d1a/f4d [4557381,13674] 0 2024-04-06T03:40:41.002 INFO:tasks.workunit.client.0.smithi094.stdout:1/580: dwrite d0/d7/d1a/da8/fa2 [0,4194304] 0 2024-04-06T03:40:41.004 INFO:tasks.workunit.client.0.smithi094.stdout:0/645: creat d1/d9/dd/d18/d1e/d86/d96/fd6 x:0 0 0 2024-04-06T03:40:41.006 INFO:tasks.workunit.client.0.smithi094.stdout:9/595: dwrite d2/dc/d21/d23/d47/d51/f8c [0,4194304] 0 2024-04-06T03:40:41.008 INFO:tasks.workunit.client.1.smithi136.stdout:8/654: unlink d3/de/c34 0 2024-04-06T03:40:41.010 INFO:tasks.workunit.client.0.smithi094.stdout:1/581: getdents d0/d1/d27/da5/db2 0 2024-04-06T03:40:41.010 INFO:tasks.workunit.client.0.smithi094.stdout:1/582: creat d0/d4/fc0 x:0 0 0 2024-04-06T03:40:41.011 INFO:tasks.workunit.client.0.smithi094.stdout:0/646: mknod d1/d9/dd/d18/d1e/d86/d96/dc0/cd7 0 2024-04-06T03:40:41.011 INFO:tasks.workunit.client.0.smithi094.stdout:0/647: truncate d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/fb1 404552 0 2024-04-06T03:40:41.012 INFO:tasks.workunit.client.1.smithi136.stdout:7/657: rename d5/df/d12/d1b/dbb/dbe/le3 to d5/df/d53/le4 0 2024-04-06T03:40:41.016 INFO:tasks.workunit.client.1.smithi136.stdout:6/578: creat d2/d92/fc1 x:0 0 0 2024-04-06T03:40:41.020 INFO:tasks.workunit.client.0.smithi094.stdout:1/583: symlink d0/d1/d27/da5/d40/db1/d2f/d76/d7d/lc1 0 2024-04-06T03:40:41.022 INFO:tasks.workunit.client.1.smithi136.stdout:1/614: mkdir d12/d1b/da7/dcc 0 2024-04-06T03:40:41.022 INFO:tasks.workunit.client.1.smithi136.stdout:1/615: write d12/d3b/d44/d5f/f63 [16562,127307] 0 2024-04-06T03:40:41.024 INFO:tasks.workunit.client.0.smithi094.stdout:0/648: mkdir d1/d9/dd/d18/dd8 0 2024-04-06T03:40:41.024 INFO:tasks.workunit.client.0.smithi094.stdout:0/649: write d1/d9/dd/d18/d1e/d86/d96/daa/fca [885805,15417] 0 2024-04-06T03:40:41.026 INFO:tasks.workunit.client.0.smithi094.stdout:1/584: symlink d0/d1/d27/da5/d40/db1/d2f/d4b/lc2 0 2024-04-06T03:40:41.028 INFO:tasks.workunit.client.1.smithi136.stdout:3/593: mknod d0/d4/d12/d3a/d51/da6/dbb/cc4 0 2024-04-06T03:40:41.029 INFO:tasks.workunit.client.1.smithi136.stdout:4/591: unlink d1d/d31/d3e/d55/d7f/fc3 0 2024-04-06T03:40:41.029 INFO:tasks.workunit.client.0.smithi094.stdout:0/650: dread d1/d9/dd/d18/d1a/f4d [4194304,4194304] 0 2024-04-06T03:40:41.031 INFO:tasks.workunit.client.1.smithi136.stdout:6/579: dwrite d2/d7/d26/d30/d6e/d58/d5d/f65 [0,4194304] 0 2024-04-06T03:40:41.032 INFO:tasks.workunit.client.1.smithi136.stdout:8/655: mknod d3/d2d/dad/cc8 0 2024-04-06T03:40:41.032 INFO:tasks.workunit.client.1.smithi136.stdout:5/643: dread d1/dc/f5b [0,4194304] 0 2024-04-06T03:40:41.033 INFO:tasks.workunit.client.1.smithi136.stdout:7/658: creat d5/df/d53/d68/da3/fe5 x:0 0 0 2024-04-06T03:40:41.037 INFO:tasks.workunit.client.1.smithi136.stdout:1/616: rename d12/d3b/d44/d5f/d5a/c8a to d12/ccd 0 2024-04-06T03:40:41.039 INFO:tasks.workunit.client.1.smithi136.stdout:3/594: link d0/d4/d12/d3a/d51/l63 d0/d4/d12/d19/d2c/lc5 0 2024-04-06T03:40:41.041 INFO:tasks.workunit.client.1.smithi136.stdout:7/659: dread d5/df/d12/d1b/d3a/d4a/d70/fc0 [0,4194304] 0 2024-04-06T03:40:41.044 INFO:tasks.workunit.client.0.smithi094.stdout:1/585: rename d0/d4/d37 to d0/d1/d27/da5/d40/db1/dc3 0 2024-04-06T03:40:41.044 INFO:tasks.workunit.client.0.smithi094.stdout:1/586: stat d0/d1/d27/da5/d40/db1/d2f 0 2024-04-06T03:40:41.044 INFO:tasks.workunit.client.0.smithi094.stdout:1/587: write d0/d1/d27/da5/d40/db1/d2f/d3c/d48/f69 [1859502,39739] 0 2024-04-06T03:40:41.044 INFO:tasks.workunit.client.0.smithi094.stdout:1/588: chown d0/d1/d27/da5/d40/db1/d2f/d3c/da0 99328 1 2024-04-06T03:40:41.045 INFO:tasks.workunit.client.0.smithi094.stdout:0/651: mknod d1/d9/dd/d18/d1a/d44/cd9 0 2024-04-06T03:40:41.047 INFO:tasks.workunit.client.0.smithi094.stdout:1/589: creat d0/d1/d27/da5/fc4 x:0 0 0 2024-04-06T03:40:41.049 INFO:tasks.workunit.client.1.smithi136.stdout:8/656: dwrite d3/da/fb [4194304,4194304] 0 2024-04-06T03:40:41.049 INFO:tasks.workunit.client.1.smithi136.stdout:8/657: fsync d3/da/d26/d35/f3c 0 2024-04-06T03:40:41.049 INFO:tasks.workunit.client.1.smithi136.stdout:8/658: chown d3/de/d18/f1e 656 1 2024-04-06T03:40:41.050 INFO:tasks.workunit.client.1.smithi136.stdout:6/580: mkdir d2/d7/d13/d79/dc2 0 2024-04-06T03:40:41.054 INFO:tasks.workunit.client.1.smithi136.stdout:4/592: rename d1d/d31/d3e/d56/d7b/f77 to d1d/d31/d3e/d55/dba/fd7 0 2024-04-06T03:40:41.054 INFO:tasks.workunit.client.1.smithi136.stdout:4/593: write d1d/d2f/d6c/f7e [340970,98346] 0 2024-04-06T03:40:41.056 INFO:tasks.workunit.client.1.smithi136.stdout:1/617: creat d12/d3b/d44/d5f/d5a/fce x:0 0 0 2024-04-06T03:40:41.056 INFO:tasks.workunit.client.1.smithi136.stdout:1/618: dread - d12/d1b/d1e/d23/f92 zero size 2024-04-06T03:40:41.058 INFO:tasks.workunit.client.1.smithi136.stdout:3/595: mkdir d0/d4/d12/d19/d55/d3c/d5a/d79/d98/dc6 0 2024-04-06T03:40:41.058 INFO:tasks.workunit.client.1.smithi136.stdout:7/660: creat d5/df/d11/fe6 x:0 0 0 2024-04-06T03:40:41.062 INFO:tasks.workunit.client.1.smithi136.stdout:5/644: creat d1/dc/d10/fc4 x:0 0 0 2024-04-06T03:40:41.063 INFO:tasks.workunit.client.1.smithi136.stdout:5/645: read d1/dc/d1c/d32/d8b/d20/d66/fb9 [538396,18067] 0 2024-04-06T03:40:41.065 INFO:tasks.workunit.client.1.smithi136.stdout:8/659: getdents d3/da/d26/d35/d53 0 2024-04-06T03:40:41.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/590: dwrite d0/d1/d27/da5/d40/db1/f55 [0,4194304] 0 2024-04-06T03:40:41.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/591: getdents d0/d7/d2d/dab 0 2024-04-06T03:40:41.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/592: write d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/f9c [55538,49879] 0 2024-04-06T03:40:41.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/593: readlink d0/d1/d27/da5/d40/d50/l6a 0 2024-04-06T03:40:41.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/594: stat d0/d1/d27/da5/d40/db1/d2f/d76/l99 0 2024-04-06T03:40:41.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/595: stat d0/d7/f26 0 2024-04-06T03:40:41.067 INFO:tasks.workunit.client.0.smithi094.stdout:1/596: chown d0/d1/d27/da5/d3f/c56 74724 1 2024-04-06T03:40:41.071 INFO:tasks.workunit.client.0.smithi094.stdout:4/663: sync 2024-04-06T03:40:41.073 INFO:tasks.workunit.client.1.smithi136.stdout:8/660: rmdir d3/da/d26/d35/d53/d86 39 2024-04-06T03:40:41.073 INFO:tasks.workunit.client.1.smithi136.stdout:8/661: fdatasync d3/de/f4c 0 2024-04-06T03:40:41.073 INFO:tasks.workunit.client.1.smithi136.stdout:8/662: stat d3/d2d/l40 0 2024-04-06T03:40:41.073 INFO:tasks.workunit.client.1.smithi136.stdout:2/700: sync 2024-04-06T03:40:41.073 INFO:tasks.workunit.client.1.smithi136.stdout:9/633: sync 2024-04-06T03:40:41.074 INFO:tasks.workunit.client.0.smithi094.stdout:1/597: unlink d0/d1/d27/da5/d40/db1/dc3/d5f/l85 0 2024-04-06T03:40:41.076 INFO:tasks.workunit.client.1.smithi136.stdout:5/646: write d1/d16/d17/f44 [987474,74618] 0 2024-04-06T03:40:41.076 INFO:tasks.workunit.client.0.smithi094.stdout:4/664: dread - d0/d77/f9e zero size 2024-04-06T03:40:41.076 INFO:tasks.workunit.client.0.smithi094.stdout:1/598: dread d0/d7/d2d/f61 [0,4194304] 0 2024-04-06T03:40:41.078 INFO:tasks.workunit.client.1.smithi136.stdout:8/663: rename d3/da/d26/d35/d3d/d48/d6c/dab/dba to d3/da/d26/d35/d3d/d48/dc9 0 2024-04-06T03:40:41.081 INFO:tasks.workunit.client.1.smithi136.stdout:2/701: unlink d3/d15/f90 0 2024-04-06T03:40:41.081 INFO:tasks.workunit.client.1.smithi136.stdout:1/619: dwrite d12/d1b/d1e/d4d/f68 [0,4194304] 0 2024-04-06T03:40:41.081 INFO:tasks.workunit.client.1.smithi136.stdout:7/661: dwrite d5/df/d12/d1b/d3a/d4a/d70/ddb/faf [0,4194304] 0 2024-04-06T03:40:41.081 INFO:tasks.workunit.client.1.smithi136.stdout:3/596: dwrite d0/d4/d12/f15 [4194304,4194304] 0 2024-04-06T03:40:41.081 INFO:tasks.workunit.client.0.smithi094.stdout:4/665: chown d0/d15/d3a/d3e/d49/f73 73335 1 2024-04-06T03:40:41.081 INFO:tasks.workunit.client.0.smithi094.stdout:4/666: chown d0/d15/c97 53523942 1 2024-04-06T03:40:41.089 INFO:tasks.workunit.client.1.smithi136.stdout:9/634: rename d17/d19/f8e to d17/d20/d25/d3d/da5/fd6 0 2024-04-06T03:40:41.089 INFO:tasks.workunit.client.0.smithi094.stdout:4/667: creat d0/d77/dc4/fe2 x:0 0 0 2024-04-06T03:40:41.091 INFO:tasks.workunit.client.1.smithi136.stdout:8/664: symlink d3/de/d18/d7d/dc4/lca 0 2024-04-06T03:40:41.091 INFO:tasks.workunit.client.1.smithi136.stdout:8/665: write d3/da/d26/d35/f3c [1807351,21624] 0 2024-04-06T03:40:41.092 INFO:tasks.workunit.client.1.smithi136.stdout:8/666: truncate d3/d2d/f67 5130911 0 2024-04-06T03:40:41.092 INFO:tasks.workunit.client.1.smithi136.stdout:8/667: dread - d3/da/d26/d35/d39/f65 zero size 2024-04-06T03:40:41.092 INFO:tasks.workunit.client.0.smithi094.stdout:1/599: mknod d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/cc5 0 2024-04-06T03:40:41.092 INFO:tasks.workunit.client.0.smithi094.stdout:1/600: chown d0/d1/d27/da5/d40/db1/dc3/d5f/d70/l97 0 1 2024-04-06T03:40:41.092 INFO:tasks.workunit.client.0.smithi094.stdout:1/601: write d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/f81 [655417,39007] 0 2024-04-06T03:40:41.093 INFO:tasks.workunit.client.1.smithi136.stdout:0/611: sync 2024-04-06T03:40:41.093 INFO:tasks.workunit.client.0.smithi094.stdout:4/668: mknod d0/d15/d3a/ce3 0 2024-04-06T03:40:41.094 INFO:tasks.workunit.client.1.smithi136.stdout:5/647: dwrite d1/dc/d10/d1e/f97 [0,4194304] 0 2024-04-06T03:40:41.094 INFO:tasks.workunit.client.1.smithi136.stdout:5/648: chown d1/dc/d1c/d32/d8b/f73 5 1 2024-04-06T03:40:41.095 INFO:tasks.workunit.client.0.smithi094.stdout:1/602: link d0/d7/d1a/la7 d0/d1/d27/da5/d40/db1/lc6 0 2024-04-06T03:40:41.098 INFO:tasks.workunit.client.1.smithi136.stdout:7/662: creat d5/df/d12/d1b/fe7 x:0 0 0 2024-04-06T03:40:41.098 INFO:tasks.workunit.client.1.smithi136.stdout:7/663: creat d5/df/d53/d8a/fe8 x:0 0 0 2024-04-06T03:40:41.098 INFO:tasks.workunit.client.1.smithi136.stdout:7/664: dread - d5/df/d1d/f7f zero size 2024-04-06T03:40:41.098 INFO:tasks.workunit.client.1.smithi136.stdout:7/665: dread - d5/df/d3d/fab zero size 2024-04-06T03:40:41.098 INFO:tasks.workunit.client.1.smithi136.stdout:7/666: creat d5/df/d12/d59/d71/fe9 x:0 0 0 2024-04-06T03:40:41.098 INFO:tasks.workunit.client.1.smithi136.stdout:7/667: dread - d5/df/d12/d1b/d3a/d4a/f86 zero size 2024-04-06T03:40:41.099 INFO:tasks.workunit.client.0.smithi094.stdout:4/669: write d0/df/d57/f86 [1279910,116681] 0 2024-04-06T03:40:41.099 INFO:tasks.workunit.client.1.smithi136.stdout:1/620: mknod d12/d3b/d44/d5f/d5a/d72/d77/d96/ccf 0 2024-04-06T03:40:41.099 INFO:tasks.workunit.client.1.smithi136.stdout:1/621: write d12/d3b/d44/d5f/fa0 [247468,87519] 0 2024-04-06T03:40:41.101 INFO:tasks.workunit.client.0.smithi094.stdout:1/603: creat d0/d1/d27/da5/d40/fc7 x:0 0 0 2024-04-06T03:40:41.101 INFO:tasks.workunit.client.0.smithi094.stdout:1/604: chown d0/d1/d27/da5/d40/db1/f55 7664996 1 2024-04-06T03:40:41.102 INFO:tasks.workunit.client.0.smithi094.stdout:1/605: fdatasync d0/d7/d2d/f61 0 2024-04-06T03:40:41.104 INFO:tasks.workunit.client.0.smithi094.stdout:4/670: readlink d0/d4/l1d 0 2024-04-06T03:40:41.106 INFO:tasks.workunit.client.1.smithi136.stdout:3/597: symlink d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/lc7 0 2024-04-06T03:40:41.108 INFO:tasks.workunit.client.0.smithi094.stdout:4/671: symlink d0/d4/d67/dcb/le4 0 2024-04-06T03:40:41.108 INFO:tasks.workunit.client.1.smithi136.stdout:2/702: dwrite d3/d15/d19/d4e/fba [0,4194304] 0 2024-04-06T03:40:41.108 INFO:tasks.workunit.client.1.smithi136.stdout:8/668: dwrite d3/da/d26/d35/d3d/f64 [0,4194304] 0 2024-04-06T03:40:41.111 INFO:tasks.workunit.client.1.smithi136.stdout:9/635: creat d17/d21/d29/d34/d46/d9a/fd7 x:0 0 0 2024-04-06T03:40:41.111 INFO:tasks.workunit.client.1.smithi136.stdout:9/636: truncate d17/d20/d25/f31 1032485 0 2024-04-06T03:40:41.112 INFO:tasks.workunit.client.1.smithi136.stdout:8/669: dread d3/f5e [0,4194304] 0 2024-04-06T03:40:41.114 INFO:tasks.workunit.client.0.smithi094.stdout:4/672: symlink d0/df/d1e/d34/db6/le5 0 2024-04-06T03:40:41.124 INFO:tasks.workunit.client.1.smithi136.stdout:3/598: dwrite d0/d4/d12/d19/d55/d3c/d5a/fac [4194304,4194304] 0 2024-04-06T03:40:41.128 INFO:tasks.workunit.client.0.smithi094.stdout:4/673: dwrite d0/d15/f8f [0,4194304] 0 2024-04-06T03:40:41.129 INFO:tasks.workunit.client.1.smithi136.stdout:0/612: dwrite d1/d6/d2c/f27 [0,4194304] 0 2024-04-06T03:40:41.189 INFO:tasks.workunit.client.1.smithi136.stdout:5/649: link d1/d1f/fbb d1/d98/dbc/fc5 0 2024-04-06T03:40:41.192 INFO:tasks.workunit.client.0.smithi094.stdout:4/674: creat d0/df/d1e/d34/db5/fe6 x:0 0 0 2024-04-06T03:40:41.194 INFO:tasks.workunit.client.1.smithi136.stdout:7/668: symlink d5/df/d12/d1b/d3a/d8d/lea 0 2024-04-06T03:40:41.194 INFO:tasks.workunit.client.1.smithi136.stdout:7/669: chown d5/df/d12/d1b/d3a/d4a/c88 58055 1 2024-04-06T03:40:41.196 INFO:tasks.workunit.client.1.smithi136.stdout:1/622: unlink d12/d3b/d74/fb4 0 2024-04-06T03:40:41.196 INFO:tasks.workunit.client.1.smithi136.stdout:5/650: write d1/dc/d1c/d32/f38 [2156745,91383] 0 2024-04-06T03:40:41.202 INFO:tasks.workunit.client.1.smithi136.stdout:2/703: mknod d3/d9/d3b/d87/ce0 0 2024-04-06T03:40:41.202 INFO:tasks.workunit.client.1.smithi136.stdout:2/704: stat d3/d4a/d9d/d18/d4b 0 2024-04-06T03:40:41.204 INFO:tasks.workunit.client.1.smithi136.stdout:5/651: dread d1/dc/d1c/d32/f8f [0,4194304] 0 2024-04-06T03:40:41.204 INFO:tasks.workunit.client.1.smithi136.stdout:8/670: unlink d3/da/d26/d35/d3d/d48/d6c/d79/l95 0 2024-04-06T03:40:41.204 INFO:tasks.workunit.client.1.smithi136.stdout:8/671: read - d3/da/d26/d35/d3d/d48/f92 zero size 2024-04-06T03:40:41.204 INFO:tasks.workunit.client.0.smithi094.stdout:4/675: dwrite d0/df/d1e/d34/db5/fe6 [0,4194304] 0 2024-04-06T03:40:41.207 INFO:tasks.workunit.client.1.smithi136.stdout:3/599: creat d0/d4/d12/dc3/fc8 x:0 0 0 2024-04-06T03:40:41.209 INFO:tasks.workunit.client.0.smithi094.stdout:4/676: write d0/d23/dc1/fcd [1017276,85268] 0 2024-04-06T03:40:41.209 INFO:tasks.workunit.client.0.smithi094.stdout:4/677: readlink d0/d4/d67/l69 0 2024-04-06T03:40:41.217 INFO:tasks.workunit.client.1.smithi136.stdout:0/613: rename d1/d6/d2c/d57/d43/d6d/d9b/d5c/f71 to d1/fc9 0 2024-04-06T03:40:41.218 INFO:tasks.workunit.client.0.smithi094.stdout:4/678: write d0/df/f19 [6589127,43728] 0 2024-04-06T03:40:41.218 INFO:tasks.workunit.client.0.smithi094.stdout:4/679: readlink d0/d15/d3a/d3e/d46/l63 0 2024-04-06T03:40:41.220 INFO:tasks.workunit.client.1.smithi136.stdout:7/670: link d5/c35 d5/df/d1d/d6f/d2a/ceb 0 2024-04-06T03:40:41.221 INFO:tasks.workunit.client.0.smithi094.stdout:4/680: fsync d0/d4/d67/d9f/fa9 0 2024-04-06T03:40:41.221 INFO:tasks.workunit.client.0.smithi094.stdout:4/681: chown d0/df/d13/d4a/l78 466 1 2024-04-06T03:40:41.221 INFO:tasks.workunit.client.1.smithi136.stdout:2/705: symlink d3/d15/d19/d43/d54/le1 0 2024-04-06T03:40:41.223 INFO:tasks.workunit.client.1.smithi136.stdout:5/652: mkdir d1/d98/dc6 0 2024-04-06T03:40:41.223 INFO:tasks.workunit.client.1.smithi136.stdout:5/653: truncate d1/d16/d4b/f95 502865 0 2024-04-06T03:40:41.223 INFO:tasks.workunit.client.1.smithi136.stdout:5/654: fsync d1/f40 0 2024-04-06T03:40:41.223 INFO:tasks.workunit.client.1.smithi136.stdout:3/600: getdents d0/d4/d12/d19/d2e/d3b/d49 0 2024-04-06T03:40:41.226 INFO:tasks.workunit.client.0.smithi094.stdout:4/682: rmdir d0/d4/d67 39 2024-04-06T03:40:41.226 INFO:tasks.workunit.client.1.smithi136.stdout:0/614: write d1/d6/fa1 [4045705,49764] 0 2024-04-06T03:40:41.229 INFO:tasks.workunit.client.0.smithi094.stdout:4/683: dread d0/f22 [0,4194304] 0 2024-04-06T03:40:41.229 INFO:tasks.workunit.client.1.smithi136.stdout:6/581: sync 2024-04-06T03:40:41.229 INFO:tasks.workunit.client.1.smithi136.stdout:9/637: sync 2024-04-06T03:40:41.229 INFO:tasks.workunit.client.1.smithi136.stdout:4/594: sync 2024-04-06T03:40:41.229 INFO:tasks.workunit.client.1.smithi136.stdout:3/601: write d0/d4/f45 [646342,25705] 0 2024-04-06T03:40:41.229 INFO:tasks.workunit.client.1.smithi136.stdout:4/595: readlink d1d/d2f/lad 0 2024-04-06T03:40:41.230 INFO:tasks.workunit.client.1.smithi136.stdout:9/638: readlink d17/d20/d25/l2b 0 2024-04-06T03:40:41.230 INFO:tasks.workunit.client.1.smithi136.stdout:6/582: fsync d2/d7/d26/d30/d6e/d3d/d47/d84/dad/fbb 0 2024-04-06T03:40:41.230 INFO:tasks.workunit.client.1.smithi136.stdout:0/615: unlink d1/d6/d2c/d57/d43/d6d/d85/da7/fbd 0 2024-04-06T03:40:41.230 INFO:tasks.workunit.client.1.smithi136.stdout:1/623: rename d12/d3b/d6b to d12/d1b/d1e/dd0 0 2024-04-06T03:40:41.232 INFO:tasks.workunit.client.1.smithi136.stdout:5/655: dread d1/dc/d1c/d32/d8b/f9f [0,4194304] 0 2024-04-06T03:40:41.233 INFO:tasks.workunit.client.1.smithi136.stdout:5/656: readlink d1/dc/d1c/d32/d70/l74 0 2024-04-06T03:40:41.233 INFO:tasks.workunit.client.1.smithi136.stdout:5/657: dread - d1/dc/d1c/d32/d8b/d20/d66/fbf zero size 2024-04-06T03:40:41.233 INFO:tasks.workunit.client.1.smithi136.stdout:5/658: creat d1/dc/d1c/d32/fc7 x:0 0 0 2024-04-06T03:40:41.233 INFO:tasks.workunit.client.1.smithi136.stdout:5/659: write d1/dc/d1c/d32/f6a [4813309,39599] 0 2024-04-06T03:40:41.233 INFO:tasks.workunit.client.1.smithi136.stdout:3/602: unlink d0/d4/d91/c92 0 2024-04-06T03:40:41.235 INFO:tasks.workunit.client.1.smithi136.stdout:4/596: link d1d/d31/d3e/d56/c8c d1d/d70/cd8 0 2024-04-06T03:40:41.235 INFO:tasks.workunit.client.1.smithi136.stdout:9/639: dread d17/d20/d28/f82 [0,4194304] 0 2024-04-06T03:40:41.236 INFO:tasks.workunit.client.0.smithi094.stdout:2/636: sync 2024-04-06T03:40:41.238 INFO:tasks.workunit.client.0.smithi094.stdout:4/684: rename d0/df/c26 to d0/d4/ce7 0 2024-04-06T03:40:41.238 INFO:tasks.workunit.client.1.smithi136.stdout:7/671: dwrite d5/df/d11/f14 [0,4194304] 0 2024-04-06T03:40:41.240 INFO:tasks.workunit.client.1.smithi136.stdout:6/583: creat d2/d7/d9e/fc3 x:0 0 0 2024-04-06T03:40:41.242 INFO:tasks.workunit.client.1.smithi136.stdout:5/660: dread d1/dc/d1c/d32/f6a [4194304,4194304] 0 2024-04-06T03:40:41.242 INFO:tasks.workunit.client.1.smithi136.stdout:5/661: chown d1/dc/d1c/d32/d8b/d20/f41 0 1 2024-04-06T03:40:41.242 INFO:tasks.workunit.client.1.smithi136.stdout:5/662: chown d1/dc/d1c/d32/d8b/d20/l35 302843222 1 2024-04-06T03:40:41.242 INFO:tasks.workunit.client.1.smithi136.stdout:5/663: chown d1/dc/d1c/d32/d8b/db8 1 1 2024-04-06T03:40:41.243 INFO:tasks.workunit.client.0.smithi094.stdout:2/637: mknod d0/d8/d4f/dab/dc3/cda 0 2024-04-06T03:40:41.244 INFO:tasks.workunit.client.1.smithi136.stdout:5/664: write d1/dc/d1c/d32/d70/fa8 [816527,104681] 0 2024-04-06T03:40:41.244 INFO:tasks.workunit.client.1.smithi136.stdout:2/706: write d3/f83 [712534,68748] 0 2024-04-06T03:40:41.244 INFO:tasks.workunit.client.1.smithi136.stdout:0/616: mknod d1/d6/d2c/d1a/cca 0 2024-04-06T03:40:41.245 INFO:tasks.workunit.client.1.smithi136.stdout:5/665: dread d1/d16/d4b/f8e [0,4194304] 0 2024-04-06T03:40:41.245 INFO:tasks.workunit.client.1.smithi136.stdout:5/666: readlink d1/dc/d10/da3/lab 0 2024-04-06T03:40:41.245 INFO:tasks.workunit.client.1.smithi136.stdout:5/667: chown d1/dc/d1c 1628221349 1 2024-04-06T03:40:41.245 INFO:tasks.workunit.client.1.smithi136.stdout:5/668: stat d1/d16/d89/d8a/cb3 0 2024-04-06T03:40:41.247 INFO:tasks.workunit.client.1.smithi136.stdout:8/672: rename d3/de/d18/d7d/dc4 to d3/da/d26/d35/dcb 0 2024-04-06T03:40:41.247 INFO:tasks.workunit.client.1.smithi136.stdout:8/673: fsync d3/d2d/f67 0 2024-04-06T03:40:41.248 INFO:tasks.workunit.client.1.smithi136.stdout:1/624: unlink d12/d3b/d44/f57 0 2024-04-06T03:40:41.248 INFO:tasks.workunit.client.1.smithi136.stdout:1/625: fsync d12/d3b/d60/d7e/f42 0 2024-04-06T03:40:41.249 INFO:tasks.workunit.client.0.smithi094.stdout:2/638: rename d0/d8/cc to d0/dd/d19/d31/cdb 0 2024-04-06T03:40:41.249 INFO:tasks.workunit.client.0.smithi094.stdout:2/639: fdatasync d0/dd/d1b/d72/d8f/fc4 0 2024-04-06T03:40:41.250 INFO:tasks.workunit.client.1.smithi136.stdout:3/603: dwrite d0/d4/d12/d2d/fbe [0,4194304] 0 2024-04-06T03:40:41.250 INFO:tasks.workunit.client.1.smithi136.stdout:3/604: write d0/d4/d12/f5c [354491,108565] 0 2024-04-06T03:40:41.251 INFO:tasks.workunit.client.1.smithi136.stdout:3/605: write d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/f9b [1031537,34699] 0 2024-04-06T03:40:41.253 INFO:tasks.workunit.client.0.smithi094.stdout:4/685: mkdir d0/de8 0 2024-04-06T03:40:41.258 INFO:tasks.workunit.client.1.smithi136.stdout:4/597: stat d1d/d31/l50 0 2024-04-06T03:40:41.259 INFO:tasks.workunit.client.1.smithi136.stdout:7/672: mkdir d5/df/d53/d68/da3/dec 0 2024-04-06T03:40:41.259 INFO:tasks.workunit.client.1.smithi136.stdout:7/673: creat d5/df/d1d/d6f/d4d/d69/fed x:0 0 0 2024-04-06T03:40:41.259 INFO:tasks.workunit.client.1.smithi136.stdout:7/674: dread - d5/df/d53/d68/d8f/fc1 zero size 2024-04-06T03:40:41.261 INFO:tasks.workunit.client.0.smithi094.stdout:2/640: write d0/dd/d19/d31/f6e [632575,19836] 0 2024-04-06T03:40:41.261 INFO:tasks.workunit.client.0.smithi094.stdout:2/641: write d0/d18/f22 [2119823,23483] 0 2024-04-06T03:40:41.264 INFO:tasks.workunit.client.1.smithi136.stdout:2/707: mkdir d3/d9/d3b/d70/de2 0 2024-04-06T03:40:41.265 INFO:tasks.workunit.client.1.smithi136.stdout:2/708: dread - d3/d4a/d9d/d18/d4b/dbd/fc3 zero size 2024-04-06T03:40:41.265 INFO:tasks.workunit.client.1.smithi136.stdout:2/709: chown d3/d9/d3b/l55 988 1 2024-04-06T03:40:41.265 INFO:tasks.workunit.client.1.smithi136.stdout:2/710: chown d3/d9/d3b/l55 515270 1 2024-04-06T03:40:41.265 INFO:tasks.workunit.client.1.smithi136.stdout:0/617: symlink d1/d6/d2c/d57/d43/d6d/d85/lcb 0 2024-04-06T03:40:41.265 INFO:tasks.workunit.client.1.smithi136.stdout:0/618: chown d1/d6/d2c/d57/d43/ca3 5530590 1 2024-04-06T03:40:41.265 INFO:tasks.workunit.client.1.smithi136.stdout:0/619: stat d1/d6/cc4 0 2024-04-06T03:40:41.266 INFO:tasks.workunit.client.1.smithi136.stdout:9/640: rename d17/d19/f1f to d17/d20/d25/d71/dd5/fd8 0 2024-04-06T03:40:41.267 INFO:tasks.workunit.client.0.smithi094.stdout:4/686: dwrite d0/d23/f32 [0,4194304] 0 2024-04-06T03:40:41.267 INFO:tasks.workunit.client.1.smithi136.stdout:3/606: dwrite d0/f32 [0,4194304] 0 2024-04-06T03:40:41.271 INFO:tasks.workunit.client.1.smithi136.stdout:1/626: mkdir d12/d1b/d1e/dd1 0 2024-04-06T03:40:41.277 INFO:tasks.workunit.client.1.smithi136.stdout:7/675: truncate d5/df/d1d/d6f/d4d/d69/f7b 8262902 0 2024-04-06T03:40:41.278 INFO:tasks.workunit.client.1.smithi136.stdout:7/676: creat d5/df/d1d/d6f/d4d/d69/fee x:0 0 0 2024-04-06T03:40:41.279 INFO:tasks.workunit.client.1.smithi136.stdout:2/711: dwrite d3/d15/d19/dac/fb9 [0,4194304] 0 2024-04-06T03:40:41.280 INFO:tasks.workunit.client.1.smithi136.stdout:6/584: rmdir d2/d7/d26/d68 39 2024-04-06T03:40:41.280 INFO:tasks.workunit.client.1.smithi136.stdout:6/585: write d2/d7/d26/d30/d6e/db7/fbf [784425,108363] 0 2024-04-06T03:40:41.280 INFO:tasks.workunit.client.1.smithi136.stdout:7/677: dread d5/f2e [0,4194304] 0 2024-04-06T03:40:41.281 INFO:tasks.workunit.client.1.smithi136.stdout:7/678: chown d5/df/d12/d1b/d3a 1 1 2024-04-06T03:40:41.281 INFO:tasks.workunit.client.1.smithi136.stdout:7/679: write d5/df/d12/d1b/d58/f7a [116727,8198] 0 2024-04-06T03:40:41.282 INFO:tasks.workunit.client.1.smithi136.stdout:0/620: getdents d1/d6/d2c/d1a 0 2024-04-06T03:40:41.284 INFO:tasks.workunit.client.1.smithi136.stdout:5/669: rename d1/dc/d1c/d32/d70/fa8 to d1/d16/d89/daf/fc8 0 2024-04-06T03:40:41.287 INFO:tasks.workunit.client.1.smithi136.stdout:9/641: symlink d17/d19/da3/dc5/ld9 0 2024-04-06T03:40:41.287 INFO:tasks.workunit.client.1.smithi136.stdout:9/642: fsync d17/d20/d25/d3d/da5/fd6 0 2024-04-06T03:40:41.293 INFO:tasks.workunit.client.1.smithi136.stdout:3/607: mkdir d0/d4/d12/d19/d2c/dc9 0 2024-04-06T03:40:41.294 INFO:tasks.workunit.client.1.smithi136.stdout:8/674: symlink d3/da/d26/d35/d53/d86/db0/lcc 0 2024-04-06T03:40:41.294 INFO:tasks.workunit.client.1.smithi136.stdout:8/675: readlink d3/da/d26/d35/l46 0 2024-04-06T03:40:41.297 INFO:tasks.workunit.client.0.smithi094.stdout:4/687: dwrite d0/d15/d3a/d3e/d49/db2/fc2 [0,4194304] 0 2024-04-06T03:40:41.297 INFO:tasks.workunit.client.1.smithi136.stdout:1/627: dwrite d12/d1b/d28/f3f [0,4194304] 0 2024-04-06T03:40:41.298 INFO:tasks.workunit.client.1.smithi136.stdout:1/628: chown d12/d3b/d60/d7e/cc9 9 1 2024-04-06T03:40:41.300 INFO:tasks.workunit.client.1.smithi136.stdout:7/680: link d5/df/d12/d1b/f30 d5/df/d53/d68/da3/dec/fef 0 2024-04-06T03:40:41.300 INFO:tasks.workunit.client.1.smithi136.stdout:7/681: stat d5/df/d12/d1b/d3a/d8d 0 2024-04-06T03:40:41.302 INFO:tasks.workunit.client.1.smithi136.stdout:0/621: rename d1/d4e/l7d to d1/d6/d2c/d57/d43/lcc 0 2024-04-06T03:40:41.304 INFO:tasks.workunit.client.1.smithi136.stdout:5/670: truncate d1/dc/d1c/d32/d8b/d20/d66/f9e 1360844 0 2024-04-06T03:40:41.304 INFO:tasks.workunit.client.1.smithi136.stdout:5/671: fdatasync d1/dc/d1c/d32/fc7 0 2024-04-06T03:40:41.305 INFO:tasks.workunit.client.1.smithi136.stdout:5/672: stat d1/d16/d4b/f4f 0 2024-04-06T03:40:41.305 INFO:tasks.workunit.client.1.smithi136.stdout:9/643: creat d17/d20/d25/d3d/da5/dce/dd1/fda x:0 0 0 2024-04-06T03:40:41.306 INFO:tasks.workunit.client.1.smithi136.stdout:9/644: write d17/d20/d25/d3c/f73 [3254384,117078] 0 2024-04-06T03:40:41.306 INFO:tasks.workunit.client.1.smithi136.stdout:1/629: write d12/d3b/d44/d5f/d5a/f62 [232107,54336] 0 2024-04-06T03:40:41.306 INFO:tasks.workunit.client.0.smithi094.stdout:4/688: rmdir d0/d15/d3a/d3e/d49/db2/dc7 39 2024-04-06T03:40:41.307 INFO:tasks.workunit.client.0.smithi094.stdout:4/689: write d0/d15/d3a/d3e/d49/f74 [8275,10540] 0 2024-04-06T03:40:41.307 INFO:tasks.workunit.client.0.smithi094.stdout:4/690: write d0/d15/d3a/f75 [1326082,31421] 0 2024-04-06T03:40:41.307 INFO:tasks.workunit.client.0.smithi094.stdout:4/691: read - d0/d15/d3a/d3e/d49/db2/dc7/fd3 zero size 2024-04-06T03:40:41.307 INFO:tasks.workunit.client.0.smithi094.stdout:4/692: readlink d0/d15/d3a/d3e/d50/l6b 0 2024-04-06T03:40:41.308 INFO:tasks.workunit.client.1.smithi136.stdout:8/676: symlink d3/de/d60/lcd 0 2024-04-06T03:40:41.308 INFO:tasks.workunit.client.1.smithi136.stdout:8/677: fdatasync d3/de/ff 0 2024-04-06T03:40:41.309 INFO:tasks.workunit.client.1.smithi136.stdout:0/622: write d1/d6/d2c/d57/d43/d6d/d85/d78/f7a [1450639,126970] 0 2024-04-06T03:40:41.310 INFO:tasks.workunit.client.1.smithi136.stdout:8/678: dread d3/da/d26/d35/d53/d86/f8f [4194304,4194304] 0 2024-04-06T03:40:41.310 INFO:tasks.workunit.client.1.smithi136.stdout:8/679: write d3/de/d60/fb1 [694080,67911] 0 2024-04-06T03:40:41.311 INFO:tasks.workunit.client.0.smithi094.stdout:4/693: mknod d0/d15/d3a/d3e/ce9 0 2024-04-06T03:40:41.311 INFO:tasks.workunit.client.0.smithi094.stdout:4/694: fdatasync d0/d4/d67/f79 0 2024-04-06T03:40:41.312 INFO:tasks.workunit.client.1.smithi136.stdout:2/712: rmdir d3/d4a/d9d/d18/d4b/dbd 39 2024-04-06T03:40:41.312 INFO:tasks.workunit.client.1.smithi136.stdout:2/713: fsync d3/d4a/d9d/d18/f81 0 2024-04-06T03:40:41.312 INFO:tasks.workunit.client.1.smithi136.stdout:2/714: readlink d3/d9/laf 0 2024-04-06T03:40:41.314 INFO:tasks.workunit.client.1.smithi136.stdout:5/673: dread d1/dc/d1c/d32/d8b/d20/f41 [0,4194304] 0 2024-04-06T03:40:41.315 INFO:tasks.workunit.client.0.smithi094.stdout:4/695: mknod d0/d15/d3a/d61/cea 0 2024-04-06T03:40:41.316 INFO:tasks.workunit.client.1.smithi136.stdout:7/682: mkdir d5/df/d53/d68/da3/dd0/df0 0 2024-04-06T03:40:41.316 INFO:tasks.workunit.client.1.smithi136.stdout:7/683: write d5/df/d11/f79 [2297388,22593] 0 2024-04-06T03:40:41.318 INFO:tasks.workunit.client.1.smithi136.stdout:0/623: dread d1/d6/d2c/d1a/d36/d59/f31 [0,4194304] 0 2024-04-06T03:40:41.319 INFO:tasks.workunit.client.1.smithi136.stdout:3/608: rename d0/d4/d12/d2d/d37/lb0 to d0/d4/d12/dc3/lca 0 2024-04-06T03:40:41.319 INFO:tasks.workunit.client.1.smithi136.stdout:3/609: chown d0/d4/d12/d2d 8331059 1 2024-04-06T03:40:41.322 INFO:tasks.workunit.client.0.smithi094.stdout:4/696: rename d0/df/d91/dbb to d0/df/d13/deb 0 2024-04-06T03:40:41.325 INFO:tasks.workunit.client.1.smithi136.stdout:9/645: creat d17/d21/d32/d43/d51/d57/fdb x:0 0 0 2024-04-06T03:40:41.326 INFO:tasks.workunit.client.0.smithi094.stdout:4/697: mkdir d0/d4/d67/dec 0 2024-04-06T03:40:41.328 INFO:tasks.workunit.client.1.smithi136.stdout:1/630: mkdir d12/d3b/d44/d5f/d5a/d72/dd2 0 2024-04-06T03:40:41.330 INFO:tasks.workunit.client.0.smithi094.stdout:4/698: creat d0/df/db3/dd8/fed x:0 0 0 2024-04-06T03:40:41.331 INFO:tasks.workunit.client.1.smithi136.stdout:8/680: mkdir d3/da/d26/d35/d3d/dc7/dce 0 2024-04-06T03:40:41.334 INFO:tasks.workunit.client.1.smithi136.stdout:5/674: unlink d1/dc/d10/d1e/l51 0 2024-04-06T03:40:41.334 INFO:tasks.workunit.client.1.smithi136.stdout:5/675: write d1/dc/d10/f28 [1907122,99633] 0 2024-04-06T03:40:41.335 INFO:tasks.workunit.client.0.smithi094.stdout:4/699: mknod d0/d4/d4b/cee 0 2024-04-06T03:40:41.337 INFO:tasks.workunit.client.1.smithi136.stdout:9/646: fsync d17/d21/d32/d43/d51/d57/fdb 0 2024-04-06T03:40:41.338 INFO:tasks.workunit.client.0.smithi094.stdout:4/700: truncate d0/d4/d4b/f6c 2946727 0 2024-04-06T03:40:41.342 INFO:tasks.workunit.client.1.smithi136.stdout:0/624: creat d1/d6/d2c/d57/d43/d6b/fcd x:0 0 0 2024-04-06T03:40:41.342 INFO:tasks.workunit.client.1.smithi136.stdout:0/625: dread d1/d6/d2c/d1a/fa0 [0,4194304] 0 2024-04-06T03:40:41.343 INFO:tasks.workunit.client.0.smithi094.stdout:6/566: sync 2024-04-06T03:40:41.343 INFO:tasks.workunit.client.1.smithi136.stdout:7/684: rename d5/df/d12/d1b/l6e to d5/df/d53/d8a/lf1 0 2024-04-06T03:40:41.344 INFO:tasks.workunit.client.1.smithi136.stdout:3/610: mkdir d0/d4/d12/d19/d55/d3c/dcb 0 2024-04-06T03:40:41.345 INFO:tasks.workunit.client.0.smithi094.stdout:4/701: dread d0/df/d13/d4a/f8b [0,4194304] 0 2024-04-06T03:40:41.345 INFO:tasks.workunit.client.0.smithi094.stdout:6/567: rename dc/l62 to dc/d3a/d7f/lcc 0 2024-04-06T03:40:41.345 INFO:tasks.workunit.client.0.smithi094.stdout:6/568: write dc/d3a/d4a/f56 [1415890,2310] 0 2024-04-06T03:40:41.347 INFO:tasks.workunit.client.1.smithi136.stdout:1/631: mkdir d12/d1b/da7/dd3 0 2024-04-06T03:40:41.350 INFO:tasks.workunit.client.0.smithi094.stdout:4/702: dread d0/d15/d3a/d61/d81/fba [0,4194304] 0 2024-04-06T03:40:41.351 INFO:tasks.workunit.client.0.smithi094.stdout:6/569: rename dc/d11/f20 to dc/d11/d1b/d24/d46/d7b/dc6/fcd 0 2024-04-06T03:40:41.351 INFO:tasks.workunit.client.0.smithi094.stdout:6/570: fdatasync dc/d11/d22/d50/f80 0 2024-04-06T03:40:41.356 INFO:tasks.workunit.client.1.smithi136.stdout:9/647: symlink d17/d21/ldc 0 2024-04-06T03:40:41.361 INFO:tasks.workunit.client.0.smithi094.stdout:6/571: dread dc/f40 [0,4194304] 0 2024-04-06T03:40:41.361 INFO:tasks.workunit.client.0.smithi094.stdout:6/572: chown dc/c13 16222 1 2024-04-06T03:40:41.361 INFO:tasks.workunit.client.1.smithi136.stdout:0/626: symlink d1/d6/d2c/d1a/d36/d59/lce 0 2024-04-06T03:40:41.363 INFO:tasks.workunit.client.1.smithi136.stdout:7/685: mkdir d5/df/d1d/d54/df2 0 2024-04-06T03:40:41.364 INFO:tasks.workunit.client.0.smithi094.stdout:6/573: truncate dc/d3a/da2/fa7 9727 0 2024-04-06T03:40:41.365 INFO:tasks.workunit.client.1.smithi136.stdout:3/611: creat d0/d4/d12/d19/d55/d3c/d5a/d79/d98/fcc x:0 0 0 2024-04-06T03:40:41.365 INFO:tasks.workunit.client.1.smithi136.stdout:3/612: read d0/d4/d12/d19/d2e/d3b/f5f [367423,130244] 0 2024-04-06T03:40:41.365 INFO:tasks.workunit.client.1.smithi136.stdout:8/681: symlink d3/da/d26/d35/d53/lcf 0 2024-04-06T03:40:41.366 INFO:tasks.workunit.client.0.smithi094.stdout:6/574: truncate dc/f28 2043418 0 2024-04-06T03:40:41.367 INFO:tasks.workunit.client.0.smithi094.stdout:6/575: write dc/d11/d22/d50/d61/f73 [47125,35434] 0 2024-04-06T03:40:41.367 INFO:tasks.workunit.client.1.smithi136.stdout:9/648: mknod d17/d20/d25/d3c/d55/d8a/cdd 0 2024-04-06T03:40:41.368 INFO:tasks.workunit.client.1.smithi136.stdout:8/682: dread d3/da/d26/d35/d39/d70/f93 [0,4194304] 0 2024-04-06T03:40:41.369 INFO:tasks.workunit.client.0.smithi094.stdout:6/576: dread dc/d3a/f89 [0,4194304] 0 2024-04-06T03:40:41.369 INFO:tasks.workunit.client.0.smithi094.stdout:4/703: dwrite d0/d15/f8f [0,4194304] 0 2024-04-06T03:40:41.369 INFO:tasks.workunit.client.0.smithi094.stdout:4/704: truncate d0/df/d1e/d34/f39 813209 0 2024-04-06T03:40:41.370 INFO:tasks.workunit.client.1.smithi136.stdout:0/627: mkdir d1/d6/d2c/d57/d43/dcf 0 2024-04-06T03:40:41.371 INFO:tasks.workunit.client.1.smithi136.stdout:5/676: rename d1/dc/d1c/d32/d70/dc1/fa2 to d1/dc/fc9 0 2024-04-06T03:40:41.372 INFO:tasks.workunit.client.0.smithi094.stdout:6/577: mknod dc/d11/d22/d50/cce 0 2024-04-06T03:40:41.375 INFO:tasks.workunit.client.1.smithi136.stdout:3/613: write d0/d4/d12/d2d/fbe [1942946,86902] 0 2024-04-06T03:40:41.375 INFO:tasks.workunit.client.1.smithi136.stdout:1/632: dwrite d12/d1b/f35 [0,4194304] 0 2024-04-06T03:40:41.375 INFO:tasks.workunit.client.1.smithi136.stdout:1/633: dread - d12/d3b/d44/d5f/d5a/fce zero size 2024-04-06T03:40:41.375 INFO:tasks.workunit.client.1.smithi136.stdout:8/683: stat d3/da/d26/d35/d39/l84 0 2024-04-06T03:40:41.375 INFO:tasks.workunit.client.1.smithi136.stdout:8/684: chown d3/da/d26/d35/d3d/d48/d6c/d79/l83 5179 1 2024-04-06T03:40:41.375 INFO:tasks.workunit.client.1.smithi136.stdout:8/685: chown d3/da/d26/d35/d3d/f64 3393709 1 2024-04-06T03:40:41.377 INFO:tasks.workunit.client.1.smithi136.stdout:9/649: dread d17/d21/d29/f92 [0,4194304] 0 2024-04-06T03:40:41.377 INFO:tasks.workunit.client.1.smithi136.stdout:0/628: symlink d1/d6/d2c/d1a/ld0 0 2024-04-06T03:40:41.377 INFO:tasks.workunit.client.1.smithi136.stdout:0/629: chown d1/d6/l61 393602 1 2024-04-06T03:40:41.378 INFO:tasks.workunit.client.1.smithi136.stdout:5/677: creat d1/dc/d1c/d32/d8b/db0/fca x:0 0 0 2024-04-06T03:40:41.379 INFO:tasks.workunit.client.1.smithi136.stdout:7/686: getdents d5/df/d12/d1b/d3a 0 2024-04-06T03:40:41.385 INFO:tasks.workunit.client.1.smithi136.stdout:3/614: unlink d0/d4/d12/d19/d2c/fb9 0 2024-04-06T03:40:41.386 INFO:tasks.workunit.client.1.smithi136.stdout:1/634: read - d12/d3b/d60/d7e/f7d zero size 2024-04-06T03:40:41.386 INFO:tasks.workunit.client.1.smithi136.stdout:1/635: fdatasync d12/d3b/d60/d7e/d37/f78 0 2024-04-06T03:40:41.387 INFO:tasks.workunit.client.1.smithi136.stdout:8/686: fsync d3/da/d26/d35/d53/fc6 0 2024-04-06T03:40:41.387 INFO:tasks.workunit.client.1.smithi136.stdout:8/687: fsync d3/de/f56 0 2024-04-06T03:40:41.387 INFO:tasks.workunit.client.0.smithi094.stdout:4/705: creat d0/df/d1e/fef x:0 0 0 2024-04-06T03:40:41.388 INFO:tasks.workunit.client.1.smithi136.stdout:9/650: rmdir d17/d21/d29/d34 39 2024-04-06T03:40:41.390 INFO:tasks.workunit.client.1.smithi136.stdout:0/630: truncate d1/d6/d2c/d57/d43/f50 309030 0 2024-04-06T03:40:41.391 INFO:tasks.workunit.client.1.smithi136.stdout:0/631: write d1/d6/fd [313480,60850] 0 2024-04-06T03:40:41.391 INFO:tasks.workunit.client.1.smithi136.stdout:5/678: mkdir d1/d16/d89/dcb 0 2024-04-06T03:40:41.391 INFO:tasks.workunit.client.0.smithi094.stdout:6/578: getdents dc/d11/d22/d50/d61 0 2024-04-06T03:40:41.391 INFO:tasks.workunit.client.1.smithi136.stdout:4/598: sync 2024-04-06T03:40:41.392 INFO:tasks.workunit.client.1.smithi136.stdout:7/687: mknod d5/df/d12/d1b/d3a/d8d/cf3 0 2024-04-06T03:40:41.392 INFO:tasks.workunit.client.1.smithi136.stdout:7/688: fdatasync d5/df/f10 0 2024-04-06T03:40:41.393 INFO:tasks.workunit.client.1.smithi136.stdout:4/599: read d1d/d31/f41 [4367925,121213] 0 2024-04-06T03:40:41.393 INFO:tasks.workunit.client.0.smithi094.stdout:4/706: getdents d0/d4 0 2024-04-06T03:40:41.397 INFO:tasks.workunit.client.0.smithi094.stdout:6/579: truncate dc/d11/d1b/f1f 3053315 0 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:8/677: sync 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:7/591: sync 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:3/643: sync 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:0/652: sync 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:7/592: readlink d2/d6/dc/dd/df/l6f 0 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:1/606: sync 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:5/578: sync 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:9/596: sync 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:2/642: sync 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:0/653: truncate d1/d9/dd/d18/d1e/d22/d35/f52 1111374 0 2024-04-06T03:40:41.398 INFO:tasks.workunit.client.0.smithi094.stdout:8/678: write d18/d3c/d76/dd9/f6d [222636,4856] 0 2024-04-06T03:40:41.399 INFO:tasks.workunit.client.0.smithi094.stdout:1/607: chown d0/d1/d27 134003266 1 2024-04-06T03:40:41.399 INFO:tasks.workunit.client.0.smithi094.stdout:1/608: dread - d0/d1/d27/da5/d40/fc7 zero size 2024-04-06T03:40:41.399 INFO:tasks.workunit.client.1.smithi136.stdout:1/636: creat d12/d1b/d1e/dd1/fd4 x:0 0 0 2024-04-06T03:40:41.399 INFO:tasks.workunit.client.1.smithi136.stdout:1/637: write d12/d53/f95 [967642,110057] 0 2024-04-06T03:40:41.400 INFO:tasks.workunit.client.0.smithi094.stdout:4/707: rename d0/df/d13/f98 to d0/df/d1e/d34/ff0 0 2024-04-06T03:40:41.400 INFO:tasks.workunit.client.0.smithi094.stdout:4/708: truncate d0/df/d1e/f7f 513954 0 2024-04-06T03:40:41.400 INFO:tasks.workunit.client.0.smithi094.stdout:0/654: dread d1/d9/fcd [0,4194304] 0 2024-04-06T03:40:41.400 INFO:tasks.workunit.client.1.smithi136.stdout:8/688: chown d3/da/d26/d35/d53/l9f 228 1 2024-04-06T03:40:41.400 INFO:tasks.workunit.client.1.smithi136.stdout:8/689: write d3/de/f61 [1054231,9400] 0 2024-04-06T03:40:41.401 INFO:tasks.workunit.client.0.smithi094.stdout:7/593: creat d2/d6/dc/d12/d3a/d56/db1/fb2 x:0 0 0 2024-04-06T03:40:41.401 INFO:tasks.workunit.client.0.smithi094.stdout:7/594: write d2/d6/dc/d12/d29/f76 [1820804,16157] 0 2024-04-06T03:40:41.401 INFO:tasks.workunit.client.0.smithi094.stdout:7/595: chown d2/d6/dc/c3c 114 1 2024-04-06T03:40:41.402 INFO:tasks.workunit.client.1.smithi136.stdout:0/632: write d1/d6/d2c/d57/d43/d6d/d85/da7/fb4 [3616928,67599] 0 2024-04-06T03:40:41.402 INFO:tasks.workunit.client.1.smithi136.stdout:9/651: symlink d17/d20/d25/d3d/da5/lde 0 2024-04-06T03:40:41.403 INFO:tasks.workunit.client.0.smithi094.stdout:6/580: read f8 [458550,119229] 0 2024-04-06T03:40:41.403 INFO:tasks.workunit.client.0.smithi094.stdout:5/579: symlink db/d7c/d8b/lc5 0 2024-04-06T03:40:41.404 INFO:tasks.workunit.client.1.smithi136.stdout:5/679: mknod d1/dc/d1c/ccc 0 2024-04-06T03:40:41.405 INFO:tasks.workunit.client.0.smithi094.stdout:9/597: mkdir d2/dc/d21/d26/d78/d69/da8/ddc 0 2024-04-06T03:40:41.407 INFO:tasks.workunit.client.1.smithi136.stdout:4/600: mknod d1d/d2f/d6c/da5/dc1/cd9 0 2024-04-06T03:40:41.407 INFO:tasks.workunit.client.1.smithi136.stdout:4/601: read - d1d/d31/d78/fbd zero size 2024-04-06T03:40:41.409 INFO:tasks.workunit.client.0.smithi094.stdout:3/644: rename d9/d1d/c31 to d9/d1e/d6e/d75/dba/d7c/da2/ced 0 2024-04-06T03:40:41.409 INFO:tasks.workunit.client.0.smithi094.stdout:3/645: chown d9/d20/d76/fac 72 1 2024-04-06T03:40:41.409 INFO:tasks.workunit.client.0.smithi094.stdout:3/646: write d9/d1d/d7d/f9c [883486,75472] 0 2024-04-06T03:40:41.409 INFO:tasks.workunit.client.0.smithi094.stdout:3/647: chown d9/d10/d7e/da3/fdc 1545 1 2024-04-06T03:40:41.410 INFO:tasks.workunit.client.0.smithi094.stdout:6/581: write dc/d11/d1b/f38 [942887,91942] 0 2024-04-06T03:40:41.411 INFO:tasks.workunit.client.0.smithi094.stdout:0/655: unlink d1/d9/dd/d18/d5d/d60/d6b/cb2 0 2024-04-06T03:40:41.411 INFO:tasks.workunit.client.0.smithi094.stdout:0/656: stat d1/d9/dd/l6d 0 2024-04-06T03:40:41.412 INFO:tasks.workunit.client.1.smithi136.stdout:3/615: rmdir d0/d4/d12/d3a/d51 39 2024-04-06T03:40:41.412 INFO:tasks.workunit.client.1.smithi136.stdout:3/616: write d0/d4/d12/d19/d55/d6f/d82/d5d/d61/f8f [736771,88253] 0 2024-04-06T03:40:41.415 INFO:tasks.workunit.client.0.smithi094.stdout:7/596: unlink d2/d6/dc/dd/df/d36/f69 0 2024-04-06T03:40:41.415 INFO:tasks.workunit.client.0.smithi094.stdout:7/597: truncate d2/d6/dc/dd/df/f66 846651 0 2024-04-06T03:40:41.415 INFO:tasks.workunit.client.0.smithi094.stdout:4/709: read d0/d15/d3a/d3e/d50/f5e [1468238,115141] 0 2024-04-06T03:40:41.416 INFO:tasks.workunit.client.1.smithi136.stdout:1/638: unlink d12/d1b/d1e/d23/f3d 0 2024-04-06T03:40:41.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:41 smithi094 ceph-mon[29713]: pgmap v20: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 222 MiB/s rd, 293 MiB/s wr, 391 op/s 2024-04-06T03:40:41.420 INFO:tasks.workunit.client.0.smithi094.stdout:2/643: rename d0/d8/d27/d2b/d76 to d0/dd/d19/d31/d67/d98/ddc 0 2024-04-06T03:40:41.422 INFO:tasks.workunit.client.1.smithi136.stdout:8/690: symlink d3/da/da0/ld0 0 2024-04-06T03:40:41.422 INFO:tasks.workunit.client.1.smithi136.stdout:8/691: chown d3/da/d26/d35/d3d/d48/d6c/d79 96 1 2024-04-06T03:40:41.423 INFO:tasks.workunit.client.0.smithi094.stdout:3/648: mkdir d9/d10/d9b/dee 0 2024-04-06T03:40:41.427 INFO:tasks.workunit.client.0.smithi094.stdout:0/657: truncate d1/d9/dd/d18/d1a/f46 3112574 0 2024-04-06T03:40:41.427 INFO:tasks.workunit.client.0.smithi094.stdout:1/609: dwrite d0/d7/d2d/f6e [0,4194304] 0 2024-04-06T03:40:41.427 INFO:tasks.workunit.client.0.smithi094.stdout:1/610: dread - d0/d1/d27/da5/d40/fc7 zero size 2024-04-06T03:40:41.427 INFO:tasks.workunit.client.0.smithi094.stdout:1/611: truncate d0/d1/fbc 354496 0 2024-04-06T03:40:41.427 INFO:tasks.workunit.client.0.smithi094.stdout:1/612: write d0/d1/d27/fa9 [763485,67978] 0 2024-04-06T03:40:41.428 INFO:tasks.workunit.client.1.smithi136.stdout:0/633: creat d1/d6/d2c/d1a/fd1 x:0 0 0 2024-04-06T03:40:41.428 INFO:tasks.workunit.client.1.smithi136.stdout:5/680: dwrite d1/f96 [0,4194304] 0 2024-04-06T03:40:41.428 INFO:tasks.workunit.client.1.smithi136.stdout:5/681: creat d1/d9b/d9c/fcd x:0 0 0 2024-04-06T03:40:41.429 INFO:tasks.workunit.client.0.smithi094.stdout:7/598: mknod d2/d6/dc/dd/df/d36/d79/cb3 0 2024-04-06T03:40:41.430 INFO:tasks.workunit.client.1.smithi136.stdout:9/652: mknod d17/d20/d28/d81/cdf 0 2024-04-06T03:40:41.432 INFO:tasks.workunit.client.0.smithi094.stdout:4/710: symlink d0/d15/d3a/d61/ddf/lf1 0 2024-04-06T03:40:41.432 INFO:tasks.workunit.client.0.smithi094.stdout:2/644: dwrite d0/d8/d1d/d77/fcf [0,4194304] 0 2024-04-06T03:40:41.433 INFO:tasks.workunit.client.1.smithi136.stdout:4/602: unlink d1d/d70/cd8 0 2024-04-06T03:40:41.434 INFO:tasks.workunit.client.1.smithi136.stdout:4/603: fdatasync d1d/d2f/d6c/da5/faa 0 2024-04-06T03:40:41.437 INFO:tasks.workunit.client.1.smithi136.stdout:3/617: link d0/d4/d12/d19/d2c/lc5 d0/d4/d12/d19/d55/lcd 0 2024-04-06T03:40:41.441 INFO:tasks.workunit.client.1.smithi136.stdout:1/639: unlink d12/d1b/d1e/d9f/fc6 0 2024-04-06T03:40:41.446 INFO:tasks.workunit.client.1.smithi136.stdout:8/692: mknod d3/da/d26/d35/d3d/dc7/cd1 0 2024-04-06T03:40:41.448 INFO:tasks.workunit.client.0.smithi094.stdout:8/679: rename d18/d2c/d37/d60/d70/f7c to d18/d3c/dc4/ff0 0 2024-04-06T03:40:41.451 INFO:tasks.workunit.client.0.smithi094.stdout:3/649: mknod d9/d1e/d6e/d75/cef 0 2024-04-06T03:40:41.451 INFO:tasks.workunit.client.0.smithi094.stdout:3/650: readlink d9/d1e/d26/l5e 0 2024-04-06T03:40:41.452 INFO:tasks.workunit.client.0.smithi094.stdout:0/658: mkdir d1/d9/dd/d18/d5d/d60/dbc/dda 0 2024-04-06T03:40:41.452 INFO:tasks.workunit.client.0.smithi094.stdout:3/651: dread d9/d1d/f39 [0,4194304] 0 2024-04-06T03:40:41.453 INFO:tasks.workunit.client.1.smithi136.stdout:3/618: dwrite d0/d4/d12/d19/d55/d6f/d82/fb4 [0,4194304] 0 2024-04-06T03:40:41.453 INFO:tasks.workunit.client.1.smithi136.stdout:3/619: write d0/d4/d12/d2d/d37/f6d [1030207,99136] 0 2024-04-06T03:40:41.453 INFO:tasks.workunit.client.1.smithi136.stdout:0/634: mknod d1/cd2 0 2024-04-06T03:40:41.453 INFO:tasks.workunit.client.1.smithi136.stdout:0/635: write d1/d6/d2c/d57/f66 [5227018,9331] 0 2024-04-06T03:40:41.454 INFO:tasks.workunit.client.1.smithi136.stdout:0/636: chown d1/d6/d2c/d1a/f4b 0 1 2024-04-06T03:40:41.457 INFO:tasks.workunit.client.0.smithi094.stdout:1/613: mknod d0/d1/d27/da5/d40/db1/d2f/d3c/cc8 0 2024-04-06T03:40:41.460 INFO:tasks.workunit.client.0.smithi094.stdout:7/599: symlink d2/d6/dc/d12/d29/lb4 0 2024-04-06T03:40:41.460 INFO:tasks.workunit.client.0.smithi094.stdout:7/600: write d2/d6/dc/dd/fae [5217527,61189] 0 2024-04-06T03:40:41.467 INFO:tasks.workunit.client.0.smithi094.stdout:1/614: dread d0/f93 [0,4194304] 0 2024-04-06T03:40:41.467 INFO:tasks.workunit.client.0.smithi094.stdout:1/615: chown d0/f93 0 1 2024-04-06T03:40:41.468 INFO:tasks.workunit.client.1.smithi136.stdout:5/682: rename d1/dc/d10/d1e/lb6 to d1/d98/dc0/lce 0 2024-04-06T03:40:41.469 INFO:tasks.workunit.client.0.smithi094.stdout:4/711: getdents d0/df/d13/deb 0 2024-04-06T03:40:41.469 INFO:tasks.workunit.client.1.smithi136.stdout:8/693: dwrite d3/de/d18/d6d/fbd [0,4194304] 0 2024-04-06T03:40:41.472 INFO:tasks.workunit.client.1.smithi136.stdout:9/653: mkdir d17/d20/d25/d3d/da5/db8/de0 0 2024-04-06T03:40:41.473 INFO:tasks.workunit.client.0.smithi094.stdout:2/645: rmdir d0/d8/d1d 39 2024-04-06T03:40:41.473 INFO:tasks.workunit.client.0.smithi094.stdout:2/646: read - d0/dd/d1b/fd3 zero size 2024-04-06T03:40:41.474 INFO:tasks.workunit.client.1.smithi136.stdout:4/604: mknod d1d/d31/d3e/d56/cda 0 2024-04-06T03:40:41.474 INFO:tasks.workunit.client.1.smithi136.stdout:4/605: dread - d1d/d31/d3e/d55/d7f/f83 zero size 2024-04-06T03:40:41.475 INFO:tasks.workunit.client.0.smithi094.stdout:8/680: dwrite d18/d2c/d37/d60/d70/fe2 [0,4194304] 0 2024-04-06T03:40:41.475 INFO:tasks.workunit.client.0.smithi094.stdout:8/681: stat d18/d3c/d49/db5/ded 0 2024-04-06T03:40:41.476 INFO:tasks.workunit.client.0.smithi094.stdout:5/580: rename db/d19/f41 to db/d7c/d98/fc6 0 2024-04-06T03:40:41.476 INFO:tasks.workunit.client.0.smithi094.stdout:5/581: readlink db/d19/d24/d28/d85/lab 0 2024-04-06T03:40:41.478 INFO:tasks.workunit.client.1.smithi136.stdout:1/640: symlink d12/d1b/d1e/dd1/ld5 0 2024-04-06T03:40:41.478 INFO:tasks.workunit.client.1.smithi136.stdout:1/641: truncate d12/d1b/d1e/d4d/f68 5060157 0 2024-04-06T03:40:41.480 INFO:tasks.workunit.client.0.smithi094.stdout:0/659: mknod d1/d9/dd/d18/d32/d42/cdb 0 2024-04-06T03:40:41.480 INFO:tasks.workunit.client.0.smithi094.stdout:0/660: stat d1/d9/dd/d18/d5d/d60/d6b/dab/ccc 0 2024-04-06T03:40:41.480 INFO:tasks.workunit.client.0.smithi094.stdout:7/601: dwrite d2/d6/dc/dd/f3b [0,4194304] 0 2024-04-06T03:40:41.480 INFO:tasks.workunit.client.1.smithi136.stdout:2/715: sync 2024-04-06T03:40:41.480 INFO:tasks.workunit.client.1.smithi136.stdout:6/586: sync 2024-04-06T03:40:41.483 INFO:tasks.workunit.client.0.smithi094.stdout:2/647: dread d0/dd/d1b/d72/d8f/fc4 [0,4194304] 0 2024-04-06T03:40:41.483 INFO:tasks.workunit.client.0.smithi094.stdout:3/652: link d9/d1e/d6e/d75/dba/f38 d9/d10/d9b/dee/ff0 0 2024-04-06T03:40:41.483 INFO:tasks.workunit.client.0.smithi094.stdout:3/653: fsync d9/d1d/f4a 0 2024-04-06T03:40:41.485 INFO:tasks.workunit.client.0.smithi094.stdout:5/582: dread db/d19/d30/d83/f87 [0,4194304] 0 2024-04-06T03:40:41.485 INFO:tasks.workunit.client.0.smithi094.stdout:1/616: mkdir d0/d1/d27/da5/d40/db1/d2f/d3c/da0/dc9 0 2024-04-06T03:40:41.486 INFO:tasks.workunit.client.0.smithi094.stdout:1/617: creat d0/d1/d27/da5/fca x:0 0 0 2024-04-06T03:40:41.487 INFO:tasks.workunit.client.1.smithi136.stdout:3/620: mknod d0/d4/d12/d3a/d51/da6/cce 0 2024-04-06T03:40:41.493 INFO:tasks.workunit.client.0.smithi094.stdout:8/682: mknod d18/d5c/cf1 0 2024-04-06T03:40:41.495 INFO:tasks.workunit.client.0.smithi094.stdout:2/648: dwrite d0/d8/d1d/d77/fa6 [0,4194304] 0 2024-04-06T03:40:41.495 INFO:tasks.workunit.client.0.smithi094.stdout:2/649: chown d0/dd/d19/d31/d67/d98/ddc/l9e 1648825 1 2024-04-06T03:40:41.495 INFO:tasks.workunit.client.1.smithi136.stdout:5/683: mknod d1/d16/d17/db2/ccf 0 2024-04-06T03:40:41.495 INFO:tasks.workunit.client.1.smithi136.stdout:5/684: readlink d1/d16/d4b/d7b/l90 0 2024-04-06T03:40:41.502 INFO:tasks.workunit.client.1.smithi136.stdout:8/694: dread d3/da/d26/d35/d53/f7c [0,4194304] 0 2024-04-06T03:40:41.504 INFO:tasks.workunit.client.0.smithi094.stdout:3/654: creat d9/d20/d30/d88/ff1 x:0 0 0 2024-04-06T03:40:41.505 INFO:tasks.workunit.client.1.smithi136.stdout:1/642: mkdir d12/d3b/d74/dd6 0 2024-04-06T03:40:41.505 INFO:tasks.workunit.client.1.smithi136.stdout:1/643: stat d12/d1b/d28/f5e 0 2024-04-06T03:40:41.506 INFO:tasks.workunit.client.0.smithi094.stdout:5/583: symlink db/d19/d24/d2e/lc7 0 2024-04-06T03:40:41.507 INFO:tasks.workunit.client.1.smithi136.stdout:2/716: symlink d3/d15/d19/le3 0 2024-04-06T03:40:41.509 INFO:tasks.workunit.client.1.smithi136.stdout:6/587: mkdir d2/d7/d26/d30/dc4 0 2024-04-06T03:40:41.509 INFO:tasks.workunit.client.1.smithi136.stdout:6/588: write d2/d92/fc1 [1029043,104264] 0 2024-04-06T03:40:41.510 INFO:tasks.workunit.client.0.smithi094.stdout:1/618: mkdir d0/d1/d27/da5/d40/db1/d2f/d3c/d48/dcb 0 2024-04-06T03:40:41.512 INFO:tasks.workunit.client.0.smithi094.stdout:3/655: write d9/d9d/fe3 [3367278,75948] 0 2024-04-06T03:40:41.512 INFO:tasks.workunit.client.0.smithi094.stdout:3/656: chown d9/d10/fd8 124414181 1 2024-04-06T03:40:41.512 INFO:tasks.workunit.client.0.smithi094.stdout:3/657: getdents d9/d1d/d7d/dd7 0 2024-04-06T03:40:41.512 INFO:tasks.workunit.client.0.smithi094.stdout:3/658: truncate d9/d1e/d6e/d75/dba/d7c/faa 1111229 0 2024-04-06T03:40:41.513 INFO:tasks.workunit.client.1.smithi136.stdout:3/621: symlink d0/d4/d91/lcf 0 2024-04-06T03:40:41.513 INFO:tasks.workunit.client.1.smithi136.stdout:3/622: readlink d0/d4/d12/d19/d2c/l8d 0 2024-04-06T03:40:41.513 INFO:tasks.workunit.client.0.smithi094.stdout:4/712: rmdir d0/df/d13 39 2024-04-06T03:40:41.514 INFO:tasks.workunit.client.1.smithi136.stdout:0/637: link d1/d6/d2c/d57/d43/d6d/d85/d78/l9d d1/d6/d2c/ld3 0 2024-04-06T03:40:41.515 INFO:tasks.workunit.client.1.smithi136.stdout:3/623: dread d0/d4/d12/d19/f50 [0,4194304] 0 2024-04-06T03:40:41.516 INFO:tasks.workunit.client.1.smithi136.stdout:5/685: dwrite d1/d9b/d9c/fbe [0,4194304] 0 2024-04-06T03:40:41.517 INFO:tasks.workunit.client.1.smithi136.stdout:9/654: rename d17/d21/d32/d43 to d17/d21/d29/d34/d66/de1 0 2024-04-06T03:40:41.517 INFO:tasks.workunit.client.0.smithi094.stdout:8/683: dwrite d18/d2c/d37/fa1 [0,4194304] 0 2024-04-06T03:40:41.518 INFO:tasks.workunit.client.1.smithi136.stdout:8/695: unlink d3/da/d26/d35/l46 0 2024-04-06T03:40:41.518 INFO:tasks.workunit.client.1.smithi136.stdout:8/696: truncate d3/da/d26/f85 656467 0 2024-04-06T03:40:41.518 INFO:tasks.workunit.client.0.smithi094.stdout:8/684: dread d18/d3c/d49/d9f/fb3 [0,4194304] 0 2024-04-06T03:40:41.518 INFO:tasks.workunit.client.0.smithi094.stdout:8/685: creat d18/d3c/d49/d91/ff2 x:0 0 0 2024-04-06T03:40:41.519 INFO:tasks.workunit.client.0.smithi094.stdout:2/650: creat d0/d8/d27/d2b/fdd x:0 0 0 2024-04-06T03:40:41.521 INFO:tasks.workunit.client.1.smithi136.stdout:2/717: rmdir d3/d9/d3b/d70/de2 0 2024-04-06T03:40:41.524 INFO:tasks.workunit.client.1.smithi136.stdout:6/589: truncate d2/d25/f59 393779 0 2024-04-06T03:40:41.524 INFO:tasks.workunit.client.1.smithi136.stdout:6/590: write d2/d25/f17 [3232733,35176] 0 2024-04-06T03:40:41.526 INFO:tasks.workunit.client.0.smithi094.stdout:5/584: truncate db/f23 4756142 0 2024-04-06T03:40:41.526 INFO:tasks.workunit.client.0.smithi094.stdout:8/686: dread d18/d2c/d37/d60/da3/f97 [0,4194304] 0 2024-04-06T03:40:41.526 INFO:tasks.workunit.client.0.smithi094.stdout:8/687: read - d18/d3c/d8b/fe0 zero size 2024-04-06T03:40:41.526 INFO:tasks.workunit.client.0.smithi094.stdout:5/585: write db/d19/d2c/d6d/f78 [4993137,45556] 0 2024-04-06T03:40:41.526 INFO:tasks.workunit.client.0.smithi094.stdout:5/586: read db/d19/d24/d28/f3f [89386,29179] 0 2024-04-06T03:40:41.526 INFO:tasks.workunit.client.0.smithi094.stdout:5/587: write db/d19/d24/f37 [4163429,26929] 0 2024-04-06T03:40:41.527 INFO:tasks.workunit.client.1.smithi136.stdout:0/638: unlink d1/fc 0 2024-04-06T03:40:41.531 INFO:tasks.workunit.client.1.smithi136.stdout:3/624: link d0/d4/d12/d19/d55/d6f/fb1 d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/fd0 0 2024-04-06T03:40:41.535 INFO:tasks.workunit.client.1.smithi136.stdout:5/686: dwrite d1/dc/d1c/f56 [0,4194304] 0 2024-04-06T03:40:41.535 INFO:tasks.workunit.client.1.smithi136.stdout:1/644: rename d12/d1b/l87 to d12/d3b/d60/d6d/ld7 0 2024-04-06T03:40:41.536 INFO:tasks.workunit.client.1.smithi136.stdout:7/689: sync 2024-04-06T03:40:41.536 INFO:tasks.workunit.client.1.smithi136.stdout:8/697: dwrite d3/de/ff [0,4194304] 0 2024-04-06T03:40:41.536 INFO:tasks.workunit.client.1.smithi136.stdout:8/698: fdatasync d3/da/d26/d35/d3d/d48/f72 0 2024-04-06T03:40:41.537 INFO:tasks.workunit.client.1.smithi136.stdout:9/655: mknod d17/d21/d29/d34/d66/de1/d51/ce2 0 2024-04-06T03:40:41.539 INFO:tasks.workunit.client.1.smithi136.stdout:2/718: creat d3/d4a/d9d/d18/d4b/dc0/fe4 x:0 0 0 2024-04-06T03:40:41.540 INFO:tasks.workunit.client.0.smithi094.stdout:1/619: symlink d0/d1/d27/da5/d40/db1/d2f/d3c/lcc 0 2024-04-06T03:40:41.540 INFO:tasks.workunit.client.0.smithi094.stdout:1/620: stat d0/d1/d27/da5/d40/db1/d2f/d3c/d48/f69 0 2024-04-06T03:40:41.543 INFO:tasks.workunit.client.1.smithi136.stdout:6/591: creat d2/d93/dab/fc5 x:0 0 0 2024-04-06T03:40:41.550 INFO:tasks.workunit.client.1.smithi136.stdout:3/625: mknod d0/d4/d12/d19/d2e/d3b/cd1 0 2024-04-06T03:40:41.550 INFO:tasks.workunit.client.0.smithi094.stdout:3/659: mkdir d9/d1e/d6e/d75/dba/d7c/da2/df2 0 2024-04-06T03:40:41.552 INFO:tasks.workunit.client.1.smithi136.stdout:5/687: dwrite d1/dc/d1c/d32/d8b/db0/fca [0,4194304] 0 2024-04-06T03:40:41.553 INFO:tasks.workunit.client.0.smithi094.stdout:4/713: link d0/d23/f32 d0/d15/d3a/d3e/ff2 0 2024-04-06T03:40:41.560 INFO:tasks.workunit.client.0.smithi094.stdout:2/651: creat d0/dd/d3d/fde x:0 0 0 2024-04-06T03:40:41.561 INFO:tasks.workunit.client.1.smithi136.stdout:1/645: unlink d12/d1b/l1c 0 2024-04-06T03:40:41.564 INFO:tasks.workunit.client.0.smithi094.stdout:9/598: rename d2/dc/d21/d26/d78/d69/d6d to d2/dc/d80/ddd 0 2024-04-06T03:40:41.566 INFO:tasks.workunit.client.1.smithi136.stdout:7/690: mkdir d5/df/d12/d1b/df4 0 2024-04-06T03:40:41.568 INFO:tasks.workunit.client.1.smithi136.stdout:3/626: dwrite d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/fb2 [0,4194304] 0 2024-04-06T03:40:41.569 INFO:tasks.workunit.client.1.smithi136.stdout:6/592: dwrite d2/d7/d26/d30/d6e/d58/d5d/f65 [0,4194304] 0 2024-04-06T03:40:41.569 INFO:tasks.workunit.client.1.smithi136.stdout:6/593: chown d2/d7/d26/d68 973 1 2024-04-06T03:40:41.569 INFO:tasks.workunit.client.0.smithi094.stdout:8/688: mkdir d18/d3c/d49/d91/d94/df3 0 2024-04-06T03:40:41.569 INFO:tasks.workunit.client.0.smithi094.stdout:8/689: chown d18/l26 181962793 1 2024-04-06T03:40:41.569 INFO:tasks.workunit.client.0.smithi094.stdout:8/690: fsync d18/d2c/fac 0 2024-04-06T03:40:41.570 INFO:tasks.workunit.client.1.smithi136.stdout:9/656: mkdir d17/d21/d29/d34/d66/de1/d51/d57/de3 0 2024-04-06T03:40:41.573 INFO:tasks.workunit.client.1.smithi136.stdout:2/719: unlink d3/f83 0 2024-04-06T03:40:41.573 INFO:tasks.workunit.client.1.smithi136.stdout:2/720: fdatasync d3/d15/f30 0 2024-04-06T03:40:41.574 INFO:tasks.workunit.client.0.smithi094.stdout:1/621: mkdir d0/d1/d27/da5/d40/db0/dcd 0 2024-04-06T03:40:41.576 INFO:tasks.workunit.client.0.smithi094.stdout:4/714: dwrite d0/d15/fe1 [0,4194304] 0 2024-04-06T03:40:41.577 INFO:tasks.workunit.client.0.smithi094.stdout:3/660: creat d9/d10/dbb/ff3 x:0 0 0 2024-04-06T03:40:41.577 INFO:tasks.workunit.client.0.smithi094.stdout:3/661: read d9/f15 [3091754,109980] 0 2024-04-06T03:40:41.577 INFO:tasks.workunit.client.0.smithi094.stdout:3/662: getdents d9/d1d/d7d/dd7 0 2024-04-06T03:40:41.578 INFO:tasks.workunit.client.0.smithi094.stdout:2/652: dread d0/dd/d19/d31/f42 [0,4194304] 0 2024-04-06T03:40:41.579 INFO:tasks.workunit.client.0.smithi094.stdout:5/588: rename db/d19/d2c/d6d/f78 to db/d7c/d98/fc8 0 2024-04-06T03:40:41.580 INFO:tasks.workunit.client.1.smithi136.stdout:7/691: mknod d5/df/d53/d8a/cf5 0 2024-04-06T03:40:41.581 INFO:tasks.workunit.client.0.smithi094.stdout:8/691: creat d18/d2c/d37/d60/da3/db8/dd8/ff4 x:0 0 0 2024-04-06T03:40:41.582 INFO:tasks.workunit.client.1.smithi136.stdout:8/699: mkdir d3/da/d26/d35/d53/dd2 0 2024-04-06T03:40:41.582 INFO:tasks.workunit.client.1.smithi136.stdout:8/700: creat d3/da/d26/d35/d39/d70/fd3 x:0 0 0 2024-04-06T03:40:41.582 INFO:tasks.workunit.client.0.smithi094.stdout:1/622: mkdir d0/d1/d27/da5/db2/dce 0 2024-04-06T03:40:41.582 INFO:tasks.workunit.client.0.smithi094.stdout:1/623: write d0/d1/d27/da5/f3a [5442225,12326] 0 2024-04-06T03:40:41.583 INFO:tasks.workunit.client.1.smithi136.stdout:3/627: write d0/f2 [3148689,109728] 0 2024-04-06T03:40:41.585 INFO:tasks.workunit.client.0.smithi094.stdout:3/663: dread d9/d20/d76/f5c [0,4194304] 0 2024-04-06T03:40:41.586 INFO:tasks.workunit.client.1.smithi136.stdout:5/688: dwrite d1/dc/d1c/d32/d8b/d20/f41 [0,4194304] 0 2024-04-06T03:40:41.586 INFO:tasks.workunit.client.1.smithi136.stdout:5/689: truncate d1/dc/d10/fc4 627333 0 2024-04-06T03:40:41.586 INFO:tasks.workunit.client.1.smithi136.stdout:1/646: dwrite d12/d1b/d1e/d9f/fa8 [0,4194304] 0 2024-04-06T03:40:41.587 INFO:tasks.workunit.client.0.smithi094.stdout:2/653: link d0/d8/d1d/d77/fd4 d0/dd/d19/d31/d67/fdf 0 2024-04-06T03:40:41.588 INFO:tasks.workunit.client.0.smithi094.stdout:9/599: dwrite d2/d2c/d99/fd9 [0,4194304] 0 2024-04-06T03:40:41.590 INFO:tasks.workunit.client.0.smithi094.stdout:9/600: dread d2/dc/d21/fad [0,4194304] 0 2024-04-06T03:40:41.590 INFO:tasks.workunit.client.1.smithi136.stdout:6/594: symlink d2/d7/d26/d30/d6e/lc6 0 2024-04-06T03:40:41.592 INFO:tasks.workunit.client.0.smithi094.stdout:5/589: symlink db/d19/d24/d28/d85/db5/lc9 0 2024-04-06T03:40:41.593 INFO:tasks.workunit.client.1.smithi136.stdout:9/657: mkdir d17/d20/d25/da4/dbb/de4 0 2024-04-06T03:40:41.595 INFO:tasks.workunit.client.1.smithi136.stdout:2/721: symlink d3/d4a/d9d/d18/d1d/d91/le5 0 2024-04-06T03:40:41.595 INFO:tasks.workunit.client.1.smithi136.stdout:2/722: truncate d3/d4a/d9d/d18/d4b/dbd/fc3 66508 0 2024-04-06T03:40:41.595 INFO:tasks.workunit.client.1.smithi136.stdout:2/723: chown d3/d9/d3b/d87/lb0 320261493 1 2024-04-06T03:40:41.596 INFO:tasks.workunit.client.0.smithi094.stdout:2/654: dread d0/d8/d27/d2b/f52 [0,4194304] 0 2024-04-06T03:40:41.598 INFO:tasks.workunit.client.0.smithi094.stdout:1/624: mknod d0/d7/ccf 0 2024-04-06T03:40:41.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:41 smithi136 ceph-mon[36436]: pgmap v20: 65 pgs: 65 active+clean; 2.1 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 222 MiB/s rd, 293 MiB/s wr, 391 op/s 2024-04-06T03:40:41.599 INFO:tasks.workunit.client.0.smithi094.stdout:4/715: dwrite d0/d15/d3a/d3e/d46/fce [0,4194304] 0 2024-04-06T03:40:41.599 INFO:tasks.workunit.client.0.smithi094.stdout:3/664: dwrite d9/f3f [0,4194304] 0 2024-04-06T03:40:41.600 INFO:tasks.workunit.client.1.smithi136.stdout:7/692: mkdir d5/df/d53/d68/da3/df6 0 2024-04-06T03:40:41.602 INFO:tasks.workunit.client.0.smithi094.stdout:3/665: dread d9/d1e/f8f [0,4194304] 0 2024-04-06T03:40:41.602 INFO:tasks.workunit.client.0.smithi094.stdout:3/666: chown d9/d1e/d6e/d75/dba/ca4 492442043 1 2024-04-06T03:40:41.604 INFO:tasks.workunit.client.1.smithi136.stdout:9/658: dread d17/d21/d29/f99 [0,4194304] 0 2024-04-06T03:40:41.604 INFO:tasks.workunit.client.0.smithi094.stdout:9/601: creat d2/dc/d46/fde x:0 0 0 2024-04-06T03:40:41.607 INFO:tasks.workunit.client.1.smithi136.stdout:8/701: fsync d3/da/d26/d35/d53/f5b 0 2024-04-06T03:40:41.607 INFO:tasks.workunit.client.1.smithi136.stdout:8/702: chown d3/d2d/c4a 305 1 2024-04-06T03:40:41.607 INFO:tasks.workunit.client.1.smithi136.stdout:8/703: stat d3/d2d/l40 0 2024-04-06T03:40:41.609 INFO:tasks.workunit.client.0.smithi094.stdout:4/716: write d0/df/d1e/d34/ff0 [3824702,16753] 0 2024-04-06T03:40:41.609 INFO:tasks.workunit.client.1.smithi136.stdout:3/628: rename d0/d4/d12/d19/d2c/f67 to d0/d4/d12/d19/d55/d6f/d82/fd2 0 2024-04-06T03:40:41.611 INFO:tasks.workunit.client.0.smithi094.stdout:8/692: getdents d18/d34/d52 0 2024-04-06T03:40:41.611 INFO:tasks.workunit.client.0.smithi094.stdout:8/693: creat d18/d3c/d49/d91/ff5 x:0 0 0 2024-04-06T03:40:41.611 INFO:tasks.workunit.client.0.smithi094.stdout:8/694: truncate d18/d3c/d49/f53 399417 0 2024-04-06T03:40:41.611 INFO:tasks.workunit.client.0.smithi094.stdout:8/695: write d18/d3c/d76/dd9/f72 [2182572,10390] 0 2024-04-06T03:40:41.613 INFO:tasks.workunit.client.0.smithi094.stdout:2/655: creat d0/d8/d4f/dab/dc3/fe0 x:0 0 0 2024-04-06T03:40:41.613 INFO:tasks.workunit.client.0.smithi094.stdout:2/656: write d0/dd/d1b/fd3 [626065,49953] 0 2024-04-06T03:40:41.613 INFO:tasks.workunit.client.0.smithi094.stdout:2/657: fdatasync d0/dd/d19/d31/f3a 0 2024-04-06T03:40:41.615 INFO:tasks.workunit.client.0.smithi094.stdout:1/625: creat d0/d1/d27/da5/d40/db1/d2f/d76/fd0 x:0 0 0 2024-04-06T03:40:41.616 INFO:tasks.workunit.client.1.smithi136.stdout:1/647: truncate d12/d3b/f84 1802906 0 2024-04-06T03:40:41.623 INFO:tasks.workunit.client.0.smithi094.stdout:9/602: mknod d2/dc/d21/d23/d5b/cdf 0 2024-04-06T03:40:41.624 INFO:tasks.workunit.client.0.smithi094.stdout:9/603: stat d2/dc/d10/d2a/c48 0 2024-04-06T03:40:41.625 INFO:tasks.workunit.client.1.smithi136.stdout:6/595: creat d2/d7/fc7 x:0 0 0 2024-04-06T03:40:41.627 INFO:tasks.workunit.client.0.smithi094.stdout:4/717: link d0/df/d13/f5a d0/d4/dde/ff3 0 2024-04-06T03:40:41.627 INFO:tasks.workunit.client.0.smithi094.stdout:4/718: getdents d0/df/d13/deb 0 2024-04-06T03:40:41.627 INFO:tasks.workunit.client.0.smithi094.stdout:4/719: write d0/d15/d3a/d3e/d46/f62 [2672448,5730] 0 2024-04-06T03:40:41.629 INFO:tasks.workunit.client.1.smithi136.stdout:2/724: symlink d3/d4a/d9d/d18/d1d/d3c/le6 0 2024-04-06T03:40:41.631 INFO:tasks.workunit.client.1.smithi136.stdout:6/596: dread d2/d7/d26/d30/d4f/f67 [0,4194304] 0 2024-04-06T03:40:41.631 INFO:tasks.workunit.client.1.smithi136.stdout:6/597: write d2/d7/d26/d30/d6e/d3d/d47/d84/dad/fbb [555268,91737] 0 2024-04-06T03:40:41.631 INFO:tasks.workunit.client.1.smithi136.stdout:6/598: write d2/d7/f12 [760976,28556] 0 2024-04-06T03:40:41.631 INFO:tasks.workunit.client.1.smithi136.stdout:6/599: truncate d2/d92/f99 336806 0 2024-04-06T03:40:41.631 INFO:tasks.workunit.client.1.smithi136.stdout:6/600: write d2/d7/d13/f76 [4458732,56657] 0 2024-04-06T03:40:41.632 INFO:tasks.workunit.client.0.smithi094.stdout:8/696: dwrite d18/d3c/d49/f53 [0,4194304] 0 2024-04-06T03:40:41.634 INFO:tasks.workunit.client.0.smithi094.stdout:5/590: truncate db/f23 5196793 0 2024-04-06T03:40:41.635 INFO:tasks.workunit.client.0.smithi094.stdout:9/604: dread d2/f7 [0,4194304] 0 2024-04-06T03:40:41.635 INFO:tasks.workunit.client.0.smithi094.stdout:9/605: chown d2/dc/d10/f66 1 1 2024-04-06T03:40:41.635 INFO:tasks.workunit.client.1.smithi136.stdout:1/648: dwrite d12/d1b/d1e/d4d/f51 [0,4194304] 0 2024-04-06T03:40:41.636 INFO:tasks.workunit.client.0.smithi094.stdout:8/697: dread d18/f25 [0,4194304] 0 2024-04-06T03:40:41.636 INFO:tasks.workunit.client.1.smithi136.stdout:2/725: write d3/d9/fad [2358042,94003] 0 2024-04-06T03:40:41.637 INFO:tasks.workunit.client.1.smithi136.stdout:7/693: creat d5/df/d1d/d54/ff7 x:0 0 0 2024-04-06T03:40:41.638 INFO:tasks.workunit.client.1.smithi136.stdout:5/690: rename d1/d16/f85 to d1/d98/dc6/fd0 0 2024-04-06T03:40:41.640 INFO:tasks.workunit.client.1.smithi136.stdout:3/629: mkdir d0/d4/d12/dc3/dd3 0 2024-04-06T03:40:41.643 INFO:tasks.workunit.client.1.smithi136.stdout:1/649: dread d12/d1b/f86 [0,4194304] 0 2024-04-06T03:40:41.645 INFO:tasks.workunit.client.0.smithi094.stdout:4/720: write d0/f83 [469936,15373] 0 2024-04-06T03:40:41.647 INFO:tasks.workunit.client.1.smithi136.stdout:6/601: unlink d2/d7/d26/d30/d6e/d3d/d47/d84/fa0 0 2024-04-06T03:40:41.647 INFO:tasks.workunit.client.1.smithi136.stdout:2/726: dread d3/d4a/d9d/d18/f2a [0,4194304] 0 2024-04-06T03:40:41.647 INFO:tasks.workunit.client.1.smithi136.stdout:2/727: write d3/d9b/f86 [755595,50658] 0 2024-04-06T03:40:41.649 INFO:tasks.workunit.client.1.smithi136.stdout:5/691: dread d1/dc/d10/d1e/f97 [0,4194304] 0 2024-04-06T03:40:41.649 INFO:tasks.workunit.client.1.smithi136.stdout:5/692: chown d1/d16/d89/d8a/cb3 117642541 1 2024-04-06T03:40:41.651 INFO:tasks.workunit.client.0.smithi094.stdout:5/591: creat db/d19/fca x:0 0 0 2024-04-06T03:40:41.651 INFO:tasks.workunit.client.0.smithi094.stdout:5/592: truncate db/d19/d24/d2e/f36 4238310 0 2024-04-06T03:40:41.651 INFO:tasks.workunit.client.1.smithi136.stdout:4/606: sync 2024-04-06T03:40:41.652 INFO:tasks.workunit.client.0.smithi094.stdout:9/606: symlink d2/dc/d21/d23/d47/d51/dd8/le0 0 2024-04-06T03:40:41.653 INFO:tasks.workunit.client.1.smithi136.stdout:7/694: rmdir d5/df/d12 39 2024-04-06T03:40:41.654 INFO:tasks.workunit.client.1.smithi136.stdout:6/602: dread d2/d25/f17 [0,4194304] 0 2024-04-06T03:40:41.656 INFO:tasks.workunit.client.0.smithi094.stdout:8/698: creat d18/d3c/d49/d91/d94/daa/ddc/ff6 x:0 0 0 2024-04-06T03:40:41.656 INFO:tasks.workunit.client.0.smithi094.stdout:8/699: readlink d18/d3c/dc4/db9/le5 0 2024-04-06T03:40:41.656 INFO:tasks.workunit.client.0.smithi094.stdout:8/700: creat d18/d2c/ff7 x:0 0 0 2024-04-06T03:40:41.656 INFO:tasks.workunit.client.0.smithi094.stdout:8/701: read - d18/d34/d52/f65 zero size 2024-04-06T03:40:41.658 INFO:tasks.workunit.client.0.smithi094.stdout:8/702: write d18/d3c/d76/dd9/f72 [969607,71887] 0 2024-04-06T03:40:41.659 INFO:tasks.workunit.client.0.smithi094.stdout:1/626: link d0/d1/d27/da5/d3f/f5c d0/d7/d1a/fd1 0 2024-04-06T03:40:41.659 INFO:tasks.workunit.client.0.smithi094.stdout:1/627: dread - d0/d1/d27/da5/d40/fc7 zero size 2024-04-06T03:40:41.661 INFO:tasks.workunit.client.1.smithi136.stdout:7/695: dread d5/df/d53/f3c [0,4194304] 0 2024-04-06T03:40:41.661 INFO:tasks.workunit.client.1.smithi136.stdout:7/696: creat d5/df/d1d/d54/d5c/ff8 x:0 0 0 2024-04-06T03:40:41.661 INFO:tasks.workunit.client.0.smithi094.stdout:8/703: write d18/d3c/dc4/dbc/fc1 [902701,51268] 0 2024-04-06T03:40:41.662 INFO:tasks.workunit.client.0.smithi094.stdout:9/607: dread d2/d2c/d33/f5c [0,4194304] 0 2024-04-06T03:40:41.662 INFO:tasks.workunit.client.1.smithi136.stdout:8/704: rename d3/f21 to d3/da/d26/d35/d3d/d48/d6c/dab/fd4 0 2024-04-06T03:40:41.663 INFO:tasks.workunit.client.1.smithi136.stdout:0/639: sync 2024-04-06T03:40:41.664 INFO:tasks.workunit.client.0.smithi094.stdout:4/721: truncate d0/d15/d3a/d3e/d50/fac 472501 0 2024-04-06T03:40:41.664 INFO:tasks.workunit.client.0.smithi094.stdout:4/722: write d0/d15/d3a/d61/d81/da0/fc5 [335646,79846] 0 2024-04-06T03:40:41.665 INFO:tasks.workunit.client.0.smithi094.stdout:5/593: dwrite db/d19/d24/f34 [0,4194304] 0 2024-04-06T03:40:41.666 INFO:tasks.workunit.client.1.smithi136.stdout:1/650: creat d12/d3b/d60/d7e/dbe/fd8 x:0 0 0 2024-04-06T03:40:41.666 INFO:tasks.workunit.client.1.smithi136.stdout:1/651: write d12/d3b/d60/d7e/f42 [438840,65697] 0 2024-04-06T03:40:41.666 INFO:tasks.workunit.client.1.smithi136.stdout:1/652: dread - d12/d3b/d44/d5f/d5a/d72/d77/d96/f97 zero size 2024-04-06T03:40:41.666 INFO:tasks.workunit.client.1.smithi136.stdout:1/653: read - d12/d3b/d44/d5f/d5a/d72/fbd zero size 2024-04-06T03:40:41.671 INFO:tasks.workunit.client.1.smithi136.stdout:5/693: creat d1/dc/d1c/fd1 x:0 0 0 2024-04-06T03:40:41.673 INFO:tasks.workunit.client.1.smithi136.stdout:4/607: rmdir d1d/d31/d8e 39 2024-04-06T03:40:41.675 INFO:tasks.workunit.client.1.smithi136.stdout:6/603: symlink d2/d7/d26/d30/d6e/d3d/d47/d84/lc8 0 2024-04-06T03:40:41.675 INFO:tasks.workunit.client.1.smithi136.stdout:6/604: readlink d2/l61 0 2024-04-06T03:40:41.676 INFO:tasks.workunit.client.1.smithi136.stdout:5/694: write d1/dc/d1c/d32/d8b/d20/f42 [1364546,101025] 0 2024-04-06T03:40:41.676 INFO:tasks.workunit.client.1.smithi136.stdout:5/695: stat d1/d16/d4b/l7d 0 2024-04-06T03:40:41.677 INFO:tasks.workunit.client.0.smithi094.stdout:5/594: creat db/d19/d2c/d6d/fcb x:0 0 0 2024-04-06T03:40:41.680 INFO:tasks.workunit.client.1.smithi136.stdout:7/697: mkdir d5/df/d12/d59/d71/df9 0 2024-04-06T03:40:41.685 INFO:tasks.workunit.client.1.smithi136.stdout:3/630: rename d0/d4/d12/d3a/d51/da6/dbb to d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/dd4 0 2024-04-06T03:40:41.685 INFO:tasks.workunit.client.1.smithi136.stdout:3/631: chown d0/d4/d12/d2d/d37/ca8 252307660 1 2024-04-06T03:40:41.685 INFO:tasks.workunit.client.1.smithi136.stdout:3/632: read d0/d4/d12/d19/d55/d6f/f9e [107129,112324] 0 2024-04-06T03:40:41.685 INFO:tasks.workunit.client.1.smithi136.stdout:3/633: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/fd5 x:0 0 0 2024-04-06T03:40:41.687 INFO:tasks.workunit.client.1.smithi136.stdout:8/705: mkdir d3/da/d26/d90/dd5 0 2024-04-06T03:40:41.688 INFO:tasks.workunit.client.0.smithi094.stdout:1/628: dwrite d0/d1/d27/fa9 [0,4194304] 0 2024-04-06T03:40:41.689 INFO:tasks.workunit.client.0.smithi094.stdout:8/704: dwrite d18/d3c/fb2 [0,4194304] 0 2024-04-06T03:40:41.690 INFO:tasks.workunit.client.1.smithi136.stdout:1/654: dwrite d12/d3b/d60/d6d/da2/fc2 [0,4194304] 0 2024-04-06T03:40:41.690 INFO:tasks.workunit.client.0.smithi094.stdout:5/595: creat db/d19/d24/d28/d85/db5/fcc x:0 0 0 2024-04-06T03:40:41.690 INFO:tasks.workunit.client.0.smithi094.stdout:5/596: chown db/d19/d2c/d57/ca4 45371555 1 2024-04-06T03:40:41.691 INFO:tasks.workunit.client.1.smithi136.stdout:2/728: getdents d3/d4a/d77 0 2024-04-06T03:40:41.695 INFO:tasks.workunit.client.1.smithi136.stdout:4/608: dwrite d1d/d2f/f7d [0,4194304] 0 2024-04-06T03:40:41.697 INFO:tasks.workunit.client.1.smithi136.stdout:6/605: creat d2/d7/d13/d79/dc2/fc9 x:0 0 0 2024-04-06T03:40:41.698 INFO:tasks.workunit.client.0.smithi094.stdout:9/608: dwrite d2/dc/d21/d23/d2f/f34 [0,4194304] 0 2024-04-06T03:40:41.698 INFO:tasks.workunit.client.0.smithi094.stdout:9/609: creat d2/dc/d21/d23/fe1 x:0 0 0 2024-04-06T03:40:41.700 INFO:tasks.workunit.client.0.smithi094.stdout:8/705: dread d18/f1c [0,4194304] 0 2024-04-06T03:40:41.700 INFO:tasks.workunit.client.1.smithi136.stdout:1/655: write d12/d53/f66 [3070495,113934] 0 2024-04-06T03:40:41.704 INFO:tasks.workunit.client.1.smithi136.stdout:1/656: dread d12/d3b/f4b [0,4194304] 0 2024-04-06T03:40:41.704 INFO:tasks.workunit.client.1.smithi136.stdout:1/657: chown d12 3195496 1 2024-04-06T03:40:41.704 INFO:tasks.workunit.client.1.smithi136.stdout:1/658: chown d12/c69 28060807 1 2024-04-06T03:40:41.705 INFO:tasks.workunit.client.1.smithi136.stdout:5/696: link d1/dc/d1c/d32/d8b/db0/fca d1/dc/d10/d99/fd2 0 2024-04-06T03:40:41.706 INFO:tasks.workunit.client.1.smithi136.stdout:5/697: write d1/dc/d1c/fd1 [754110,100675] 0 2024-04-06T03:40:41.706 INFO:tasks.workunit.client.1.smithi136.stdout:5/698: dread - d1/d9b/faa zero size 2024-04-06T03:40:41.706 INFO:tasks.workunit.client.1.smithi136.stdout:5/699: chown d1/dc/d1c/d32/d70/dc1 529272134 1 2024-04-06T03:40:41.706 INFO:tasks.workunit.client.1.smithi136.stdout:5/700: chown d1/dc/d1c/fd1 0 1 2024-04-06T03:40:41.709 INFO:tasks.workunit.client.0.smithi094.stdout:5/597: dwrite db/f1e [0,4194304] 0 2024-04-06T03:40:41.710 INFO:tasks.workunit.client.0.smithi094.stdout:1/629: truncate d0/d1/f10 1756619 0 2024-04-06T03:40:41.714 INFO:tasks.workunit.client.1.smithi136.stdout:2/729: dwrite d3/d15/f30 [0,4194304] 0 2024-04-06T03:40:41.714 INFO:tasks.workunit.client.1.smithi136.stdout:0/640: rmdir d1/d6/d2c/d84 39 2024-04-06T03:40:41.714 INFO:tasks.workunit.client.1.smithi136.stdout:2/730: stat d3/d9/d3b/fd7 0 2024-04-06T03:40:41.714 INFO:tasks.workunit.client.1.smithi136.stdout:2/731: fdatasync d3/d4a/d9d/d18/d1d/f20 0 2024-04-06T03:40:41.715 INFO:tasks.workunit.client.0.smithi094.stdout:8/706: dwrite d18/d2c/f47 [0,4194304] 0 2024-04-06T03:40:41.716 INFO:tasks.workunit.client.0.smithi094.stdout:1/630: dread d0/d7/f78 [0,4194304] 0 2024-04-06T03:40:41.720 INFO:tasks.workunit.client.0.smithi094.stdout:9/610: rename d2/dc/d21/d26/dba to d2/dc/d21/d23/d5b/de2 0 2024-04-06T03:40:41.720 INFO:tasks.workunit.client.1.smithi136.stdout:1/659: dwrite d12/d3b/d74/fba [0,4194304] 0 2024-04-06T03:40:41.725 INFO:tasks.workunit.client.0.smithi094.stdout:8/707: dread d18/f1b [0,4194304] 0 2024-04-06T03:40:41.726 INFO:tasks.workunit.client.0.smithi094.stdout:5/598: creat db/d40/d77/fcd x:0 0 0 2024-04-06T03:40:41.728 INFO:tasks.workunit.client.0.smithi094.stdout:1/631: symlink d0/d1/d27/da5/d3f/ld2 0 2024-04-06T03:40:41.729 INFO:tasks.workunit.client.0.smithi094.stdout:9/611: write d2/dc/d21/d26/f29 [1324621,130452] 0 2024-04-06T03:40:41.729 INFO:tasks.workunit.client.0.smithi094.stdout:9/612: chown d2/dc/d10/f66 0 1 2024-04-06T03:40:41.731 INFO:tasks.workunit.client.0.smithi094.stdout:8/708: unlink d18/d2c/d37/d60/da3/f92 0 2024-04-06T03:40:41.731 INFO:tasks.workunit.client.0.smithi094.stdout:8/709: truncate d18/d3c/d49/d91/d94/daa/ddc/ff6 194744 0 2024-04-06T03:40:41.731 INFO:tasks.workunit.client.0.smithi094.stdout:9/613: dread d2/dc/d80/db7/fc7 [0,4194304] 0 2024-04-06T03:40:41.731 INFO:tasks.workunit.client.0.smithi094.stdout:9/614: write d2/d2c/d99/fb4 [655158,76751] 0 2024-04-06T03:40:41.732 INFO:tasks.workunit.client.1.smithi136.stdout:6/606: truncate d2/d7/f2c 1601289 0 2024-04-06T03:40:41.732 INFO:tasks.workunit.client.1.smithi136.stdout:6/607: fsync d2/d7/d26/d30/d4f/f67 0 2024-04-06T03:40:41.735 INFO:tasks.workunit.client.0.smithi094.stdout:1/632: rmdir d0/d1/d27/da5/d40/db1/dc3 39 2024-04-06T03:40:41.739 INFO:tasks.workunit.client.0.smithi094.stdout:9/615: symlink d2/dc/d80/db7/le3 0 2024-04-06T03:40:41.739 INFO:tasks.workunit.client.1.smithi136.stdout:5/701: dread d1/dc/f91 [0,4194304] 0 2024-04-06T03:40:41.739 INFO:tasks.workunit.client.1.smithi136.stdout:5/702: write d1/d1f/f21 [5116531,129411] 0 2024-04-06T03:40:41.739 INFO:tasks.workunit.client.1.smithi136.stdout:5/703: chown d1/c67 1813275156 1 2024-04-06T03:40:41.742 INFO:tasks.workunit.client.1.smithi136.stdout:3/634: mkdir d0/d4/dd6 0 2024-04-06T03:40:41.742 INFO:tasks.workunit.client.0.smithi094.stdout:8/710: read d18/d30/f40 [3863003,10273] 0 2024-04-06T03:40:41.743 INFO:tasks.workunit.client.0.smithi094.stdout:5/599: rmdir db/d19/d2c/d57/d8f 39 2024-04-06T03:40:41.745 INFO:tasks.workunit.client.0.smithi094.stdout:1/633: write d0/d1/d27/da5/f3a [482224,60672] 0 2024-04-06T03:40:41.745 INFO:tasks.workunit.client.1.smithi136.stdout:8/706: getdents d3/da/d26/d35/d3d/d48 0 2024-04-06T03:40:41.745 INFO:tasks.workunit.client.1.smithi136.stdout:8/707: write d3/da/d26/d35/d3d/d48/d6c/dab/fd4 [2814937,59158] 0 2024-04-06T03:40:41.746 INFO:tasks.workunit.client.0.smithi094.stdout:8/711: symlink d18/d5c/lf8 0 2024-04-06T03:40:41.749 INFO:tasks.workunit.client.0.smithi094.stdout:9/616: dread d2/d2c/fa5 [0,4194304] 0 2024-04-06T03:40:41.750 INFO:tasks.workunit.client.1.smithi136.stdout:2/732: getdents d3/d4a 0 2024-04-06T03:40:41.750 INFO:tasks.workunit.client.0.smithi094.stdout:5/600: dread db/fd [0,4194304] 0 2024-04-06T03:40:41.751 INFO:tasks.workunit.client.0.smithi094.stdout:9/617: dread d2/dc/d21/fad [0,4194304] 0 2024-04-06T03:40:41.752 INFO:tasks.workunit.client.1.smithi136.stdout:9/659: sync 2024-04-06T03:40:41.754 INFO:tasks.workunit.client.0.smithi094.stdout:1/634: dread d0/d1/d27/da5/d40/db1/d2f/d76/d7d/fae [0,4194304] 0 2024-04-06T03:40:41.756 INFO:tasks.workunit.client.1.smithi136.stdout:6/608: dwrite d2/d7/d26/d30/d6e/d58/fbe [0,4194304] 0 2024-04-06T03:40:41.756 INFO:tasks.workunit.client.1.smithi136.stdout:0/641: dread d1/d6/d2c/f87 [0,4194304] 0 2024-04-06T03:40:41.760 INFO:tasks.workunit.client.1.smithi136.stdout:1/660: mknod d12/d3b/d60/cd9 0 2024-04-06T03:40:41.761 INFO:tasks.workunit.client.0.smithi094.stdout:9/618: dread d2/f1d [0,4194304] 0 2024-04-06T03:40:41.761 INFO:tasks.workunit.client.1.smithi136.stdout:5/704: creat d1/d16/d89/d8a/fd3 x:0 0 0 2024-04-06T03:40:41.762 INFO:tasks.workunit.client.0.smithi094.stdout:8/712: dwrite d18/d30/f3e [0,4194304] 0 2024-04-06T03:40:41.763 INFO:tasks.workunit.client.1.smithi136.stdout:3/635: symlink d0/d4/d12/d3a/d51/d8e/ld7 0 2024-04-06T03:40:41.765 INFO:tasks.workunit.client.0.smithi094.stdout:8/713: dread d18/d3c/dc4/db9/fc0 [0,4194304] 0 2024-04-06T03:40:41.765 INFO:tasks.workunit.client.0.smithi094.stdout:8/714: truncate d18/d2c/d37/fa1 4995055 0 2024-04-06T03:40:41.765 INFO:tasks.workunit.client.1.smithi136.stdout:2/733: creat d3/d15/fe7 x:0 0 0 2024-04-06T03:40:41.769 INFO:tasks.workunit.client.0.smithi094.stdout:1/635: mkdir d0/d1/dd3 0 2024-04-06T03:40:41.769 INFO:tasks.workunit.client.0.smithi094.stdout:1/636: chown d0/d1/d27/da5/d40/db1/d2f/d76/d7d 219 1 2024-04-06T03:40:41.771 INFO:tasks.workunit.client.0.smithi094.stdout:9/619: rmdir d2/dc/d10/d6a 39 2024-04-06T03:40:41.771 INFO:tasks.workunit.client.0.smithi094.stdout:9/620: write d2/dc/d21/d23/d47/f77 [1011683,100181] 0 2024-04-06T03:40:41.772 INFO:tasks.workunit.client.0.smithi094.stdout:9/621: read d2/dc/d46/f60 [12424,42819] 0 2024-04-06T03:40:41.772 INFO:tasks.workunit.client.0.smithi094.stdout:9/622: stat d2/dc/d46/cc1 0 2024-04-06T03:40:41.772 INFO:tasks.workunit.client.0.smithi094.stdout:9/623: read d2/dc/d21/f43 [5044342,90583] 0 2024-04-06T03:40:41.772 INFO:tasks.workunit.client.1.smithi136.stdout:8/708: dwrite d3/de/f8d [0,4194304] 0 2024-04-06T03:40:41.773 INFO:tasks.workunit.client.0.smithi094.stdout:1/637: rename d0/d4/f39 to d0/d1/d27/da5/d40/db1/d2f/d3c/d48/fd4 0 2024-04-06T03:40:41.774 INFO:tasks.workunit.client.0.smithi094.stdout:9/624: getdents d2/d2c/d33 0 2024-04-06T03:40:41.776 INFO:tasks.workunit.client.0.smithi094.stdout:1/638: rename d0/d1/d27/c47 to d0/d1/d27/da5/d40/db1/d2f/d76/d30/cd5 0 2024-04-06T03:40:41.778 INFO:tasks.workunit.client.1.smithi136.stdout:6/609: symlink d2/d7/d26/d30/dc4/lca 0 2024-04-06T03:40:41.779 INFO:tasks.workunit.client.0.smithi094.stdout:9/625: creat d2/dc/d80/db7/fe4 x:0 0 0 2024-04-06T03:40:41.779 INFO:tasks.workunit.client.0.smithi094.stdout:9/626: truncate d2/dc/d46/fde 387698 0 2024-04-06T03:40:41.779 INFO:tasks.workunit.client.0.smithi094.stdout:9/627: creat d2/dc/d21/d23/db0/fe5 x:0 0 0 2024-04-06T03:40:41.781 INFO:tasks.workunit.client.1.smithi136.stdout:0/642: mknod d1/d6/d2c/d57/d43/d6d/d85/d78/cd4 0 2024-04-06T03:40:41.781 INFO:tasks.workunit.client.1.smithi136.stdout:0/643: fdatasync d1/d6/d2c/d57/d43/d6d/d85/d78/f9a 0 2024-04-06T03:40:41.782 INFO:tasks.workunit.client.0.smithi094.stdout:8/715: dread d18/d34/f3d [0,4194304] 0 2024-04-06T03:40:41.783 INFO:tasks.workunit.client.1.smithi136.stdout:0/644: dread d1/d6/d2c/d57/d43/d6d/d85/f7f [0,4194304] 0 2024-04-06T03:40:41.783 INFO:tasks.workunit.client.1.smithi136.stdout:0/645: creat d1/d6/d2c/d1a/d36/fd5 x:0 0 0 2024-04-06T03:40:41.783 INFO:tasks.workunit.client.1.smithi136.stdout:0/646: truncate d1/d6/d2c/d1a/d36/f8f 5057227 0 2024-04-06T03:40:41.783 INFO:tasks.workunit.client.1.smithi136.stdout:0/647: chown d1/d6/d2c/d1a/l32 2684 1 2024-04-06T03:40:41.783 INFO:tasks.workunit.client.1.smithi136.stdout:0/648: fsync d1/d6/d2c/d57/d43/d6d/d85/d78/fad 0 2024-04-06T03:40:41.784 INFO:tasks.workunit.client.1.smithi136.stdout:5/705: creat d1/d9b/d9c/da9/fd4 x:0 0 0 2024-04-06T03:40:41.785 INFO:tasks.workunit.client.0.smithi094.stdout:9/628: truncate d2/d2c/f85 1423311 0 2024-04-06T03:40:41.787 INFO:tasks.workunit.client.1.smithi136.stdout:2/734: link d3/d9/d3b/d70/dc8/ccc d3/d4a/d9d/d18/ce8 0 2024-04-06T03:40:41.787 INFO:tasks.workunit.client.1.smithi136.stdout:2/735: fsync d3/d4a/d9d/d18/d4b/f65 0 2024-04-06T03:40:41.789 INFO:tasks.workunit.client.0.smithi094.stdout:8/716: write d18/d3c/d49/d91/d94/daa/fe9 [154616,3826] 0 2024-04-06T03:40:41.789 INFO:tasks.workunit.client.0.smithi094.stdout:9/629: symlink d2/dc/d21/d23/d47/d51/le6 0 2024-04-06T03:40:41.791 INFO:tasks.workunit.client.1.smithi136.stdout:8/709: dwrite d3/da/d26/d35/d3d/f64 [4194304,4194304] 0 2024-04-06T03:40:41.791 INFO:tasks.workunit.client.1.smithi136.stdout:8/710: dread - d3/da/d26/d35/d53/f5b zero size 2024-04-06T03:40:41.791 INFO:tasks.workunit.client.1.smithi136.stdout:8/711: write d3/de/f3a [4829299,9221] 0 2024-04-06T03:40:41.791 INFO:tasks.workunit.client.0.smithi094.stdout:1/639: dwrite d0/d4/fc [0,4194304] 0 2024-04-06T03:40:41.792 INFO:tasks.workunit.client.1.smithi136.stdout:9/660: rename d17/d20/d25/d3d to d17/d21/d29/de5 0 2024-04-06T03:40:41.793 INFO:tasks.workunit.client.1.smithi136.stdout:8/712: fdatasync d3/f2a 0 2024-04-06T03:40:41.793 INFO:tasks.workunit.client.1.smithi136.stdout:8/713: chown d3/d2d/f67 238362046 1 2024-04-06T03:40:41.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/630: link d2/dc/c5e d2/dc/da6/ce7 0 2024-04-06T03:40:41.801 INFO:tasks.workunit.client.1.smithi136.stdout:8/714: dread d3/de/d18/f74 [0,4194304] 0 2024-04-06T03:40:41.811 INFO:tasks.workunit.client.1.smithi136.stdout:8/715: read d3/da/d26/d35/d3d/d48/d6c/dab/fbe [3201473,82803] 0 2024-04-06T03:40:41.811 INFO:tasks.workunit.client.1.smithi136.stdout:8/716: write d3/da/d26/f7a [5207855,57802] 0 2024-04-06T03:40:41.815 INFO:tasks.workunit.client.1.smithi136.stdout:6/610: creat d2/d7/d26/d30/d4f/fcb x:0 0 0 2024-04-06T03:40:41.818 INFO:tasks.workunit.client.0.smithi094.stdout:8/717: dwrite d18/d3c/dc4/dbc/fc1 [0,4194304] 0 2024-04-06T03:40:41.820 INFO:tasks.workunit.client.1.smithi136.stdout:1/661: rmdir d12/d3b/d74 39 2024-04-06T03:40:41.826 INFO:tasks.workunit.client.1.smithi136.stdout:0/649: symlink d1/d6/d2c/d57/d43/d6d/d85/d78/da4/ld6 0 2024-04-06T03:40:41.829 INFO:tasks.workunit.client.1.smithi136.stdout:5/706: mknod d1/d9b/db4/cd5 0 2024-04-06T03:40:41.829 INFO:tasks.workunit.client.1.smithi136.stdout:5/707: fdatasync d1/dc/d1c/d32/d8b/d20/f41 0 2024-04-06T03:40:41.830 INFO:tasks.workunit.client.1.smithi136.stdout:7/698: sync 2024-04-06T03:40:41.830 INFO:tasks.workunit.client.1.smithi136.stdout:4/609: sync 2024-04-06T03:40:41.830 INFO:tasks.workunit.client.1.smithi136.stdout:7/699: stat d5/df/d53/d68/da3 0 2024-04-06T03:40:41.830 INFO:tasks.workunit.client.1.smithi136.stdout:7/700: creat d5/df/d12/d1b/ffa x:0 0 0 2024-04-06T03:40:41.830 INFO:tasks.workunit.client.1.smithi136.stdout:2/736: mknod d3/d15/ce9 0 2024-04-06T03:40:41.831 INFO:tasks.workunit.client.0.smithi094.stdout:1/640: symlink d0/d1/d27/da5/d40/db1/d2f/d4b/ld6 0 2024-04-06T03:40:41.832 INFO:tasks.workunit.client.1.smithi136.stdout:6/611: dwrite d2/d7/d26/d30/d6e/f69 [0,4194304] 0 2024-04-06T03:40:41.835 INFO:tasks.workunit.client.1.smithi136.stdout:3/636: rename d0/d4/d12/d19/d2e/d3b/d49/f5e to d0/d4/d12/d19/d55/d6f/d82/d5d/fd8 0 2024-04-06T03:40:41.835 INFO:tasks.workunit.client.1.smithi136.stdout:3/637: write d0/d4/d12/d19/d55/d3c/d5a/d79/fa4 [1752630,14741] 0 2024-04-06T03:40:41.835 INFO:tasks.workunit.client.1.smithi136.stdout:9/661: rmdir d17/d21/d29/d34 39 2024-04-06T03:40:41.836 INFO:tasks.workunit.client.1.smithi136.stdout:8/717: creat d3/da/d26/d35/d3d/dc7/dce/fd6 x:0 0 0 2024-04-06T03:40:41.836 INFO:tasks.workunit.client.1.smithi136.stdout:8/718: creat d3/de/fd7 x:0 0 0 2024-04-06T03:40:41.837 INFO:tasks.workunit.client.0.smithi094.stdout:8/718: creat d18/ff9 x:0 0 0 2024-04-06T03:40:41.839 INFO:tasks.workunit.client.1.smithi136.stdout:1/662: creat d12/d3b/d60/fda x:0 0 0 2024-04-06T03:40:41.842 INFO:tasks.workunit.client.0.smithi094.stdout:1/641: rmdir d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/dbe 0 2024-04-06T03:40:41.843 INFO:tasks.workunit.client.1.smithi136.stdout:0/650: creat d1/d6/d2c/d84/fd7 x:0 0 0 2024-04-06T03:40:41.846 INFO:tasks.workunit.client.1.smithi136.stdout:5/708: link d1/d16/d4b/l77 d1/d98/ld6 0 2024-04-06T03:40:41.849 INFO:tasks.workunit.client.1.smithi136.stdout:9/662: read d17/d21/d29/d34/d66/de1/f8c [1988221,58741] 0 2024-04-06T03:40:41.849 INFO:tasks.workunit.client.1.smithi136.stdout:4/610: symlink d1d/d31/d78/dcb/ldb 0 2024-04-06T03:40:41.849 INFO:tasks.workunit.client.1.smithi136.stdout:9/663: write d17/d20/d28/f98 [529704,102903] 0 2024-04-06T03:40:41.849 INFO:tasks.workunit.client.1.smithi136.stdout:9/664: readlink d17/d21/ldc 0 2024-04-06T03:40:41.852 INFO:tasks.workunit.client.1.smithi136.stdout:7/701: rename d5/df/d1d/d6f/f56 to d5/df/d53/d68/ffb 0 2024-04-06T03:40:41.852 INFO:tasks.workunit.client.0.smithi094.stdout:6/582: sync 2024-04-06T03:40:41.853 INFO:tasks.workunit.client.1.smithi136.stdout:2/737: dwrite d3/d15/d19/f9a [8388608,4194304] 0 2024-04-06T03:40:41.853 INFO:tasks.workunit.client.1.smithi136.stdout:7/702: write d5/df/d12/d1b/d3a/d4a/f64 [295305,27084] 0 2024-04-06T03:40:41.854 INFO:tasks.workunit.client.1.smithi136.stdout:7/703: readlink d5/df/d1d/l45 0 2024-04-06T03:40:41.854 INFO:tasks.workunit.client.1.smithi136.stdout:8/719: creat d3/da/d26/d35/d53/fd8 x:0 0 0 2024-04-06T03:40:41.855 INFO:tasks.workunit.client.1.smithi136.stdout:1/663: creat d12/d3b/d44/d5f/fdb x:0 0 0 2024-04-06T03:40:41.856 INFO:tasks.workunit.client.1.smithi136.stdout:1/664: write d12/d3b/d60/d7e/dbe/fd8 [754065,101463] 0 2024-04-06T03:40:41.856 INFO:tasks.workunit.client.1.smithi136.stdout:1/665: fdatasync d12/d3b/d44/d5f/f63 0 2024-04-06T03:40:41.856 INFO:tasks.workunit.client.1.smithi136.stdout:1/666: creat d12/d3b/d44/d5f/d5a/d72/d77/fdc x:0 0 0 2024-04-06T03:40:41.856 INFO:tasks.workunit.client.1.smithi136.stdout:1/667: dread - d12/d3b/d44/d5f/d5a/d72/d77/fc1 zero size 2024-04-06T03:40:41.856 INFO:tasks.workunit.client.1.smithi136.stdout:1/668: chown d12/d3b/d60/d7e/f7d 6 1 2024-04-06T03:40:41.857 INFO:tasks.workunit.client.0.smithi094.stdout:6/583: creat dc/d11/d1b/d4e/fcf x:0 0 0 2024-04-06T03:40:41.857 INFO:tasks.workunit.client.0.smithi094.stdout:6/584: creat dc/d11/d1b/d72/d88/fd0 x:0 0 0 2024-04-06T03:40:41.857 INFO:tasks.workunit.client.0.smithi094.stdout:1/642: dread d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:41.859 INFO:tasks.workunit.client.1.smithi136.stdout:0/651: link d1/d6/d2c/d1a/l32 d1/d6/d2c/d57/d43/d6b/ld8 0 2024-04-06T03:40:41.859 INFO:tasks.workunit.client.1.smithi136.stdout:0/652: readlink d1/d6/d2c/d57/d43/d6d/d85/la5 0 2024-04-06T03:40:41.860 INFO:tasks.workunit.client.1.smithi136.stdout:5/709: truncate d1/dc/d10/d99/fd2 231957 0 2024-04-06T03:40:41.860 INFO:tasks.workunit.client.1.smithi136.stdout:5/710: chown d1/dc/d1c/d32/d8b/d20/f42 3 1 2024-04-06T03:40:41.861 INFO:tasks.workunit.client.0.smithi094.stdout:8/719: dwrite d18/d3c/d49/d91/ff5 [0,4194304] 0 2024-04-06T03:40:41.861 INFO:tasks.workunit.client.1.smithi136.stdout:6/612: dwrite d2/d7/d26/d30/d6e/f5e [0,4194304] 0 2024-04-06T03:40:41.861 INFO:tasks.workunit.client.1.smithi136.stdout:0/653: dread d1/d6/d2c/d1a/f94 [0,4194304] 0 2024-04-06T03:40:41.862 INFO:tasks.workunit.client.1.smithi136.stdout:5/711: dread d1/d16/d4b/f95 [0,4194304] 0 2024-04-06T03:40:41.862 INFO:tasks.workunit.client.1.smithi136.stdout:5/712: chown d1/d9b 312429 1 2024-04-06T03:40:41.862 INFO:tasks.workunit.client.1.smithi136.stdout:5/713: write d1/d16/d4b/f80 [1475214,46999] 0 2024-04-06T03:40:41.865 INFO:tasks.workunit.client.1.smithi136.stdout:9/665: symlink d17/d21/d29/d34/dbd/le6 0 2024-04-06T03:40:41.866 INFO:tasks.workunit.client.1.smithi136.stdout:7/704: dread d5/df/d53/d68/da3/dec/fef [0,4194304] 0 2024-04-06T03:40:41.867 INFO:tasks.workunit.client.0.smithi094.stdout:6/585: mknod dc/d3a/d7f/cd1 0 2024-04-06T03:40:41.867 INFO:tasks.workunit.client.0.smithi094.stdout:6/586: fsync dc/d11/d1b/f52 0 2024-04-06T03:40:41.868 INFO:tasks.workunit.client.1.smithi136.stdout:3/638: rename d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/dd4/cc4 to d0/d4/d12/d3a/d69/cd9 0 2024-04-06T03:40:41.871 INFO:tasks.workunit.client.1.smithi136.stdout:4/611: dwrite d1d/d31/d3e/f5a [0,4194304] 0 2024-04-06T03:40:41.872 INFO:tasks.workunit.client.1.smithi136.stdout:2/738: link d3/d15/d19/fc7 d3/d4a/d9d/d18/d1d/d91/fea 0 2024-04-06T03:40:41.872 INFO:tasks.workunit.client.1.smithi136.stdout:6/613: dread d2/d7/f12 [0,4194304] 0 2024-04-06T03:40:41.872 INFO:tasks.workunit.client.1.smithi136.stdout:6/614: truncate d2/d7/d26/d30/d6e/db7/fbd 400423 0 2024-04-06T03:40:41.873 INFO:tasks.workunit.client.1.smithi136.stdout:6/615: dread d2/d7/d26/d30/d6e/d3d/d47/f8c [0,4194304] 0 2024-04-06T03:40:41.873 INFO:tasks.workunit.client.1.smithi136.stdout:6/616: stat d2/d7/d26/d30/d4f/d75/f82 0 2024-04-06T03:40:41.873 INFO:tasks.workunit.client.0.smithi094.stdout:1/643: dwrite d0/d1/d27/da5/d40/db1/d2f/d3c/d48/f89 [0,4194304] 0 2024-04-06T03:40:41.875 INFO:tasks.workunit.client.1.smithi136.stdout:6/617: dread d2/d7/d26/d30/d4f/f67 [0,4194304] 0 2024-04-06T03:40:41.876 INFO:tasks.workunit.client.0.smithi094.stdout:8/720: creat d18/d2c/d37/d60/da3/deb/ffa x:0 0 0 2024-04-06T03:40:41.877 INFO:tasks.workunit.client.1.smithi136.stdout:8/720: mknod d3/da/d26/cd9 0 2024-04-06T03:40:41.879 INFO:tasks.workunit.client.0.smithi094.stdout:6/587: mknod dc/d11/d1b/d72/d87/cd2 0 2024-04-06T03:40:41.879 INFO:tasks.workunit.client.0.smithi094.stdout:6/588: stat dc/d11/d1b/d72/d88/fad 0 2024-04-06T03:40:41.879 INFO:tasks.workunit.client.0.smithi094.stdout:6/589: readlink dc/d11/d1b/lc3 0 2024-04-06T03:40:41.880 INFO:tasks.workunit.client.1.smithi136.stdout:7/705: dwrite d5/df/d12/d1b/d3a/d8d/f9e [0,4194304] 0 2024-04-06T03:40:41.883 INFO:tasks.workunit.client.1.smithi136.stdout:1/669: creat d12/d3b/d44/d5f/db7/fdd x:0 0 0 2024-04-06T03:40:41.885 INFO:tasks.workunit.client.0.smithi094.stdout:1/644: mknod d0/d7/d1a/d64/cd7 0 2024-04-06T03:40:41.886 INFO:tasks.workunit.client.1.smithi136.stdout:0/654: symlink d1/d6/d2c/d57/d62/ld9 0 2024-04-06T03:40:41.887 INFO:tasks.workunit.client.1.smithi136.stdout:2/739: write d3/d4a/d9d/d18/f23 [710462,27120] 0 2024-04-06T03:40:41.888 INFO:tasks.workunit.client.0.smithi094.stdout:8/721: rename d18/d30/f77 to d18/d2c/d37/d60/ffb 0 2024-04-06T03:40:41.890 INFO:tasks.workunit.client.0.smithi094.stdout:6/590: truncate dc/d11/d1b/d24/d46/fa5 330636 0 2024-04-06T03:40:41.893 INFO:tasks.workunit.client.1.smithi136.stdout:5/714: creat d1/d16/d17/db2/fd7 x:0 0 0 2024-04-06T03:40:41.893 INFO:tasks.workunit.client.1.smithi136.stdout:5/715: stat d1/dc/d1c/d32/d8b/d20/f42 0 2024-04-06T03:40:41.893 INFO:tasks.workunit.client.1.smithi136.stdout:5/716: fsync d1/d16/d89/daf/fc8 0 2024-04-06T03:40:41.893 INFO:tasks.workunit.client.1.smithi136.stdout:5/717: write d1/dc/d1c/d32/fc7 [213132,39622] 0 2024-04-06T03:40:41.893 INFO:tasks.workunit.client.0.smithi094.stdout:1/645: mknod d0/d1/d27/da5/d40/db1/dc3/cd8 0 2024-04-06T03:40:41.894 INFO:tasks.workunit.client.1.smithi136.stdout:8/721: dwrite d3/da/d26/d35/d53/fd8 [0,4194304] 0 2024-04-06T03:40:41.896 INFO:tasks.workunit.client.1.smithi136.stdout:8/722: dread d3/da/d26/d35/f38 [4194304,4194304] 0 2024-04-06T03:40:41.896 INFO:tasks.workunit.client.1.smithi136.stdout:3/639: rename d0/d4/d12/d19/d2c/f9c to d0/d4/d12/d19/d55/d3c/d5a/d79/d98/fda 0 2024-04-06T03:40:41.896 INFO:tasks.workunit.client.1.smithi136.stdout:3/640: fsync d0/d4/d12/d19/d2e/d3b/f5f 0 2024-04-06T03:40:41.898 INFO:tasks.workunit.client.1.smithi136.stdout:9/666: dread f16 [0,4194304] 0 2024-04-06T03:40:41.898 INFO:tasks.workunit.client.0.smithi094.stdout:1/646: creat d0/d1/d27/da5/d40/db1/fd9 x:0 0 0 2024-04-06T03:40:41.903 INFO:tasks.workunit.client.0.smithi094.stdout:1/647: creat d0/d1/d27/da5/d40/db1/d2f/d76/d30/fda x:0 0 0 2024-04-06T03:40:41.905 INFO:tasks.workunit.client.1.smithi136.stdout:4/612: creat d1d/d31/d3e/d55/d7f/fdc x:0 0 0 2024-04-06T03:40:41.906 INFO:tasks.workunit.client.1.smithi136.stdout:6/618: symlink d2/d7/d13/d79/lcc 0 2024-04-06T03:40:41.908 INFO:tasks.workunit.client.0.smithi094.stdout:8/722: dread d18/f21 [0,4194304] 0 2024-04-06T03:40:41.908 INFO:tasks.workunit.client.0.smithi094.stdout:8/723: creat d18/d2c/ffc x:0 0 0 2024-04-06T03:40:41.908 INFO:tasks.workunit.client.0.smithi094.stdout:8/724: readlink l12 0 2024-04-06T03:40:41.910 INFO:tasks.workunit.client.0.smithi094.stdout:1/648: mknod d0/d7/d1a/cdb 0 2024-04-06T03:40:41.910 INFO:tasks.workunit.client.0.smithi094.stdout:1/649: chown d0/d4/c75 12161593 1 2024-04-06T03:40:41.910 INFO:tasks.workunit.client.0.smithi094.stdout:1/650: write d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/fb7 [111877,128744] 0 2024-04-06T03:40:41.911 INFO:tasks.workunit.client.0.smithi094.stdout:8/725: dread d18/d3c/d76/dd9/f6d [0,4194304] 0 2024-04-06T03:40:41.915 INFO:tasks.workunit.client.0.smithi094.stdout:8/726: write f17 [5112066,26480] 0 2024-04-06T03:40:41.915 INFO:tasks.workunit.client.0.smithi094.stdout:8/727: truncate d18/d2c/d37/fa2 648771 0 2024-04-06T03:40:41.915 INFO:tasks.workunit.client.0.smithi094.stdout:8/728: creat d18/d3c/d49/d91/d94/daa/ffd x:0 0 0 2024-04-06T03:40:41.916 INFO:tasks.workunit.client.1.smithi136.stdout:2/740: symlink d3/d4a/d9d/d18/d1d/d3c/leb 0 2024-04-06T03:40:41.918 INFO:tasks.workunit.client.0.smithi094.stdout:6/591: dwrite dc/d11/d1b/d72/f8c [0,4194304] 0 2024-04-06T03:40:41.918 INFO:tasks.workunit.client.0.smithi094.stdout:1/651: truncate d0/d1/d27/da5/d40/f7a 4110224 0 2024-04-06T03:40:41.920 INFO:tasks.workunit.client.0.smithi094.stdout:1/652: dread d0/d7/f78 [0,4194304] 0 2024-04-06T03:40:41.923 INFO:tasks.workunit.client.1.smithi136.stdout:5/718: truncate d1/d16/d4b/f7e 637735 0 2024-04-06T03:40:41.926 INFO:tasks.workunit.client.1.smithi136.stdout:8/723: dwrite d3/da/d26/d35/f3c [0,4194304] 0 2024-04-06T03:40:41.927 INFO:tasks.workunit.client.1.smithi136.stdout:6/619: dwrite d2/d7/d26/d30/d4f/f57 [0,4194304] 0 2024-04-06T03:40:41.932 INFO:tasks.workunit.client.1.smithi136.stdout:5/719: write d1/dc/d10/d1e/f97 [951827,62865] 0 2024-04-06T03:40:41.936 INFO:tasks.workunit.client.0.smithi094.stdout:8/729: dwrite d18/ff9 [0,4194304] 0 2024-04-06T03:40:41.936 INFO:tasks.workunit.client.0.smithi094.stdout:8/730: fsync d18/d2c/d37/d60/da3/db8/dd8/ff4 0 2024-04-06T03:40:41.937 INFO:tasks.workunit.client.1.smithi136.stdout:6/620: dread d2/f21 [0,4194304] 0 2024-04-06T03:40:41.938 INFO:tasks.workunit.client.1.smithi136.stdout:5/720: dread d1/d1f/f21 [4194304,4194304] 0 2024-04-06T03:40:41.943 INFO:tasks.workunit.client.1.smithi136.stdout:5/721: write d1/dc/d1c/d32/f8f [2228592,104813] 0 2024-04-06T03:40:41.948 INFO:tasks.workunit.client.1.smithi136.stdout:5/722: dread d1/dc/d1c/d32/d8b/d20/d66/f9e [0,4194304] 0 2024-04-06T03:40:41.949 INFO:tasks.workunit.client.0.smithi094.stdout:8/731: write d18/ff9 [1002443,59680] 0 2024-04-06T03:40:41.949 INFO:tasks.workunit.client.0.smithi094.stdout:1/653: dwrite d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/f9c [0,4194304] 0 2024-04-06T03:40:41.949 INFO:tasks.workunit.client.0.smithi094.stdout:1/654: creat d0/d1/d27/da5/d40/fdc x:0 0 0 2024-04-06T03:40:41.953 INFO:tasks.workunit.client.1.smithi136.stdout:6/621: dwrite d2/d7/f5a [0,4194304] 0 2024-04-06T03:40:41.958 INFO:tasks.workunit.client.1.smithi136.stdout:3/641: mkdir d0/d4/d12/d19/d55/d6f/ddb 0 2024-04-06T03:40:41.959 INFO:tasks.workunit.client.1.smithi136.stdout:3/642: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/fdc x:0 0 0 2024-04-06T03:40:41.959 INFO:tasks.workunit.client.1.smithi136.stdout:0/655: rename d1/d6/l61 to d1/d6/d2c/d1a/d36/lda 0 2024-04-06T03:40:41.960 INFO:tasks.workunit.client.1.smithi136.stdout:4/613: mkdir d1d/d2f/d6c/d91/ddd 0 2024-04-06T03:40:41.961 INFO:tasks.workunit.client.1.smithi136.stdout:2/741: mkdir d3/d15/d19/d43/daa/dec 0 2024-04-06T03:40:41.962 INFO:tasks.workunit.client.1.smithi136.stdout:2/742: creat d3/d15/fed x:0 0 0 2024-04-06T03:40:41.966 INFO:tasks.workunit.client.1.smithi136.stdout:8/724: getdents d3/da/d26/d35/d3d/d48/d6c 0 2024-04-06T03:40:41.969 INFO:tasks.workunit.client.0.smithi094.stdout:1/655: truncate d0/d7/d1a/d64/f72 990418 0 2024-04-06T03:40:41.969 INFO:tasks.workunit.client.0.smithi094.stdout:1/656: read - d0/d1/d27/da5/fca zero size 2024-04-06T03:40:41.972 INFO:tasks.workunit.client.0.smithi094.stdout:6/592: rename dc/d11/d1b to dc/dd3 0 2024-04-06T03:40:41.977 INFO:tasks.workunit.client.1.smithi136.stdout:5/723: chown d1/dc/l86 65 1 2024-04-06T03:40:41.980 INFO:tasks.workunit.client.0.smithi094.stdout:1/657: rename d0/d1/d27/da5/db2/lbd to d0/d1/d27/da5/d40/db1/d2f/d76/daf/ldd 0 2024-04-06T03:40:41.980 INFO:tasks.workunit.client.0.smithi094.stdout:1/658: chown d0/d1/d27/da5/d3f 1093992874 1 2024-04-06T03:40:41.980 INFO:tasks.workunit.client.0.smithi094.stdout:1/659: fsync d0/d1/d27/da5/d40/db1/d2f/d76/d30/f32 0 2024-04-06T03:40:41.980 INFO:tasks.workunit.client.0.smithi094.stdout:1/660: write d0/d7/f36 [3867237,4764] 0 2024-04-06T03:40:41.981 INFO:tasks.workunit.client.1.smithi136.stdout:7/706: sync 2024-04-06T03:40:41.981 INFO:tasks.workunit.client.1.smithi136.stdout:1/670: sync 2024-04-06T03:40:41.981 INFO:tasks.workunit.client.1.smithi136.stdout:7/707: readlink d5/df/d12/d1b/d3a/d4a/lbd 0 2024-04-06T03:40:41.981 INFO:tasks.workunit.client.1.smithi136.stdout:1/671: creat d12/d3b/d44/d5f/fde x:0 0 0 2024-04-06T03:40:41.981 INFO:tasks.workunit.client.1.smithi136.stdout:1/672: creat d12/d1b/d1e/dd1/fdf x:0 0 0 2024-04-06T03:40:41.982 INFO:tasks.workunit.client.0.smithi094.stdout:8/732: getdents d18/d3c/d49/d80 0 2024-04-06T03:40:41.983 INFO:tasks.workunit.client.0.smithi094.stdout:8/733: truncate d18/d3c/d49/d91/ff2 333214 0 2024-04-06T03:40:41.983 INFO:tasks.workunit.client.0.smithi094.stdout:8/734: chown d18/d2c/d37/d60/da3/db1/lb4 197174 1 2024-04-06T03:40:41.983 INFO:tasks.workunit.client.0.smithi094.stdout:8/735: fsync d18/f82 0 2024-04-06T03:40:41.986 INFO:tasks.workunit.client.0.smithi094.stdout:0/661: sync 2024-04-06T03:40:41.986 INFO:tasks.workunit.client.1.smithi136.stdout:6/622: rmdir d2/d7/d26/d68 39 2024-04-06T03:40:41.988 INFO:tasks.workunit.client.0.smithi094.stdout:1/661: rename d0/d1/d27/da5/f3a to d0/fde 0 2024-04-06T03:40:41.991 INFO:tasks.workunit.client.0.smithi094.stdout:8/736: symlink d18/d3c/d49/d80/lfe 0 2024-04-06T03:40:41.998 INFO:tasks.workunit.client.1.smithi136.stdout:6/623: dread d2/d7/d26/d30/f7e [0,4194304] 0 2024-04-06T03:40:41.998 INFO:tasks.workunit.client.1.smithi136.stdout:6/624: truncate d2/d7/d26/d68/d9b/fae 53406 0 2024-04-06T03:40:42.000 INFO:tasks.workunit.client.1.smithi136.stdout:9/667: rename d17/d21/ldc to d17/d20/d25/da4/dbb/le7 0 2024-04-06T03:40:42.001 INFO:tasks.workunit.client.1.smithi136.stdout:0/656: link d1/d4e/fb0 d1/d6/d2c/d1a/d36/fdb 0 2024-04-06T03:40:42.001 INFO:tasks.workunit.client.1.smithi136.stdout:0/657: stat d1/d6/d2c/d57/d43/d6d/d85/d78 0 2024-04-06T03:40:42.001 INFO:tasks.workunit.client.1.smithi136.stdout:4/614: link d1d/d31/d3e/d56/d7b/c7c d1d/d2f/da6/dbf/cde 0 2024-04-06T03:40:42.003 INFO:tasks.workunit.client.0.smithi094.stdout:6/593: dwrite dc/dd3/f31 [0,4194304] 0 2024-04-06T03:40:42.003 INFO:tasks.workunit.client.0.smithi094.stdout:6/594: truncate dc/d3a/d4a/d94/dac/fb8 929322 0 2024-04-06T03:40:42.003 INFO:tasks.workunit.client.1.smithi136.stdout:2/743: creat d3/d15/d19/d43/daa/dec/fee x:0 0 0 2024-04-06T03:40:42.008 INFO:tasks.workunit.client.0.smithi094.stdout:6/595: truncate f7 153189 0 2024-04-06T03:40:42.009 INFO:tasks.workunit.client.0.smithi094.stdout:6/596: write dc/d11/d22/f9e [3784962,46137] 0 2024-04-06T03:40:42.010 INFO:tasks.workunit.client.1.smithi136.stdout:5/724: dwrite d1/d16/d17/f44 [4194304,4194304] 0 2024-04-06T03:40:42.010 INFO:tasks.workunit.client.1.smithi136.stdout:5/725: write d1/d1f/fbb [475339,50490] 0 2024-04-06T03:40:42.011 INFO:tasks.workunit.client.0.smithi094.stdout:1/662: dwrite d0/d1/d27/da5/d40/fdc [0,4194304] 0 2024-04-06T03:40:42.011 INFO:tasks.workunit.client.0.smithi094.stdout:0/662: dwrite d1/d9/dd/d18/d5d/d60/d6b/f59 [0,4194304] 0 2024-04-06T03:40:42.011 INFO:tasks.workunit.client.1.smithi136.stdout:1/673: creat d12/fe0 x:0 0 0 2024-04-06T03:40:42.014 INFO:tasks.workunit.client.1.smithi136.stdout:9/668: creat d17/d21/d29/de5/da5/dce/dd1/fe8 x:0 0 0 2024-04-06T03:40:42.015 INFO:tasks.workunit.client.1.smithi136.stdout:6/625: dwrite d2/d7/d26/d30/d6e/d58/d5d/f65 [0,4194304] 0 2024-04-06T03:40:42.015 INFO:tasks.workunit.client.1.smithi136.stdout:6/626: write d2/d7/d13/f76 [5317507,88406] 0 2024-04-06T03:40:42.017 INFO:tasks.workunit.client.1.smithi136.stdout:0/658: symlink d1/d6/d2c/d57/d43/d6d/d85/d86/ldc 0 2024-04-06T03:40:42.017 INFO:tasks.workunit.client.1.smithi136.stdout:0/659: write d1/d6/d2c/d1a/d36/fd5 [802147,95156] 0 2024-04-06T03:40:42.022 INFO:tasks.workunit.client.1.smithi136.stdout:4/615: link d1d/d70/f6a d1d/d31/d3e/d56/d7b/fdf 0 2024-04-06T03:40:42.023 INFO:tasks.workunit.client.0.smithi094.stdout:6/597: mkdir dc/d3a/d7f/dd4 0 2024-04-06T03:40:42.025 INFO:tasks.workunit.client.1.smithi136.stdout:9/669: dread d17/d20/d25/f84 [0,4194304] 0 2024-04-06T03:40:42.025 INFO:tasks.workunit.client.1.smithi136.stdout:2/744: stat d3/d15/d19/c38 0 2024-04-06T03:40:42.025 INFO:tasks.workunit.client.1.smithi136.stdout:9/670: write d17/d21/d29/d34/d66/de1/d51/d57/fdb [140559,114117] 0 2024-04-06T03:40:42.031 INFO:tasks.workunit.client.1.smithi136.stdout:8/725: link d3/da/d26/d35/d53/fac d3/da/d26/d90/dae/fda 0 2024-04-06T03:40:42.031 INFO:tasks.workunit.client.1.smithi136.stdout:8/726: creat d3/de/d18/d6d/fdb x:0 0 0 2024-04-06T03:40:42.031 INFO:tasks.workunit.client.1.smithi136.stdout:8/727: chown d3/da/d26/d35/d53/d86/f8f 357363735 1 2024-04-06T03:40:42.035 INFO:tasks.workunit.client.1.smithi136.stdout:5/726: dwrite d1/f4 [0,4194304] 0 2024-04-06T03:40:42.035 INFO:tasks.workunit.client.1.smithi136.stdout:5/727: chown d1/dc/d1c/d32/d8b/c2e 2 1 2024-04-06T03:40:42.035 INFO:tasks.workunit.client.1.smithi136.stdout:5/728: chown d1/f96 480794639 1 2024-04-06T03:40:42.035 INFO:tasks.workunit.client.1.smithi136.stdout:5/729: readlink d1/d1f/l8c 0 2024-04-06T03:40:42.036 INFO:tasks.workunit.client.0.smithi094.stdout:1/663: creat d0/d1/d27/da5/d40/db1/d2f/d3c/da0/fdf x:0 0 0 2024-04-06T03:40:42.037 INFO:tasks.workunit.client.1.smithi136.stdout:7/708: rmdir d5/df/d1d/d6f 39 2024-04-06T03:40:42.039 INFO:tasks.workunit.client.0.smithi094.stdout:0/663: dwrite d1/d9/dd/d18/d1e/f71 [0,4194304] 0 2024-04-06T03:40:42.039 INFO:tasks.workunit.client.0.smithi094.stdout:0/664: fsync d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/f9e 0 2024-04-06T03:40:42.040 INFO:tasks.workunit.client.1.smithi136.stdout:6/627: dwrite d2/d25/f3b [4194304,4194304] 0 2024-04-06T03:40:42.041 INFO:tasks.workunit.client.0.smithi094.stdout:1/664: symlink d0/d1/d27/da5/d40/db0/dcd/le0 0 2024-04-06T03:40:42.044 INFO:tasks.workunit.client.1.smithi136.stdout:8/728: dread d3/d2d/f55 [4194304,4194304] 0 2024-04-06T03:40:42.044 INFO:tasks.workunit.client.1.smithi136.stdout:8/729: stat d3/da/d26/d35/d53/dd2 0 2024-04-06T03:40:42.044 INFO:tasks.workunit.client.1.smithi136.stdout:0/660: mknod d1/d6/d2c/d57/d43/d6d/d85/d78/da4/cdd 0 2024-04-06T03:40:42.046 INFO:tasks.workunit.client.1.smithi136.stdout:0/661: dread d1/d6/d2c/d1a/d36/f76 [0,4194304] 0 2024-04-06T03:40:42.049 INFO:tasks.workunit.client.1.smithi136.stdout:2/745: creat d3/fef x:0 0 0 2024-04-06T03:40:42.049 INFO:tasks.workunit.client.1.smithi136.stdout:2/746: write d3/d4a/d9d/d18/d4b/fb3 [2103122,66070] 0 2024-04-06T03:40:42.050 INFO:tasks.workunit.client.1.smithi136.stdout:6/628: dread d2/d25/f59 [0,4194304] 0 2024-04-06T03:40:42.050 INFO:tasks.workunit.client.0.smithi094.stdout:0/665: rename d1/d9/dd/d18/d1e/l26 to d1/d9/dd/d18/d1e/d22/ldc 0 2024-04-06T03:40:42.052 INFO:tasks.workunit.client.1.smithi136.stdout:9/671: rename d17/d19/f1e to d17/d20/d25/da4/fe9 0 2024-04-06T03:40:42.053 INFO:tasks.workunit.client.0.smithi094.stdout:1/665: creat d0/d7/d1a/fe1 x:0 0 0 2024-04-06T03:40:42.053 INFO:tasks.workunit.client.0.smithi094.stdout:1/666: write d0/d7/f52 [1738011,40698] 0 2024-04-06T03:40:42.053 INFO:tasks.workunit.client.0.smithi094.stdout:6/598: dwrite dc/d3a/d4a/d94/dac/fb8 [0,4194304] 0 2024-04-06T03:40:42.053 INFO:tasks.workunit.client.0.smithi094.stdout:1/667: stat d0/d1/d27/da5/d40/db0 0 2024-04-06T03:40:42.053 INFO:tasks.workunit.client.0.smithi094.stdout:8/737: dwrite d18/d30/f99 [0,4194304] 0 2024-04-06T03:40:42.054 INFO:tasks.workunit.client.0.smithi094.stdout:8/738: write d18/d3c/d76/f5b [683344,19022] 0 2024-04-06T03:40:42.054 INFO:tasks.workunit.client.1.smithi136.stdout:4/616: dwrite d1d/d31/f3f [0,4194304] 0 2024-04-06T03:40:42.054 INFO:tasks.workunit.client.1.smithi136.stdout:4/617: write d1d/d31/d78/fa9 [2781996,114802] 0 2024-04-06T03:40:42.059 INFO:tasks.workunit.client.0.smithi094.stdout:0/666: dread d1/d9/dd/d18/d5d/d60/d6b/dab/f73 [0,4194304] 0 2024-04-06T03:40:42.059 INFO:tasks.workunit.client.0.smithi094.stdout:0/667: creat d1/d9/dd/d18/d1e/d86/fdd x:0 0 0 2024-04-06T03:40:42.059 INFO:tasks.workunit.client.0.smithi094.stdout:0/668: creat d1/d9/dd/fde x:0 0 0 2024-04-06T03:40:42.059 INFO:tasks.workunit.client.1.smithi136.stdout:5/730: unlink d1/d9b/db4/cd5 0 2024-04-06T03:40:42.060 INFO:tasks.workunit.client.0.smithi094.stdout:6/599: write dc/f40 [3697142,63142] 0 2024-04-06T03:40:42.060 INFO:tasks.workunit.client.0.smithi094.stdout:6/600: dread - dc/dd3/fb4 zero size 2024-04-06T03:40:42.060 INFO:tasks.workunit.client.0.smithi094.stdout:6/601: chown dc/dd3/d24/f32 7716672 1 2024-04-06T03:40:42.063 INFO:tasks.workunit.client.1.smithi136.stdout:7/709: symlink d5/df/d12/d1b/d3a/lfc 0 2024-04-06T03:40:42.068 INFO:tasks.workunit.client.0.smithi094.stdout:1/668: dread d0/d1/d27/da5/d40/db1/d2f/d76/d30/f32 [0,4194304] 0 2024-04-06T03:40:42.068 INFO:tasks.workunit.client.0.smithi094.stdout:1/669: write d0/fde [965014,49995] 0 2024-04-06T03:40:42.069 INFO:tasks.workunit.client.1.smithi136.stdout:1/674: getdents d12/d3b/d60 0 2024-04-06T03:40:42.070 INFO:tasks.workunit.client.1.smithi136.stdout:6/629: dwrite d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/fa9 [0,4194304] 0 2024-04-06T03:40:42.070 INFO:tasks.workunit.client.1.smithi136.stdout:6/630: write d2/d7/d26/d30/d6e/f35 [4422106,57549] 0 2024-04-06T03:40:42.070 INFO:tasks.workunit.client.1.smithi136.stdout:6/631: chown d2/d7/d26/d30/d4f/f5f 0 1 2024-04-06T03:40:42.070 INFO:tasks.workunit.client.1.smithi136.stdout:6/632: write d2/d7/d26/faa [367811,55183] 0 2024-04-06T03:40:42.076 INFO:tasks.workunit.client.1.smithi136.stdout:5/731: dwrite d1/dc/d1c/d32/d8b/d20/d66/fbf [0,4194304] 0 2024-04-06T03:40:42.086 INFO:tasks.workunit.client.1.smithi136.stdout:0/662: mknod d1/d6/d2c/d57/d43/d6d/d9b/cde 0 2024-04-06T03:40:42.087 INFO:tasks.workunit.client.1.smithi136.stdout:3/643: sync 2024-04-06T03:40:42.091 INFO:tasks.workunit.client.1.smithi136.stdout:9/672: rename d17/d19/l1b to d17/d20/d28/d49/d75/d60/lea 0 2024-04-06T03:40:42.091 INFO:tasks.workunit.client.1.smithi136.stdout:9/673: chown d17/d21/d29/de5 436150 1 2024-04-06T03:40:42.091 INFO:tasks.workunit.client.1.smithi136.stdout:9/674: stat d17/d20/d25/d71/fa2 0 2024-04-06T03:40:42.093 INFO:tasks.workunit.client.0.smithi094.stdout:0/669: rename d1/d9/dd/d18/d5d/d88 to d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/ddf 0 2024-04-06T03:40:42.093 INFO:tasks.workunit.client.0.smithi094.stdout:0/670: fdatasync d1/d9/dd/d18/d1e/d86/d96/daa/fca 0 2024-04-06T03:40:42.094 INFO:tasks.workunit.client.0.smithi094.stdout:0/671: chown d1/d9/dd/d18/d78/l7f 241917086 1 2024-04-06T03:40:42.095 INFO:tasks.workunit.client.1.smithi136.stdout:1/675: symlink d12/d53/le1 0 2024-04-06T03:40:42.095 INFO:tasks.workunit.client.1.smithi136.stdout:1/676: chown d12/c19 7421 1 2024-04-06T03:40:42.096 INFO:tasks.workunit.client.0.smithi094.stdout:6/602: mknod dc/dd3/d24/d46/cd5 0 2024-04-06T03:40:42.098 INFO:tasks.workunit.client.0.smithi094.stdout:4/723: sync 2024-04-06T03:40:42.098 INFO:tasks.workunit.client.0.smithi094.stdout:5/601: sync 2024-04-06T03:40:42.098 INFO:tasks.workunit.client.0.smithi094.stdout:9/631: sync 2024-04-06T03:40:42.098 INFO:tasks.workunit.client.0.smithi094.stdout:3/667: sync 2024-04-06T03:40:42.098 INFO:tasks.workunit.client.0.smithi094.stdout:5/602: fdatasync db/f2f 0 2024-04-06T03:40:42.098 INFO:tasks.workunit.client.0.smithi094.stdout:2/658: sync 2024-04-06T03:40:42.098 INFO:tasks.workunit.client.0.smithi094.stdout:7/602: sync 2024-04-06T03:40:42.098 INFO:tasks.workunit.client.0.smithi094.stdout:9/632: chown d2/f30 1 1 2024-04-06T03:40:42.100 INFO:tasks.workunit.client.0.smithi094.stdout:1/670: symlink d0/d1/d27/da5/d3f/d8c/le2 0 2024-04-06T03:40:42.105 INFO:tasks.workunit.client.1.smithi136.stdout:9/675: dwrite d17/d20/d28/d49/d75/f91 [0,4194304] 0 2024-04-06T03:40:42.107 INFO:tasks.workunit.client.0.smithi094.stdout:3/668: write d9/d20/d30/f4d [7214165,13172] 0 2024-04-06T03:40:42.107 INFO:tasks.workunit.client.0.smithi094.stdout:0/672: creat d1/d9/dd/d18/d5d/fe0 x:0 0 0 2024-04-06T03:40:42.108 INFO:tasks.workunit.client.1.smithi136.stdout:5/732: symlink d1/dc/d10/da3/ld8 0 2024-04-06T03:40:42.110 INFO:tasks.workunit.client.0.smithi094.stdout:6/603: mknod dc/dab/cd6 0 2024-04-06T03:40:42.110 INFO:tasks.workunit.client.0.smithi094.stdout:6/604: dread - dc/dd3/d24/fbe zero size 2024-04-06T03:40:42.111 INFO:tasks.workunit.client.0.smithi094.stdout:4/724: symlink d0/d4/dde/lf4 0 2024-04-06T03:40:42.111 INFO:tasks.workunit.client.0.smithi094.stdout:4/725: fdatasync d0/d15/fe1 0 2024-04-06T03:40:42.111 INFO:tasks.workunit.client.0.smithi094.stdout:4/726: write d0/d23/f6e [466606,85585] 0 2024-04-06T03:40:42.112 INFO:tasks.workunit.client.0.smithi094.stdout:2/659: link d0/dd/d19/d31/l89 d0/dd/d1b/d72/d8f/le1 0 2024-04-06T03:40:42.115 INFO:tasks.workunit.client.0.smithi094.stdout:5/603: mkdir db/d19/d21/dce 0 2024-04-06T03:40:42.115 INFO:tasks.workunit.client.0.smithi094.stdout:5/604: write db/d80/f97 [4028106,123460] 0 2024-04-06T03:40:42.118 INFO:tasks.workunit.client.0.smithi094.stdout:7/603: link d2/d6/dc/f15 d2/d6/dc/d12/d3a/d56/db1/fb5 0 2024-04-06T03:40:42.121 INFO:tasks.workunit.client.0.smithi094.stdout:9/633: unlink d2/dc/d21/d23/fa3 0 2024-04-06T03:40:42.123 INFO:tasks.workunit.client.0.smithi094.stdout:1/671: creat d0/d1/dd3/fe3 x:0 0 0 2024-04-06T03:40:42.123 INFO:tasks.workunit.client.0.smithi094.stdout:1/672: fdatasync d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/f8d 0 2024-04-06T03:40:42.123 INFO:tasks.workunit.client.0.smithi094.stdout:8/739: rmdir d18/d3c/d49/db5 39 2024-04-06T03:40:42.125 INFO:tasks.workunit.client.0.smithi094.stdout:3/669: link d9/d1e/d6e/d75/dba/ca4 d9/d20/da6/cf4 0 2024-04-06T03:40:42.125 INFO:tasks.workunit.client.0.smithi094.stdout:3/670: readlink d9/d20/l24 0 2024-04-06T03:40:42.125 INFO:tasks.workunit.client.0.smithi094.stdout:3/671: write d9/d1e/d26/d55/d62/db8/fca [690282,111860] 0 2024-04-06T03:40:42.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/673: mknod d1/d9/dd/d18/d5d/d60/d6b/d67/ce1 0 2024-04-06T03:40:42.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/674: write d1/d9/dd/d18/d1e/d86/d96/daa/fca [1195237,51967] 0 2024-04-06T03:40:42.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/675: creat d1/d9/dd/d18/d1e/fe2 x:0 0 0 2024-04-06T03:40:42.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/676: chown d1/d9/dd/d18/d5d/d60/d6b/dab/f73 892036 1 2024-04-06T03:40:42.127 INFO:tasks.workunit.client.0.smithi094.stdout:0/677: dread - d1/d9/dd/d18/d32/d42/fd3 zero size 2024-04-06T03:40:42.127 INFO:tasks.workunit.client.0.smithi094.stdout:2/660: dwrite d0/dd/d19/f4c [0,4194304] 0 2024-04-06T03:40:42.127 INFO:tasks.workunit.client.0.smithi094.stdout:2/661: write d0/dd/d19/d31/d67/d98/fb7 [1844723,92650] 0 2024-04-06T03:40:42.127 INFO:tasks.workunit.client.0.smithi094.stdout:2/662: fsync d0/f5 0 2024-04-06T03:40:42.127 INFO:tasks.workunit.client.0.smithi094.stdout:6/605: creat dc/dd3/d4e/fd7 x:0 0 0 2024-04-06T03:40:42.129 INFO:tasks.workunit.client.0.smithi094.stdout:4/727: creat d0/d4/d67/dcb/ff5 x:0 0 0 2024-04-06T03:40:42.129 INFO:tasks.workunit.client.0.smithi094.stdout:4/728: chown d0/df/d13 1 1 2024-04-06T03:40:42.129 INFO:tasks.workunit.client.0.smithi094.stdout:4/729: chown d0/df/d13/f8c 1867249 1 2024-04-06T03:40:42.129 INFO:tasks.workunit.client.0.smithi094.stdout:4/730: chown d0/de8 18 1 2024-04-06T03:40:42.131 INFO:tasks.workunit.client.0.smithi094.stdout:7/604: symlink d2/d6/dc/d12/d9a/da0/lb6 0 2024-04-06T03:40:42.131 INFO:tasks.workunit.client.0.smithi094.stdout:7/605: write d2/d6/dc/dd/df/d36/f4d [1198372,81239] 0 2024-04-06T03:40:42.132 INFO:tasks.workunit.client.0.smithi094.stdout:9/634: getdents d2/dc/d10/d6a 0 2024-04-06T03:40:42.132 INFO:tasks.workunit.client.0.smithi094.stdout:9/635: stat d2/dc/d21/d26/d78/d69/da8/f55 0 2024-04-06T03:40:42.135 INFO:tasks.workunit.client.0.smithi094.stdout:1/673: link d0/d1/d27/da5/d40/db1/d2f/d76/d7d/fae d0/d1/d27/da5/d40/db1/d2f/d76/d30/fe4 0 2024-04-06T03:40:42.136 INFO:tasks.workunit.client.1.smithi136.stdout:8/730: rmdir d3 39 2024-04-06T03:40:42.136 INFO:tasks.workunit.client.1.smithi136.stdout:8/731: truncate d3/de/fc5 1173999 0 2024-04-06T03:40:42.137 INFO:tasks.workunit.client.0.smithi094.stdout:8/740: mknod d18/d2c/d37/d60/da3/deb/cff 0 2024-04-06T03:40:42.139 INFO:tasks.workunit.client.1.smithi136.stdout:0/663: symlink d1/d6/d2c/d57/d43/d6b/ldf 0 2024-04-06T03:40:42.140 INFO:tasks.workunit.client.0.smithi094.stdout:3/672: mknod d9/d20/d30/cf5 0 2024-04-06T03:40:42.140 INFO:tasks.workunit.client.0.smithi094.stdout:3/673: fsync d9/d20/d30/f69 0 2024-04-06T03:40:42.141 INFO:tasks.workunit.client.0.smithi094.stdout:5/605: dwrite db/d40/d77/fcd [0,4194304] 0 2024-04-06T03:40:42.141 INFO:tasks.workunit.client.1.smithi136.stdout:2/747: mkdir d3/d4a/d9d/d8a/d93/d5f/df0 0 2024-04-06T03:40:42.146 INFO:tasks.workunit.client.0.smithi094.stdout:9/636: write d2/dc/d21/d23/d47/d51/f8c [2423776,76776] 0 2024-04-06T03:40:42.148 INFO:tasks.workunit.client.0.smithi094.stdout:6/606: truncate dc/f28 1532510 0 2024-04-06T03:40:42.150 INFO:tasks.workunit.client.0.smithi094.stdout:0/678: dwrite d1/d9/dd/d18/d1e/d86/d96/fd6 [0,4194304] 0 2024-04-06T03:40:42.150 INFO:tasks.workunit.client.0.smithi094.stdout:4/731: creat d0/df/d13/ff6 x:0 0 0 2024-04-06T03:40:42.150 INFO:tasks.workunit.client.0.smithi094.stdout:4/732: chown d0/d4/d67/dcb 3724732 1 2024-04-06T03:40:42.152 INFO:tasks.workunit.client.1.smithi136.stdout:1/677: mkdir d12/d3b/d44/d5f/d5a/de2 0 2024-04-06T03:40:42.152 INFO:tasks.workunit.client.0.smithi094.stdout:1/674: truncate d0/d1/d27/da5/d40/db1/d2f/d76/d7d/fae 4016497 0 2024-04-06T03:40:42.156 INFO:tasks.workunit.client.0.smithi094.stdout:3/674: symlink d9/d1e/d26/d54/lf6 0 2024-04-06T03:40:42.157 INFO:tasks.workunit.client.0.smithi094.stdout:2/663: dwrite d0/d8/fb3 [0,4194304] 0 2024-04-06T03:40:42.157 INFO:tasks.workunit.client.0.smithi094.stdout:2/664: write d0/d8/d4f/dab/dc3/fe0 [149714,18333] 0 2024-04-06T03:40:42.157 INFO:tasks.workunit.client.0.smithi094.stdout:7/606: dwrite d2/d6/dc/dd/df/fa9 [0,4194304] 0 2024-04-06T03:40:42.158 INFO:tasks.workunit.client.0.smithi094.stdout:8/741: dwrite d18/d2c/d37/fa2 [0,4194304] 0 2024-04-06T03:40:42.161 INFO:tasks.workunit.client.1.smithi136.stdout:5/733: creat d1/d16/d17/fd9 x:0 0 0 2024-04-06T03:40:42.161 INFO:tasks.workunit.client.1.smithi136.stdout:5/734: creat d1/dc/d1c/d32/d8b/d20/d66/fda x:0 0 0 2024-04-06T03:40:42.162 INFO:tasks.workunit.client.1.smithi136.stdout:8/732: creat d3/da/d26/d35/d53/d86/fdc x:0 0 0 2024-04-06T03:40:42.164 INFO:tasks.workunit.client.1.smithi136.stdout:0/664: mknod d1/d6/d2c/d57/d43/d6d/d9b/d5c/d96/ce0 0 2024-04-06T03:40:42.165 INFO:tasks.workunit.client.0.smithi094.stdout:4/733: dwrite d0/d4/d67/fe0 [0,4194304] 0 2024-04-06T03:40:42.166 INFO:tasks.workunit.client.0.smithi094.stdout:5/606: mkdir db/d40/dcf 0 2024-04-06T03:40:42.166 INFO:tasks.workunit.client.1.smithi136.stdout:0/665: read d1/d6/d2c/d57/d62/f65 [270401,64813] 0 2024-04-06T03:40:42.166 INFO:tasks.workunit.client.1.smithi136.stdout:0/666: fdatasync d1/d6/d2c/d57/d62/f65 0 2024-04-06T03:40:42.166 INFO:tasks.workunit.client.1.smithi136.stdout:0/667: truncate d1/d6/d2c/d57/d43/d6d/d9b/d5c/fc7 590922 0 2024-04-06T03:40:42.166 INFO:tasks.workunit.client.1.smithi136.stdout:0/668: chown d1/d6/d2c/d1a/d36/fdb 23 1 2024-04-06T03:40:42.167 INFO:tasks.workunit.client.1.smithi136.stdout:2/748: unlink d3/d4a/d9d/d8a/d93/c99 0 2024-04-06T03:40:42.168 INFO:tasks.workunit.client.1.smithi136.stdout:5/735: dread d1/dc/d1c/d32/f38 [0,4194304] 0 2024-04-06T03:40:42.172 INFO:tasks.workunit.client.0.smithi094.stdout:6/607: mkdir dc/dd8 0 2024-04-06T03:40:42.174 INFO:tasks.workunit.client.0.smithi094.stdout:5/607: dread db/d40/d77/fcd [0,4194304] 0 2024-04-06T03:40:42.174 INFO:tasks.workunit.client.0.smithi094.stdout:5/608: truncate db/d7c/d8b/fc3 992450 0 2024-04-06T03:40:42.174 INFO:tasks.workunit.client.0.smithi094.stdout:5/609: chown db/f11 1291 1 2024-04-06T03:40:42.175 INFO:tasks.workunit.client.0.smithi094.stdout:5/610: dread db/d40/f43 [0,4194304] 0 2024-04-06T03:40:42.176 INFO:tasks.workunit.client.1.smithi136.stdout:4/618: rename d1d/d31/l82 to d1d/d31/d8e/le0 0 2024-04-06T03:40:42.176 INFO:tasks.workunit.client.1.smithi136.stdout:4/619: fsync d1d/d31/f67 0 2024-04-06T03:40:42.176 INFO:tasks.workunit.client.1.smithi136.stdout:4/620: write d1d/d70/f68 [4574120,67664] 0 2024-04-06T03:40:42.179 INFO:tasks.workunit.client.1.smithi136.stdout:1/678: unlink d12/d3b/d60/d7e/cb5 0 2024-04-06T03:40:42.179 INFO:tasks.workunit.client.1.smithi136.stdout:1/679: read d12/d1b/d1e/f99 [42418,101076] 0 2024-04-06T03:40:42.181 INFO:tasks.workunit.client.1.smithi136.stdout:0/669: dwrite d1/d53/f6e [0,4194304] 0 2024-04-06T03:40:42.182 INFO:tasks.workunit.client.1.smithi136.stdout:5/736: read d1/dc/d10/d1e/f69 [4146024,108453] 0 2024-04-06T03:40:42.183 INFO:tasks.workunit.client.0.smithi094.stdout:0/679: mknod d1/d9/dd/d18/dad/ce3 0 2024-04-06T03:40:42.184 INFO:tasks.workunit.client.1.smithi136.stdout:8/733: symlink d3/da/ldd 0 2024-04-06T03:40:42.184 INFO:tasks.workunit.client.1.smithi136.stdout:8/734: fdatasync d3/da/d26/d90/dae/fda 0 2024-04-06T03:40:42.185 INFO:tasks.workunit.client.0.smithi094.stdout:1/675: creat d0/d1/d27/da5/d40/db1/d2f/d3c/d48/fe5 x:0 0 0 2024-04-06T03:40:42.186 INFO:tasks.workunit.client.1.smithi136.stdout:2/749: symlink d3/d4a/d9d/d18/d4b/dc0/dd1/lf1 0 2024-04-06T03:40:42.188 INFO:tasks.workunit.client.0.smithi094.stdout:3/675: link d9/d10/fd8 d9/d20/d76/ff7 0 2024-04-06T03:40:42.193 INFO:tasks.workunit.client.1.smithi136.stdout:4/621: mkdir d1d/d2f/da6/de1 0 2024-04-06T03:40:42.194 INFO:tasks.workunit.client.0.smithi094.stdout:1/676: read d0/d1/d27/da5/d40/db1/d2f/d3c/d48/f88 [2480061,20759] 0 2024-04-06T03:40:42.195 INFO:tasks.workunit.client.1.smithi136.stdout:5/737: dwrite d1/dc/d10/d1e/f69 [8388608,4194304] 0 2024-04-06T03:40:42.195 INFO:tasks.workunit.client.1.smithi136.stdout:5/738: dread - d1/d9b/d9c/da9/fd4 zero size 2024-04-06T03:40:42.195 INFO:tasks.workunit.client.1.smithi136.stdout:2/750: write d3/d4a/d9d/d18/d1d/d3c/fa0 [497322,60435] 0 2024-04-06T03:40:42.198 INFO:tasks.workunit.client.1.smithi136.stdout:0/670: mkdir d1/d6/d2c/d57/d43/dcf/de1 0 2024-04-06T03:40:42.200 INFO:tasks.workunit.client.1.smithi136.stdout:8/735: creat d3/da/d26/d90/fde x:0 0 0 2024-04-06T03:40:42.200 INFO:tasks.workunit.client.1.smithi136.stdout:8/736: fdatasync d3/da/d26/d90/dae/fda 0 2024-04-06T03:40:42.200 INFO:tasks.workunit.client.1.smithi136.stdout:8/737: readlink d3/de/d60/lbb 0 2024-04-06T03:40:42.203 INFO:tasks.workunit.client.1.smithi136.stdout:4/622: dread d1d/d31/d8e/f9a [0,4194304] 0 2024-04-06T03:40:42.203 INFO:tasks.workunit.client.1.smithi136.stdout:4/623: write d1d/d31/d3e/d56/f5e [2267812,122799] 0 2024-04-06T03:40:42.207 INFO:tasks.workunit.client.1.smithi136.stdout:5/739: mknod d1/d16/d4b/d7b/cdb 0 2024-04-06T03:40:42.207 INFO:tasks.workunit.client.1.smithi136.stdout:5/740: dread - d1/dc/fc9 zero size 2024-04-06T03:40:42.207 INFO:tasks.workunit.client.0.smithi094.stdout:8/742: rmdir d18/d2c/d37 39 2024-04-06T03:40:42.208 INFO:tasks.workunit.client.1.smithi136.stdout:0/671: dread d1/d6/f58 [0,4194304] 0 2024-04-06T03:40:42.208 INFO:tasks.workunit.client.1.smithi136.stdout:0/672: write d1/d6/d2c/d1a/d36/fdb [495831,108177] 0 2024-04-06T03:40:42.208 INFO:tasks.workunit.client.1.smithi136.stdout:3/644: sync 2024-04-06T03:40:42.208 INFO:tasks.workunit.client.1.smithi136.stdout:2/751: creat d3/d4a/ff2 x:0 0 0 2024-04-06T03:40:42.208 INFO:tasks.workunit.client.1.smithi136.stdout:2/752: write d3/d4a/d9d/d18/d1d/d3c/fa1 [715823,89807] 0 2024-04-06T03:40:42.209 INFO:tasks.workunit.client.0.smithi094.stdout:8/743: dread d18/d3c/d49/d91/d94/daa/fd1 [0,4194304] 0 2024-04-06T03:40:42.210 INFO:tasks.workunit.client.0.smithi094.stdout:8/744: readlink d18/d3c/dc4/dbc/lc5 0 2024-04-06T03:40:42.212 INFO:tasks.workunit.client.1.smithi136.stdout:5/741: dread d1/dc/d10/f28 [0,4194304] 0 2024-04-06T03:40:42.212 INFO:tasks.workunit.client.1.smithi136.stdout:5/742: stat d1/f40 0 2024-04-06T03:40:42.212 INFO:tasks.workunit.client.1.smithi136.stdout:4/624: rmdir d1d/d31/d3e/dd2 0 2024-04-06T03:40:42.212 INFO:tasks.workunit.client.1.smithi136.stdout:4/625: write d1d/d31/fa2 [1048642,94031] 0 2024-04-06T03:40:42.213 INFO:tasks.workunit.client.1.smithi136.stdout:4/626: write d1d/d31/f44 [680232,73503] 0 2024-04-06T03:40:42.217 INFO:tasks.workunit.client.1.smithi136.stdout:7/710: rename d5/df/d12/d59 to d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd 0 2024-04-06T03:40:42.221 INFO:tasks.workunit.client.1.smithi136.stdout:5/743: write d1/dc/d1c/d32/d8b/d20/f41 [4059200,8605] 0 2024-04-06T03:40:42.221 INFO:tasks.workunit.client.1.smithi136.stdout:5/744: truncate d1/dc/d1c/f34 2548594 0 2024-04-06T03:40:42.222 INFO:tasks.workunit.client.1.smithi136.stdout:3/645: creat d0/d4/d12/d19/d55/fdd x:0 0 0 2024-04-06T03:40:42.222 INFO:tasks.workunit.client.1.smithi136.stdout:3/646: write d0/d4/d12/d19/d55/fdd [240209,85942] 0 2024-04-06T03:40:42.222 INFO:tasks.workunit.client.1.smithi136.stdout:3/647: fsync d0/d4/d12/d19/d55/d3c/d5a/f71 0 2024-04-06T03:40:42.223 INFO:tasks.workunit.client.0.smithi094.stdout:9/637: mknod d2/ce8 0 2024-04-06T03:40:42.223 INFO:tasks.workunit.client.0.smithi094.stdout:9/638: read d2/f14 [5067871,35930] 0 2024-04-06T03:40:42.225 INFO:tasks.workunit.client.1.smithi136.stdout:2/753: mkdir d3/d9/df3 0 2024-04-06T03:40:42.225 INFO:tasks.workunit.client.1.smithi136.stdout:2/754: write d3/d4a/d9d/d18/f23 [1960145,72676] 0 2024-04-06T03:40:42.227 INFO:tasks.workunit.client.0.smithi094.stdout:6/608: mkdir dc/d3a/d7f/dd9 0 2024-04-06T03:40:42.227 INFO:tasks.workunit.client.0.smithi094.stdout:6/609: write dc/d11/d22/d50/f69 [1377191,92733] 0 2024-04-06T03:40:42.227 INFO:tasks.workunit.client.1.smithi136.stdout:5/745: write d1/d9b/d9c/fbe [1710180,30623] 0 2024-04-06T03:40:42.228 INFO:tasks.workunit.client.0.smithi094.stdout:6/610: dread dc/dd3/d24/d46/fa5 [0,4194304] 0 2024-04-06T03:40:42.230 INFO:tasks.workunit.client.1.smithi136.stdout:5/746: dread d1/dc/d1c/fd1 [0,4194304] 0 2024-04-06T03:40:42.230 INFO:tasks.workunit.client.0.smithi094.stdout:5/611: symlink db/d19/d2c/d57/ld0 0 2024-04-06T03:40:42.234 INFO:tasks.workunit.client.1.smithi136.stdout:7/711: creat d5/df/d53/d68/da3/dd0/df0/ffe x:0 0 0 2024-04-06T03:40:42.236 INFO:tasks.workunit.client.0.smithi094.stdout:3/676: mkdir d9/d1e/d26/d6c/d8e/df8 0 2024-04-06T03:40:42.236 INFO:tasks.workunit.client.0.smithi094.stdout:3/677: dread - d9/d20/d30/d88/ff1 zero size 2024-04-06T03:40:42.236 INFO:tasks.workunit.client.0.smithi094.stdout:9/639: dwrite d2/dc/d10/f66 [0,4194304] 0 2024-04-06T03:40:42.237 INFO:tasks.workunit.client.1.smithi136.stdout:2/755: read d3/d15/f73 [1359059,68724] 0 2024-04-06T03:40:42.238 INFO:tasks.workunit.client.0.smithi094.stdout:1/677: creat d0/d7/d1a/d74/fe6 x:0 0 0 2024-04-06T03:40:42.240 INFO:tasks.workunit.client.0.smithi094.stdout:2/665: mkdir d0/dd/d1b/de2 0 2024-04-06T03:40:42.240 INFO:tasks.workunit.client.0.smithi094.stdout:2/666: fsync d0/dd/da4/fcb 0 2024-04-06T03:40:42.242 INFO:tasks.workunit.client.0.smithi094.stdout:5/612: dread db/d19/d21/f26 [0,4194304] 0 2024-04-06T03:40:42.243 INFO:tasks.workunit.client.0.smithi094.stdout:3/678: dread d9/d20/d76/f5c [0,4194304] 0 2024-04-06T03:40:42.243 INFO:tasks.workunit.client.0.smithi094.stdout:3/679: fdatasync d9/d1d/d7d/f9c 0 2024-04-06T03:40:42.245 INFO:tasks.workunit.client.0.smithi094.stdout:7/607: rmdir d2/d6/dc/d12/d3a/d56 39 2024-04-06T03:40:42.248 INFO:tasks.workunit.client.0.smithi094.stdout:8/745: mkdir d18/d3c/d49/d91/d94/df3/d100 0 2024-04-06T03:40:42.250 INFO:tasks.workunit.client.0.smithi094.stdout:6/611: dwrite dc/d11/d22/f57 [0,4194304] 0 2024-04-06T03:40:42.252 INFO:tasks.workunit.client.0.smithi094.stdout:4/734: getdents d0/d15/d3a/d61/d81 0 2024-04-06T03:40:42.252 INFO:tasks.workunit.client.0.smithi094.stdout:4/735: readlink d0/d15/d3a/l4f 0 2024-04-06T03:40:42.252 INFO:tasks.workunit.client.0.smithi094.stdout:4/736: chown d0/d15/d3a/d61/d81/da0/cd5 452 1 2024-04-06T03:40:42.252 INFO:tasks.workunit.client.0.smithi094.stdout:4/737: creat d0/d4/d4b/ff7 x:0 0 0 2024-04-06T03:40:42.258 INFO:tasks.workunit.client.0.smithi094.stdout:0/680: truncate d1/d9/dd/d18/d1e/d86/d96/fd6 3637157 0 2024-04-06T03:40:42.261 INFO:tasks.workunit.client.0.smithi094.stdout:5/613: dwrite db/d19/d24/f9d [0,4194304] 0 2024-04-06T03:40:42.261 INFO:tasks.workunit.client.0.smithi094.stdout:9/640: mkdir d2/dc/d21/d26/d8a/de9 0 2024-04-06T03:40:42.263 INFO:tasks.workunit.client.1.smithi136.stdout:3/648: mknod d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/cde 0 2024-04-06T03:40:42.263 INFO:tasks.workunit.client.1.smithi136.stdout:3/649: dread - d0/d4/d12/d19/d55/d3c/d5a/d79/d98/fcc zero size 2024-04-06T03:40:42.264 INFO:tasks.workunit.client.0.smithi094.stdout:1/678: mkdir d0/d1/dd3/de7 0 2024-04-06T03:40:42.264 INFO:tasks.workunit.client.0.smithi094.stdout:1/679: read d0/d1/d27/da5/d40/db1/d2f/d3c/d48/f69 [1800905,43559] 0 2024-04-06T03:40:42.265 INFO:tasks.workunit.client.0.smithi094.stdout:7/608: dwrite d2/d6/dac/f70 [4194304,4194304] 0 2024-04-06T03:40:42.267 INFO:tasks.workunit.client.0.smithi094.stdout:2/667: unlink d0/c7 0 2024-04-06T03:40:42.269 INFO:tasks.workunit.client.0.smithi094.stdout:3/680: mknod d9/d10/dd4/cf9 0 2024-04-06T03:40:42.269 INFO:tasks.workunit.client.0.smithi094.stdout:3/681: readlink d9/d10/l17 0 2024-04-06T03:40:42.271 INFO:tasks.workunit.client.0.smithi094.stdout:7/609: dread d2/d6/dc/d12/d61/f83 [0,4194304] 0 2024-04-06T03:40:42.272 INFO:tasks.workunit.client.0.smithi094.stdout:8/746: dwrite fd [0,4194304] 0 2024-04-06T03:40:42.272 INFO:tasks.workunit.client.0.smithi094.stdout:6/612: dwrite dc/d3a/d4a/d54/d71/fcb [0,4194304] 0 2024-04-06T03:40:42.272 INFO:tasks.workunit.client.0.smithi094.stdout:6/613: creat dc/d3a/fda x:0 0 0 2024-04-06T03:40:42.273 INFO:tasks.workunit.client.0.smithi094.stdout:6/614: truncate dc/d11/d22/d50/f8f 754930 0 2024-04-06T03:40:42.273 INFO:tasks.workunit.client.0.smithi094.stdout:6/615: write dc/d3a/da2/fa7 [131716,63349] 0 2024-04-06T03:40:42.273 INFO:tasks.workunit.client.1.smithi136.stdout:5/747: unlink d1/d1f/fbb 0 2024-04-06T03:40:42.274 INFO:tasks.workunit.client.0.smithi094.stdout:6/616: write dc/d11/d22/d50/d61/f68 [35008,118190] 0 2024-04-06T03:40:42.280 INFO:tasks.workunit.client.0.smithi094.stdout:4/738: rmdir d0/d4/d67/d9f 39 2024-04-06T03:40:42.280 INFO:tasks.workunit.client.0.smithi094.stdout:2/668: dwrite d0/dd/f40 [4194304,4194304] 0 2024-04-06T03:40:42.280 INFO:tasks.workunit.client.0.smithi094.stdout:2/669: dread - d0/dd/d1b/d72/d8f/f93 zero size 2024-04-06T03:40:42.280 INFO:tasks.workunit.client.0.smithi094.stdout:2/670: write d0/d8/d1d/f38 [138838,39929] 0 2024-04-06T03:40:42.280 INFO:tasks.workunit.client.0.smithi094.stdout:2/671: creat d0/dd/d1b/d72/fe3 x:0 0 0 2024-04-06T03:40:42.280 INFO:tasks.workunit.client.1.smithi136.stdout:7/712: truncate d5/df/d12/d1b/d3a/d4a/d70/ddb/fb1 556490 0 2024-04-06T03:40:42.281 INFO:tasks.workunit.client.0.smithi094.stdout:8/747: write d18/f82 [2471435,5234] 0 2024-04-06T03:40:42.281 INFO:tasks.workunit.client.0.smithi094.stdout:8/748: readlink d18/d3c/lcc 0 2024-04-06T03:40:42.283 INFO:tasks.workunit.client.1.smithi136.stdout:2/756: symlink d3/d4a/lf4 0 2024-04-06T03:40:42.285 INFO:tasks.workunit.client.1.smithi136.stdout:3/650: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d61/fdf x:0 0 0 2024-04-06T03:40:42.286 INFO:tasks.workunit.client.1.smithi136.stdout:5/748: write d1/dc/f6e [1552974,77898] 0 2024-04-06T03:40:42.287 INFO:tasks.workunit.client.1.smithi136.stdout:5/749: dread d1/d16/d4b/f8e [0,4194304] 0 2024-04-06T03:40:42.287 INFO:tasks.workunit.client.1.smithi136.stdout:5/750: write d1/dc/d1c/d32/d8b/f73 [179278,112422] 0 2024-04-06T03:40:42.288 INFO:tasks.workunit.client.0.smithi094.stdout:5/614: symlink db/d19/d24/d2e/ld1 0 2024-04-06T03:40:42.289 INFO:tasks.workunit.client.1.smithi136.stdout:6/633: rename d2/d7/d26/d30/d6e/d3d/d47/d84 to d2/d7/d26/dcd 0 2024-04-06T03:40:42.290 INFO:tasks.workunit.client.0.smithi094.stdout:9/641: mknod d2/dc/d21/d23/d5b/de2/cea 0 2024-04-06T03:40:42.291 INFO:tasks.workunit.client.1.smithi136.stdout:7/713: mknod d5/df/cff 0 2024-04-06T03:40:42.291 INFO:tasks.workunit.client.1.smithi136.stdout:7/714: fsync d5/df/d12/d1b/dbb/fd3 0 2024-04-06T03:40:42.293 INFO:tasks.workunit.client.1.smithi136.stdout:2/757: rmdir d3/d9/d3b/d70 39 2024-04-06T03:40:42.294 INFO:tasks.workunit.client.1.smithi136.stdout:3/651: dread d0/d4/d12/d19/d55/d6f/d82/fb4 [0,4194304] 0 2024-04-06T03:40:42.294 INFO:tasks.workunit.client.1.smithi136.stdout:5/751: dread d1/d16/d4b/f4f [0,4194304] 0 2024-04-06T03:40:42.300 INFO:tasks.workunit.client.0.smithi094.stdout:3/682: creat d9/d1e/d6e/d75/dba/d3a/ffa x:0 0 0 2024-04-06T03:40:42.302 INFO:tasks.workunit.client.1.smithi136.stdout:5/752: read d1/d16/d4b/d7b/f83 [3432851,58741] 0 2024-04-06T03:40:42.303 INFO:tasks.workunit.client.0.smithi094.stdout:7/610: mkdir d2/d6/dc/d12/d29/d35/da4/db7 0 2024-04-06T03:40:42.303 INFO:tasks.workunit.client.0.smithi094.stdout:7/611: chown d2/d6/dc/d12/d29/d35/f59 124601 1 2024-04-06T03:40:42.303 INFO:tasks.workunit.client.0.smithi094.stdout:7/612: creat d2/d6/dc/dd/df/fb8 x:0 0 0 2024-04-06T03:40:42.303 INFO:tasks.workunit.client.0.smithi094.stdout:7/613: dread - d2/d6/dc/dd/df/fb8 zero size 2024-04-06T03:40:42.304 INFO:tasks.workunit.client.1.smithi136.stdout:6/634: symlink d2/d93/dab/lce 0 2024-04-06T03:40:42.304 INFO:tasks.workunit.client.1.smithi136.stdout:7/715: truncate d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/f5e 863289 0 2024-04-06T03:40:42.305 INFO:tasks.workunit.client.1.smithi136.stdout:7/716: write d5/df/d12/d1b/d3a/d4a/f86 [474524,120662] 0 2024-04-06T03:40:42.310 INFO:tasks.workunit.client.1.smithi136.stdout:7/717: dread d5/df/d12/f37 [0,4194304] 0 2024-04-06T03:40:42.316 INFO:tasks.workunit.client.1.smithi136.stdout:3/652: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/fe0 x:0 0 0 2024-04-06T03:40:42.316 INFO:tasks.workunit.client.1.smithi136.stdout:3/653: fdatasync d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/fb5 0 2024-04-06T03:40:42.317 INFO:tasks.workunit.client.1.smithi136.stdout:3/654: chown d0/d4/d12/d19/d55/d3c/d5a/d79/d98 2 1 2024-04-06T03:40:42.318 INFO:tasks.workunit.client.0.smithi094.stdout:6/617: mknod dc/d3a/d4a/d94/cdb 0 2024-04-06T03:40:42.324 INFO:tasks.workunit.client.0.smithi094.stdout:4/739: truncate d0/d15/f8f 3396599 0 2024-04-06T03:40:42.324 INFO:tasks.workunit.client.0.smithi094.stdout:4/740: read - d0/d4/d67/d9f/fdb zero size 2024-04-06T03:40:42.325 INFO:tasks.workunit.client.1.smithi136.stdout:5/753: dwrite d1/d16/d4b/f95 [0,4194304] 0 2024-04-06T03:40:42.325 INFO:tasks.workunit.client.1.smithi136.stdout:9/676: rename d17/d20/d25/d3b to d17/d19/da3/dc5/deb 0 2024-04-06T03:40:42.326 INFO:tasks.workunit.client.1.smithi136.stdout:9/677: fsync d17/d21/d29/f92 0 2024-04-06T03:40:42.326 INFO:tasks.workunit.client.1.smithi136.stdout:9/678: chown d17/d20/f95 335392 1 2024-04-06T03:40:42.329 INFO:tasks.workunit.client.0.smithi094.stdout:1/680: dwrite d0/d7/d1a/d64/f72 [0,4194304] 0 2024-04-06T03:40:42.330 INFO:tasks.workunit.client.1.smithi136.stdout:6/635: link d2/d7/d26/d30/d6e/d3d/l55 d2/d7/d26/dcd/lcf 0 2024-04-06T03:40:42.334 INFO:tasks.workunit.client.1.smithi136.stdout:3/655: mkdir d0/d4/d12/dc3/dd3/de1 0 2024-04-06T03:40:42.335 INFO:tasks.workunit.client.0.smithi094.stdout:2/672: mknod d0/d8/d27/d2b/d8e/ce4 0 2024-04-06T03:40:42.338 INFO:tasks.workunit.client.0.smithi094.stdout:6/618: dwrite dc/dd3/d24/d46/d7b/dc6/fcd [0,4194304] 0 2024-04-06T03:40:42.338 INFO:tasks.workunit.client.0.smithi094.stdout:6/619: chown dc/dd3/d72/f81 3297 1 2024-04-06T03:40:42.343 INFO:tasks.workunit.client.0.smithi094.stdout:8/749: mknod d18/d3c/d49/d91/d94/daa/ddc/c101 0 2024-04-06T03:40:42.343 INFO:tasks.workunit.client.0.smithi094.stdout:8/750: readlink d18/d2c/dd2/le8 0 2024-04-06T03:40:42.345 INFO:tasks.workunit.client.1.smithi136.stdout:4/627: sync 2024-04-06T03:40:42.348 INFO:tasks.workunit.client.1.smithi136.stdout:1/680: rename d12/d3b/d74/d7a/cb9 to d12/d3b/d44/d5f/d5a/d72/d77/d96/ce3 0 2024-04-06T03:40:42.351 INFO:tasks.workunit.client.1.smithi136.stdout:9/679: truncate d17/d21/d32/fc7 266293 0 2024-04-06T03:40:42.352 INFO:tasks.workunit.client.0.smithi094.stdout:5/615: mknod db/d19/d24/d2e/cd2 0 2024-04-06T03:40:42.352 INFO:tasks.workunit.client.0.smithi094.stdout:5/616: write db/d80/f97 [222713,121216] 0 2024-04-06T03:40:42.352 INFO:tasks.workunit.client.0.smithi094.stdout:5/617: stat db/d40/d77/fa9 0 2024-04-06T03:40:42.353 INFO:tasks.workunit.client.1.smithi136.stdout:6/636: creat d2/d7/d26/d30/d6e/d58/d5d/fd0 x:0 0 0 2024-04-06T03:40:42.354 INFO:tasks.workunit.client.0.smithi094.stdout:9/642: creat d2/dc/d80/db7/dd0/feb x:0 0 0 2024-04-06T03:40:42.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/643: dread d2/dc/d21/d23/d47/d51/f92 [0,4194304] 0 2024-04-06T03:40:42.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/644: fsync d2/dc/d21/d26/d78/d69/da8/f3a 0 2024-04-06T03:40:42.356 INFO:tasks.workunit.client.1.smithi136.stdout:5/754: unlink d1/c39 0 2024-04-06T03:40:42.359 INFO:tasks.workunit.client.1.smithi136.stdout:8/738: rename d3/da/d26/d35/d3d/d48 to d3/de/d18/d7d/ddf 0 2024-04-06T03:40:42.361 INFO:tasks.workunit.client.1.smithi136.stdout:3/656: dwrite d0/d4/d12/d19/d55/d3c/d46/f72 [0,4194304] 0 2024-04-06T03:40:42.365 INFO:tasks.workunit.client.1.smithi136.stdout:1/681: mknod d12/d3b/d44/d5f/db7/ce4 0 2024-04-06T03:40:42.366 INFO:tasks.workunit.client.0.smithi094.stdout:2/673: dwrite d0/d8/da0/fd9 [0,4194304] 0 2024-04-06T03:40:42.366 INFO:tasks.workunit.client.0.smithi094.stdout:7/614: truncate d2/d6/dc/dd/f28 1609663 0 2024-04-06T03:40:42.366 INFO:tasks.workunit.client.0.smithi094.stdout:7/615: write d2/d6/fa [3278035,99210] 0 2024-04-06T03:40:42.368 INFO:tasks.workunit.client.1.smithi136.stdout:9/680: truncate d17/d20/f48 1520824 0 2024-04-06T03:40:42.373 INFO:tasks.workunit.client.0.smithi094.stdout:5/618: read db/fa8 [1157556,16612] 0 2024-04-06T03:40:42.374 INFO:tasks.workunit.client.1.smithi136.stdout:0/673: rename d1/d6/d2c/d1a/d9c/fae to d1/d6/d2c/d57/d43/d6d/d85/d86/fe2 0 2024-04-06T03:40:42.374 INFO:tasks.workunit.client.1.smithi136.stdout:0/674: chown d1/d6/d2c/d1a/d36/l8c 1573846 1 2024-04-06T03:40:42.380 INFO:tasks.workunit.client.1.smithi136.stdout:6/637: dwrite d2/fa [0,4194304] 0 2024-04-06T03:40:42.383 INFO:tasks.workunit.client.1.smithi136.stdout:5/755: dwrite d1/d16/d4b/f4f [0,4194304] 0 2024-04-06T03:40:42.383 INFO:tasks.workunit.client.1.smithi136.stdout:5/756: write d1/dc/d10/d1e/f97 [310369,35474] 0 2024-04-06T03:40:42.383 INFO:tasks.workunit.client.1.smithi136.stdout:1/682: symlink d12/d3b/d60/d6d/da2/le5 0 2024-04-06T03:40:42.384 INFO:tasks.workunit.client.1.smithi136.stdout:1/683: read - d12/d3b/d44/d5f/db7/fdd zero size 2024-04-06T03:40:42.387 INFO:tasks.workunit.client.1.smithi136.stdout:8/739: dwrite d3/f10 [0,4194304] 0 2024-04-06T03:40:42.388 INFO:tasks.workunit.client.1.smithi136.stdout:8/740: write d3/de/d18/f1e [230730,113608] 0 2024-04-06T03:40:42.388 INFO:tasks.workunit.client.1.smithi136.stdout:2/758: rename d3/d4a/d9d/c27 to d3/d9/cf5 0 2024-04-06T03:40:42.391 INFO:tasks.workunit.client.1.smithi136.stdout:0/675: mkdir d1/d6/d2c/d57/de3 0 2024-04-06T03:40:42.393 INFO:tasks.workunit.client.0.smithi094.stdout:5/619: dwrite db/fae [0,4194304] 0 2024-04-06T03:40:42.393 INFO:tasks.workunit.client.0.smithi094.stdout:5/620: fsync db/d19/d30/f44 0 2024-04-06T03:40:42.395 INFO:tasks.workunit.client.1.smithi136.stdout:6/638: symlink d2/d7/d13/d79/dc2/ld1 0 2024-04-06T03:40:42.398 INFO:tasks.workunit.client.1.smithi136.stdout:9/681: dwrite fa [0,4194304] 0 2024-04-06T03:40:42.399 INFO:tasks.workunit.client.1.smithi136.stdout:9/682: readlink d17/d19/da3/dc5/ld9 0 2024-04-06T03:40:42.400 INFO:tasks.workunit.client.1.smithi136.stdout:5/757: creat d1/d98/dbc/fdc x:0 0 0 2024-04-06T03:40:42.400 INFO:tasks.workunit.client.0.smithi094.stdout:2/674: dwrite d0/dd/d3d/f88 [0,4194304] 0 2024-04-06T03:40:42.404 INFO:tasks.workunit.client.1.smithi136.stdout:8/741: mkdir d3/de/d18/d7d/de0 0 2024-04-06T03:40:42.406 INFO:tasks.workunit.client.0.smithi094.stdout:4/741: dread d0/f44 [0,4194304] 0 2024-04-06T03:40:42.407 INFO:tasks.workunit.client.1.smithi136.stdout:4/628: rename d1d/d2f/f60 to d1d/fe2 0 2024-04-06T03:40:42.407 INFO:tasks.workunit.client.1.smithi136.stdout:4/629: creat d1d/d31/d3e/d55/fe3 x:0 0 0 2024-04-06T03:40:42.408 INFO:tasks.workunit.client.1.smithi136.stdout:4/630: readlink d1d/d70/l54 0 2024-04-06T03:40:42.408 INFO:tasks.workunit.client.1.smithi136.stdout:6/639: read d2/fa [3458917,5846] 0 2024-04-06T03:40:42.410 INFO:tasks.workunit.client.1.smithi136.stdout:2/759: creat d3/d9/d3b/d87/ff6 x:0 0 0 2024-04-06T03:40:42.410 INFO:tasks.workunit.client.1.smithi136.stdout:2/760: readlink d3/d4a/d9d/d8a/d93/l59 0 2024-04-06T03:40:42.410 INFO:tasks.workunit.client.1.smithi136.stdout:9/683: read d17/d20/d28/d49/d75/d60/f7a [2634599,116376] 0 2024-04-06T03:40:42.410 INFO:tasks.workunit.client.1.smithi136.stdout:4/631: dread d1d/d2f/f43 [0,4194304] 0 2024-04-06T03:40:42.411 INFO:tasks.workunit.client.1.smithi136.stdout:0/676: mknod d1/d6/d2c/d57/d43/d6d/d85/ce4 0 2024-04-06T03:40:42.412 INFO:tasks.workunit.client.0.smithi094.stdout:5/621: dwrite db/d40/d77/fcd [0,4194304] 0 2024-04-06T03:40:42.412 INFO:tasks.workunit.client.0.smithi094.stdout:1/681: creat d0/d1/d27/da5/d40/db1/dc3/d5f/d70/fe8 x:0 0 0 2024-04-06T03:40:42.413 INFO:tasks.workunit.client.0.smithi094.stdout:1/682: readlink d0/d1/d27/da5/d40/db1/lb3 0 2024-04-06T03:40:42.415 INFO:tasks.workunit.client.0.smithi094.stdout:6/620: creat dc/dd3/fdc x:0 0 0 2024-04-06T03:40:42.415 INFO:tasks.workunit.client.0.smithi094.stdout:6/621: stat dc/d11/d22/d50/cce 0 2024-04-06T03:40:42.417 INFO:tasks.workunit.client.0.smithi094.stdout:8/751: creat d18/d2c/d37/d60/da3/db8/f102 x:0 0 0 2024-04-06T03:40:42.419 INFO:tasks.workunit.client.0.smithi094.stdout:0/681: rename d1/d9/dd/d18/d1d to d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/de4 0 2024-04-06T03:40:42.420 INFO:tasks.workunit.client.1.smithi136.stdout:5/758: mknod d1/d16/d89/d8a/cdd 0 2024-04-06T03:40:42.422 INFO:tasks.workunit.client.0.smithi094.stdout:9/645: mknod d2/dc/d21/d23/d5b/cec 0 2024-04-06T03:40:42.422 INFO:tasks.workunit.client.0.smithi094.stdout:9/646: chown d2/fe 253964 1 2024-04-06T03:40:42.422 INFO:tasks.workunit.client.0.smithi094.stdout:9/647: read - d2/dc/d21/d23/fe1 zero size 2024-04-06T03:40:42.422 INFO:tasks.workunit.client.0.smithi094.stdout:9/648: creat d2/dc/d21/d23/d47/d51/fed x:0 0 0 2024-04-06T03:40:42.423 INFO:tasks.workunit.client.1.smithi136.stdout:8/742: mkdir d3/de/d60/de1 0 2024-04-06T03:40:42.425 INFO:tasks.workunit.client.0.smithi094.stdout:8/752: dread d18/d2c/f2e [0,4194304] 0 2024-04-06T03:40:42.426 INFO:tasks.workunit.client.1.smithi136.stdout:3/657: rename d0/d4/d12/d3a/d69/cd9 to d0/d4/d12/dc3/dd3/ce2 0 2024-04-06T03:40:42.426 INFO:tasks.workunit.client.1.smithi136.stdout:3/658: creat d0/d4/d12/d19/d55/d3c/d5a/fe3 x:0 0 0 2024-04-06T03:40:42.427 INFO:tasks.workunit.client.1.smithi136.stdout:3/659: chown d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/cde 11980 1 2024-04-06T03:40:42.429 INFO:tasks.workunit.client.1.smithi136.stdout:6/640: mknod d2/d7/d26/d30/d6e/d58/cd2 0 2024-04-06T03:40:42.429 INFO:tasks.workunit.client.1.smithi136.stdout:6/641: creat d2/d7/d9e/fd3 x:0 0 0 2024-04-06T03:40:42.430 INFO:tasks.workunit.client.0.smithi094.stdout:7/616: mkdir d2/d6/dc/d12/d29/d3f/db9 0 2024-04-06T03:40:42.432 INFO:tasks.workunit.client.1.smithi136.stdout:2/761: mkdir d3/d4a/d9d/d18/d1d/d3c/df7 0 2024-04-06T03:40:42.437 INFO:tasks.workunit.client.1.smithi136.stdout:0/677: readlink d1/d6/d2c/l41 0 2024-04-06T03:40:42.437 INFO:tasks.workunit.client.1.smithi136.stdout:0/678: stat d1/d6/d2c/d57/d43/d6d/d85/d78 0 2024-04-06T03:40:42.438 INFO:tasks.workunit.client.1.smithi136.stdout:7/718: sync 2024-04-06T03:40:42.440 INFO:tasks.workunit.client.1.smithi136.stdout:8/743: creat d3/de/d18/d7d/ddf/d6c/dab/fe2 x:0 0 0 2024-04-06T03:40:42.441 INFO:tasks.workunit.client.0.smithi094.stdout:2/675: creat d0/dd/d19/fe5 x:0 0 0 2024-04-06T03:40:42.441 INFO:tasks.workunit.client.0.smithi094.stdout:2/676: fsync d0/d8/da0/fd9 0 2024-04-06T03:40:42.442 INFO:tasks.workunit.client.0.smithi094.stdout:7/617: dwrite d2/d6/dc/d12/d29/fa7 [0,4194304] 0 2024-04-06T03:40:42.442 INFO:tasks.workunit.client.0.smithi094.stdout:7/618: readlink d2/d6/dc/dd/df/l8e 0 2024-04-06T03:40:42.444 INFO:tasks.workunit.client.1.smithi136.stdout:9/684: rename f4 to d17/d19/da3/dc5/fec 0 2024-04-06T03:40:42.444 INFO:tasks.workunit.client.1.smithi136.stdout:9/685: chown d17/d21/d32/d3f/f63 9 1 2024-04-06T03:40:42.444 INFO:tasks.workunit.client.1.smithi136.stdout:9/686: truncate d17/d21/d32/d3f/f5b 737753 0 2024-04-06T03:40:42.445 INFO:tasks.workunit.client.1.smithi136.stdout:4/632: dread d1d/d31/f58 [0,4194304] 0 2024-04-06T03:40:42.445 INFO:tasks.workunit.client.1.smithi136.stdout:3/660: dwrite d0/d4/d12/d19/d55/fdd [0,4194304] 0 2024-04-06T03:40:42.445 INFO:tasks.workunit.client.0.smithi094.stdout:4/742: mkdir d0/d15/d3a/d61/df8 0 2024-04-06T03:40:42.445 INFO:tasks.workunit.client.0.smithi094.stdout:4/743: truncate d0/d4/d4b/ff7 214477 0 2024-04-06T03:40:42.448 INFO:tasks.workunit.client.0.smithi094.stdout:5/622: unlink db/d7c/d98/lad 0 2024-04-06T03:40:42.453 INFO:tasks.workunit.client.0.smithi094.stdout:1/683: mkdir d0/d1/d27/da5/d3f/de9 0 2024-04-06T03:40:42.457 INFO:tasks.workunit.client.1.smithi136.stdout:2/762: mkdir d3/d4a/d9d/d18/df8 0 2024-04-06T03:40:42.458 INFO:tasks.workunit.client.1.smithi136.stdout:8/744: dwrite d3/da/d26/f7a [0,4194304] 0 2024-04-06T03:40:42.460 INFO:tasks.workunit.client.1.smithi136.stdout:0/679: mkdir d1/d6/d2c/d57/d43/d6d/d85/da7/de5 0 2024-04-06T03:40:42.460 INFO:tasks.workunit.client.1.smithi136.stdout:0/680: read d1/d6/d2c/d1a/f94 [228635,82364] 0 2024-04-06T03:40:42.460 INFO:tasks.workunit.client.1.smithi136.stdout:0/681: stat d1/d6/d2c/d1a/d36/d59 0 2024-04-06T03:40:42.460 INFO:tasks.workunit.client.0.smithi094.stdout:4/744: dread d0/d15/d3a/d3e/d46/fa3 [0,4194304] 0 2024-04-06T03:40:42.461 INFO:tasks.workunit.client.0.smithi094.stdout:0/682: symlink d1/d9/dd/d18/d1e/d86/d96/le5 0 2024-04-06T03:40:42.461 INFO:tasks.workunit.client.1.smithi136.stdout:9/687: write d17/d20/d28/d81/d88/fac [3337481,83670] 0 2024-04-06T03:40:42.462 INFO:tasks.workunit.client.1.smithi136.stdout:9/688: chown d17/d20/d28/d81/f83 2 1 2024-04-06T03:40:42.462 INFO:tasks.workunit.client.1.smithi136.stdout:9/689: readlink d17/d20/d25/d3c/l8b 0 2024-04-06T03:40:42.462 INFO:tasks.workunit.client.1.smithi136.stdout:5/759: symlink d1/dc/lde 0 2024-04-06T03:40:42.464 INFO:tasks.workunit.client.1.smithi136.stdout:7/719: symlink d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/l100 0 2024-04-06T03:40:42.466 INFO:tasks.workunit.client.0.smithi094.stdout:9/649: mkdir d2/dc/d21/d23/d47/d51/dee 0 2024-04-06T03:40:42.468 INFO:tasks.workunit.client.0.smithi094.stdout:8/753: link d18/d30/c4c d18/d2c/d37/d60/da3/c103 0 2024-04-06T03:40:42.470 INFO:tasks.workunit.client.0.smithi094.stdout:2/677: symlink d0/dd/d19/d31/dcc/le6 0 2024-04-06T03:40:42.472 INFO:tasks.workunit.client.1.smithi136.stdout:3/661: dwrite d0/d4/d12/d19/d55/d3c/d5a/d79/fa4 [0,4194304] 0 2024-04-06T03:40:42.478 INFO:tasks.workunit.client.0.smithi094.stdout:7/619: creat d2/d6/dc/d12/d29/d35/da4/db7/fba x:0 0 0 2024-04-06T03:40:42.479 INFO:tasks.workunit.client.1.smithi136.stdout:2/763: dwrite d3/d15/f2b [0,4194304] 0 2024-04-06T03:40:42.479 INFO:tasks.workunit.client.1.smithi136.stdout:2/764: truncate d3/d15/fed 8056 0 2024-04-06T03:40:42.479 INFO:tasks.workunit.client.1.smithi136.stdout:2/765: write d3/d4a/d9d/d18/d1d/fd8 [75731,59382] 0 2024-04-06T03:40:42.479 INFO:tasks.workunit.client.1.smithi136.stdout:2/766: creat d3/d4a/d9d/d18/d4b/dc0/ff9 x:0 0 0 2024-04-06T03:40:42.481 INFO:tasks.workunit.client.0.smithi094.stdout:5/623: creat db/d19/d30/d83/fd3 x:0 0 0 2024-04-06T03:40:42.483 INFO:tasks.workunit.client.1.smithi136.stdout:7/720: dwrite d5/df/d12/d1b/d3a/fa6 [0,4194304] 0 2024-04-06T03:40:42.484 INFO:tasks.workunit.client.0.smithi094.stdout:1/684: creat d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/db9/fea x:0 0 0 2024-04-06T03:40:42.484 INFO:tasks.workunit.client.0.smithi094.stdout:1/685: write d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/fb7 [620901,99231] 0 2024-04-06T03:40:42.484 INFO:tasks.workunit.client.0.smithi094.stdout:1/686: chown d0/d7/c98 744283 1 2024-04-06T03:40:42.484 INFO:tasks.workunit.client.0.smithi094.stdout:1/687: fdatasync d0/d7/d1a/da8/fa2 0 2024-04-06T03:40:42.484 INFO:tasks.workunit.client.0.smithi094.stdout:1/688: write d0/d1/f73 [903479,27804] 0 2024-04-06T03:40:42.485 INFO:tasks.workunit.client.1.smithi136.stdout:4/633: creat d1d/d2f/da6/dbf/fe4 x:0 0 0 2024-04-06T03:40:42.486 INFO:tasks.workunit.client.0.smithi094.stdout:4/745: unlink d0/d15/d3a/f75 0 2024-04-06T03:40:42.488 INFO:tasks.workunit.client.1.smithi136.stdout:6/642: getdents d2/d93/dab 0 2024-04-06T03:40:42.488 INFO:tasks.workunit.client.1.smithi136.stdout:6/643: write d2/d7/d26/d30/f4c [5041111,60214] 0 2024-04-06T03:40:42.489 INFO:tasks.workunit.client.0.smithi094.stdout:0/683: rename d1/d9/fcd to d1/d9/dd/d18/d5d/d60/fe6 0 2024-04-06T03:40:42.493 INFO:tasks.workunit.client.1.smithi136.stdout:0/682: unlink d1/d6/d2c/d57/d43/d6b/ldf 0 2024-04-06T03:40:42.495 INFO:tasks.workunit.client.0.smithi094.stdout:9/650: rmdir d2/dc/d21/d23/db0/dbe 0 2024-04-06T03:40:42.496 INFO:tasks.workunit.client.1.smithi136.stdout:1/684: sync 2024-04-06T03:40:42.496 INFO:tasks.workunit.client.1.smithi136.stdout:1/685: creat d12/d3b/d8e/fe6 x:0 0 0 2024-04-06T03:40:42.497 INFO:tasks.workunit.client.0.smithi094.stdout:8/754: write d18/d3c/d49/d80/fea [275362,62571] 0 2024-04-06T03:40:42.499 INFO:tasks.workunit.client.1.smithi136.stdout:9/690: link d17/d21/fa7 d17/d20/d25/d71/dd5/fed 0 2024-04-06T03:40:42.499 INFO:tasks.workunit.client.1.smithi136.stdout:9/691: getdents d17/d20/d28/d81/d88/dcb 0 2024-04-06T03:40:42.499 INFO:tasks.workunit.client.1.smithi136.stdout:9/692: truncate d17/d20/d28/f98 969151 0 2024-04-06T03:40:42.499 INFO:tasks.workunit.client.1.smithi136.stdout:9/693: truncate d17/d21/d32/fa6 446286 0 2024-04-06T03:40:42.499 INFO:tasks.workunit.client.1.smithi136.stdout:9/694: creat d17/d20/d25/da4/dbb/fee x:0 0 0 2024-04-06T03:40:42.500 INFO:tasks.workunit.client.0.smithi094.stdout:2/678: mkdir d0/dd/d1b/de2/de7 0 2024-04-06T03:40:42.500 INFO:tasks.workunit.client.0.smithi094.stdout:2/679: chown d0/d8/d27/c80 261 1 2024-04-06T03:40:42.502 INFO:tasks.workunit.client.1.smithi136.stdout:5/760: unlink d1/dc/d1c/d32/f6a 0 2024-04-06T03:40:42.502 INFO:tasks.workunit.client.1.smithi136.stdout:2/767: dwrite d3/d4a/d9d/d18/d1d/d91/fea [0,4194304] 0 2024-04-06T03:40:42.502 INFO:tasks.workunit.client.1.smithi136.stdout:2/768: write d3/d4a/d9d/d8a/d93/d5f/f78 [1174360,5300] 0 2024-04-06T03:40:42.502 INFO:tasks.workunit.client.0.smithi094.stdout:6/622: dwrite f7 [0,4194304] 0 2024-04-06T03:40:42.502 INFO:tasks.workunit.client.0.smithi094.stdout:6/623: chown dc/dd3/fdc 4731460 1 2024-04-06T03:40:42.503 INFO:tasks.workunit.client.0.smithi094.stdout:6/624: creat dc/d11/d22/d50/daf/fdd x:0 0 0 2024-04-06T03:40:42.504 INFO:tasks.workunit.client.1.smithi136.stdout:3/662: link d0/d4/d12/d2d/fbe d0/fe4 0 2024-04-06T03:40:42.504 INFO:tasks.workunit.client.1.smithi136.stdout:3/663: readlink d0/d4/d12/dc3/lca 0 2024-04-06T03:40:42.509 INFO:tasks.workunit.client.0.smithi094.stdout:4/746: readlink d0/l28 0 2024-04-06T03:40:42.512 INFO:tasks.workunit.client.1.smithi136.stdout:5/761: write d1/dc/d1c/d32/d8b/fa7 [414834,72763] 0 2024-04-06T03:40:42.512 INFO:tasks.workunit.client.1.smithi136.stdout:5/762: readlink d1/d98/la5 0 2024-04-06T03:40:42.513 INFO:tasks.workunit.client.1.smithi136.stdout:1/686: dwrite d12/d3b/d74/d7a/f8d [0,4194304] 0 2024-04-06T03:40:42.513 INFO:tasks.workunit.client.1.smithi136.stdout:7/721: symlink d5/df/d53/l101 0 2024-04-06T03:40:42.513 INFO:tasks.workunit.client.1.smithi136.stdout:7/722: chown d5/df/d12/d1b/d3a/d4a/dc4 61769 1 2024-04-06T03:40:42.514 INFO:tasks.workunit.client.0.smithi094.stdout:2/680: dwrite d0/d8/d27/d2b/fdd [0,4194304] 0 2024-04-06T03:40:42.515 INFO:tasks.workunit.client.1.smithi136.stdout:4/634: mknod d1d/d31/d3e/d56/d7b/ce5 0 2024-04-06T03:40:42.518 INFO:tasks.workunit.client.1.smithi136.stdout:6/644: rename d2/d7/f12 to d2/d7/d26/d30/fd4 0 2024-04-06T03:40:42.518 INFO:tasks.workunit.client.1.smithi136.stdout:6/645: write d2/d7/d26/d30/d6e/d3d/f81 [4671156,88742] 0 2024-04-06T03:40:42.519 INFO:tasks.workunit.client.0.smithi094.stdout:9/651: mkdir d2/dc/d80/ddd/def 0 2024-04-06T03:40:42.520 INFO:tasks.workunit.client.1.smithi136.stdout:0/683: creat d1/d6/d2c/d57/fe6 x:0 0 0 2024-04-06T03:40:42.520 INFO:tasks.workunit.client.1.smithi136.stdout:0/684: stat d1/lb7 0 2024-04-06T03:40:42.520 INFO:tasks.workunit.client.1.smithi136.stdout:0/685: chown d1/f18 633231197 1 2024-04-06T03:40:42.525 INFO:tasks.workunit.client.1.smithi136.stdout:9/695: truncate d17/d20/d25/f26 6034042 0 2024-04-06T03:40:42.526 INFO:tasks.workunit.client.0.smithi094.stdout:1/689: getdents d0/d1/d27/da5/d40/db1/d2f/d76/d30 0 2024-04-06T03:40:42.529 INFO:tasks.workunit.client.0.smithi094.stdout:4/747: mkdir d0/d4/d67/df9 0 2024-04-06T03:40:42.532 INFO:tasks.workunit.client.1.smithi136.stdout:2/769: mknod d3/d15/d19/d43/daa/dec/cfa 0 2024-04-06T03:40:42.536 INFO:tasks.workunit.client.1.smithi136.stdout:5/763: creat d1/dc/d1c/d32/d8b/db8/fdf x:0 0 0 2024-04-06T03:40:42.538 INFO:tasks.workunit.client.0.smithi094.stdout:2/681: creat d0/d8/d4f/dab/dc3/fe8 x:0 0 0 2024-04-06T03:40:42.540 INFO:tasks.workunit.client.0.smithi094.stdout:9/652: link d2/f7 d2/dc/d21/d23/d5b/ff0 0 2024-04-06T03:40:42.548 INFO:tasks.workunit.client.0.smithi094.stdout:1/690: dread d0/d1/d27/da5/d40/f7a [0,4194304] 0 2024-04-06T03:40:42.552 INFO:tasks.workunit.client.1.smithi136.stdout:1/687: link l7 d12/d53/le7 0 2024-04-06T03:40:42.553 INFO:tasks.workunit.client.0.smithi094.stdout:5/624: rename db/d40 to db/d19/dc1/dd4 0 2024-04-06T03:40:42.555 INFO:tasks.workunit.client.1.smithi136.stdout:7/723: link d5/df/d12/d1b/d58/f7a d5/df/d1d/d54/d5c/dc7/f102 0 2024-04-06T03:40:42.557 INFO:tasks.workunit.client.1.smithi136.stdout:4/635: symlink d1d/d2f/d6c/d91/le6 0 2024-04-06T03:40:42.557 INFO:tasks.workunit.client.0.smithi094.stdout:2/682: getdents d0/d8/d4f/dab 0 2024-04-06T03:40:42.560 INFO:tasks.workunit.client.0.smithi094.stdout:9/653: mknod d2/dc/d21/d23/d47/d51/dee/cf1 0 2024-04-06T03:40:42.560 INFO:tasks.workunit.client.0.smithi094.stdout:9/654: dread - d2/dc/d10/fb2 zero size 2024-04-06T03:40:42.560 INFO:tasks.workunit.client.0.smithi094.stdout:9/655: stat d2/dc/d46/fde 0 2024-04-06T03:40:42.560 INFO:tasks.workunit.client.0.smithi094.stdout:9/656: readlink d2/dc/d21/d23/d47/d51/le6 0 2024-04-06T03:40:42.561 INFO:tasks.workunit.client.1.smithi136.stdout:6/646: mknod d2/d7/d26/d97/cd5 0 2024-04-06T03:40:42.561 INFO:tasks.workunit.client.1.smithi136.stdout:6/647: write d2/d7/d26/d30/d6e/f2f [4271647,47981] 0 2024-04-06T03:40:42.561 INFO:tasks.workunit.client.1.smithi136.stdout:6/648: read - d2/d7/d26/d30/d6e/d58/f7d zero size 2024-04-06T03:40:42.562 INFO:tasks.workunit.client.0.smithi094.stdout:4/748: truncate d0/f4e 3878805 0 2024-04-06T03:40:42.563 INFO:tasks.workunit.client.0.smithi094.stdout:1/691: dwrite d0/d7/d2d/f6e [0,4194304] 0 2024-04-06T03:40:42.564 INFO:tasks.workunit.client.1.smithi136.stdout:0/686: link d1/d53/f6e d1/d6/d2c/d1a/fe7 0 2024-04-06T03:40:42.564 INFO:tasks.workunit.client.1.smithi136.stdout:0/687: chown d1/d6/d2c/d57/d43/d6d/d85/f7f 3362 1 2024-04-06T03:40:42.564 INFO:tasks.workunit.client.1.smithi136.stdout:0/688: truncate d1/d6/d2c/d57/d43/fa2 933505 0 2024-04-06T03:40:42.564 INFO:tasks.workunit.client.1.smithi136.stdout:0/689: getdents d1/d6/d2c/d57/d43/dcf/de1 0 2024-04-06T03:40:42.564 INFO:tasks.workunit.client.1.smithi136.stdout:0/690: chown d1/d6/d2c/d57/d43/fa2 31807981 1 2024-04-06T03:40:42.565 INFO:tasks.workunit.client.0.smithi094.stdout:8/755: rename d18/d3c/d76/dd9/f72 to d18/d3c/dc4/f104 0 2024-04-06T03:40:42.566 INFO:tasks.workunit.client.1.smithi136.stdout:9/696: mkdir d17/d21/d29/d34/d66/de1/d51/d57/def 0 2024-04-06T03:40:42.567 INFO:tasks.workunit.client.1.smithi136.stdout:9/697: dread - d17/d19/fb3 zero size 2024-04-06T03:40:42.567 INFO:tasks.workunit.client.0.smithi094.stdout:9/657: creat d2/dc/d21/d26/d8a/ff2 x:0 0 0 2024-04-06T03:40:42.567 INFO:tasks.workunit.client.0.smithi094.stdout:9/658: creat d2/dc/d21/d23/d47/d51/ff3 x:0 0 0 2024-04-06T03:40:42.569 INFO:tasks.workunit.client.0.smithi094.stdout:5/625: dwrite db/f2a [0,4194304] 0 2024-04-06T03:40:42.569 INFO:tasks.workunit.client.0.smithi094.stdout:5/626: dread - db/f76 zero size 2024-04-06T03:40:42.570 INFO:tasks.workunit.client.1.smithi136.stdout:2/770: symlink d3/d4a/d9d/lfb 0 2024-04-06T03:40:42.570 INFO:tasks.workunit.client.1.smithi136.stdout:3/664: rmdir d0/d4/d12/dc3 39 2024-04-06T03:40:42.570 INFO:tasks.workunit.client.1.smithi136.stdout:3/665: write d0/d4/d12/d19/d55/d6f/d82/d5d/d61/fdf [607607,19866] 0 2024-04-06T03:40:42.571 INFO:tasks.workunit.client.0.smithi094.stdout:4/749: mkdir d0/df/d13/d4a/dfa 0 2024-04-06T03:40:42.572 INFO:tasks.workunit.client.1.smithi136.stdout:5/764: mknod d1/dc/d1c/ce0 0 2024-04-06T03:40:42.575 INFO:tasks.workunit.client.1.smithi136.stdout:1/688: rename d12/d3b/d44/d5f/d5a/d72/d77/l7c to d12/d3b/d44/le8 0 2024-04-06T03:40:42.576 INFO:tasks.workunit.client.0.smithi094.stdout:1/692: mknod d0/d4/ceb 0 2024-04-06T03:40:42.576 INFO:tasks.workunit.client.0.smithi094.stdout:1/693: creat d0/d1/d27/da5/d40/d50/fec x:0 0 0 2024-04-06T03:40:42.577 INFO:tasks.workunit.client.1.smithi136.stdout:7/724: mkdir d5/df/d53/d68/da3/dd0/d103 0 2024-04-06T03:40:42.580 INFO:tasks.workunit.client.1.smithi136.stdout:4/636: rmdir d1d/d31/d3e/d56 39 2024-04-06T03:40:42.580 INFO:tasks.workunit.client.1.smithi136.stdout:4/637: stat f7 0 2024-04-06T03:40:42.580 INFO:tasks.workunit.client.1.smithi136.stdout:4/638: creat d1d/d31/d3e/d55/fe7 x:0 0 0 2024-04-06T03:40:42.581 INFO:tasks.workunit.client.0.smithi094.stdout:4/750: write d0/d77/f80 [2118294,32446] 0 2024-04-06T03:40:42.582 INFO:tasks.workunit.client.1.smithi136.stdout:6/649: creat d2/d7/d13/fd6 x:0 0 0 2024-04-06T03:40:42.582 INFO:tasks.workunit.client.1.smithi136.stdout:6/650: readlink d2/d7/d26/dcd/lc8 0 2024-04-06T03:40:42.583 INFO:tasks.workunit.client.1.smithi136.stdout:8/745: sync 2024-04-06T03:40:42.585 INFO:tasks.workunit.client.0.smithi094.stdout:9/659: dwrite d2/dc/d21/d23/d5b/f89 [0,4194304] 0 2024-04-06T03:40:42.585 INFO:tasks.workunit.client.1.smithi136.stdout:9/698: link d17/d20/d25/da4/dbb/cd4 d17/d20/d28/d49/d75/d60/cf0 0 2024-04-06T03:40:42.587 INFO:tasks.workunit.client.1.smithi136.stdout:2/771: creat d3/d15/d19/d43/daa/ffc x:0 0 0 2024-04-06T03:40:42.588 INFO:tasks.workunit.client.1.smithi136.stdout:3/666: mknod d0/d4/d12/d19/d55/d3c/d5a/ce5 0 2024-04-06T03:40:42.588 INFO:tasks.workunit.client.1.smithi136.stdout:3/667: chown d0/d4/d12/d19/d55/d3c/d5a/f71 55298 1 2024-04-06T03:40:42.588 INFO:tasks.workunit.client.1.smithi136.stdout:1/689: dread d12/d1b/d22/f43 [0,4194304] 0 2024-04-06T03:40:42.589 INFO:tasks.workunit.client.0.smithi094.stdout:5/627: unlink db/d19/dc1/dd4/d77/f66 0 2024-04-06T03:40:42.590 INFO:tasks.workunit.client.1.smithi136.stdout:5/765: symlink d1/d16/d17/le1 0 2024-04-06T03:40:42.590 INFO:tasks.workunit.client.1.smithi136.stdout:5/766: truncate d1/dc/fc9 232406 0 2024-04-06T03:40:42.591 INFO:tasks.workunit.client.1.smithi136.stdout:0/691: rename d1/d6/d2c/d57/d43/d6b/c83 to d1/d6/d2c/d57/ce8 0 2024-04-06T03:40:42.592 INFO:tasks.workunit.client.0.smithi094.stdout:1/694: truncate d0/d1/d27/da5/d40/db1/d2f/d3c/d48/fd4 1710553 0 2024-04-06T03:40:42.593 INFO:tasks.workunit.client.1.smithi136.stdout:7/725: creat d5/df/d12/d1b/d3a/d4a/ddf/f104 x:0 0 0 2024-04-06T03:40:42.594 INFO:tasks.workunit.client.1.smithi136.stdout:9/699: dread d17/d20/d28/d81/f83 [0,4194304] 0 2024-04-06T03:40:42.594 INFO:tasks.workunit.client.1.smithi136.stdout:9/700: dread - d17/d19/da3/dc5/fd3 zero size 2024-04-06T03:40:42.595 INFO:tasks.workunit.client.0.smithi094.stdout:8/756: dwrite d18/d3c/d49/fd5 [0,4194304] 0 2024-04-06T03:40:42.595 INFO:tasks.workunit.client.0.smithi094.stdout:8/757: chown d18/d3c/d76/c50 14572 1 2024-04-06T03:40:42.595 INFO:tasks.workunit.client.0.smithi094.stdout:6/625: rename dc/d11/d22 to dc/d3a/d4a/d54/dde 0 2024-04-06T03:40:42.596 INFO:tasks.workunit.client.1.smithi136.stdout:4/639: creat d1d/d31/d78/dcb/fe8 x:0 0 0 2024-04-06T03:40:42.597 INFO:tasks.workunit.client.1.smithi136.stdout:6/651: rmdir d2/d7/d26/d30/d6e/d58/d5d/d62/d53 39 2024-04-06T03:40:42.597 INFO:tasks.workunit.client.1.smithi136.stdout:6/652: truncate d2/d7/d26/f54 992130 0 2024-04-06T03:40:42.598 INFO:tasks.workunit.client.0.smithi094.stdout:9/660: creat d2/dc/d10/ff4 x:0 0 0 2024-04-06T03:40:42.598 INFO:tasks.workunit.client.0.smithi094.stdout:9/661: getdents d2/dc/d21/d26/d8a/de9 0 2024-04-06T03:40:42.598 INFO:tasks.workunit.client.1.smithi136.stdout:8/746: creat d3/da/d26/d35/d53/dd2/fe3 x:0 0 0 2024-04-06T03:40:42.599 INFO:tasks.workunit.client.0.smithi094.stdout:5/628: mkdir db/d19/d24/d28/d85/dd5 0 2024-04-06T03:40:42.600 INFO:tasks.workunit.client.1.smithi136.stdout:2/772: mknod d3/d9/d3b/d87/cfd 0 2024-04-06T03:40:42.601 INFO:tasks.workunit.client.1.smithi136.stdout:8/747: dread d3/de/d18/d6d/f7b [0,4194304] 0 2024-04-06T03:40:42.602 INFO:tasks.workunit.client.1.smithi136.stdout:3/668: truncate d0/f32 2331129 0 2024-04-06T03:40:42.602 INFO:tasks.workunit.client.1.smithi136.stdout:8/748: fsync d3/d2d/f67 0 2024-04-06T03:40:42.602 INFO:tasks.workunit.client.1.smithi136.stdout:3/669: readlink d0/l4f 0 2024-04-06T03:40:42.602 INFO:tasks.workunit.client.0.smithi094.stdout:1/695: mknod d0/d1/d27/da5/d40/db1/d2f/d76/ced 0 2024-04-06T03:40:42.603 INFO:tasks.workunit.client.0.smithi094.stdout:8/758: dread d18/d34/f3d [0,4194304] 0 2024-04-06T03:40:42.604 INFO:tasks.workunit.client.1.smithi136.stdout:1/690: link d12/d1b/d1e/d9f/fa8 d12/d3b/d74/d7a/fe9 0 2024-04-06T03:40:42.606 INFO:tasks.workunit.client.1.smithi136.stdout:5/767: mknod d1/d9b/db4/ce2 0 2024-04-06T03:40:42.607 INFO:tasks.workunit.client.1.smithi136.stdout:0/692: truncate d1/d53/f6e 2015343 0 2024-04-06T03:40:42.608 INFO:tasks.workunit.client.1.smithi136.stdout:0/693: creat d1/d6/d2c/d57/d43/d6d/d85/d78/da4/fe9 x:0 0 0 2024-04-06T03:40:42.610 INFO:tasks.workunit.client.0.smithi094.stdout:2/683: rename d0/d8/d4f/dab/dc3/fe0 to d0/dd/d19/dbe/fe9 0 2024-04-06T03:40:42.610 INFO:tasks.workunit.client.0.smithi094.stdout:2/684: chown d0/d18/f55 113 1 2024-04-06T03:40:42.612 INFO:tasks.workunit.client.0.smithi094.stdout:1/696: dread d0/d7/d1a/fd1 [0,4194304] 0 2024-04-06T03:40:42.612 INFO:tasks.workunit.client.1.smithi136.stdout:7/726: rename d5/df/d11/fa2 to d5/df/d12/d1b/d58/f105 0 2024-04-06T03:40:42.614 INFO:tasks.workunit.client.1.smithi136.stdout:9/701: symlink d17/d20/d28/d49/lf1 0 2024-04-06T03:40:42.616 INFO:tasks.workunit.client.0.smithi094.stdout:8/759: symlink d18/d2c/l105 0 2024-04-06T03:40:42.618 INFO:tasks.workunit.client.1.smithi136.stdout:4/640: mknod d1d/d31/d78/dcb/ce9 0 2024-04-06T03:40:42.618 INFO:tasks.workunit.client.1.smithi136.stdout:4/641: readlink l6 0 2024-04-06T03:40:42.618 INFO:tasks.workunit.client.1.smithi136.stdout:4/642: write d1d/d31/d3e/dd4/fd1 [1010770,89542] 0 2024-04-06T03:40:42.619 INFO:tasks.workunit.client.1.smithi136.stdout:4/643: write d1d/d31/d3e/d55/d89/db3/fca [710017,6596] 0 2024-04-06T03:40:42.619 INFO:tasks.workunit.client.0.smithi094.stdout:5/629: dwrite db/f11 [0,4194304] 0 2024-04-06T03:40:42.620 INFO:tasks.workunit.client.0.smithi094.stdout:1/697: rename d0/d7/d1a/da8/fa2 to d0/d1/d27/da5/d40/db1/d2f/d3c/d48/fee 0 2024-04-06T03:40:42.621 INFO:tasks.workunit.client.1.smithi136.stdout:2/773: mknod d3/d15/d19/d43/daa/dce/cfe 0 2024-04-06T03:40:42.621 INFO:tasks.workunit.client.1.smithi136.stdout:2/774: write d3/fef [494694,3203] 0 2024-04-06T03:40:42.622 INFO:tasks.workunit.client.0.smithi094.stdout:6/626: getdents dc/dd3 0 2024-04-06T03:40:42.623 INFO:tasks.workunit.client.0.smithi094.stdout:6/627: fsync dc/d3a/d4a/d54/f58 0 2024-04-06T03:40:42.624 INFO:tasks.workunit.client.1.smithi136.stdout:8/749: creat d3/da/d26/d35/d3d/dc7/dce/fe4 x:0 0 0 2024-04-06T03:40:42.626 INFO:tasks.workunit.client.0.smithi094.stdout:5/630: mknod db/d19/d2c/d6d/cd6 0 2024-04-06T03:40:42.626 INFO:tasks.workunit.client.0.smithi094.stdout:5/631: read - db/d19/d30/f81 zero size 2024-04-06T03:40:42.627 INFO:tasks.workunit.client.0.smithi094.stdout:2/685: dwrite d0/dd/d19/f81 [8388608,4194304] 0 2024-04-06T03:40:42.627 INFO:tasks.workunit.client.0.smithi094.stdout:2/686: write d0/dd/f95 [311823,104660] 0 2024-04-06T03:40:42.627 INFO:tasks.workunit.client.1.smithi136.stdout:3/670: mkdir d0/d4/d12/d19/d55/d3c/d53/de6 0 2024-04-06T03:40:42.627 INFO:tasks.workunit.client.1.smithi136.stdout:3/671: fdatasync d0/d4/d12/d2d/d37/f62 0 2024-04-06T03:40:42.629 INFO:tasks.workunit.client.0.smithi094.stdout:5/632: mkdir db/d7c/dd7 0 2024-04-06T03:40:42.630 INFO:tasks.workunit.client.1.smithi136.stdout:7/727: dwrite d5/df/d1d/d6f/d4d/fd4 [0,4194304] 0 2024-04-06T03:40:42.632 INFO:tasks.workunit.client.1.smithi136.stdout:0/694: symlink d1/d6/d2c/d57/d43/d6d/d9b/d5c/d96/lea 0 2024-04-06T03:40:42.634 INFO:tasks.workunit.client.1.smithi136.stdout:6/653: rename d2/d7/c11 to d2/d25/cd7 0 2024-04-06T03:40:42.635 INFO:tasks.workunit.client.1.smithi136.stdout:4/644: mkdir d1d/d31/d3e/d55/d89/dea 0 2024-04-06T03:40:42.635 INFO:tasks.workunit.client.1.smithi136.stdout:4/645: chown d1d/d31/d78 121561086 1 2024-04-06T03:40:42.636 INFO:tasks.workunit.client.0.smithi094.stdout:2/687: dread d0/d8/d27/d2b/fdd [0,4194304] 0 2024-04-06T03:40:42.637 INFO:tasks.workunit.client.0.smithi094.stdout:2/688: write d0/d18/f55 [3123226,85052] 0 2024-04-06T03:40:42.637 INFO:tasks.workunit.client.0.smithi094.stdout:2/689: write d0/d8/d27/fb4 [746996,24591] 0 2024-04-06T03:40:42.637 INFO:tasks.workunit.client.1.smithi136.stdout:8/750: creat d3/da/d26/d90/dae/db5/fe5 x:0 0 0 2024-04-06T03:40:42.638 INFO:tasks.workunit.client.1.smithi136.stdout:3/672: creat d0/d4/d12/d19/d55/d6f/ddb/fe7 x:0 0 0 2024-04-06T03:40:42.640 INFO:tasks.workunit.client.0.smithi094.stdout:2/690: symlink d0/dd/d1b/d72/d8f/lea 0 2024-04-06T03:40:42.640 INFO:tasks.workunit.client.0.smithi094.stdout:2/691: chown d0/d8/d1d/d77/cd7 10 1 2024-04-06T03:40:42.641 INFO:tasks.workunit.client.0.smithi094.stdout:5/633: dwrite db/fa8 [4194304,4194304] 0 2024-04-06T03:40:42.644 INFO:tasks.workunit.client.1.smithi136.stdout:7/728: mkdir d5/df/d53/d68/da3/dd0/d106 0 2024-04-06T03:40:42.645 INFO:tasks.workunit.client.1.smithi136.stdout:7/729: write d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/fb6 [371070,5310] 0 2024-04-06T03:40:42.645 INFO:tasks.workunit.client.1.smithi136.stdout:7/730: read - d5/df/d53/d68/da3/fe5 zero size 2024-04-06T03:40:42.645 INFO:tasks.workunit.client.1.smithi136.stdout:7/731: fdatasync d5/df/f41 0 2024-04-06T03:40:42.645 INFO:tasks.workunit.client.1.smithi136.stdout:0/695: mkdir d1/d6/d2c/d1a/d36/d59/deb 0 2024-04-06T03:40:42.645 INFO:tasks.workunit.client.1.smithi136.stdout:0/696: stat d1/d6/l2d 0 2024-04-06T03:40:42.646 INFO:tasks.workunit.client.1.smithi136.stdout:0/697: write d1/f5e [4276198,121248] 0 2024-04-06T03:40:42.647 INFO:tasks.workunit.client.1.smithi136.stdout:5/768: rename d1/d16/d89/dcb to d1/dc/d1c/d32/d8b/de3 0 2024-04-06T03:40:42.648 INFO:tasks.workunit.client.1.smithi136.stdout:8/751: truncate d3/de/f8d 644586 0 2024-04-06T03:40:42.648 INFO:tasks.workunit.client.1.smithi136.stdout:7/732: creat d5/d6d/f107 x:0 0 0 2024-04-06T03:40:42.649 INFO:tasks.workunit.client.1.smithi136.stdout:0/698: mkdir d1/d6/d2c/d57/d43/d6d/d85/da7/de5/dec 0 2024-04-06T03:40:42.649 INFO:tasks.workunit.client.1.smithi136.stdout:0/699: write d1/d4e/f75 [5161867,20286] 0 2024-04-06T03:40:42.651 INFO:tasks.workunit.client.1.smithi136.stdout:2/775: rename d3/d15/d19/d43/d54/l7a to d3/lff 0 2024-04-06T03:40:42.654 INFO:tasks.workunit.client.1.smithi136.stdout:5/769: creat d1/d98/dc6/fe4 x:0 0 0 2024-04-06T03:40:42.654 INFO:tasks.workunit.client.1.smithi136.stdout:8/752: creat d3/de/d60/fe6 x:0 0 0 2024-04-06T03:40:42.654 INFO:tasks.workunit.client.1.smithi136.stdout:8/753: creat d3/da/d26/d90/fe7 x:0 0 0 2024-04-06T03:40:42.654 INFO:tasks.workunit.client.1.smithi136.stdout:8/754: readlink d3/de/d18/d7d/ddf/d6c/d79/l83 0 2024-04-06T03:40:42.654 INFO:tasks.workunit.client.1.smithi136.stdout:8/755: write d3/de/f3a [8199886,106366] 0 2024-04-06T03:40:42.655 INFO:tasks.workunit.client.1.smithi136.stdout:8/756: fdatasync d3/da/d26/d35/d39/f99 0 2024-04-06T03:40:42.655 INFO:tasks.workunit.client.1.smithi136.stdout:0/700: truncate d1/d6/d2c/d57/d43/d6d/d85/d86/fba 711373 0 2024-04-06T03:40:42.657 INFO:tasks.workunit.client.1.smithi136.stdout:6/654: rename d2/d7/d26/d30/d6e/d3d/l55 to d2/d7/d26/d97/ld8 0 2024-04-06T03:40:42.658 INFO:tasks.workunit.client.1.smithi136.stdout:8/757: symlink d3/da/d26/d90/dd5/le8 0 2024-04-06T03:40:42.660 INFO:tasks.workunit.client.1.smithi136.stdout:0/701: mkdir d1/d6/ded 0 2024-04-06T03:40:42.660 INFO:tasks.workunit.client.1.smithi136.stdout:0/702: chown d1/d6/d2c/d57/d43/d6d/d85/d78/f7a 0 1 2024-04-06T03:40:42.662 INFO:tasks.workunit.client.1.smithi136.stdout:5/770: write d1/d1f/f21 [1156203,61330] 0 2024-04-06T03:40:42.662 INFO:tasks.workunit.client.1.smithi136.stdout:5/771: fdatasync d1/dc/d1c/d32/d8b/d20/d66/fbf 0 2024-04-06T03:40:42.662 INFO:tasks.workunit.client.1.smithi136.stdout:6/655: link d2/d7/d26/d30/d6e/d3d/d47/c52 d2/d7/d26/d30/d6e/d85/cd9 0 2024-04-06T03:40:42.662 INFO:tasks.workunit.client.1.smithi136.stdout:8/758: symlink d3/de/d18/d7d/de0/le9 0 2024-04-06T03:40:42.663 INFO:tasks.workunit.client.1.smithi136.stdout:4/646: rename d1d/d31/f35 to d1d/d31/d78/feb 0 2024-04-06T03:40:42.663 INFO:tasks.workunit.client.1.smithi136.stdout:4/647: chown cb 96 1 2024-04-06T03:40:42.666 INFO:tasks.workunit.client.1.smithi136.stdout:8/759: rmdir d3/de/d18 39 2024-04-06T03:40:42.666 INFO:tasks.workunit.client.1.smithi136.stdout:4/648: symlink d1d/dd0/lec 0 2024-04-06T03:40:42.668 INFO:tasks.workunit.client.1.smithi136.stdout:0/703: rename d1/d6/d2c/d57/d43/d6d/d85/d78/cc1 to d1/d6/d2c/d1a/cee 0 2024-04-06T03:40:42.668 INFO:tasks.workunit.client.1.smithi136.stdout:5/772: rmdir d1/dc/d1c/d32/d8b/db8 39 2024-04-06T03:40:42.668 INFO:tasks.workunit.client.1.smithi136.stdout:5/773: write d1/f96 [4968259,26758] 0 2024-04-06T03:40:42.669 INFO:tasks.workunit.client.1.smithi136.stdout:5/774: truncate d1/dc/d1c/d32/d8b/f73 309511 0 2024-04-06T03:40:42.669 INFO:tasks.workunit.client.1.smithi136.stdout:8/760: mknod d3/da/d26/d90/dae/cea 0 2024-04-06T03:40:42.669 INFO:tasks.workunit.client.1.smithi136.stdout:8/761: write d3/da/d26/d90/fde [165411,14490] 0 2024-04-06T03:40:42.669 INFO:tasks.workunit.client.1.smithi136.stdout:8/762: truncate d3/da/d26/d35/d39/f99 1100598 0 2024-04-06T03:40:42.669 INFO:tasks.workunit.client.1.smithi136.stdout:8/763: truncate d3/da/d26/d35/f57 3835266 0 2024-04-06T03:40:42.670 INFO:tasks.workunit.client.1.smithi136.stdout:7/733: dwrite d5/f2e [0,4194304] 0 2024-04-06T03:40:42.670 INFO:tasks.workunit.client.1.smithi136.stdout:0/704: mkdir d1/d6/d2c/d1a/d36/d59/def 0 2024-04-06T03:40:42.670 INFO:tasks.workunit.client.1.smithi136.stdout:7/734: write d5/df/d53/d8a/fbc [1224724,107013] 0 2024-04-06T03:40:42.670 INFO:tasks.workunit.client.1.smithi136.stdout:7/735: chown d5/df/d11/f38 45249 1 2024-04-06T03:40:42.672 INFO:tasks.workunit.client.1.smithi136.stdout:5/775: rename d1/dc/d1c/d32/d8b/l4a to d1/dc/d1c/d32/d8b/db0/le5 0 2024-04-06T03:40:42.673 INFO:tasks.workunit.client.1.smithi136.stdout:8/764: getdents d3/da/d26/d35/dcb 0 2024-04-06T03:40:42.677 INFO:tasks.workunit.client.1.smithi136.stdout:0/705: symlink d1/d6/d2c/d57/d43/lf0 0 2024-04-06T03:40:42.677 INFO:tasks.workunit.client.1.smithi136.stdout:0/706: creat d1/d6/d2c/d84/ff1 x:0 0 0 2024-04-06T03:40:42.677 INFO:tasks.workunit.client.1.smithi136.stdout:0/707: stat d1/d6/d2c/d57/d43/d6d/d85/d78/f97 0 2024-04-06T03:40:42.677 INFO:tasks.workunit.client.1.smithi136.stdout:0/708: dread - d1/d6/d2c/d1a/d36/d59/fbe zero size 2024-04-06T03:40:42.677 INFO:tasks.workunit.client.1.smithi136.stdout:0/709: dread - d1/d6/d2c/d57/d43/d6d/d85/d86/fbc zero size 2024-04-06T03:40:42.677 INFO:tasks.workunit.client.1.smithi136.stdout:3/673: dread d0/d4/d12/d2d/d37/f6d [0,4194304] 0 2024-04-06T03:40:42.681 INFO:tasks.workunit.client.1.smithi136.stdout:8/765: truncate d3/de/d18/d7d/ddf/f92 131302 0 2024-04-06T03:40:42.683 INFO:tasks.workunit.client.1.smithi136.stdout:3/674: link d0/f23 d0/d4/d12/d19/d2c/fe8 0 2024-04-06T03:40:42.683 INFO:tasks.workunit.client.1.smithi136.stdout:3/675: fdatasync d0/d4/d12/d19/f29 0 2024-04-06T03:40:42.684 INFO:tasks.workunit.client.1.smithi136.stdout:0/710: dread d1/d6/d2c/d57/f3f [0,4194304] 0 2024-04-06T03:40:42.685 INFO:tasks.workunit.client.1.smithi136.stdout:5/776: symlink d1/d98/dbc/le6 0 2024-04-06T03:40:42.685 INFO:tasks.workunit.client.1.smithi136.stdout:5/777: chown d1/d9b/d9c 6043877 1 2024-04-06T03:40:42.691 INFO:tasks.workunit.client.1.smithi136.stdout:9/702: dwrite d17/d20/d28/d49/d75/f9d [0,4194304] 0 2024-04-06T03:40:42.691 INFO:tasks.workunit.client.1.smithi136.stdout:9/703: truncate d17/d20/d25/d71/fa2 4558339 0 2024-04-06T03:40:42.691 INFO:tasks.workunit.client.1.smithi136.stdout:4/649: dread d1d/f20 [0,4194304] 0 2024-04-06T03:40:42.691 INFO:tasks.workunit.client.1.smithi136.stdout:3/676: symlink d0/d4/d12/d19/d55/d3c/le9 0 2024-04-06T03:40:42.692 INFO:tasks.workunit.client.1.smithi136.stdout:8/766: dread d3/da/d26/f75 [0,4194304] 0 2024-04-06T03:40:42.692 INFO:tasks.workunit.client.1.smithi136.stdout:8/767: readlink d3/l14 0 2024-04-06T03:40:42.692 INFO:tasks.workunit.client.0.smithi094.stdout:5/634: dwrite db/d19/d24/f72 [0,4194304] 0 2024-04-06T03:40:42.692 INFO:tasks.workunit.client.0.smithi094.stdout:5/635: getdents db/d7c/dd7 0 2024-04-06T03:40:42.692 INFO:tasks.workunit.client.0.smithi094.stdout:6/628: dwrite dc/d3a/d4a/d54/dde/d50/f80 [0,4194304] 0 2024-04-06T03:40:42.693 INFO:tasks.workunit.client.1.smithi136.stdout:0/711: link d1/d6/d2c/d57/d43/d6d/d85/d78/f7a d1/d6/d2c/d57/d43/ff2 0 2024-04-06T03:40:42.694 INFO:tasks.workunit.client.1.smithi136.stdout:9/704: write d17/d21/fa7 [718748,59240] 0 2024-04-06T03:40:42.694 INFO:tasks.workunit.client.1.smithi136.stdout:7/736: dwrite d5/df/d12/d1b/fe7 [0,4194304] 0 2024-04-06T03:40:42.694 INFO:tasks.workunit.client.1.smithi136.stdout:7/737: chown d5/f82 3 1 2024-04-06T03:40:42.695 INFO:tasks.workunit.client.1.smithi136.stdout:5/778: link d1/c27 d1/d16/d17/ce7 0 2024-04-06T03:40:42.695 INFO:tasks.workunit.client.1.smithi136.stdout:5/779: fsync d1/dc/d1c/d32/d8b/db8/fdf 0 2024-04-06T03:40:42.696 INFO:tasks.workunit.client.1.smithi136.stdout:9/705: dread d17/d21/d29/f5f [0,4194304] 0 2024-04-06T03:40:42.696 INFO:tasks.workunit.client.1.smithi136.stdout:9/706: write d17/d20/d25/da4/dbb/fc4 [81571,21274] 0 2024-04-06T03:40:42.701 INFO:tasks.workunit.client.0.smithi094.stdout:5/636: dread db/d19/d2c/f2d [0,4194304] 0 2024-04-06T03:40:42.709 INFO:tasks.workunit.client.1.smithi136.stdout:3/677: dwrite d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/fdc [0,4194304] 0 2024-04-06T03:40:42.710 INFO:tasks.workunit.client.1.smithi136.stdout:3/678: dread d0/d4/d12/d19/d55/d6f/fb1 [0,4194304] 0 2024-04-06T03:40:42.710 INFO:tasks.workunit.client.1.smithi136.stdout:3/679: stat d0/d4/d12/d19/d55/d3c/d46/f72 0 2024-04-06T03:40:42.711 INFO:tasks.workunit.client.1.smithi136.stdout:3/680: read d0/d4/d12/d19/d55/d3c/faa [339299,16831] 0 2024-04-06T03:40:42.711 INFO:tasks.workunit.client.1.smithi136.stdout:4/650: stat d1d/cc0 0 2024-04-06T03:40:42.711 INFO:tasks.workunit.client.1.smithi136.stdout:4/651: write d1d/d31/d3e/d56/d7b/fce [1041215,42051] 0 2024-04-06T03:40:42.712 INFO:tasks.workunit.client.0.smithi094.stdout:6/629: symlink dc/dd3/d72/d88/ldf 0 2024-04-06T03:40:42.713 INFO:tasks.workunit.client.1.smithi136.stdout:1/691: getdents d12/d3b/d44/d5f/d5a/d72/d77 0 2024-04-06T03:40:42.715 INFO:tasks.workunit.client.1.smithi136.stdout:7/738: dwrite d5/df/d1d/d6f/d4d/d69/fc8 [0,4194304] 0 2024-04-06T03:40:42.715 INFO:tasks.workunit.client.1.smithi136.stdout:5/780: dwrite d1/dc/d1c/f78 [0,4194304] 0 2024-04-06T03:40:42.717 INFO:tasks.workunit.client.0.smithi094.stdout:5/637: rename db/d7c/d8b/fc3 to db/d19/dc1/dd4/d77/fd8 0 2024-04-06T03:40:42.717 INFO:tasks.workunit.client.0.smithi094.stdout:5/638: chown db/d19/d2c/d6d/cb9 230938160 1 2024-04-06T03:40:42.720 INFO:tasks.workunit.client.1.smithi136.stdout:3/681: write d0/d4/d12/d19/d55/d3c/d46/f72 [74464,78065] 0 2024-04-06T03:40:42.720 INFO:tasks.workunit.client.1.smithi136.stdout:3/682: chown d0/d4/d91/lcf 479995099 1 2024-04-06T03:40:42.720 INFO:tasks.workunit.client.1.smithi136.stdout:3/683: fdatasync d0/f11 0 2024-04-06T03:40:42.720 INFO:tasks.workunit.client.1.smithi136.stdout:3/684: fsync d0/d4/f22 0 2024-04-06T03:40:42.720 INFO:tasks.workunit.client.1.smithi136.stdout:3/685: chown d0/d4/d12/d19/d2e/f87 0 1 2024-04-06T03:40:42.725 INFO:tasks.workunit.client.0.smithi094.stdout:6/630: truncate dc/dd3/f1f 1473749 0 2024-04-06T03:40:42.725 INFO:tasks.workunit.client.0.smithi094.stdout:6/631: stat dc/d3a/d4a/d54/dde/f57 0 2024-04-06T03:40:42.727 INFO:tasks.workunit.client.0.smithi094.stdout:5/639: symlink db/d19/d2c/ld9 0 2024-04-06T03:40:42.729 INFO:tasks.workunit.client.0.smithi094.stdout:6/632: mkdir dc/d3a/d4a/d54/dde/d50/de0 0 2024-04-06T03:40:42.732 INFO:tasks.workunit.client.0.smithi094.stdout:5/640: creat db/d19/d21/fda x:0 0 0 2024-04-06T03:40:42.734 INFO:tasks.workunit.client.1.smithi136.stdout:8/768: mkdir d3/de/d60/deb 0 2024-04-06T03:40:42.734 INFO:tasks.workunit.client.1.smithi136.stdout:8/769: write d3/f77 [630616,130758] 0 2024-04-06T03:40:42.738 INFO:tasks.workunit.client.1.smithi136.stdout:0/712: rename d1/f4c to d1/d6/d2c/d1a/d36/ff3 0 2024-04-06T03:40:42.739 INFO:tasks.workunit.client.0.smithi094.stdout:6/633: truncate dc/d3a/d4a/d54/dde/f9e 585908 0 2024-04-06T03:40:42.745 INFO:tasks.workunit.client.1.smithi136.stdout:9/707: mknod d17/d20/d28/d81/d88/cf2 0 2024-04-06T03:40:42.745 INFO:tasks.workunit.client.1.smithi136.stdout:9/708: chown d17/d21/d29 131632 1 2024-04-06T03:40:42.751 INFO:tasks.workunit.client.0.smithi094.stdout:6/634: write dc/d3a/d4a/d54/dde/f39 [2651033,6133] 0 2024-04-06T03:40:42.751 INFO:tasks.workunit.client.0.smithi094.stdout:6/635: stat dc/dd3/d72/d87/ca6 0 2024-04-06T03:40:42.753 INFO:tasks.workunit.client.1.smithi136.stdout:8/770: dwrite d3/d2d/d5f/fb4 [0,4194304] 0 2024-04-06T03:40:42.754 INFO:tasks.workunit.client.0.smithi094.stdout:5/641: dwrite db/d19/d21/f62 [0,4194304] 0 2024-04-06T03:40:42.754 INFO:tasks.workunit.client.0.smithi094.stdout:5/642: stat db/d19/d2c/c94 0 2024-04-06T03:40:42.759 INFO:tasks.workunit.client.0.smithi094.stdout:5/643: dread db/d19/d21/f8a [0,4194304] 0 2024-04-06T03:40:42.760 INFO:tasks.workunit.client.0.smithi094.stdout:5/644: write db/d19/d30/f63 [481438,19151] 0 2024-04-06T03:40:42.761 INFO:tasks.workunit.client.0.smithi094.stdout:5/645: dread db/d19/d24/d28/f8c [0,4194304] 0 2024-04-06T03:40:42.840 INFO:tasks.workunit.client.1.smithi136.stdout:7/739: mkdir d5/df/d1d/d54/d108 0 2024-04-06T03:40:42.847 INFO:tasks.workunit.client.0.smithi094.stdout:6/636: getdents dc/d3a/d4a/d94/dac 0 2024-04-06T03:40:42.849 INFO:tasks.workunit.client.1.smithi136.stdout:5/781: creat d1/dc/d1c/d32/d70/fe8 x:0 0 0 2024-04-06T03:40:42.849 INFO:tasks.workunit.client.0.smithi094.stdout:6/637: link dc/d3a/d7f/cd1 dc/db6/ce1 0 2024-04-06T03:40:42.850 INFO:tasks.workunit.client.0.smithi094.stdout:6/638: read - dc/dd3/d24/fbe zero size 2024-04-06T03:40:42.850 INFO:tasks.workunit.client.0.smithi094.stdout:6/639: fdatasync dc/d3a/d4a/d54/dde/d50/f8f 0 2024-04-06T03:40:42.852 INFO:tasks.workunit.client.1.smithi136.stdout:3/686: creat d0/d4/d12/d19/d55/d3c/d53/de6/fea x:0 0 0 2024-04-06T03:40:42.859 INFO:tasks.workunit.client.1.smithi136.stdout:0/713: rmdir d1/d6/d2c/d1a/d36/d59/deb 0 2024-04-06T03:40:42.861 INFO:tasks.workunit.client.1.smithi136.stdout:9/709: truncate d17/d20/d28/d49/d75/f91 491536 0 2024-04-06T03:40:42.862 INFO:tasks.workunit.client.1.smithi136.stdout:3/687: dread d0/d4/d12/d19/d55/d6f/fad [0,4194304] 0 2024-04-06T03:40:42.864 INFO:tasks.workunit.client.1.smithi136.stdout:5/782: dwrite d1/d98/dc6/fe4 [0,4194304] 0 2024-04-06T03:40:42.864 INFO:tasks.workunit.client.1.smithi136.stdout:5/783: readlink d1/d16/d4b/d7b/l90 0 2024-04-06T03:40:42.865 INFO:tasks.workunit.client.1.smithi136.stdout:8/771: read - d3/da/d26/d35/d53/fa9 zero size 2024-04-06T03:40:42.868 INFO:tasks.workunit.client.1.smithi136.stdout:7/740: link d5/df/d53/l101 d5/df/d53/d68/d8f/l109 0 2024-04-06T03:40:42.870 INFO:tasks.workunit.client.0.smithi094.stdout:6/640: dwrite dc/dd3/d24/d46/fa5 [0,4194304] 0 2024-04-06T03:40:42.870 INFO:tasks.workunit.client.1.smithi136.stdout:0/714: write d1/d6/f58 [3047958,29733] 0 2024-04-06T03:40:42.870 INFO:tasks.workunit.client.1.smithi136.stdout:0/715: chown d1/d6/d2c/d57/d62/l90 1 1 2024-04-06T03:40:42.870 INFO:tasks.workunit.client.1.smithi136.stdout:1/692: rename d12/d3b/d44/d5f/d5a/d72 to d12/d3b/d74/d7a/dea 0 2024-04-06T03:40:42.872 INFO:tasks.workunit.client.1.smithi136.stdout:3/688: mkdir d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/dd4/deb 0 2024-04-06T03:40:42.872 INFO:tasks.workunit.client.1.smithi136.stdout:3/689: creat d0/d4/d12/d19/d55/fec x:0 0 0 2024-04-06T03:40:42.873 INFO:tasks.workunit.client.1.smithi136.stdout:3/690: chown d0/d4/d12/d19/d55/fdd 31094 1 2024-04-06T03:40:42.873 INFO:tasks.workunit.client.1.smithi136.stdout:3/691: chown d0/d4/d12/d19/d55/d3c/d5a/fac 15 1 2024-04-06T03:40:42.873 INFO:tasks.workunit.client.1.smithi136.stdout:7/741: dread d5/df/d12/d1b/f47 [0,4194304] 0 2024-04-06T03:40:42.877 INFO:tasks.workunit.client.1.smithi136.stdout:9/710: dwrite d17/d21/d32/fa6 [0,4194304] 0 2024-04-06T03:40:42.879 INFO:tasks.workunit.client.0.smithi094.stdout:6/641: creat dc/dd3/d24/d46/fe2 x:0 0 0 2024-04-06T03:40:42.881 INFO:tasks.workunit.client.1.smithi136.stdout:3/692: write d0/d4/d12/d19/f29 [3546857,123860] 0 2024-04-06T03:40:42.881 INFO:tasks.workunit.client.1.smithi136.stdout:3/693: stat d0/d4/d12/d19/d55/d3c/d5a/d79/fa4 0 2024-04-06T03:40:42.882 INFO:tasks.workunit.client.1.smithi136.stdout:0/716: rmdir d1/d6/d2c/d1a/d36/d59/def 0 2024-04-06T03:40:42.883 INFO:tasks.workunit.client.1.smithi136.stdout:8/772: dwrite d3/de/f4b [0,4194304] 0 2024-04-06T03:40:42.883 INFO:tasks.workunit.client.1.smithi136.stdout:8/773: stat d3/da/d26/d35/d53/d86/f8f 0 2024-04-06T03:40:42.883 INFO:tasks.workunit.client.1.smithi136.stdout:8/774: chown d3/da/d26/d35/d39/l84 1899 1 2024-04-06T03:40:42.884 INFO:tasks.workunit.client.0.smithi094.stdout:6/642: rename dc/d11/l35 to dc/dd3/d72/d88/da3/le3 0 2024-04-06T03:40:42.887 INFO:tasks.workunit.client.1.smithi136.stdout:5/784: rename d1/dc/d10/da3/lab to d1/dc/d1c/d32/d8b/le9 0 2024-04-06T03:40:42.887 INFO:tasks.workunit.client.1.smithi136.stdout:5/785: write d1/dc/d1c/d32/d70/fe8 [955804,23106] 0 2024-04-06T03:40:42.887 INFO:tasks.workunit.client.1.smithi136.stdout:5/786: chown d1/d16/d4b/f95 37670 1 2024-04-06T03:40:42.890 INFO:tasks.workunit.client.0.smithi094.stdout:6/643: mkdir dc/dd3/d24/de4 0 2024-04-06T03:40:42.891 INFO:tasks.workunit.client.1.smithi136.stdout:8/775: dread d3/de/f4b [0,4194304] 0 2024-04-06T03:40:42.892 INFO:tasks.workunit.client.1.smithi136.stdout:7/742: creat d5/df/d1d/d6f/d2a/f10a x:0 0 0 2024-04-06T03:40:42.894 INFO:tasks.workunit.client.0.smithi094.stdout:6/644: link dc/d3a/d4a/d54/fc7 dc/dd3/d24/fe5 0 2024-04-06T03:40:42.900 INFO:tasks.workunit.client.1.smithi136.stdout:9/711: creat d17/d20/d25/d3c/d55/d8a/ff3 x:0 0 0 2024-04-06T03:40:42.901 INFO:tasks.workunit.client.1.smithi136.stdout:3/694: mkdir d0/d4/d12/d19/d2c/ded 0 2024-04-06T03:40:42.903 INFO:tasks.workunit.client.1.smithi136.stdout:8/776: dread d3/fc [0,4194304] 0 2024-04-06T03:40:42.906 INFO:tasks.workunit.client.1.smithi136.stdout:5/787: mkdir d1/d9b/dea 0 2024-04-06T03:40:42.906 INFO:tasks.workunit.client.1.smithi136.stdout:5/788: creat d1/dc/d1c/d32/feb x:0 0 0 2024-04-06T03:40:42.907 INFO:tasks.workunit.client.1.smithi136.stdout:1/693: getdents d12/d1b/d1e/d9f 0 2024-04-06T03:40:42.908 INFO:tasks.workunit.client.1.smithi136.stdout:1/694: truncate d12/d1b/d1e/dd0/f70 1491259 0 2024-04-06T03:40:42.908 INFO:tasks.workunit.client.1.smithi136.stdout:1/695: write d12/d1b/f29 [1294774,12055] 0 2024-04-06T03:40:42.908 INFO:tasks.workunit.client.1.smithi136.stdout:7/743: mknod d5/df/d12/d1b/d58/c10b 0 2024-04-06T03:40:42.909 INFO:tasks.workunit.client.1.smithi136.stdout:7/744: write d5/df/d1d/d6f/d4d/fcc [812503,47777] 0 2024-04-06T03:40:42.910 INFO:tasks.workunit.client.1.smithi136.stdout:9/712: truncate f16 2442209 0 2024-04-06T03:40:42.910 INFO:tasks.workunit.client.1.smithi136.stdout:9/713: stat d17/d21/d29/de5/d4b 0 2024-04-06T03:40:42.912 INFO:tasks.workunit.client.1.smithi136.stdout:3/695: mkdir d0/d4/d12/dee 0 2024-04-06T03:40:42.912 INFO:tasks.workunit.client.1.smithi136.stdout:3/696: fsync d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/fdc 0 2024-04-06T03:40:42.912 INFO:tasks.workunit.client.1.smithi136.stdout:3/697: dread - d0/d4/d12/d19/d55/f9f zero size 2024-04-06T03:40:42.912 INFO:tasks.workunit.client.1.smithi136.stdout:3/698: chown d0/d4/d12/d19/d55/d3c/d46/c48 15088 1 2024-04-06T03:40:42.913 INFO:tasks.workunit.client.1.smithi136.stdout:0/717: dread d1/d6/d2c/f20 [0,4194304] 0 2024-04-06T03:40:42.913 INFO:tasks.workunit.client.1.smithi136.stdout:0/718: creat d1/d6/d2c/d57/d43/d6d/ff4 x:0 0 0 2024-04-06T03:40:42.914 INFO:tasks.workunit.client.1.smithi136.stdout:8/777: rename d3/de/d60/deb to d3/da/d26/d90/dae/db5/dec 0 2024-04-06T03:40:42.916 INFO:tasks.workunit.client.0.smithi094.stdout:6/645: dwrite dc/d3a/d4a/d54/dde/f57 [0,4194304] 0 2024-04-06T03:40:42.916 INFO:tasks.workunit.client.0.smithi094.stdout:6/646: chown dc/dd3/d24/d46/d7b/l9a 0 1 2024-04-06T03:40:42.916 INFO:tasks.workunit.client.0.smithi094.stdout:6/647: chown dc/d3a/d4a/d54/dde/l47 30 1 2024-04-06T03:40:42.917 INFO:tasks.workunit.client.1.smithi136.stdout:1/696: rmdir d12/d3b/d74/d7a 39 2024-04-06T03:40:42.920 INFO:tasks.workunit.client.1.smithi136.stdout:9/714: symlink d17/d20/lf4 0 2024-04-06T03:40:42.921 INFO:tasks.workunit.client.0.smithi094.stdout:6/648: creat dc/dd3/d24/de4/fe6 x:0 0 0 2024-04-06T03:40:42.921 INFO:tasks.workunit.client.0.smithi094.stdout:6/649: read - dc/dd3/d72/f7c zero size 2024-04-06T03:40:42.923 INFO:tasks.workunit.client.1.smithi136.stdout:3/699: link d0/l9 d0/d4/d12/d19/d55/d6f/ddb/lef 0 2024-04-06T03:40:42.923 INFO:tasks.workunit.client.1.smithi136.stdout:3/700: chown d0/d4 17031 1 2024-04-06T03:40:42.923 INFO:tasks.workunit.client.1.smithi136.stdout:3/701: stat d0/d4/d12/dc3/dd3/de1 0 2024-04-06T03:40:42.926 INFO:tasks.workunit.client.1.smithi136.stdout:0/719: rmdir d1/d6/d2c/d57/d43/d6d/d9b 39 2024-04-06T03:40:42.926 INFO:tasks.workunit.client.1.smithi136.stdout:0/720: stat d1/d6/d2c/d57/d43/d6d/d85/f7f 0 2024-04-06T03:40:42.928 INFO:tasks.workunit.client.1.smithi136.stdout:5/789: dwrite d1/d9b/faa [0,4194304] 0 2024-04-06T03:40:42.928 INFO:tasks.workunit.client.1.smithi136.stdout:7/745: dwrite d5/df/d1d/f57 [0,4194304] 0 2024-04-06T03:40:42.929 INFO:tasks.workunit.client.1.smithi136.stdout:8/778: rename d3/da/f42 to d3/da/d26/d90/dae/fed 0 2024-04-06T03:40:42.930 INFO:tasks.workunit.client.1.smithi136.stdout:1/697: mkdir d12/d1b/d28/deb 0 2024-04-06T03:40:42.930 INFO:tasks.workunit.client.1.smithi136.stdout:1/698: write d12/d3b/d8e/fbc [409788,36212] 0 2024-04-06T03:40:42.930 INFO:tasks.workunit.client.1.smithi136.stdout:5/790: dread d1/dc/d1c/d32/d8b/db0/fca [0,4194304] 0 2024-04-06T03:40:42.930 INFO:tasks.workunit.client.1.smithi136.stdout:5/791: chown d1/dc/d1c/d32/d8b/l65 54666 1 2024-04-06T03:40:42.931 INFO:tasks.workunit.client.0.smithi094.stdout:6/650: dread dc/dd3/f31 [0,4194304] 0 2024-04-06T03:40:42.931 INFO:tasks.workunit.client.1.smithi136.stdout:9/715: stat d17/d20/d25/ld2 0 2024-04-06T03:40:42.932 INFO:tasks.workunit.client.1.smithi136.stdout:3/702: mkdir d0/d4/d12/d19/d55/d6f/df0 0 2024-04-06T03:40:42.932 INFO:tasks.workunit.client.0.smithi094.stdout:6/651: write dc/d3a/d4a/f56 [978557,14487] 0 2024-04-06T03:40:42.933 INFO:tasks.workunit.client.1.smithi136.stdout:0/721: mkdir d1/d6/df5 0 2024-04-06T03:40:42.934 INFO:tasks.workunit.client.1.smithi136.stdout:0/722: read d1/f18 [723915,27293] 0 2024-04-06T03:40:42.937 INFO:tasks.workunit.client.0.smithi094.stdout:6/652: rename dc/l3e to dc/d3a/d7f/dd4/le7 0 2024-04-06T03:40:42.940 INFO:tasks.workunit.client.0.smithi094.stdout:6/653: creat dc/dd3/d24/fe8 x:0 0 0 2024-04-06T03:40:42.942 INFO:tasks.workunit.client.1.smithi136.stdout:7/746: rename d5/df/d1d/d6f/fc2 to d5/df/d12/d1b/f10c 0 2024-04-06T03:40:42.942 INFO:tasks.workunit.client.1.smithi136.stdout:1/699: rename d12/d3b/d74/d7a to d12/d3b/d74/d7a/dea/dd2/dec 22 2024-04-06T03:40:42.942 INFO:tasks.workunit.client.1.smithi136.stdout:7/747: readlink d5/df/d1d/d54/lb3 0 2024-04-06T03:40:42.943 INFO:tasks.workunit.client.1.smithi136.stdout:1/700: creat d12/d3b/d44/d5f/db7/fed x:0 0 0 2024-04-06T03:40:42.943 INFO:tasks.workunit.client.1.smithi136.stdout:1/701: readlink d12/d1b/d1e/d6f/l83 0 2024-04-06T03:40:42.943 INFO:tasks.workunit.client.1.smithi136.stdout:1/702: truncate d12/d3b/d74/fba 4690227 0 2024-04-06T03:40:42.943 INFO:tasks.workunit.client.1.smithi136.stdout:1/703: getdents d12/d3b/d74/d7a/dea/d77 0 2024-04-06T03:40:42.943 INFO:tasks.workunit.client.1.smithi136.stdout:1/704: chown d12/d1b/d22 105 1 2024-04-06T03:40:42.943 INFO:tasks.workunit.client.1.smithi136.stdout:1/705: chown d12/d1b/c90 477893394 1 2024-04-06T03:40:42.943 INFO:tasks.workunit.client.1.smithi136.stdout:8/779: mknod d3/da/d26/d35/d53/d86/db0/cee 0 2024-04-06T03:40:42.943 INFO:tasks.workunit.client.0.smithi094.stdout:6/654: symlink dc/d3a/d4a/d54/dde/d50/de0/le9 0 2024-04-06T03:40:42.944 INFO:tasks.workunit.client.1.smithi136.stdout:5/792: symlink d1/d98/dc0/lec 0 2024-04-06T03:40:42.944 INFO:tasks.workunit.client.1.smithi136.stdout:5/793: write d1/dc/d1c/d32/d8b/db0/fca [710103,67921] 0 2024-04-06T03:40:42.945 INFO:tasks.workunit.client.1.smithi136.stdout:9/716: mkdir d17/d21/d29/de5/da5/df5 0 2024-04-06T03:40:42.947 INFO:tasks.workunit.client.1.smithi136.stdout:0/723: mknod d1/d53/cf6 0 2024-04-06T03:40:42.950 INFO:tasks.workunit.client.1.smithi136.stdout:3/703: rename d0/d4/d12/d19/d55/d3c/f4e to d0/d4/d12/d19/d55/d3c/d5a/d79/ff1 0 2024-04-06T03:40:42.951 INFO:tasks.workunit.client.1.smithi136.stdout:7/748: symlink d5/df/d12/dd6/l10d 0 2024-04-06T03:40:42.951 INFO:tasks.workunit.client.1.smithi136.stdout:7/749: truncate d5/d6d/f107 483601 0 2024-04-06T03:40:42.952 INFO:tasks.workunit.client.1.smithi136.stdout:5/794: dread d1/d16/d4b/f80 [0,4194304] 0 2024-04-06T03:40:42.953 INFO:tasks.workunit.client.0.smithi094.stdout:6/655: write f7 [3466486,90547] 0 2024-04-06T03:40:42.953 INFO:tasks.workunit.client.1.smithi136.stdout:1/706: mknod d12/d3b/d74/d7a/dea/d77/d96/cee 0 2024-04-06T03:40:42.953 INFO:tasks.workunit.client.1.smithi136.stdout:8/780: mknod d3/de/d18/d7d/cef 0 2024-04-06T03:40:42.957 INFO:tasks.workunit.client.0.smithi094.stdout:6/656: symlink dc/dd3/d72/d88/da3/dc4/lea 0 2024-04-06T03:40:42.957 INFO:tasks.workunit.client.0.smithi094.stdout:6/657: fdatasync dc/dd3/d72/d88/da3/faa 0 2024-04-06T03:40:42.960 INFO:tasks.workunit.client.1.smithi136.stdout:3/704: mkdir d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/df2 0 2024-04-06T03:40:42.960 INFO:tasks.workunit.client.1.smithi136.stdout:3/705: write d0/f2 [527768,114765] 0 2024-04-06T03:40:42.960 INFO:tasks.workunit.client.0.smithi094.stdout:6/658: creat dc/dd3/d24/d46/d7b/dc6/feb x:0 0 0 2024-04-06T03:40:42.962 INFO:tasks.workunit.client.1.smithi136.stdout:5/795: mknod d1/d16/ced 0 2024-04-06T03:40:42.963 INFO:tasks.workunit.client.1.smithi136.stdout:5/796: write d1/dc/d1c/d32/d8b/d20/d66/fb9 [4797,45685] 0 2024-04-06T03:40:42.967 INFO:tasks.workunit.client.1.smithi136.stdout:3/706: rename d0/d4/d12/d19/d2e/f87 to d0/d4/d12/d3a/d51/ff3 0 2024-04-06T03:40:42.967 INFO:tasks.workunit.client.1.smithi136.stdout:3/707: chown d0/d4/f45 2 1 2024-04-06T03:40:42.967 INFO:tasks.workunit.client.1.smithi136.stdout:3/708: readlink d0/d4/d12/d3a/l68 0 2024-04-06T03:40:42.968 INFO:tasks.workunit.client.0.smithi094.stdout:6/659: dread dc/dd3/d24/d46/f48 [0,4194304] 0 2024-04-06T03:40:42.972 INFO:tasks.workunit.client.1.smithi136.stdout:7/750: dwrite d5/df/d53/d68/ffb [0,4194304] 0 2024-04-06T03:40:42.972 INFO:tasks.workunit.client.1.smithi136.stdout:7/751: chown d5/df/d12/d1b/d3a/d4a/d70/ddb/faf 70564724 1 2024-04-06T03:40:42.973 INFO:tasks.workunit.client.1.smithi136.stdout:7/752: write d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/fb6 [800680,77534] 0 2024-04-06T03:40:42.973 INFO:tasks.workunit.client.1.smithi136.stdout:1/707: dwrite f10 [4194304,4194304] 0 2024-04-06T03:40:42.974 INFO:tasks.workunit.client.0.smithi094.stdout:6/660: mknod dc/dd3/d72/d88/da3/cec 0 2024-04-06T03:40:42.978 INFO:tasks.workunit.client.1.smithi136.stdout:0/724: dwrite d1/d6/d2c/f5a [0,4194304] 0 2024-04-06T03:40:42.978 INFO:tasks.workunit.client.1.smithi136.stdout:0/725: write d1/d6/d2c/d57/d43/d6d/d85/d86/fbc [267101,29179] 0 2024-04-06T03:40:42.978 INFO:tasks.workunit.client.1.smithi136.stdout:9/717: truncate d17/d20/d25/d71/dd5/fed 26600 0 2024-04-06T03:40:42.978 INFO:tasks.workunit.client.1.smithi136.stdout:5/797: mknod d1/d98/dbc/cee 0 2024-04-06T03:40:42.978 INFO:tasks.workunit.client.1.smithi136.stdout:5/798: chown d1/dc/d1c/d32/feb 47208 1 2024-04-06T03:40:42.979 INFO:tasks.workunit.client.0.smithi094.stdout:6/661: creat dc/fed x:0 0 0 2024-04-06T03:40:42.979 INFO:tasks.workunit.client.0.smithi094.stdout:6/662: chown dc/d3a/l67 514987 1 2024-04-06T03:40:42.980 INFO:tasks.workunit.client.1.smithi136.stdout:7/753: rename d5/df/d12/d1b/d3a/d4a/le2 to d5/df/d53/d8a/l10e 0 2024-04-06T03:40:42.980 INFO:tasks.workunit.client.1.smithi136.stdout:7/754: chown d5/fa1 32808 1 2024-04-06T03:40:42.980 INFO:tasks.workunit.client.1.smithi136.stdout:7/755: dread - d5/df/d12/d1b/dbb/fd3 zero size 2024-04-06T03:40:42.981 INFO:tasks.workunit.client.1.smithi136.stdout:1/708: creat d12/d53/dc5/fef x:0 0 0 2024-04-06T03:40:42.981 INFO:tasks.workunit.client.0.smithi094.stdout:6/663: mknod dc/db6/cee 0 2024-04-06T03:40:42.983 INFO:tasks.workunit.client.1.smithi136.stdout:3/709: dwrite d0/d4/d12/d3a/d51/da6/fb6 [0,4194304] 0 2024-04-06T03:40:42.983 INFO:tasks.workunit.client.1.smithi136.stdout:3/710: fdatasync d0/d4/d12/d19/d55/d3c/d5a/d79/ff1 0 2024-04-06T03:40:42.984 INFO:tasks.workunit.client.1.smithi136.stdout:0/726: link d1/d6/d2c/d57/fe6 d1/d53/ff7 0 2024-04-06T03:40:42.986 INFO:tasks.workunit.client.0.smithi094.stdout:6/664: creat dc/d3a/d4a/d54/dde/d50/d61/fef x:0 0 0 2024-04-06T03:40:42.987 INFO:tasks.workunit.client.1.smithi136.stdout:9/718: creat d17/d20/d28/d81/d88/dcb/ff6 x:0 0 0 2024-04-06T03:40:42.987 INFO:tasks.workunit.client.1.smithi136.stdout:9/719: write f13 [3651592,128938] 0 2024-04-06T03:40:42.988 INFO:tasks.workunit.client.1.smithi136.stdout:5/799: creat d1/dc/d1c/d32/d8b/d20/fef x:0 0 0 2024-04-06T03:40:42.988 INFO:tasks.workunit.client.1.smithi136.stdout:5/800: write d1/dc/d10/fc4 [358198,22950] 0 2024-04-06T03:40:42.990 INFO:tasks.workunit.client.1.smithi136.stdout:2/776: sync 2024-04-06T03:40:42.990 INFO:tasks.workunit.client.1.smithi136.stdout:6/656: sync 2024-04-06T03:40:42.991 INFO:tasks.workunit.client.1.smithi136.stdout:1/709: symlink d12/d3b/d74/d7a/dea/d77/d96/lf0 0 2024-04-06T03:40:42.991 INFO:tasks.workunit.client.1.smithi136.stdout:6/657: write d2/d7/d26/d30/d6e/d3d/f6a [926583,94664] 0 2024-04-06T03:40:42.992 INFO:tasks.workunit.client.0.smithi094.stdout:3/683: sync 2024-04-06T03:40:42.994 INFO:tasks.workunit.client.1.smithi136.stdout:0/727: mknod d1/d6/d2c/d1a/d9c/cf8 0 2024-04-06T03:40:42.994 INFO:tasks.workunit.client.1.smithi136.stdout:0/728: write d1/d6/d2c/d57/d43/d6d/d85/d78/f9a [5214382,1929] 0 2024-04-06T03:40:42.999 INFO:tasks.workunit.client.1.smithi136.stdout:9/720: mknod d17/d20/d28/d49/d75/d60/cf7 0 2024-04-06T03:40:43.002 INFO:tasks.workunit.client.1.smithi136.stdout:7/756: dwrite d5/df/d1d/d54/d5c/dc7/fd8 [0,4194304] 0 2024-04-06T03:40:43.002 INFO:tasks.workunit.client.1.smithi136.stdout:7/757: creat d5/df/d12/d1b/dbb/f10f x:0 0 0 2024-04-06T03:40:43.003 INFO:tasks.workunit.client.1.smithi136.stdout:7/758: chown d5/df/d1d 3418062 1 2024-04-06T03:40:43.006 INFO:tasks.workunit.client.1.smithi136.stdout:4/652: dwrite d1d/f26 [0,4194304] 0 2024-04-06T03:40:43.016 INFO:tasks.workunit.client.0.smithi094.stdout:3/684: dwrite d9/d1e/d6e/d75/dba/f38 [0,4194304] 0 2024-04-06T03:40:43.017 INFO:tasks.workunit.client.1.smithi136.stdout:0/729: rename d1/d6/d2c/d57/f3f to d1/d6/d2c/d57/d43/d6d/d9b/d5c/ff9 0 2024-04-06T03:40:43.017 INFO:tasks.workunit.client.1.smithi136.stdout:0/730: read d1/d4e/f55 [3538662,53205] 0 2024-04-06T03:40:43.017 INFO:tasks.workunit.client.1.smithi136.stdout:0/731: creat d1/d6/d2c/d1a/d82/ffa x:0 0 0 2024-04-06T03:40:43.017 INFO:tasks.workunit.client.1.smithi136.stdout:0/732: truncate d1/d6/d2c/d57/fe6 497657 0 2024-04-06T03:40:43.017 INFO:tasks.workunit.client.1.smithi136.stdout:0/733: chown d1/d6/d2c/d57/d43/d6d/d85/d78/f7a 501 1 2024-04-06T03:40:43.017 INFO:tasks.workunit.client.1.smithi136.stdout:1/710: dwrite d12/d3b/f84 [0,4194304] 0 2024-04-06T03:40:43.017 INFO:tasks.workunit.client.1.smithi136.stdout:1/711: fsync d12/d1b/d28/f91 0 2024-04-06T03:40:43.021 INFO:tasks.workunit.client.0.smithi094.stdout:3/685: dread d9/d1e/d26/d55/d62/db8/fca [0,4194304] 0 2024-04-06T03:40:43.026 INFO:tasks.workunit.client.1.smithi136.stdout:9/721: dwrite d17/d21/d32/d3f/f41 [0,4194304] 0 2024-04-06T03:40:43.026 INFO:tasks.workunit.client.1.smithi136.stdout:4/653: dwrite d1d/d2f/f80 [0,4194304] 0 2024-04-06T03:40:43.027 INFO:tasks.workunit.client.0.smithi094.stdout:6/665: dwrite dc/d3a/d4a/d54/dde/d50/f8f [0,4194304] 0 2024-04-06T03:40:43.027 INFO:tasks.workunit.client.0.smithi094.stdout:3/686: rename d9/d1d/f45 to d9/d1e/d6e/d75/d90/ffb 0 2024-04-06T03:40:43.031 INFO:tasks.workunit.client.1.smithi136.stdout:1/712: dread d12/d3b/d74/fba [0,4194304] 0 2024-04-06T03:40:43.038 INFO:tasks.workunit.client.0.smithi094.stdout:3/687: creat d9/d1e/d26/d6c/ddd/ffc x:0 0 0 2024-04-06T03:40:43.041 INFO:tasks.workunit.client.1.smithi136.stdout:1/713: dread d12/d3b/d44/d5f/d5a/f62 [0,4194304] 0 2024-04-06T03:40:43.043 INFO:tasks.workunit.client.1.smithi136.stdout:5/801: getdents d1/d9b/d9c 0 2024-04-06T03:40:43.043 INFO:tasks.workunit.client.1.smithi136.stdout:5/802: chown d1/d16/d89/d8a/cb3 368 1 2024-04-06T03:40:43.043 INFO:tasks.workunit.client.1.smithi136.stdout:5/803: readlink d1/dc/d1c/d32/d8b/d20/l35 0 2024-04-06T03:40:43.045 INFO:tasks.workunit.client.1.smithi136.stdout:6/658: getdents d2 0 2024-04-06T03:40:43.045 INFO:tasks.workunit.client.1.smithi136.stdout:6/659: dread - d2/d7/d26/d30/d6e/d58/f7d zero size 2024-04-06T03:40:43.045 INFO:tasks.workunit.client.0.smithi094.stdout:6/666: dwrite dc/dd3/d24/f2e [4194304,4194304] 0 2024-04-06T03:40:43.050 INFO:tasks.workunit.client.0.smithi094.stdout:6/667: mkdir dc/d3a/d7f/df0 0 2024-04-06T03:40:43.052 INFO:tasks.workunit.client.1.smithi136.stdout:6/660: write d2/d7/d26/d30/f7e [1997567,70769] 0 2024-04-06T03:40:43.058 INFO:tasks.workunit.client.1.smithi136.stdout:8/781: getdents d3/de/d60 0 2024-04-06T03:40:43.063 INFO:tasks.workunit.client.1.smithi136.stdout:5/804: dwrite d1/dc/d1c/d32/feb [0,4194304] 0 2024-04-06T03:40:43.065 INFO:tasks.workunit.client.0.smithi094.stdout:6/668: dwrite dc/dd3/d72/f77 [0,4194304] 0 2024-04-06T03:40:43.066 INFO:tasks.workunit.client.1.smithi136.stdout:9/722: rmdir d17/d20/d25/da4 39 2024-04-06T03:40:43.073 INFO:tasks.workunit.client.0.smithi094.stdout:6/669: mkdir dc/dd3/d24/df1 0 2024-04-06T03:40:43.074 INFO:tasks.workunit.client.1.smithi136.stdout:7/759: getdents d5/df/d11 0 2024-04-06T03:40:43.074 INFO:tasks.workunit.client.1.smithi136.stdout:7/760: write d5/df/d12/d1b/d3a/d4a/d70/ddb/fb1 [772651,23062] 0 2024-04-06T03:40:43.075 INFO:tasks.workunit.client.1.smithi136.stdout:1/714: mknod d12/d1b/d1e/dd1/cf1 0 2024-04-06T03:40:43.075 INFO:tasks.workunit.client.1.smithi136.stdout:1/715: stat d12/d1b/d28/f91 0 2024-04-06T03:40:43.076 INFO:tasks.workunit.client.1.smithi136.stdout:1/716: dread d12/d3b/d44/fa5 [0,4194304] 0 2024-04-06T03:40:43.077 INFO:tasks.workunit.client.0.smithi094.stdout:6/670: mkdir dc/dd3/d24/d46/df2 0 2024-04-06T03:40:43.078 INFO:tasks.workunit.client.1.smithi136.stdout:0/734: link d1/d6/d2c/c47 d1/d6/d2c/d1a/cfb 0 2024-04-06T03:40:43.078 INFO:tasks.workunit.client.1.smithi136.stdout:0/735: write d1/d6/d2c/d1a/d36/d59/fbe [175611,127406] 0 2024-04-06T03:40:43.080 INFO:tasks.workunit.client.1.smithi136.stdout:6/661: truncate d2/d7/d26/d30/d6e/d3d/f46 1926585 0 2024-04-06T03:40:43.083 INFO:tasks.workunit.client.1.smithi136.stdout:8/782: rename d3/de/d18/d7d/ddf/d6c/dab/fbe to d3/d2d/d5f/ff0 0 2024-04-06T03:40:43.084 INFO:tasks.workunit.client.1.smithi136.stdout:7/761: write d5/df/d12/d1b/d58/f105 [3291593,92635] 0 2024-04-06T03:40:43.084 INFO:tasks.workunit.client.1.smithi136.stdout:7/762: write d5/df/d12/d1b/d3a/d4a/d70/ddb/faf [5222851,95958] 0 2024-04-06T03:40:43.087 INFO:tasks.workunit.client.1.smithi136.stdout:5/805: symlink d1/dc/d1c/d32/d8b/de3/lf0 0 2024-04-06T03:40:43.089 INFO:tasks.workunit.client.1.smithi136.stdout:9/723: mknod d17/d21/d29/de5/da5/db8/cf8 0 2024-04-06T03:40:43.090 INFO:tasks.workunit.client.1.smithi136.stdout:9/724: creat d17/d21/d29/d34/d46/d9a/ff9 x:0 0 0 2024-04-06T03:40:43.092 INFO:tasks.workunit.client.1.smithi136.stdout:3/711: getdents d0/d4/d12/d19/d55/d3c 0 2024-04-06T03:40:43.092 INFO:tasks.workunit.client.1.smithi136.stdout:7/763: dread d5/df/d1d/d54/fa4 [0,4194304] 0 2024-04-06T03:40:43.092 INFO:tasks.workunit.client.1.smithi136.stdout:7/764: chown d5/df/d53/d8a/fbc 336 1 2024-04-06T03:40:43.095 INFO:tasks.workunit.client.1.smithi136.stdout:1/717: rename d12/d3b/lb6 to d12/d3b/d74/d7a/dea/dd2/lf2 0 2024-04-06T03:40:43.098 INFO:tasks.workunit.client.1.smithi136.stdout:5/806: mkdir d1/d16/d89/d8a/df1 0 2024-04-06T03:40:43.101 INFO:tasks.workunit.client.1.smithi136.stdout:9/725: link d17/d20/d28/d49/d75/d60/cf0 d17/d21/d29/de5/da5/db8/de0/cfa 0 2024-04-06T03:40:43.101 INFO:tasks.workunit.client.1.smithi136.stdout:9/726: write d17/d21/d32/d3f/f63 [3930132,27344] 0 2024-04-06T03:40:43.102 INFO:tasks.workunit.client.1.smithi136.stdout:6/662: dwrite d2/d7/d26/d30/d6e/f6b [0,4194304] 0 2024-04-06T03:40:43.103 INFO:tasks.workunit.client.1.smithi136.stdout:6/663: stat d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/d94/lb4 0 2024-04-06T03:40:43.103 INFO:tasks.workunit.client.1.smithi136.stdout:8/783: dwrite d3/da/d26/d35/d3d/f82 [4194304,4194304] 0 2024-04-06T03:40:43.103 INFO:tasks.workunit.client.1.smithi136.stdout:3/712: creat d0/ff4 x:0 0 0 2024-04-06T03:40:43.106 INFO:tasks.workunit.client.1.smithi136.stdout:7/765: unlink d5/df/d1d/d6f/c7e 0 2024-04-06T03:40:43.106 INFO:tasks.workunit.client.1.smithi136.stdout:7/766: read - d5/df/d12/d1b/d3a/fc3 zero size 2024-04-06T03:40:43.106 INFO:tasks.workunit.client.1.smithi136.stdout:7/767: chown d5/df/d12/d1b/f47 8201 1 2024-04-06T03:40:43.106 INFO:tasks.workunit.client.1.smithi136.stdout:7/768: creat d5/df/d1d/d54/d5c/f110 x:0 0 0 2024-04-06T03:40:43.106 INFO:tasks.workunit.client.1.smithi136.stdout:1/718: dread d12/d3b/d74/d7a/f58 [0,4194304] 0 2024-04-06T03:40:43.109 INFO:tasks.workunit.client.1.smithi136.stdout:0/736: rename d1/d6/d2c/d57/d43/d6d/d9b/d5c/d96/lea to d1/d6/d2c/d1a/d82/lfc 0 2024-04-06T03:40:43.109 INFO:tasks.workunit.client.1.smithi136.stdout:0/737: creat d1/d6/d2c/d1a/d36/ffd x:0 0 0 2024-04-06T03:40:43.112 INFO:tasks.workunit.client.1.smithi136.stdout:9/727: getdents d17/d19/da3/dc5/deb/d9e 0 2024-04-06T03:40:43.117 INFO:tasks.workunit.client.1.smithi136.stdout:8/784: dwrite d3/da/d26/d35/d39/d70/f9e [0,4194304] 0 2024-04-06T03:40:43.117 INFO:tasks.workunit.client.1.smithi136.stdout:3/713: link d0/d4/d12/d19/d2c/f7f d0/d4/d12/d3a/d51/d8e/ff5 0 2024-04-06T03:40:43.117 INFO:tasks.workunit.client.1.smithi136.stdout:5/807: dwrite d1/dc/d1c/d32/d8b/f9f [0,4194304] 0 2024-04-06T03:40:43.119 INFO:tasks.workunit.client.1.smithi136.stdout:1/719: mknod d12/d3b/d60/d6d/da2/cf3 0 2024-04-06T03:40:43.121 INFO:tasks.workunit.client.1.smithi136.stdout:6/664: rename d2/f3c to d2/d7/d26/d30/dc4/fda 0 2024-04-06T03:40:43.123 INFO:tasks.workunit.client.1.smithi136.stdout:0/738: symlink d1/d6/d2c/d57/d43/d6d/d85/da7/lfe 0 2024-04-06T03:40:43.124 INFO:tasks.workunit.client.1.smithi136.stdout:9/728: mkdir d17/d19/da3/dc5/deb/d9e/dfb 0 2024-04-06T03:40:43.124 INFO:tasks.workunit.client.1.smithi136.stdout:9/729: write d17/d21/d29/de5/da5/db8/fd0 [990203,127226] 0 2024-04-06T03:40:43.125 INFO:tasks.workunit.client.0.smithi094.stdout:7/620: sync 2024-04-06T03:40:43.125 INFO:tasks.workunit.client.0.smithi094.stdout:7/621: truncate d2/d6/dc/d12/d29/d35/f59 113261 0 2024-04-06T03:40:43.126 INFO:tasks.workunit.client.1.smithi136.stdout:3/714: symlink d0/d4/d12/d19/d55/d3c/d5a/d79/d98/dc6/lf6 0 2024-04-06T03:40:43.127 INFO:tasks.workunit.client.1.smithi136.stdout:8/785: write f1 [759320,101979] 0 2024-04-06T03:40:43.127 INFO:tasks.workunit.client.0.smithi094.stdout:7/622: symlink d2/d6/dc/dd/df/d36/lbb 0 2024-04-06T03:40:43.127 INFO:tasks.workunit.client.0.smithi094.stdout:7/623: stat d2/d6/dc/d12/d3a/d56/db1 0 2024-04-06T03:40:43.128 INFO:tasks.workunit.client.1.smithi136.stdout:8/786: dread d3/da/f13 [0,4194304] 0 2024-04-06T03:40:43.128 INFO:tasks.workunit.client.1.smithi136.stdout:8/787: fsync d3/da/d26/d35/d53/fac 0 2024-04-06T03:40:43.131 INFO:tasks.workunit.client.1.smithi136.stdout:5/808: creat d1/d9b/dea/ff2 x:0 0 0 2024-04-06T03:40:43.131 INFO:tasks.workunit.client.1.smithi136.stdout:5/809: readlink d1/dc/d1c/d32/d8b/d20/l92 0 2024-04-06T03:40:43.131 INFO:tasks.workunit.client.1.smithi136.stdout:1/720: dwrite d12/d3b/d44/d5f/f7f [0,4194304] 0 2024-04-06T03:40:43.131 INFO:tasks.workunit.client.1.smithi136.stdout:1/721: fsync d12/d3b/d44/d5f/f7f 0 2024-04-06T03:40:43.132 INFO:tasks.workunit.client.0.smithi094.stdout:7/624: rename d2/d6/dc/dd/df/d36/f4d to d2/d6/dc/d12/d29/d35/fbc 0 2024-04-06T03:40:43.134 INFO:tasks.workunit.client.1.smithi136.stdout:7/769: rename d5/df/d12/ld9 to d5/df/d1d/d54/df2/l111 0 2024-04-06T03:40:43.135 INFO:tasks.workunit.client.1.smithi136.stdout:6/665: mknod d2/d7/d13/cdb 0 2024-04-06T03:40:43.136 INFO:tasks.workunit.client.0.smithi094.stdout:7/625: mkdir d2/d6/dc/d12/d9a/dbd 0 2024-04-06T03:40:43.136 INFO:tasks.workunit.client.0.smithi094.stdout:7/626: chown d2/d6/dc/dd/f3e 3628 1 2024-04-06T03:40:43.138 INFO:tasks.workunit.client.1.smithi136.stdout:0/739: dwrite d1/d6/d2c/d57/d43/d6d/d9b/d5c/f91 [0,4194304] 0 2024-04-06T03:40:43.139 INFO:tasks.workunit.client.1.smithi136.stdout:9/730: creat d17/d20/d25/da4/ffc x:0 0 0 2024-04-06T03:40:43.139 INFO:tasks.workunit.client.1.smithi136.stdout:9/731: getdents d17/d19/da3/dc5/deb/d9e/dfb 0 2024-04-06T03:40:43.139 INFO:tasks.workunit.client.1.smithi136.stdout:9/732: creat d17/d21/d29/d34/d46/d9a/ffd x:0 0 0 2024-04-06T03:40:43.140 INFO:tasks.workunit.client.1.smithi136.stdout:9/733: fsync d17/d20/d25/d71/fa9 0 2024-04-06T03:40:43.140 INFO:tasks.workunit.client.1.smithi136.stdout:9/734: stat d17/d21/d29/d34/cad 0 2024-04-06T03:40:43.140 INFO:tasks.workunit.client.1.smithi136.stdout:3/715: rename d0/l4f to d0/lf7 0 2024-04-06T03:40:43.141 INFO:tasks.workunit.client.0.smithi094.stdout:7/627: mknod d2/d6/dc/dd/df/d36/cbe 0 2024-04-06T03:40:43.143 INFO:tasks.workunit.client.1.smithi136.stdout:8/788: creat d3/da/d26/d90/dae/ff1 x:0 0 0 2024-04-06T03:40:43.143 INFO:tasks.workunit.client.1.smithi136.stdout:8/789: truncate d3/de/d18/d7d/ddf/d6c/dab/fd4 4443960 0 2024-04-06T03:40:43.143 INFO:tasks.workunit.client.1.smithi136.stdout:8/790: dread - d3/da/d26/d35/d39/f65 zero size 2024-04-06T03:40:43.143 INFO:tasks.workunit.client.1.smithi136.stdout:8/791: dread - d3/fc3 zero size 2024-04-06T03:40:43.146 INFO:tasks.workunit.client.1.smithi136.stdout:5/810: dwrite d1/dc/d10/d1e/f97 [0,4194304] 0 2024-04-06T03:40:43.147 INFO:tasks.workunit.client.1.smithi136.stdout:5/811: write d1/dc/d1c/d32/d8b/db0/fca [175592,101518] 0 2024-04-06T03:40:43.147 INFO:tasks.workunit.client.1.smithi136.stdout:1/722: creat d12/d3b/d74/d7a/dea/ff4 x:0 0 0 2024-04-06T03:40:43.148 INFO:tasks.workunit.client.1.smithi136.stdout:1/723: dread - d12/d3b/d74/d7a/dea/d77/fc1 zero size 2024-04-06T03:40:43.148 INFO:tasks.workunit.client.0.smithi094.stdout:7/628: getdents d2/d6/dc/d12/d9a/da0 0 2024-04-06T03:40:43.150 INFO:tasks.workunit.client.0.smithi094.stdout:7/629: mkdir d2/d6/dc/d12/d9a/dbf 0 2024-04-06T03:40:43.151 INFO:tasks.workunit.client.1.smithi136.stdout:7/770: mknod d5/df/d12/d1b/d3a/d4a/d70/ddb/c112 0 2024-04-06T03:40:43.151 INFO:tasks.workunit.client.1.smithi136.stdout:7/771: fdatasync d5/df/d11/f1c 0 2024-04-06T03:40:43.153 INFO:tasks.workunit.client.1.smithi136.stdout:6/666: rmdir d2/d7/d13/d79 39 2024-04-06T03:40:43.153 INFO:tasks.workunit.client.0.smithi094.stdout:7/630: creat d2/d6/dc/d12/d29/daa/fc0 x:0 0 0 2024-04-06T03:40:43.153 INFO:tasks.workunit.client.0.smithi094.stdout:7/631: fdatasync d2/d6/dc/f15 0 2024-04-06T03:40:43.154 INFO:tasks.workunit.client.1.smithi136.stdout:7/772: dread d5/df/d1d/fc6 [0,4194304] 0 2024-04-06T03:40:43.155 INFO:tasks.workunit.client.1.smithi136.stdout:8/792: dread f1 [8388608,4194304] 0 2024-04-06T03:40:43.159 INFO:tasks.workunit.client.1.smithi136.stdout:0/740: unlink d1/d6/d2c/d57/d43/d6d/d85/d78/da4/fb6 0 2024-04-06T03:40:43.160 INFO:tasks.workunit.client.0.smithi094.stdout:7/632: symlink d2/d6/d93/lc1 0 2024-04-06T03:40:43.160 INFO:tasks.workunit.client.0.smithi094.stdout:7/633: readlink d2/d6/dc/dd/df/d36/lbb 0 2024-04-06T03:40:43.162 INFO:tasks.workunit.client.1.smithi136.stdout:9/735: rmdir d17/d21/d29/d34/d66/de1/d51/d57/def 0 2024-04-06T03:40:43.162 INFO:tasks.workunit.client.1.smithi136.stdout:9/736: chown d17/d19/da3/dc5/deb/d9e/dfb 1 1 2024-04-06T03:40:43.163 INFO:tasks.workunit.client.0.smithi094.stdout:7/634: link d2/d6/dc/dd/df/d36/lbb d2/d6/dc/d12/d9a/lc2 0 2024-04-06T03:40:43.164 INFO:tasks.workunit.client.1.smithi136.stdout:3/716: symlink d0/d4/d12/d19/d55/d3c/d53/de6/lf8 0 2024-04-06T03:40:43.165 INFO:tasks.workunit.client.1.smithi136.stdout:3/717: write d0/d4/f18 [2962269,12557] 0 2024-04-06T03:40:43.165 INFO:tasks.workunit.client.1.smithi136.stdout:3/718: dread - d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/fe0 zero size 2024-04-06T03:40:43.166 INFO:tasks.workunit.client.1.smithi136.stdout:0/741: dread d1/d6/d2c/f27 [0,4194304] 0 2024-04-06T03:40:43.166 INFO:tasks.workunit.client.0.smithi094.stdout:7/635: rename d2/d6/dc/f15 to d2/d6/dc/d12/d9a/da0/fc3 0 2024-04-06T03:40:43.166 INFO:tasks.workunit.client.0.smithi094.stdout:7/636: fdatasync d2/d6/dc/dd/f31 0 2024-04-06T03:40:43.172 INFO:tasks.workunit.client.1.smithi136.stdout:1/724: rename d12/d3b/d60/d7e/cc9 to d12/d1b/da7/daa/cf5 0 2024-04-06T03:40:43.174 INFO:tasks.workunit.client.1.smithi136.stdout:7/773: symlink d5/df/d12/d1b/l113 0 2024-04-06T03:40:43.174 INFO:tasks.workunit.client.1.smithi136.stdout:7/774: chown d5/df/d3d/f63 23116337 1 2024-04-06T03:40:43.176 INFO:tasks.workunit.client.0.smithi094.stdout:7/637: dread d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:43.177 INFO:tasks.workunit.client.1.smithi136.stdout:8/793: getdents d3/da/d26/d35/d39/d70/d7e 0 2024-04-06T03:40:43.187 INFO:tasks.workunit.client.0.smithi094.stdout:7/638: rename d2/d6/dc/d12/d29/daa/fc0 to d2/d6/dc/d12/d9a/dbd/fc4 0 2024-04-06T03:40:43.188 INFO:tasks.workunit.client.1.smithi136.stdout:9/737: symlink d17/d21/d29/d34/d66/de1/d51/d57/de3/lfe 0 2024-04-06T03:40:43.188 INFO:tasks.workunit.client.1.smithi136.stdout:7/775: dwrite d5/df/d1d/d6f/d2a/f5d [4194304,4194304] 0 2024-04-06T03:40:43.192 INFO:tasks.workunit.client.1.smithi136.stdout:3/719: mkdir d0/d4/d12/d19/d55/d6f/ddb/df9 0 2024-04-06T03:40:43.192 INFO:tasks.workunit.client.1.smithi136.stdout:3/720: stat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/l6e 0 2024-04-06T03:40:43.193 INFO:tasks.workunit.client.1.smithi136.stdout:5/812: link d1/dc/c3c d1/dc/d1c/d32/d70/cf3 0 2024-04-06T03:40:43.194 INFO:tasks.workunit.client.0.smithi094.stdout:7/639: dread d2/d6/fa [0,4194304] 0 2024-04-06T03:40:43.194 INFO:tasks.workunit.client.1.smithi136.stdout:4/654: dwrite d1d/d31/d3e/f5f [0,4194304] 0 2024-04-06T03:40:43.194 INFO:tasks.workunit.client.1.smithi136.stdout:4/655: fsync d1d/d31/d78/fb0 0 2024-04-06T03:40:43.195 INFO:tasks.workunit.client.1.smithi136.stdout:1/725: creat d12/d1b/da7/ff6 x:0 0 0 2024-04-06T03:40:43.197 INFO:tasks.workunit.client.1.smithi136.stdout:8/794: rename d3/da/d26/d88 to d3/da/d26/d35/d3d/dc7/dce/df2 0 2024-04-06T03:40:43.197 INFO:tasks.workunit.client.1.smithi136.stdout:2/777: sync 2024-04-06T03:40:43.197 INFO:tasks.workunit.client.1.smithi136.stdout:9/738: creat d17/d20/d25/d71/dd5/fff x:0 0 0 2024-04-06T03:40:43.199 INFO:tasks.workunit.client.1.smithi136.stdout:7/776: truncate d5/df/d11/f1c 45810 0 2024-04-06T03:40:43.201 INFO:tasks.workunit.client.1.smithi136.stdout:3/721: mknod d0/d4/d12/d19/d55/d6f/d82/d5d/d61/cfa 0 2024-04-06T03:40:43.202 INFO:tasks.workunit.client.1.smithi136.stdout:5/813: symlink d1/dc/d1c/d32/d8b/d20/lf4 0 2024-04-06T03:40:43.203 INFO:tasks.workunit.client.1.smithi136.stdout:1/726: mknod d12/d3b/d60/d7e/d37/cf7 0 2024-04-06T03:40:43.203 INFO:tasks.workunit.client.1.smithi136.stdout:1/727: chown d12/d1b/d28/f5e 146642 1 2024-04-06T03:40:43.204 INFO:tasks.workunit.client.1.smithi136.stdout:8/795: rename d3/da/d26/d35/d3d/f82 to d3/d2d/dad/ff3 0 2024-04-06T03:40:43.206 INFO:tasks.workunit.client.1.smithi136.stdout:2/778: mknod d3/d4a/d9d/d18/d1d/d3c/d58/c100 0 2024-04-06T03:40:43.207 INFO:tasks.workunit.client.1.smithi136.stdout:9/739: creat d17/d20/d25/f100 x:0 0 0 2024-04-06T03:40:43.208 INFO:tasks.workunit.client.1.smithi136.stdout:7/777: symlink d5/df/d12/l114 0 2024-04-06T03:40:43.209 INFO:tasks.workunit.client.1.smithi136.stdout:3/722: dread d0/d4/d12/d3a/d51/da6/fb6 [0,4194304] 0 2024-04-06T03:40:43.209 INFO:tasks.workunit.client.1.smithi136.stdout:1/728: symlink d12/d1b/d1e/d23/lf8 0 2024-04-06T03:40:43.210 INFO:tasks.workunit.client.1.smithi136.stdout:2/779: getdents d3/d4a/d9d/d18/d4b/dc0/dd1 0 2024-04-06T03:40:43.211 INFO:tasks.workunit.client.1.smithi136.stdout:9/740: mkdir d17/d19/da3/d101 0 2024-04-06T03:40:43.212 INFO:tasks.workunit.client.1.smithi136.stdout:7/778: creat d5/df/d12/dd6/f115 x:0 0 0 2024-04-06T03:40:43.214 INFO:tasks.workunit.client.1.smithi136.stdout:3/723: rename d0/d4/f18 to d0/d4/d12/d19/d2e/d3b/ffb 0 2024-04-06T03:40:43.215 INFO:tasks.workunit.client.1.smithi136.stdout:1/729: creat d12/d1b/da7/daa/ff9 x:0 0 0 2024-04-06T03:40:43.215 INFO:tasks.workunit.client.1.smithi136.stdout:1/730: chown d12/d53/l64 359 1 2024-04-06T03:40:43.215 INFO:tasks.workunit.client.1.smithi136.stdout:1/731: chown d12/d1b/d22/f43 1 1 2024-04-06T03:40:43.220 INFO:tasks.workunit.client.1.smithi136.stdout:9/741: mkdir d17/d21/d29/de5/da5/dce/d102 0 2024-04-06T03:40:43.220 INFO:tasks.workunit.client.1.smithi136.stdout:9/742: chown d17/d19/da3/dc5/deb/f4e 216432 1 2024-04-06T03:40:43.221 INFO:tasks.workunit.client.1.smithi136.stdout:7/779: mkdir d5/df/d1d/d54/d5c/dc7/d116 0 2024-04-06T03:40:43.221 INFO:tasks.workunit.client.1.smithi136.stdout:7/780: stat d5/df/d1d/d6f/d2a 0 2024-04-06T03:40:43.229 INFO:tasks.workunit.client.1.smithi136.stdout:2/780: dread d3/d4a/d9d/d8a/fb2 [0,4194304] 0 2024-04-06T03:40:43.229 INFO:tasks.workunit.client.1.smithi136.stdout:2/781: dread - d3/d9/d3b/d87/ff6 zero size 2024-04-06T03:40:43.229 INFO:tasks.workunit.client.1.smithi136.stdout:1/732: write d12/d1b/d1e/d9f/fa8 [3579047,38163] 0 2024-04-06T03:40:43.229 INFO:tasks.workunit.client.1.smithi136.stdout:1/733: dread - d12/d3b/d44/d5f/db7/fdd zero size 2024-04-06T03:40:43.230 INFO:tasks.workunit.client.1.smithi136.stdout:1/734: getdents d12/d3b/d60/d7e/d37/dbb 0 2024-04-06T03:40:43.230 INFO:tasks.workunit.client.1.smithi136.stdout:1/735: creat d12/d3b/d44/ffa x:0 0 0 2024-04-06T03:40:43.230 INFO:tasks.workunit.client.1.smithi136.stdout:1/736: write d12/d3b/d60/d7e/f42 [1424826,91785] 0 2024-04-06T03:40:43.230 INFO:tasks.workunit.client.1.smithi136.stdout:1/737: stat d12/d1b/da7 0 2024-04-06T03:40:43.230 INFO:tasks.workunit.client.1.smithi136.stdout:1/738: chown d12/d3b/d44/d5f/l65 917131 1 2024-04-06T03:40:43.230 INFO:tasks.workunit.client.1.smithi136.stdout:1/739: creat d12/d3b/d60/d7e/d37/ffb x:0 0 0 2024-04-06T03:40:43.233 INFO:tasks.workunit.client.1.smithi136.stdout:2/782: mknod d3/d4a/d9d/d18/d1d/d3c/df7/c101 0 2024-04-06T03:40:43.236 INFO:tasks.workunit.client.1.smithi136.stdout:1/740: truncate d12/d3b/d60/d6d/da2/fc2 201013 0 2024-04-06T03:40:43.237 INFO:tasks.workunit.client.1.smithi136.stdout:1/741: chown d12/d3b/d60/d7e/d37/f4e 926326 1 2024-04-06T03:40:43.238 INFO:tasks.workunit.client.1.smithi136.stdout:7/781: dwrite d5/df/d12/d1b/f10c [0,4194304] 0 2024-04-06T03:40:43.239 INFO:tasks.workunit.client.1.smithi136.stdout:9/743: dwrite d17/d20/d28/d81/f83 [0,4194304] 0 2024-04-06T03:40:43.239 INFO:tasks.workunit.client.1.smithi136.stdout:9/744: fdatasync d17/d20/d25/d71/fa9 0 2024-04-06T03:40:43.239 INFO:tasks.workunit.client.1.smithi136.stdout:9/745: dread - d17/d20/d28/f9c zero size 2024-04-06T03:40:43.240 INFO:tasks.workunit.client.1.smithi136.stdout:2/783: mknod d3/d9/dde/c102 0 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:5/646: sync 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:2/692: sync 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:4/751: sync 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:1/698: sync 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:9/662: sync 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:9/663: truncate d2/dc/d10/d2a/fa2 812902 0 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:8/760: sync 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:8/761: stat d18/d2c/d37/d60/d70/c55 0 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:0/684: sync 2024-04-06T03:40:43.241 INFO:tasks.workunit.client.0.smithi094.stdout:8/762: creat d18/d3c/dc4/db9/f106 x:0 0 0 2024-04-06T03:40:43.243 INFO:tasks.workunit.client.1.smithi136.stdout:1/742: write d12/d3b/d74/d7a/f8d [1697488,75984] 0 2024-04-06T03:40:43.244 INFO:tasks.workunit.client.0.smithi094.stdout:9/664: dread d2/dc/d46/f7e [0,4194304] 0 2024-04-06T03:40:43.247 INFO:tasks.workunit.client.1.smithi136.stdout:7/782: link d5/df/d11/f26 d5/df/d53/d68/da3/dd0/d106/f117 0 2024-04-06T03:40:43.247 INFO:tasks.workunit.client.1.smithi136.stdout:7/783: fdatasync d5/df/d11/f89 0 2024-04-06T03:40:43.248 INFO:tasks.workunit.client.1.smithi136.stdout:7/784: chown d5/df/d1d/d6f/d4d/d69/fee 607 1 2024-04-06T03:40:43.251 INFO:tasks.workunit.client.0.smithi094.stdout:4/752: dread d0/d15/d3a/d3e/d46/fa3 [0,4194304] 0 2024-04-06T03:40:43.251 INFO:tasks.workunit.client.0.smithi094.stdout:1/699: creat d0/d1/d27/da5/d40/db1/d2f/d4b/fef x:0 0 0 2024-04-06T03:40:43.251 INFO:tasks.workunit.client.0.smithi094.stdout:4/753: creat d0/d4/d67/dcb/ffb x:0 0 0 2024-04-06T03:40:43.252 INFO:tasks.workunit.client.1.smithi136.stdout:9/746: mknod d17/d19/da3/d101/c103 0 2024-04-06T03:40:43.255 INFO:tasks.workunit.client.1.smithi136.stdout:2/784: write d3/d4a/d9d/d8a/fd9 [3154513,48448] 0 2024-04-06T03:40:43.255 INFO:tasks.workunit.client.1.smithi136.stdout:2/785: creat d3/d4a/f103 x:0 0 0 2024-04-06T03:40:43.255 INFO:tasks.workunit.client.0.smithi094.stdout:9/665: dread d2/dc/f68 [0,4194304] 0 2024-04-06T03:40:43.257 INFO:tasks.workunit.client.0.smithi094.stdout:8/763: unlink d18/f1b 0 2024-04-06T03:40:43.257 INFO:tasks.workunit.client.0.smithi094.stdout:8/764: creat d18/d3c/d49/d91/d94/daa/f107 x:0 0 0 2024-04-06T03:40:43.258 INFO:tasks.workunit.client.1.smithi136.stdout:1/743: creat d12/d3b/d60/d7e/ffc x:0 0 0 2024-04-06T03:40:43.258 INFO:tasks.workunit.client.1.smithi136.stdout:1/744: write d12/d3b/d44/d5f/db7/fed [64559,71691] 0 2024-04-06T03:40:43.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/693: rmdir d0/d8/d4f/dab/dc3 39 2024-04-06T03:40:43.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/694: chown d0/dd/d3d/d49/f53 22 1 2024-04-06T03:40:43.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/695: chown d0/dd/d1b/lc1 7 1 2024-04-06T03:40:43.259 INFO:tasks.workunit.client.0.smithi094.stdout:2/696: readlink d0/dd/d19/d31/dcc/le6 0 2024-04-06T03:40:43.262 INFO:tasks.workunit.client.0.smithi094.stdout:1/700: mkdir d0/d1/d27/da5/d40/db1/dc3/df0 0 2024-04-06T03:40:43.263 INFO:tasks.workunit.client.0.smithi094.stdout:5/647: dwrite db/d19/d24/f34 [0,4194304] 0 2024-04-06T03:40:43.263 INFO:tasks.workunit.client.0.smithi094.stdout:5/648: write db/d19/d30/f81 [548450,33069] 0 2024-04-06T03:40:43.263 INFO:tasks.workunit.client.0.smithi094.stdout:0/685: dwrite d1/d9/fc [4194304,4194304] 0 2024-04-06T03:40:43.264 INFO:tasks.workunit.client.1.smithi136.stdout:2/786: dread d3/d9/fad [0,4194304] 0 2024-04-06T03:40:43.264 INFO:tasks.workunit.client.1.smithi136.stdout:2/787: write d3/d4a/ff2 [871452,58463] 0 2024-04-06T03:40:43.264 INFO:tasks.workunit.client.1.smithi136.stdout:2/788: creat d3/d4a/d9d/f104 x:0 0 0 2024-04-06T03:40:43.264 INFO:tasks.workunit.client.1.smithi136.stdout:7/785: unlink d5/df/d12/d1b/d3a/d4a/l60 0 2024-04-06T03:40:43.269 INFO:tasks.workunit.client.0.smithi094.stdout:4/754: truncate d0/d15/d3a/d3e/d46/fa3 3887949 0 2024-04-06T03:40:43.272 INFO:tasks.workunit.client.0.smithi094.stdout:9/666: rmdir d2/dc/d21/d26/d78 39 2024-04-06T03:40:43.275 INFO:tasks.workunit.client.0.smithi094.stdout:0/686: dwrite d1/d9/dd/d18/d1e/d22/fd5 [0,4194304] 0 2024-04-06T03:40:43.276 INFO:tasks.workunit.client.1.smithi136.stdout:9/747: mknod d17/d21/d29/de5/c104 0 2024-04-06T03:40:43.276 INFO:tasks.workunit.client.1.smithi136.stdout:7/786: dwrite d5/f80 [0,4194304] 0 2024-04-06T03:40:43.277 INFO:tasks.workunit.client.0.smithi094.stdout:8/765: symlink d18/d3c/d49/db5/l108 0 2024-04-06T03:40:43.278 INFO:tasks.workunit.client.1.smithi136.stdout:7/787: dread d5/df/d12/d1b/d58/f7a [0,4194304] 0 2024-04-06T03:40:43.278 INFO:tasks.workunit.client.0.smithi094.stdout:0/687: dread d1/d9/dd/d18/dad/f93 [0,4194304] 0 2024-04-06T03:40:43.278 INFO:tasks.workunit.client.0.smithi094.stdout:0/688: creat d1/d9/dd/d18/d1e/d22/d35/dac/fe7 x:0 0 0 2024-04-06T03:40:43.279 INFO:tasks.workunit.client.0.smithi094.stdout:2/697: creat d0/dd/d3d/d49/feb x:0 0 0 2024-04-06T03:40:43.283 INFO:tasks.workunit.client.0.smithi094.stdout:1/701: mknod d0/d1/d27/cf1 0 2024-04-06T03:40:43.283 INFO:tasks.workunit.client.0.smithi094.stdout:1/702: fsync d0/d1/d27/da5/f9f 0 2024-04-06T03:40:43.289 INFO:tasks.workunit.client.1.smithi136.stdout:7/788: dread d5/df/d11/f79 [0,4194304] 0 2024-04-06T03:40:43.291 INFO:tasks.workunit.client.0.smithi094.stdout:5/649: rename db/d19/dc1/dd4/d77/fa9 to db/d19/d24/fdb 0 2024-04-06T03:40:43.291 INFO:tasks.workunit.client.0.smithi094.stdout:5/650: truncate db/d19/d21/f52 898298 0 2024-04-06T03:40:43.291 INFO:tasks.workunit.client.0.smithi094.stdout:5/651: truncate db/d19/d24/f53 5055557 0 2024-04-06T03:40:43.291 INFO:tasks.workunit.client.0.smithi094.stdout:5/652: write db/d19/d30/d83/fd3 [852613,21892] 0 2024-04-06T03:40:43.293 INFO:tasks.workunit.client.1.smithi136.stdout:9/748: dwrite d17/d20/d28/f89 [4194304,4194304] 0 2024-04-06T03:40:43.293 INFO:tasks.workunit.client.1.smithi136.stdout:9/749: truncate d17/d20/d28/d49/d75/f91 587781 0 2024-04-06T03:40:43.293 INFO:tasks.workunit.client.0.smithi094.stdout:2/698: dwrite d0/d8/d27/fb4 [0,4194304] 0 2024-04-06T03:40:43.294 INFO:tasks.workunit.client.0.smithi094.stdout:2/699: read - d0/dd/d1b/d72/fca zero size 2024-04-06T03:40:43.294 INFO:tasks.workunit.client.1.smithi136.stdout:2/789: mkdir d3/d4a/d9d/d105 0 2024-04-06T03:40:43.294 INFO:tasks.workunit.client.1.smithi136.stdout:2/790: readlink d3/d4a/d9d/d18/d1d/l66 0 2024-04-06T03:40:43.296 INFO:tasks.workunit.client.0.smithi094.stdout:4/755: creat d0/d23/ffc x:0 0 0 2024-04-06T03:40:43.296 INFO:tasks.workunit.client.0.smithi094.stdout:4/756: chown d0/d15/d3a/d3e/l90 21 1 2024-04-06T03:40:43.301 INFO:tasks.workunit.client.0.smithi094.stdout:1/703: dwrite d0/d1/d27/da5/d40/db1/d2f/d3c/d48/fe5 [0,4194304] 0 2024-04-06T03:40:43.301 INFO:tasks.workunit.client.0.smithi094.stdout:1/704: readlink d0/d1/d27/da5/d40/db1/lc6 0 2024-04-06T03:40:43.308 INFO:tasks.workunit.client.0.smithi094.stdout:5/653: dwrite db/fa8 [0,4194304] 0 2024-04-06T03:40:43.309 INFO:tasks.workunit.client.1.smithi136.stdout:7/789: dwrite d5/df/d12/d1b/d3a/d4a/f64 [0,4194304] 0 2024-04-06T03:40:43.309 INFO:tasks.workunit.client.1.smithi136.stdout:2/791: symlink d3/d9/d3b/l106 0 2024-04-06T03:40:43.309 INFO:tasks.workunit.client.1.smithi136.stdout:7/790: write d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/fb6 [437,61520] 0 2024-04-06T03:40:43.316 INFO:tasks.workunit.client.1.smithi136.stdout:2/792: dread d3/d15/f50 [0,4194304] 0 2024-04-06T03:40:43.318 INFO:tasks.workunit.client.0.smithi094.stdout:0/689: truncate d1/d9/dd/d18/d1e/d22/d35/f52 1620743 0 2024-04-06T03:40:43.319 INFO:tasks.workunit.client.1.smithi136.stdout:7/791: rmdir d5/df/d1d/d6f 39 2024-04-06T03:40:43.320 INFO:tasks.workunit.client.0.smithi094.stdout:9/667: rename d2/dc/l1f to d2/dc/d21/dc8/lf5 0 2024-04-06T03:40:43.320 INFO:tasks.workunit.client.0.smithi094.stdout:9/668: write d2/d2c/d99/faf [483397,91902] 0 2024-04-06T03:40:43.324 INFO:tasks.workunit.client.1.smithi136.stdout:7/792: rename d5/df/d1d/d6f/d2a/c75 to d5/df/d1d/d6f/d4d/d69/c118 0 2024-04-06T03:40:43.324 INFO:tasks.workunit.client.1.smithi136.stdout:7/793: write d5/df/d12/d1b/d3a/fc3 [945803,104014] 0 2024-04-06T03:40:43.324 INFO:tasks.workunit.client.1.smithi136.stdout:7/794: fdatasync d5/df/d1d/d6f/d2a/fda 0 2024-04-06T03:40:43.325 INFO:tasks.workunit.client.0.smithi094.stdout:2/700: stat d0/dd/d1b/c51 0 2024-04-06T03:40:43.326 INFO:tasks.workunit.client.1.smithi136.stdout:6/667: sync 2024-04-06T03:40:43.327 INFO:tasks.workunit.client.0.smithi094.stdout:4/757: mknod d0/d15/d3a/d61/cfd 0 2024-04-06T03:40:43.329 INFO:tasks.workunit.client.1.smithi136.stdout:7/795: getdents d5/df/d12/d1b/d58 0 2024-04-06T03:40:43.330 INFO:tasks.workunit.client.1.smithi136.stdout:2/793: write d3/d15/d19/dac/fb9 [3771178,84571] 0 2024-04-06T03:40:43.330 INFO:tasks.workunit.client.1.smithi136.stdout:2/794: write d3/d4a/d9d/d8a/fd9 [3276605,61994] 0 2024-04-06T03:40:43.330 INFO:tasks.workunit.client.1.smithi136.stdout:2/795: chown d3/d4a/d9d/d18/d1d/d3c/d58/lca 32280353 1 2024-04-06T03:40:43.334 INFO:tasks.workunit.client.0.smithi094.stdout:4/758: dread d0/df/d1e/d34/db5/fe6 [0,4194304] 0 2024-04-06T03:40:43.334 INFO:tasks.workunit.client.0.smithi094.stdout:4/759: chown d0/d15/d3a/d61/d81/fc0 10 1 2024-04-06T03:40:43.335 INFO:tasks.workunit.client.0.smithi094.stdout:4/760: creat d0/df/d1e/d34/db5/ffe x:0 0 0 2024-04-06T03:40:43.335 INFO:tasks.workunit.client.0.smithi094.stdout:4/761: stat d0/d15/c8d 0 2024-04-06T03:40:43.335 INFO:tasks.workunit.client.0.smithi094.stdout:4/762: read - d0/d15/d3a/d61/d81/da0/fdd zero size 2024-04-06T03:40:43.337 INFO:tasks.workunit.client.0.smithi094.stdout:5/654: mknod db/d19/dc1/dd4/dcf/cdc 0 2024-04-06T03:40:43.338 INFO:tasks.workunit.client.1.smithi136.stdout:7/796: write d5/df/d12/d1b/d3a/d4a/d70/ddb/faf [737562,27709] 0 2024-04-06T03:40:43.340 INFO:tasks.workunit.client.0.smithi094.stdout:8/766: mkdir d18/d3c/d49/d9f/d109 0 2024-04-06T03:40:43.343 INFO:tasks.workunit.client.1.smithi136.stdout:2/796: mknod d3/d4a/d9d/c107 0 2024-04-06T03:40:43.344 INFO:tasks.workunit.client.0.smithi094.stdout:0/690: link d1/d9/dd/d18/d1e/d22/d35/f5c d1/d9/dd/d18/d1e/d22/fe8 0 2024-04-06T03:40:43.344 INFO:tasks.workunit.client.0.smithi094.stdout:0/691: read - d1/d9/dd/d18/d5d/fe0 zero size 2024-04-06T03:40:43.347 INFO:tasks.workunit.client.0.smithi094.stdout:5/655: write db/d19/d30/d83/f87 [2390662,28405] 0 2024-04-06T03:40:43.347 INFO:tasks.workunit.client.0.smithi094.stdout:5/656: write db/d19/d30/f44 [2881748,19135] 0 2024-04-06T03:40:43.347 INFO:tasks.workunit.client.0.smithi094.stdout:5/657: chown db/d19/d24/f72 2016092627 1 2024-04-06T03:40:43.347 INFO:tasks.workunit.client.0.smithi094.stdout:5/658: creat db/d19/d30/d83/fdd x:0 0 0 2024-04-06T03:40:43.348 INFO:tasks.workunit.client.1.smithi136.stdout:7/797: rename d5/fcf to d5/d6d/f119 0 2024-04-06T03:40:43.349 INFO:tasks.workunit.client.0.smithi094.stdout:9/669: mknod d2/d2c/d33/d64/cf6 0 2024-04-06T03:40:43.351 INFO:tasks.workunit.client.0.smithi094.stdout:4/763: dwrite d0/d4/f9 [0,4194304] 0 2024-04-06T03:40:43.352 INFO:tasks.workunit.client.0.smithi094.stdout:2/701: link d0/dd/f95 d0/dd/d19/d31/d67/dae/fec 0 2024-04-06T03:40:43.354 INFO:tasks.workunit.client.1.smithi136.stdout:7/798: rename d5/df/d1d/d54/d108 to d5/df/d53/d68/da3/dd0/d106/d11a 0 2024-04-06T03:40:43.356 INFO:tasks.workunit.client.0.smithi094.stdout:8/767: dwrite d18/d2c/d37/fa2 [0,4194304] 0 2024-04-06T03:40:43.357 INFO:tasks.workunit.client.1.smithi136.stdout:2/797: dread d3/d15/d19/d4e/fba [0,4194304] 0 2024-04-06T03:40:43.357 INFO:tasks.workunit.client.0.smithi094.stdout:1/705: link d0/d1/d27/da5/f9f d0/d7/d1a/ff2 0 2024-04-06T03:40:43.357 INFO:tasks.workunit.client.0.smithi094.stdout:1/706: write d0/d1/d27/da5/d40/d50/fbf [934041,62581] 0 2024-04-06T03:40:43.359 INFO:tasks.workunit.client.1.smithi136.stdout:0/742: sync 2024-04-06T03:40:43.360 INFO:tasks.workunit.client.1.smithi136.stdout:7/799: truncate d5/df/d12/d1b/f47 295640 0 2024-04-06T03:40:43.363 INFO:tasks.workunit.client.1.smithi136.stdout:2/798: unlink d3/d4a/d9d/d18/d4b/dc0/ff9 0 2024-04-06T03:40:43.363 INFO:tasks.workunit.client.1.smithi136.stdout:2/799: write d3/d4a/d9d/d18/d4b/dbd/fc3 [262855,58363] 0 2024-04-06T03:40:43.363 INFO:tasks.workunit.client.1.smithi136.stdout:2/800: write d3/d4a/d9d/fb6 [1132312,38892] 0 2024-04-06T03:40:43.363 INFO:tasks.workunit.client.1.smithi136.stdout:2/801: chown d3/d15/d19/c38 1827397 1 2024-04-06T03:40:43.369 INFO:tasks.workunit.client.0.smithi094.stdout:1/707: write d0/d1/d27/fa9 [979577,41282] 0 2024-04-06T03:40:43.370 INFO:tasks.workunit.client.0.smithi094.stdout:1/708: chown d0/d1/d27/cf1 15759096 1 2024-04-06T03:40:43.371 INFO:tasks.workunit.client.0.smithi094.stdout:5/659: rename db/d19/d24/c7d to db/d19/dc1/dd4/dcf/cde 0 2024-04-06T03:40:43.371 INFO:tasks.workunit.client.0.smithi094.stdout:5/660: read db/d19/d30/d83/f87 [960550,119776] 0 2024-04-06T03:40:43.374 INFO:tasks.workunit.client.1.smithi136.stdout:7/800: rmdir d5/df/d1d/d54/d5c/dc7/d116 0 2024-04-06T03:40:43.376 INFO:tasks.workunit.client.0.smithi094.stdout:0/692: dwrite d1/d9/dd/d18/d1e/d22/fd5 [4194304,4194304] 0 2024-04-06T03:40:43.376 INFO:tasks.workunit.client.0.smithi094.stdout:0/693: chown d1/d9/dd/d18/d1e/d22/d35/f3c 21001 1 2024-04-06T03:40:43.378 INFO:tasks.workunit.client.1.smithi136.stdout:0/743: unlink d1/d6/d2c/d57/d43/d6d/d85/d78/da4/ld6 0 2024-04-06T03:40:43.381 INFO:tasks.workunit.client.0.smithi094.stdout:4/764: mknod d0/df/dbd/cff 0 2024-04-06T03:40:43.383 INFO:tasks.workunit.client.0.smithi094.stdout:2/702: mkdir d0/d8/da0/ded 0 2024-04-06T03:40:43.383 INFO:tasks.workunit.client.0.smithi094.stdout:2/703: chown d0/dd/d3d/d49/l68 7 1 2024-04-06T03:40:43.384 INFO:tasks.workunit.client.0.smithi094.stdout:5/661: dwrite db/d19/d30/fb8 [0,4194304] 0 2024-04-06T03:40:43.386 INFO:tasks.workunit.client.0.smithi094.stdout:8/768: mkdir d18/d5c/de3/d10a 0 2024-04-06T03:40:43.393 INFO:tasks.workunit.client.0.smithi094.stdout:2/704: write d0/d8/d1d/fc0 [2659659,68995] 0 2024-04-06T03:40:43.393 INFO:tasks.workunit.client.0.smithi094.stdout:4/765: dwrite d0/f44 [0,4194304] 0 2024-04-06T03:40:43.393 INFO:tasks.workunit.client.0.smithi094.stdout:4/766: write d0/d15/fd7 [367130,48190] 0 2024-04-06T03:40:43.394 INFO:tasks.workunit.client.1.smithi136.stdout:7/801: rename d5/df/d1d/d6f/d4d/d69/c118 to d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/df9/c11b 0 2024-04-06T03:40:43.396 INFO:tasks.workunit.client.1.smithi136.stdout:0/744: mkdir d1/d6/d2c/d1a/d36/d59/dff 0 2024-04-06T03:40:43.399 INFO:tasks.workunit.client.0.smithi094.stdout:9/670: getdents d2/dc/d10/d2a 0 2024-04-06T03:40:43.399 INFO:tasks.workunit.client.0.smithi094.stdout:9/671: write d2/d2c/d99/faf [1316293,66416] 0 2024-04-06T03:40:43.403 INFO:tasks.workunit.client.1.smithi136.stdout:0/745: rename d1/d6/d2c/d57/l2f to d1/d6/l100 0 2024-04-06T03:40:43.403 INFO:tasks.workunit.client.1.smithi136.stdout:0/746: write d1/f29 [1062173,111891] 0 2024-04-06T03:40:43.405 INFO:tasks.workunit.client.1.smithi136.stdout:7/802: dread d5/df/d1d/f1f [0,4194304] 0 2024-04-06T03:40:43.405 INFO:tasks.workunit.client.1.smithi136.stdout:7/803: chown d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/fe9 110612901 1 2024-04-06T03:40:43.406 INFO:tasks.workunit.client.1.smithi136.stdout:4/656: sync 2024-04-06T03:40:43.407 INFO:tasks.workunit.client.1.smithi136.stdout:8/796: sync 2024-04-06T03:40:43.407 INFO:tasks.workunit.client.1.smithi136.stdout:5/814: sync 2024-04-06T03:40:43.407 INFO:tasks.workunit.client.1.smithi136.stdout:1/745: sync 2024-04-06T03:40:43.407 INFO:tasks.workunit.client.1.smithi136.stdout:3/724: sync 2024-04-06T03:40:43.408 INFO:tasks.workunit.client.1.smithi136.stdout:7/804: unlink d5/df/d12/d1b/d3a/d8d/lea 0 2024-04-06T03:40:43.411 INFO:tasks.workunit.client.1.smithi136.stdout:4/657: rmdir d1d/d2f/d6c/d91/ddd 0 2024-04-06T03:40:43.411 INFO:tasks.workunit.client.1.smithi136.stdout:4/658: chown d1d/d70/f94 10268261 1 2024-04-06T03:40:43.413 INFO:tasks.workunit.client.1.smithi136.stdout:1/746: symlink d12/d3b/d74/d7a/dea/d77/d96/lfd 0 2024-04-06T03:40:43.414 INFO:tasks.workunit.client.1.smithi136.stdout:0/747: write d1/d6/d2c/d1a/f81 [3081153,20541] 0 2024-04-06T03:40:43.416 INFO:tasks.workunit.client.1.smithi136.stdout:5/815: creat d1/dc/d10/d99/ff5 x:0 0 0 2024-04-06T03:40:43.416 INFO:tasks.workunit.client.1.smithi136.stdout:5/816: fdatasync d1/dc/d1c/d32/f8f 0 2024-04-06T03:40:43.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:43 smithi094 ceph-mon[29713]: pgmap v21: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 339 MiB/s rd, 433 MiB/s wr, 598 op/s 2024-04-06T03:40:43.418 INFO:tasks.workunit.client.0.smithi094.stdout:0/694: mkdir d1/d9/dd/d18/d5d/de9 0 2024-04-06T03:40:43.418 INFO:tasks.workunit.client.0.smithi094.stdout:0/695: stat d1/d9/dd/d18/d1e/d86/fa3 0 2024-04-06T03:40:43.420 INFO:tasks.workunit.client.1.smithi136.stdout:3/725: mkdir d0/d4/d12/d19/d55/d6f/d82/d5d/dfc 0 2024-04-06T03:40:43.420 INFO:tasks.workunit.client.1.smithi136.stdout:3/726: fdatasync d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 0 2024-04-06T03:40:43.421 INFO:tasks.workunit.client.0.smithi094.stdout:9/672: dwrite d2/dc/d21/d23/d2f/f90 [0,4194304] 0 2024-04-06T03:40:43.422 INFO:tasks.workunit.client.0.smithi094.stdout:9/673: dread d2/dc/d21/d23/d47/d51/f92 [0,4194304] 0 2024-04-06T03:40:43.422 INFO:tasks.workunit.client.0.smithi094.stdout:9/674: chown d2/dc/d21/d26/d8a/ff2 7483309 1 2024-04-06T03:40:43.425 INFO:tasks.workunit.client.1.smithi136.stdout:7/805: mkdir d5/df/d53/d68/d11c 0 2024-04-06T03:40:43.427 INFO:tasks.workunit.client.1.smithi136.stdout:1/747: symlink d12/d1b/da7/dcc/lfe 0 2024-04-06T03:40:43.427 INFO:tasks.workunit.client.1.smithi136.stdout:1/748: read d12/d1b/d1e/dd0/f9e [968404,38279] 0 2024-04-06T03:40:43.427 INFO:tasks.workunit.client.0.smithi094.stdout:5/662: truncate db/f12 3312352 0 2024-04-06T03:40:43.429 INFO:tasks.workunit.client.0.smithi094.stdout:7/640: sync 2024-04-06T03:40:43.429 INFO:tasks.workunit.client.0.smithi094.stdout:3/688: sync 2024-04-06T03:40:43.429 INFO:tasks.workunit.client.0.smithi094.stdout:6/671: sync 2024-04-06T03:40:43.431 INFO:tasks.workunit.client.0.smithi094.stdout:8/769: rename d18/d34/d52/f65 to d18/d3c/d8b/f10b 0 2024-04-06T03:40:43.431 INFO:tasks.workunit.client.0.smithi094.stdout:8/770: truncate d18/d2c/d37/d60/d70/f84 1505733 0 2024-04-06T03:40:43.431 INFO:tasks.workunit.client.0.smithi094.stdout:8/771: readlink d18/d34/d52/l7f 0 2024-04-06T03:40:43.433 INFO:tasks.workunit.client.1.smithi136.stdout:5/817: dread d1/f4 [0,4194304] 0 2024-04-06T03:40:43.434 INFO:tasks.workunit.client.1.smithi136.stdout:7/806: write d5/df/d12/d1b/d3a/d4a/d70/ddb/f72 [339401,93178] 0 2024-04-06T03:40:43.435 INFO:tasks.workunit.client.1.smithi136.stdout:7/807: read - d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/fe9 zero size 2024-04-06T03:40:43.435 INFO:tasks.workunit.client.1.smithi136.stdout:7/808: chown d5/df/d12/l6a 11596014 1 2024-04-06T03:40:43.435 INFO:tasks.workunit.client.1.smithi136.stdout:7/809: chown d5/df/d12/d1b/d3a/d4a/d70/ddb/l8c 4 1 2024-04-06T03:40:43.436 INFO:tasks.workunit.client.1.smithi136.stdout:7/810: dread d5/df/d12/d1b/f47 [0,4194304] 0 2024-04-06T03:40:43.436 INFO:tasks.workunit.client.1.smithi136.stdout:7/811: write d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/f5e [918075,54275] 0 2024-04-06T03:40:43.436 INFO:tasks.workunit.client.1.smithi136.stdout:5/818: creat d1/d16/d4b/d7b/ff6 x:0 0 0 2024-04-06T03:40:43.437 INFO:tasks.workunit.client.1.smithi136.stdout:5/819: truncate d1/d98/dc6/fd0 5064881 0 2024-04-06T03:40:43.438 INFO:tasks.workunit.client.0.smithi094.stdout:0/696: creat d1/d9/dd/d18/d78/fea x:0 0 0 2024-04-06T03:40:43.440 INFO:tasks.workunit.client.1.smithi136.stdout:7/812: link d5/df/d12/d1b/d3a/d4a/f86 d5/df/d53/d8a/f11d 0 2024-04-06T03:40:43.440 INFO:tasks.workunit.client.1.smithi136.stdout:7/813: fsync d5/df/d11/f26 0 2024-04-06T03:40:43.443 INFO:tasks.workunit.client.1.smithi136.stdout:7/814: rmdir d5/df/d53/d68/da3/dec 39 2024-04-06T03:40:43.444 INFO:tasks.workunit.client.0.smithi094.stdout:9/675: creat d2/ff7 x:0 0 0 2024-04-06T03:40:43.446 INFO:tasks.workunit.client.0.smithi094.stdout:6/672: link dc/dd3/d72/d87/cd2 dc/dd3/d24/d46/cf3 0 2024-04-06T03:40:43.446 INFO:tasks.workunit.client.0.smithi094.stdout:6/673: stat dc/d3a/d4a/d54/d71 0 2024-04-06T03:40:43.447 INFO:tasks.workunit.client.1.smithi136.stdout:4/659: dwrite d1d/d70/f74 [0,4194304] 0 2024-04-06T03:40:43.448 INFO:tasks.workunit.client.1.smithi136.stdout:1/749: dwrite d12/d1b/d1e/d4d/f51 [4194304,4194304] 0 2024-04-06T03:40:43.448 INFO:tasks.workunit.client.1.smithi136.stdout:1/750: write d12/d3b/d60/d7e/f7d [411386,118725] 0 2024-04-06T03:40:43.448 INFO:tasks.workunit.client.1.smithi136.stdout:1/751: chown d12/d3b/d60/d7e/dbe/fd8 6997 1 2024-04-06T03:40:43.448 INFO:tasks.workunit.client.0.smithi094.stdout:7/641: mknod d2/d6/dc/d12/d29/d3f/db9/cc5 0 2024-04-06T03:40:43.449 INFO:tasks.workunit.client.1.smithi136.stdout:9/750: sync 2024-04-06T03:40:43.449 INFO:tasks.workunit.client.0.smithi094.stdout:3/689: mknod d9/d10/cfd 0 2024-04-06T03:40:43.449 INFO:tasks.workunit.client.0.smithi094.stdout:3/690: write d9/d1d/d7d/f85 [74820,91857] 0 2024-04-06T03:40:43.450 INFO:tasks.workunit.client.0.smithi094.stdout:3/691: write d9/d20/d76/ff7 [590473,27800] 0 2024-04-06T03:40:43.450 INFO:tasks.workunit.client.1.smithi136.stdout:0/748: dwrite d1/d6/d2c/d57/fe6 [0,4194304] 0 2024-04-06T03:40:43.450 INFO:tasks.workunit.client.1.smithi136.stdout:0/749: creat d1/d6/d2c/d57/d43/d9e/f101 x:0 0 0 2024-04-06T03:40:43.455 INFO:tasks.workunit.client.0.smithi094.stdout:2/705: rename d0/dd/d3d/d49/l5a to d0/dd/lee 0 2024-04-06T03:40:43.455 INFO:tasks.workunit.client.0.smithi094.stdout:2/706: creat d0/dd/d1b/d72/fef x:0 0 0 2024-04-06T03:40:43.458 INFO:tasks.workunit.client.0.smithi094.stdout:8/772: creat d18/d3c/d49/d9f/d109/f10c x:0 0 0 2024-04-06T03:40:43.458 INFO:tasks.workunit.client.0.smithi094.stdout:8/773: chown d18/d2c/d37/d60/da3/db8/dc3/fd4 0 1 2024-04-06T03:40:43.460 INFO:tasks.workunit.client.0.smithi094.stdout:0/697: getdents d1/d9/dd/d18/d5d/d60/d6b/dab 0 2024-04-06T03:40:43.462 INFO:tasks.workunit.client.1.smithi136.stdout:1/752: mknod d12/d3b/d44/d5f/d5a/de2/cff 0 2024-04-06T03:40:43.462 INFO:tasks.workunit.client.1.smithi136.stdout:1/753: fdatasync d12/d1b/d1e/d9f/fa8 0 2024-04-06T03:40:43.462 INFO:tasks.workunit.client.1.smithi136.stdout:1/754: readlink d12/d3b/d60/d7e/d37/l6a 0 2024-04-06T03:40:43.463 INFO:tasks.workunit.client.1.smithi136.stdout:7/815: dwrite d5/df/d53/d68/d8f/fc1 [0,4194304] 0 2024-04-06T03:40:43.463 INFO:tasks.workunit.client.1.smithi136.stdout:7/816: chown d5/df/d3d/cd1 63845 1 2024-04-06T03:40:43.463 INFO:tasks.workunit.client.1.smithi136.stdout:7/817: write d5/df/d53/d68/da3/fe5 [991489,96402] 0 2024-04-06T03:40:43.463 INFO:tasks.workunit.client.1.smithi136.stdout:7/818: creat d5/df/d1d/d6f/d2a/f11e x:0 0 0 2024-04-06T03:40:43.464 INFO:tasks.workunit.client.0.smithi094.stdout:9/676: symlink d2/dc/d21/d23/d2f/lf8 0 2024-04-06T03:40:43.467 INFO:tasks.workunit.client.0.smithi094.stdout:6/674: mkdir dc/d3a/df4 0 2024-04-06T03:40:43.470 INFO:tasks.workunit.client.0.smithi094.stdout:7/642: mknod d2/d6/cc6 0 2024-04-06T03:40:43.475 INFO:tasks.workunit.client.0.smithi094.stdout:4/767: rename d0/d23/c56 to d0/df/dbd/c100 0 2024-04-06T03:40:43.475 INFO:tasks.workunit.client.0.smithi094.stdout:4/768: readlink d0/d15/d3a/d3e/d50/l6b 0 2024-04-06T03:40:43.476 INFO:tasks.workunit.client.0.smithi094.stdout:3/692: dwrite d9/d20/f33 [8388608,4194304] 0 2024-04-06T03:40:43.476 INFO:tasks.workunit.client.1.smithi136.stdout:0/750: dwrite d1/d6/d2c/d1a/f1d [4194304,4194304] 0 2024-04-06T03:40:43.476 INFO:tasks.workunit.client.1.smithi136.stdout:0/751: chown d1/d6/d2c/d1a/d82 65927139 1 2024-04-06T03:40:43.478 INFO:tasks.workunit.client.1.smithi136.stdout:4/660: dwrite d1d/d31/d3e/d55/d89/db3/fc6 [0,4194304] 0 2024-04-06T03:40:43.478 INFO:tasks.workunit.client.1.smithi136.stdout:4/661: chown d1d/d31/d3e/d56/f81 9297277 1 2024-04-06T03:40:43.479 INFO:tasks.workunit.client.0.smithi094.stdout:2/707: symlink d0/dd/d19/d31/d67/lf0 0 2024-04-06T03:40:43.479 INFO:tasks.workunit.client.0.smithi094.stdout:2/708: chown d0/dd/d3d/fde 19609521 1 2024-04-06T03:40:43.480 INFO:tasks.workunit.client.1.smithi136.stdout:9/751: dwrite d17/d20/d25/d3c/d55/f64 [4194304,4194304] 0 2024-04-06T03:40:43.480 INFO:tasks.workunit.client.1.smithi136.stdout:9/752: write d17/d20/d28/f98 [880640,59593] 0 2024-04-06T03:40:43.481 INFO:tasks.workunit.client.0.smithi094.stdout:8/774: getdents d18/d3c 0 2024-04-06T03:40:43.482 INFO:tasks.workunit.client.0.smithi094.stdout:8/775: chown d18/d2c/d37/d60/da3/db1/lc6 163671 1 2024-04-06T03:40:43.483 INFO:tasks.workunit.client.0.smithi094.stdout:0/698: mkdir d1/d9/dd/d18/d78/deb 0 2024-04-06T03:40:43.485 INFO:tasks.workunit.client.1.smithi136.stdout:1/755: creat d12/d3b/d74/dd6/f100 x:0 0 0 2024-04-06T03:40:43.485 INFO:tasks.workunit.client.0.smithi094.stdout:9/677: dread d2/dc/d21/d23/d47/d51/f8c [0,4194304] 0 2024-04-06T03:40:43.486 INFO:tasks.workunit.client.0.smithi094.stdout:9/678: chown d2/dc/d10/c53 6794 1 2024-04-06T03:40:43.486 INFO:tasks.workunit.client.0.smithi094.stdout:6/675: dwrite dc/dd3/d24/f43 [0,4194304] 0 2024-04-06T03:40:43.489 INFO:tasks.workunit.client.0.smithi094.stdout:5/663: rename db/d19/d2c/d57/d8f to db/d19/dc1/ddf 0 2024-04-06T03:40:43.494 INFO:tasks.workunit.client.0.smithi094.stdout:4/769: link d0/d23/c52 d0/d15/d3a/d3e/d49/c101 0 2024-04-06T03:40:43.494 INFO:tasks.workunit.client.0.smithi094.stdout:4/770: readlink d0/d4/d67/dcb/le4 0 2024-04-06T03:40:43.500 INFO:tasks.workunit.client.1.smithi136.stdout:7/819: getdents d5/df/d53/d68/da3/dd0 0 2024-04-06T03:40:43.504 INFO:tasks.workunit.client.0.smithi094.stdout:2/709: mknod d0/d8/d27/d2b/d8e/cf1 0 2024-04-06T03:40:43.504 INFO:tasks.workunit.client.0.smithi094.stdout:2/710: fdatasync d0/d8/d1d/f73 0 2024-04-06T03:40:43.504 INFO:tasks.workunit.client.0.smithi094.stdout:2/711: write d0/dd/d3d/d49/feb [801543,43572] 0 2024-04-06T03:40:43.505 INFO:tasks.workunit.client.1.smithi136.stdout:0/752: getdents d1/d6/d2c/d57/d43/d6d/d9b 0 2024-04-06T03:40:43.505 INFO:tasks.workunit.client.1.smithi136.stdout:0/753: fdatasync d1/d6/d2c/d1a/f1d 0 2024-04-06T03:40:43.505 INFO:tasks.workunit.client.1.smithi136.stdout:0/754: creat d1/d6/d2c/d57/d43/d9e/f102 x:0 0 0 2024-04-06T03:40:43.505 INFO:tasks.workunit.client.1.smithi136.stdout:4/662: dwrite d1d/d31/d3e/d56/f81 [0,4194304] 0 2024-04-06T03:40:43.505 INFO:tasks.workunit.client.1.smithi136.stdout:4/663: creat d1d/fed x:0 0 0 2024-04-06T03:40:43.505 INFO:tasks.workunit.client.1.smithi136.stdout:4/664: fdatasync d1d/d70/f6a 0 2024-04-06T03:40:43.505 INFO:tasks.workunit.client.0.smithi094.stdout:4/771: write d0/d4/f21 [3253917,80827] 0 2024-04-06T03:40:43.508 INFO:tasks.workunit.client.0.smithi094.stdout:6/676: rmdir dc/d3a/d7f/dd4 39 2024-04-06T03:40:43.509 INFO:tasks.workunit.client.0.smithi094.stdout:6/677: dread dc/d3a/d4a/f51 [0,4194304] 0 2024-04-06T03:40:43.509 INFO:tasks.workunit.client.0.smithi094.stdout:6/678: write dc/d3a/d4a/d54/dde/f9e [617046,129540] 0 2024-04-06T03:40:43.510 INFO:tasks.workunit.client.1.smithi136.stdout:7/820: read d5/df/f41 [224734,28816] 0 2024-04-06T03:40:43.510 INFO:tasks.workunit.client.1.smithi136.stdout:7/821: dread - d5/df/d12/dd6/f115 zero size 2024-04-06T03:40:43.510 INFO:tasks.workunit.client.0.smithi094.stdout:9/679: dwrite d2/dc/d21/f43 [0,4194304] 0 2024-04-06T03:40:43.514 INFO:tasks.workunit.client.0.smithi094.stdout:5/664: mknod db/d7c/d8b/ce0 0 2024-04-06T03:40:43.514 INFO:tasks.workunit.client.0.smithi094.stdout:5/665: creat db/d19/d30/d83/fe1 x:0 0 0 2024-04-06T03:40:43.516 INFO:tasks.workunit.client.1.smithi136.stdout:0/755: write d1/d6/d2c/d57/d43/d6d/d9b/d5c/f91 [2143900,35874] 0 2024-04-06T03:40:43.516 INFO:tasks.workunit.client.1.smithi136.stdout:0/756: fdatasync d1/f7 0 2024-04-06T03:40:43.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/756: dwrite d12/d3b/d60/d7e/dbe/fd8 [0,4194304] 0 2024-04-06T03:40:43.517 INFO:tasks.workunit.client.1.smithi136.stdout:7/822: dread d5/df/d11/f1c [0,4194304] 0 2024-04-06T03:40:43.517 INFO:tasks.workunit.client.0.smithi094.stdout:0/699: rename d1/d9/l3f to d1/d9/dd/d18/d5d/d60/d6b/dab/lec 0 2024-04-06T03:40:43.520 INFO:tasks.workunit.client.0.smithi094.stdout:3/693: rmdir d9/d1e/d6e/d75/dba/d97 39 2024-04-06T03:40:43.521 INFO:tasks.workunit.client.0.smithi094.stdout:0/700: dread d1/d9/dd/d18/d1e/d86/fa3 [0,4194304] 0 2024-04-06T03:40:43.521 INFO:tasks.workunit.client.0.smithi094.stdout:0/701: chown d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/de4/fc3 215091 1 2024-04-06T03:40:43.523 INFO:tasks.workunit.client.0.smithi094.stdout:2/712: creat d0/d18/ff2 x:0 0 0 2024-04-06T03:40:43.524 INFO:tasks.workunit.client.1.smithi136.stdout:9/753: symlink d17/d20/d25/da4/l105 0 2024-04-06T03:40:43.525 INFO:tasks.workunit.client.1.smithi136.stdout:9/754: getdents d17/d20/d25/da4/dbb/de4 0 2024-04-06T03:40:43.529 INFO:tasks.workunit.client.1.smithi136.stdout:6/668: sync 2024-04-06T03:40:43.529 INFO:tasks.workunit.client.1.smithi136.stdout:2/802: sync 2024-04-06T03:40:43.529 INFO:tasks.workunit.client.1.smithi136.stdout:6/669: chown d2/d7/d9e/cb1 50449899 1 2024-04-06T03:40:43.531 INFO:tasks.workunit.client.0.smithi094.stdout:9/680: dwrite d2/dc/d21/d23/d5b/ff0 [0,4194304] 0 2024-04-06T03:40:43.531 INFO:tasks.workunit.client.0.smithi094.stdout:6/679: link dc/d3a/d4a/d54/dde/f34 dc/d3a/d4a/d54/dde/d50/d61/ff5 0 2024-04-06T03:40:43.534 INFO:tasks.workunit.client.1.smithi136.stdout:9/755: write d17/d20/d25/da4/fe9 [3568252,100106] 0 2024-04-06T03:40:43.534 INFO:tasks.workunit.client.1.smithi136.stdout:9/756: creat d17/d21/d29/d34/d46/d9a/f106 x:0 0 0 2024-04-06T03:40:43.538 INFO:tasks.workunit.client.0.smithi094.stdout:2/713: dwrite d0/dd/f62 [0,4194304] 0 2024-04-06T03:40:43.538 INFO:tasks.workunit.client.0.smithi094.stdout:2/714: truncate d0/d8/d27/f78 917906 0 2024-04-06T03:40:43.548 INFO:tasks.workunit.client.1.smithi136.stdout:9/757: dwrite f13 [0,4194304] 0 2024-04-06T03:40:43.558 INFO:tasks.workunit.client.1.smithi136.stdout:9/758: dread d17/d21/d29/d34/d66/de1/f8c [0,4194304] 0 2024-04-06T03:40:43.566 INFO:tasks.workunit.client.1.smithi136.stdout:9/759: dread d17/d21/d29/de5/d4b/f80 [0,4194304] 0 2024-04-06T03:40:43.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:43 smithi136 ceph-mon[36436]: pgmap v21: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 339 MiB/s rd, 433 MiB/s wr, 598 op/s 2024-04-06T03:40:43.706 INFO:tasks.workunit.client.0.smithi094.stdout:4/772: link d0/l45 d0/d77/l102 0 2024-04-06T03:40:43.708 INFO:tasks.workunit.client.0.smithi094.stdout:4/773: dread d0/f22 [0,4194304] 0 2024-04-06T03:40:43.718 INFO:tasks.workunit.client.1.smithi136.stdout:7/823: rename d5/df/d11/f79 to d5/df/d1d/d6f/f11f 0 2024-04-06T03:40:43.721 INFO:tasks.workunit.client.1.smithi136.stdout:2/803: unlink d3/d9/fad 0 2024-04-06T03:40:43.722 INFO:tasks.workunit.client.0.smithi094.stdout:2/715: truncate d0/d8/d27/d2b/f5d 1873249 0 2024-04-06T03:40:43.722 INFO:tasks.workunit.client.0.smithi094.stdout:2/716: fdatasync d0/d8/d27/d2b/f52 0 2024-04-06T03:40:43.722 INFO:tasks.workunit.client.0.smithi094.stdout:2/717: fsync d0/dd/d19/f4c 0 2024-04-06T03:40:43.722 INFO:tasks.workunit.client.0.smithi094.stdout:4/774: read d0/df/f1b [2900880,16680] 0 2024-04-06T03:40:43.723 INFO:tasks.workunit.client.1.smithi136.stdout:0/757: dread d1/d6/f58 [0,4194304] 0 2024-04-06T03:40:43.724 INFO:tasks.workunit.client.1.smithi136.stdout:6/670: mkdir d2/d7/d26/d68/ddc 0 2024-04-06T03:40:43.724 INFO:tasks.workunit.client.1.smithi136.stdout:6/671: write d2/d7/d26/d30/d6e/d58/d5d/fd0 [910832,67860] 0 2024-04-06T03:40:43.730 INFO:tasks.workunit.client.0.smithi094.stdout:2/718: dread d0/d8/d27/d2b/d8e/fac [0,4194304] 0 2024-04-06T03:40:43.785 INFO:tasks.workunit.client.1.smithi136.stdout:9/760: creat d17/d19/da3/dc5/deb/d9e/dfb/f107 x:0 0 0 2024-04-06T03:40:43.785 INFO:tasks.workunit.client.1.smithi136.stdout:9/761: truncate d17/d19/fc8 300121 0 2024-04-06T03:40:43.801 INFO:tasks.workunit.client.1.smithi136.stdout:5/820: sync 2024-04-06T03:40:43.801 INFO:tasks.workunit.client.1.smithi136.stdout:8/797: sync 2024-04-06T03:40:43.801 INFO:tasks.workunit.client.1.smithi136.stdout:3/727: sync 2024-04-06T03:40:43.801 INFO:tasks.workunit.client.1.smithi136.stdout:5/821: readlink d1/dc/d1c/d32/d8b/d20/lb5 0 2024-04-06T03:40:43.802 INFO:tasks.workunit.client.1.smithi136.stdout:3/728: fsync d0/d4/d12/d19/d55/d6f/fb1 0 2024-04-06T03:40:43.802 INFO:tasks.workunit.client.1.smithi136.stdout:5/822: dread d1/dc/d10/fc4 [0,4194304] 0 2024-04-06T03:40:43.803 INFO:tasks.workunit.client.1.smithi136.stdout:5/823: fdatasync d1/dc/d1c/d32/d8b/d20/f41 0 2024-04-06T03:40:43.803 INFO:tasks.workunit.client.1.smithi136.stdout:5/824: stat d1/d98/dbc/fdc 0 2024-04-06T03:40:43.809 INFO:tasks.workunit.client.1.smithi136.stdout:5/825: dread d1/dc/d1c/d32/f8f [0,4194304] 0 2024-04-06T03:40:43.809 INFO:tasks.workunit.client.1.smithi136.stdout:5/826: creat d1/dc/d1c/d32/d70/ff7 x:0 0 0 2024-04-06T03:40:43.892 INFO:tasks.workunit.client.1.smithi136.stdout:7/824: unlink d5/df/d3d/f63 0 2024-04-06T03:40:43.898 INFO:tasks.workunit.client.1.smithi136.stdout:6/672: mkdir d2/d7/d13/d79/dc2/ddd 0 2024-04-06T03:40:43.907 INFO:tasks.workunit.client.1.smithi136.stdout:7/825: dwrite d5/df/d53/d68/d8f/fc1 [0,4194304] 0 2024-04-06T03:40:43.907 INFO:tasks.workunit.client.1.smithi136.stdout:9/762: mknod d17/d21/d29/de5/d4b/c108 0 2024-04-06T03:40:43.911 INFO:tasks.workunit.client.1.smithi136.stdout:8/798: creat d3/da/d26/d35/dcb/ff4 x:0 0 0 2024-04-06T03:40:43.911 INFO:tasks.workunit.client.1.smithi136.stdout:8/799: stat d3/da/d26/d90/dae/fda 0 2024-04-06T03:40:43.921 INFO:tasks.workunit.client.1.smithi136.stdout:2/804: rmdir d3/d15/d19/dac 39 2024-04-06T03:40:43.922 INFO:tasks.workunit.client.1.smithi136.stdout:2/805: truncate d3/d4a/d9d/d18/fa4 753567 0 2024-04-06T03:40:43.922 INFO:tasks.workunit.client.1.smithi136.stdout:2/806: readlink d3/d4a/d9d/d18/d1d/d3c/d58/lcf 0 2024-04-06T03:40:43.923 INFO:tasks.workunit.client.0.smithi094.stdout:4/775: dread d0/f8 [0,4194304] 0 2024-04-06T03:40:43.924 INFO:tasks.workunit.client.1.smithi136.stdout:6/673: rename d2/d7/d9e/fd3 to d2/d7/d26/d68/fde 0 2024-04-06T03:40:43.924 INFO:tasks.workunit.client.1.smithi136.stdout:6/674: readlink d2/l78 0 2024-04-06T03:40:43.924 INFO:tasks.workunit.client.1.smithi136.stdout:6/675: fdatasync d2/d7/d26/d30/d6e/f35 0 2024-04-06T03:40:43.925 INFO:tasks.workunit.client.0.smithi094.stdout:3/694: creat d9/d10/ffe x:0 0 0 2024-04-06T03:40:43.925 INFO:tasks.workunit.client.0.smithi094.stdout:3/695: fdatasync d9/f8d 0 2024-04-06T03:40:43.925 INFO:tasks.workunit.client.0.smithi094.stdout:3/696: creat d9/d10/dbb/fff x:0 0 0 2024-04-06T03:40:43.925 INFO:tasks.workunit.client.0.smithi094.stdout:3/697: readlink d9/d10/l17 0 2024-04-06T03:40:43.925 INFO:tasks.workunit.client.0.smithi094.stdout:3/698: truncate d9/d1e/d6e/fc8 506668 0 2024-04-06T03:40:43.930 INFO:tasks.workunit.client.1.smithi136.stdout:7/826: symlink d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/l120 0 2024-04-06T03:40:43.933 INFO:tasks.workunit.client.1.smithi136.stdout:9/763: truncate d17/d20/d28/f82 2467061 0 2024-04-06T03:40:43.935 INFO:tasks.workunit.client.1.smithi136.stdout:6/676: write d2/d7/f5a [1577915,119561] 0 2024-04-06T03:40:43.935 INFO:tasks.workunit.client.1.smithi136.stdout:8/800: mkdir d3/da/d26/d35/d39/df5 0 2024-04-06T03:40:43.935 INFO:tasks.workunit.client.1.smithi136.stdout:8/801: write d3/da/fd [466450,43384] 0 2024-04-06T03:40:43.939 INFO:tasks.workunit.client.1.smithi136.stdout:2/807: symlink d3/d9/d3b/d87/db1/l108 0 2024-04-06T03:40:43.939 INFO:tasks.workunit.client.1.smithi136.stdout:2/808: readlink d3/d4a/d9d/d18/l6e 0 2024-04-06T03:40:43.941 INFO:tasks.workunit.client.1.smithi136.stdout:7/827: mknod d5/df/d12/d1b/d3a/c121 0 2024-04-06T03:40:43.941 INFO:tasks.workunit.client.1.smithi136.stdout:7/828: write d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/f5e [1712852,105898] 0 2024-04-06T03:40:43.943 INFO:tasks.workunit.client.0.smithi094.stdout:4/776: truncate d0/df/d1e/d34/f39 584165 0 2024-04-06T03:40:43.945 INFO:tasks.workunit.client.0.smithi094.stdout:3/699: dwrite d9/d1e/d6e/d75/dba/d3a/ffa [0,4194304] 0 2024-04-06T03:40:43.945 INFO:tasks.workunit.client.0.smithi094.stdout:3/700: write d9/fc [3760392,70817] 0 2024-04-06T03:40:43.945 INFO:tasks.workunit.client.0.smithi094.stdout:3/701: chown d9/d20/d2d/da9/fe0 0 1 2024-04-06T03:40:43.948 INFO:tasks.workunit.client.1.smithi136.stdout:6/677: truncate d2/d7/d26/d30/f4c 4758102 0 2024-04-06T03:40:43.948 INFO:tasks.workunit.client.1.smithi136.stdout:6/678: write d2/d7/d26/d30/f7e [1904741,94364] 0 2024-04-06T03:40:43.952 INFO:tasks.workunit.client.1.smithi136.stdout:9/764: dwrite d17/d21/d29/de5/da5/fd6 [4194304,4194304] 0 2024-04-06T03:40:43.953 INFO:tasks.workunit.client.1.smithi136.stdout:7/829: dread d5/df/d12/d1b/f10c [0,4194304] 0 2024-04-06T03:40:43.954 INFO:tasks.workunit.client.0.smithi094.stdout:4/777: mknod d0/d4/dde/c103 0 2024-04-06T03:40:43.954 INFO:tasks.workunit.client.0.smithi094.stdout:4/778: write d0/d15/d3a/d3e/d50/f5e [2422893,47835] 0 2024-04-06T03:40:43.954 INFO:tasks.workunit.client.0.smithi094.stdout:4/779: write d0/df/db3/dd8/fed [870359,68197] 0 2024-04-06T03:40:43.954 INFO:tasks.workunit.client.0.smithi094.stdout:4/780: creat d0/df/d1e/d34/db6/f104 x:0 0 0 2024-04-06T03:40:43.954 INFO:tasks.workunit.client.1.smithi136.stdout:2/809: creat d3/d9/d3b/d70/dc8/f109 x:0 0 0 2024-04-06T03:40:43.961 INFO:tasks.workunit.client.1.smithi136.stdout:7/830: rename d5/df/d12/d1b/d3a/d4a/d70/ddb/fb1 to d5/df/d12/f122 0 2024-04-06T03:40:43.961 INFO:tasks.workunit.client.1.smithi136.stdout:8/802: dwrite d3/da/d26/d35/d53/f5b [0,4194304] 0 2024-04-06T03:40:43.961 INFO:tasks.workunit.client.1.smithi136.stdout:8/803: chown d3/f77 244352 1 2024-04-06T03:40:43.961 INFO:tasks.workunit.client.1.smithi136.stdout:8/804: fdatasync d3/da/d26/f85 0 2024-04-06T03:40:43.962 INFO:tasks.workunit.client.1.smithi136.stdout:8/805: truncate d3/f77 1592267 0 2024-04-06T03:40:43.962 INFO:tasks.workunit.client.1.smithi136.stdout:8/806: stat d3/da/d26/l59 0 2024-04-06T03:40:43.962 INFO:tasks.workunit.client.0.smithi094.stdout:3/702: dwrite d9/d1d/f36 [0,4194304] 0 2024-04-06T03:40:43.962 INFO:tasks.workunit.client.1.smithi136.stdout:6/679: dread d2/d7/d26/d30/d6e/f69 [0,4194304] 0 2024-04-06T03:40:43.964 INFO:tasks.workunit.client.1.smithi136.stdout:2/810: mknod d3/d4a/d77/c10a 0 2024-04-06T03:40:43.969 INFO:tasks.workunit.client.0.smithi094.stdout:4/781: creat d0/df/d13/d4a/dfa/f105 x:0 0 0 2024-04-06T03:40:43.970 INFO:tasks.workunit.client.1.smithi136.stdout:6/680: dread d2/d7/d26/d30/d6e/f69 [0,4194304] 0 2024-04-06T03:40:43.970 INFO:tasks.workunit.client.1.smithi136.stdout:9/765: dwrite d17/d20/d28/f89 [0,4194304] 0 2024-04-06T03:40:43.971 INFO:tasks.workunit.client.1.smithi136.stdout:7/831: link d5/df/d1d/d54/fa4 d5/df/d12/f123 0 2024-04-06T03:40:43.972 INFO:tasks.workunit.client.0.smithi094.stdout:2/719: mkdir d0/dd/d1b/d72/df3 0 2024-04-06T03:40:43.975 INFO:tasks.workunit.client.0.smithi094.stdout:3/703: symlink d9/d1e/d6e/l100 0 2024-04-06T03:40:43.975 INFO:tasks.workunit.client.0.smithi094.stdout:3/704: readlink d9/d20/l51 0 2024-04-06T03:40:43.975 INFO:tasks.workunit.client.0.smithi094.stdout:3/705: chown d9/d1e/d6e/d75/cd3 17404039 1 2024-04-06T03:40:43.975 INFO:tasks.workunit.client.0.smithi094.stdout:3/706: chown d9/d20/d76/f5c 0 1 2024-04-06T03:40:43.976 INFO:tasks.workunit.client.0.smithi094.stdout:3/707: write d9/d20/f4e [4261249,95429] 0 2024-04-06T03:40:43.977 INFO:tasks.workunit.client.1.smithi136.stdout:8/807: rename d3/da/d26/d35/ca7 to d3/de/d18/d7d/ddf/dc9/cf6 0 2024-04-06T03:40:43.978 INFO:tasks.workunit.client.0.smithi094.stdout:4/782: creat d0/d15/d3a/d61/d81/f106 x:0 0 0 2024-04-06T03:40:43.980 INFO:tasks.workunit.client.1.smithi136.stdout:2/811: mkdir d3/d15/d19/d43/d54/d10b 0 2024-04-06T03:40:43.981 INFO:tasks.workunit.client.0.smithi094.stdout:2/720: creat d0/dd/d19/d31/dcc/ff4 x:0 0 0 2024-04-06T03:40:43.983 INFO:tasks.workunit.client.1.smithi136.stdout:6/681: mkdir d2/d7/d9e/ddf 0 2024-04-06T03:40:43.983 INFO:tasks.workunit.client.1.smithi136.stdout:6/682: write d2/d7/d26/d30/d4f/f67 [1201820,101521] 0 2024-04-06T03:40:43.984 INFO:tasks.workunit.client.1.smithi136.stdout:9/766: rmdir d17/d20/d25/d3c 39 2024-04-06T03:40:43.985 INFO:tasks.workunit.client.1.smithi136.stdout:9/767: write d17/d21/d29/d34/d46/d9a/fd7 [150216,94760] 0 2024-04-06T03:40:43.986 INFO:tasks.workunit.client.1.smithi136.stdout:7/832: creat d5/df/d11/f124 x:0 0 0 2024-04-06T03:40:43.987 INFO:tasks.workunit.client.1.smithi136.stdout:8/808: symlink d3/de/d18/d7d/ddf/d6c/lf7 0 2024-04-06T03:40:43.989 INFO:tasks.workunit.client.1.smithi136.stdout:6/683: creat d2/d7/d13/d79/dc2/ddd/fe0 x:0 0 0 2024-04-06T03:40:43.990 INFO:tasks.workunit.client.0.smithi094.stdout:5/666: rename db/c15 to db/d19/d2c/ce2 0 2024-04-06T03:40:43.993 INFO:tasks.workunit.client.1.smithi136.stdout:9/768: symlink d17/d21/d29/de5/da5/db8/l109 0 2024-04-06T03:40:43.994 INFO:tasks.workunit.client.0.smithi094.stdout:5/667: creat db/d19/d24/d28/d85/dd5/fe3 x:0 0 0 2024-04-06T03:40:43.994 INFO:tasks.workunit.client.0.smithi094.stdout:5/668: creat db/d19/d30/d83/fe4 x:0 0 0 2024-04-06T03:40:44.071 INFO:tasks.workunit.client.0.smithi094.stdout:6/680: rename dc/dd3/d24/d46 to dc/d3a/df4/df6 0 2024-04-06T03:40:44.071 INFO:tasks.workunit.client.0.smithi094.stdout:6/681: write dc/d3a/d4a/f51 [1220708,38992] 0 2024-04-06T03:40:44.075 INFO:tasks.workunit.client.0.smithi094.stdout:3/708: rename d9/d20/d2d/da9/cbd to d9/d1e/d26/d6c/d8e/c101 0 2024-04-06T03:40:44.075 INFO:tasks.workunit.client.0.smithi094.stdout:5/669: dwrite db/d19/d2c/f31 [4194304,4194304] 0 2024-04-06T03:40:44.080 INFO:tasks.workunit.client.0.smithi094.stdout:6/682: dread dc/d3a/d4a/d94/dac/fb8 [0,4194304] 0 2024-04-06T03:40:44.084 INFO:tasks.workunit.client.1.smithi136.stdout:7/833: mknod d5/df/d12/d1b/d3a/d4a/d70/c125 0 2024-04-06T03:40:44.084 INFO:tasks.workunit.client.1.smithi136.stdout:7/834: read d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/fb6 [126907,124403] 0 2024-04-06T03:40:44.085 INFO:tasks.workunit.client.1.smithi136.stdout:8/809: truncate d3/de/f4c 4797531 0 2024-04-06T03:40:44.088 INFO:tasks.workunit.client.0.smithi094.stdout:4/783: rename d0/d15/d3a/d61/ddf to d0/df/dbd/d107 0 2024-04-06T03:40:44.090 INFO:tasks.workunit.client.1.smithi136.stdout:6/684: getdents d2/d7/d26/d30/d6e/d3d/d47 0 2024-04-06T03:40:44.090 INFO:tasks.workunit.client.1.smithi136.stdout:6/685: chown d2/d7/d26/d30/dc4 0 1 2024-04-06T03:40:44.091 INFO:tasks.workunit.client.0.smithi094.stdout:3/709: symlink d9/d1e/d26/d54/l102 0 2024-04-06T03:40:44.092 INFO:tasks.workunit.client.0.smithi094.stdout:3/710: write d9/d1d/f4a [82608,89926] 0 2024-04-06T03:40:44.093 INFO:tasks.workunit.client.1.smithi136.stdout:9/769: getdents d17/d21/d29/de5/da5/db8/de0 0 2024-04-06T03:40:44.093 INFO:tasks.workunit.client.1.smithi136.stdout:9/770: write d17/d21/d29/de5/d4b/fab [472754,88353] 0 2024-04-06T03:40:44.094 INFO:tasks.workunit.client.0.smithi094.stdout:3/711: dread d9/d1e/f8f [0,4194304] 0 2024-04-06T03:40:44.094 INFO:tasks.workunit.client.0.smithi094.stdout:3/712: readlink d9/d1e/d6e/d75/dba/d7c/da2/lb4 0 2024-04-06T03:40:44.094 INFO:tasks.workunit.client.0.smithi094.stdout:3/713: chown d9/d1e/d6e/d75/dba/d97/l99 556879 1 2024-04-06T03:40:44.094 INFO:tasks.workunit.client.0.smithi094.stdout:3/714: read - d9/d1e/d6e/d75/dba/d7c/fe8 zero size 2024-04-06T03:40:44.097 INFO:tasks.workunit.client.1.smithi136.stdout:7/835: mkdir d5/df/d12/d1b/d3a/d4a/dc4/d126 0 2024-04-06T03:40:44.099 INFO:tasks.workunit.client.0.smithi094.stdout:5/670: dwrite db/d19/d30/f44 [0,4194304] 0 2024-04-06T03:40:44.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/683: dwrite dc/dd3/d24/fe8 [0,4194304] 0 2024-04-06T03:40:44.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/684: truncate dc/d3a/df4/df6/fc2 549605 0 2024-04-06T03:40:44.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/685: fsync dc/d3a/da2/fa7 0 2024-04-06T03:40:44.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/686: write dc/d3a/f41 [2278521,78695] 0 2024-04-06T03:40:44.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/687: truncate dc/d3a/d4a/d54/dde/d50/d61/fef 777135 0 2024-04-06T03:40:44.099 INFO:tasks.workunit.client.0.smithi094.stdout:3/715: symlink d9/d10/d9b/db7/dc0/l103 0 2024-04-06T03:40:44.101 INFO:tasks.workunit.client.1.smithi136.stdout:8/810: mknod d3/de/d18/cf8 0 2024-04-06T03:40:44.103 INFO:tasks.workunit.client.0.smithi094.stdout:5/671: rename db/d7c/d98/fa3 to db/d19/d30/d83/fe5 0 2024-04-06T03:40:44.104 INFO:tasks.workunit.client.1.smithi136.stdout:6/686: rmdir d2/d7/d26/d30 39 2024-04-06T03:40:44.106 INFO:tasks.workunit.client.1.smithi136.stdout:7/836: write d5/df/d1d/f57 [295790,38136] 0 2024-04-06T03:40:44.108 INFO:tasks.workunit.client.1.smithi136.stdout:9/771: rename d17/d20/d25/da4/dbb/le7 to d17/d20/d25/da4/dbb/l10a 0 2024-04-06T03:40:44.110 INFO:tasks.workunit.client.1.smithi136.stdout:8/811: mknod d3/da/d26/d35/d39/cf9 0 2024-04-06T03:40:44.113 INFO:tasks.workunit.client.1.smithi136.stdout:6/687: rmdir d2/d7/d26/d30/d6e/d3d/d47 39 2024-04-06T03:40:44.113 INFO:tasks.workunit.client.1.smithi136.stdout:6/688: readlink d2/d7/d13/d79/dc2/ld1 0 2024-04-06T03:40:44.115 INFO:tasks.workunit.client.0.smithi094.stdout:3/716: link d9/d1e/d6e/fc8 d9/d1e/d6e/d75/f104 0 2024-04-06T03:40:44.115 INFO:tasks.workunit.client.0.smithi094.stdout:3/717: truncate d9/d1e/d6e/d75/dba/d7c/da2/fc9 868420 0 2024-04-06T03:40:44.115 INFO:tasks.workunit.client.0.smithi094.stdout:3/718: creat d9/d1e/d6e/d75/d90/f105 x:0 0 0 2024-04-06T03:40:44.116 INFO:tasks.workunit.client.1.smithi136.stdout:5/827: creat d1/d9b/ff8 x:0 0 0 2024-04-06T03:40:44.117 INFO:tasks.workunit.client.1.smithi136.stdout:7/837: dread d5/df/d12/d1b/d3a/d4a/d70/ddb/faf [0,4194304] 0 2024-04-06T03:40:44.120 INFO:tasks.workunit.client.0.smithi094.stdout:6/688: rename dc/dd3/d72/d88/da3/dc4/lea to dc/d3a/d4a/d54/lf7 0 2024-04-06T03:40:44.122 INFO:tasks.workunit.client.0.smithi094.stdout:5/672: mkdir db/d19/d2c/d6d/de6 0 2024-04-06T03:40:44.123 INFO:tasks.workunit.client.1.smithi136.stdout:5/828: mknod d1/dc/d1c/cf9 0 2024-04-06T03:40:44.123 INFO:tasks.workunit.client.1.smithi136.stdout:5/829: dread - d1/d9b/dea/ff2 zero size 2024-04-06T03:40:44.128 INFO:tasks.workunit.client.1.smithi136.stdout:8/812: dwrite d3/de/d18/d7d/ddf/f54 [0,4194304] 0 2024-04-06T03:40:44.128 INFO:tasks.workunit.client.1.smithi136.stdout:7/838: mkdir d5/df/d1d/d6f/d127 0 2024-04-06T03:40:44.128 INFO:tasks.workunit.client.1.smithi136.stdout:7/839: chown d5/df/d12/d1b/d3a/d4a/lb2 111 1 2024-04-06T03:40:44.129 INFO:tasks.workunit.client.0.smithi094.stdout:5/673: mkdir db/d19/d2c/d6d/de7 0 2024-04-06T03:40:44.129 INFO:tasks.workunit.client.0.smithi094.stdout:5/674: chown db/d19/d24/d28/d85/lab 8705490 1 2024-04-06T03:40:44.131 INFO:tasks.workunit.client.1.smithi136.stdout:5/830: unlink d1/d9b/d9c/fbe 0 2024-04-06T03:40:44.133 INFO:tasks.workunit.client.0.smithi094.stdout:5/675: mknod db/d19/d2c/d57/ce8 0 2024-04-06T03:40:44.134 INFO:tasks.workunit.client.1.smithi136.stdout:6/689: dwrite d2/d7/d26/d30/f7e [4194304,4194304] 0 2024-04-06T03:40:44.134 INFO:tasks.workunit.client.1.smithi136.stdout:6/690: chown d2/d7/lc 29167896 1 2024-04-06T03:40:44.134 INFO:tasks.workunit.client.1.smithi136.stdout:7/840: dread d5/df/d53/d8a/f11d [0,4194304] 0 2024-04-06T03:40:44.136 INFO:tasks.workunit.client.1.smithi136.stdout:8/813: creat d3/d2d/dad/ffa x:0 0 0 2024-04-06T03:40:44.136 INFO:tasks.workunit.client.1.smithi136.stdout:8/814: stat d3/de/f3a 0 2024-04-06T03:40:44.137 INFO:tasks.workunit.client.1.smithi136.stdout:8/815: write d3/f2a [1371693,19074] 0 2024-04-06T03:40:44.137 INFO:tasks.workunit.client.1.smithi136.stdout:5/831: dread d1/dc/d1c/d32/d70/fe8 [0,4194304] 0 2024-04-06T03:40:44.137 INFO:tasks.workunit.client.1.smithi136.stdout:9/772: dwrite d17/d20/d28/d49/d75/f91 [0,4194304] 0 2024-04-06T03:40:44.140 INFO:tasks.workunit.client.0.smithi094.stdout:3/719: dwrite d9/d1e/d26/d6c/ddd/ffc [0,4194304] 0 2024-04-06T03:40:44.140 INFO:tasks.workunit.client.1.smithi136.stdout:8/816: mkdir d3/da/d26/dfb 0 2024-04-06T03:40:44.140 INFO:tasks.workunit.client.1.smithi136.stdout:8/817: chown d3/da/fb 12 1 2024-04-06T03:40:44.142 INFO:tasks.workunit.client.1.smithi136.stdout:5/832: mknod d1/d9b/d9c/cfa 0 2024-04-06T03:40:44.143 INFO:tasks.workunit.client.0.smithi094.stdout:3/720: dread d9/d1e/d26/d55/f65 [0,4194304] 0 2024-04-06T03:40:44.144 INFO:tasks.workunit.client.0.smithi094.stdout:6/689: dwrite dc/d3a/d4a/f56 [0,4194304] 0 2024-04-06T03:40:44.144 INFO:tasks.workunit.client.0.smithi094.stdout:6/690: creat dc/d3a/d4a/d54/dde/d50/d61/ff8 x:0 0 0 2024-04-06T03:40:44.144 INFO:tasks.workunit.client.0.smithi094.stdout:6/691: chown dc/l15 85909746 1 2024-04-06T03:40:44.148 INFO:tasks.workunit.client.1.smithi136.stdout:9/773: creat d17/d21/d29/d34/d66/de1/d51/d57/f10b x:0 0 0 2024-04-06T03:40:44.149 INFO:tasks.workunit.client.0.smithi094.stdout:5/676: dread db/d80/f97 [0,4194304] 0 2024-04-06T03:40:44.150 INFO:tasks.workunit.client.1.smithi136.stdout:5/833: rmdir d1/dc/d1c/d32/d8b/db8 39 2024-04-06T03:40:44.153 INFO:tasks.workunit.client.1.smithi136.stdout:9/774: link d17/d20/d28/d49/d75/d60/lea d17/d20/d25/d3c/l10c 0 2024-04-06T03:40:44.153 INFO:tasks.workunit.client.1.smithi136.stdout:9/775: write d17/d20/d25/da4/dbb/fc4 [282168,23255] 0 2024-04-06T03:40:44.154 INFO:tasks.workunit.client.1.smithi136.stdout:7/841: dwrite d5/df/d53/d68/da3/dd0/d106/f117 [12582912,4194304] 0 2024-04-06T03:40:44.154 INFO:tasks.workunit.client.1.smithi136.stdout:7/842: chown d5/df/d1d/d6f/d2a/f10a 4116518 1 2024-04-06T03:40:44.154 INFO:tasks.workunit.client.0.smithi094.stdout:3/721: rename d9/d1e/c1f to d9/d20/d76/c106 0 2024-04-06T03:40:44.154 INFO:tasks.workunit.client.0.smithi094.stdout:3/722: fsync d9/d20/f34 0 2024-04-06T03:40:44.155 INFO:tasks.workunit.client.1.smithi136.stdout:6/691: dwrite d2/d7/d26/d30/d6e/d3d/d47/f8c [0,4194304] 0 2024-04-06T03:40:44.156 INFO:tasks.workunit.client.1.smithi136.stdout:5/834: creat d1/ffb x:0 0 0 2024-04-06T03:40:44.156 INFO:tasks.workunit.client.1.smithi136.stdout:5/835: stat d1/d16/d89/daf 0 2024-04-06T03:40:44.156 INFO:tasks.workunit.client.1.smithi136.stdout:7/843: dread d5/df/d53/d68/da3/fe5 [0,4194304] 0 2024-04-06T03:40:44.156 INFO:tasks.workunit.client.1.smithi136.stdout:7/844: dread - d5/df/d3d/f8e zero size 2024-04-06T03:40:44.157 INFO:tasks.workunit.client.1.smithi136.stdout:8/818: dwrite d3/da/d26/d35/d39/f65 [0,4194304] 0 2024-04-06T03:40:44.157 INFO:tasks.workunit.client.1.smithi136.stdout:8/819: chown d3/d2d/l5d 172 1 2024-04-06T03:40:44.157 INFO:tasks.workunit.client.1.smithi136.stdout:1/757: mkdir d12/d101 0 2024-04-06T03:40:44.157 INFO:tasks.workunit.client.1.smithi136.stdout:1/758: chown d12/d3b/d60/d6d/da2 6943206 1 2024-04-06T03:40:44.160 INFO:tasks.workunit.client.1.smithi136.stdout:9/776: link d17/d21/d29/d34/d66/de1/d51/d57/f10b d17/d21/d29/de5/da5/f10d 0 2024-04-06T03:40:44.160 INFO:tasks.workunit.client.1.smithi136.stdout:1/759: dread d12/d3b/d44/d5f/fa0 [0,4194304] 0 2024-04-06T03:40:44.160 INFO:tasks.workunit.client.1.smithi136.stdout:9/777: write d17/d21/d29/d34/d66/de1/d51/d57/f10b [388991,37372] 0 2024-04-06T03:40:44.160 INFO:tasks.workunit.client.1.smithi136.stdout:1/760: fsync d12/d3b/d44/d5f/f63 0 2024-04-06T03:40:44.160 INFO:tasks.workunit.client.1.smithi136.stdout:9/778: write d17/d20/d28/d49/d75/d60/f7d [6160446,60128] 0 2024-04-06T03:40:44.166 INFO:tasks.workunit.client.1.smithi136.stdout:6/692: write d2/d7/d26/d30/dc4/fda [613228,50056] 0 2024-04-06T03:40:44.167 INFO:tasks.workunit.client.1.smithi136.stdout:8/820: dread d3/da/d26/d35/f89 [4194304,4194304] 0 2024-04-06T03:40:44.167 INFO:tasks.workunit.client.0.smithi094.stdout:6/692: unlink f8 0 2024-04-06T03:40:44.167 INFO:tasks.workunit.client.0.smithi094.stdout:6/693: fsync dc/dd3/fb4 0 2024-04-06T03:40:44.170 INFO:tasks.workunit.client.0.smithi094.stdout:9/681: mkdir d2/df9 0 2024-04-06T03:40:44.170 INFO:tasks.workunit.client.0.smithi094.stdout:9/682: truncate d2/dc/d80/db7/fe4 86095 0 2024-04-06T03:40:44.170 INFO:tasks.workunit.client.0.smithi094.stdout:9/683: chown d2/dc/d21/d26/d78/d69/da8/d96/f9f 222251937 1 2024-04-06T03:40:44.170 INFO:tasks.workunit.client.0.smithi094.stdout:5/677: creat db/d19/dc1/dd4/fe9 x:0 0 0 2024-04-06T03:40:44.173 INFO:tasks.workunit.client.1.smithi136.stdout:5/836: unlink d1/d16/d4b/l55 0 2024-04-06T03:40:44.176 INFO:tasks.workunit.client.0.smithi094.stdout:6/694: creat dc/d3a/df4/df6/ff9 x:0 0 0 2024-04-06T03:40:44.176 INFO:tasks.workunit.client.0.smithi094.stdout:6/695: truncate dc/dd3/fdc 1027681 0 2024-04-06T03:40:44.177 INFO:tasks.workunit.client.1.smithi136.stdout:7/845: dwrite d5/df/f41 [0,4194304] 0 2024-04-06T03:40:44.177 INFO:tasks.workunit.client.0.smithi094.stdout:3/723: dwrite d9/d1e/d6e/d75/dba/d3a/f40 [0,4194304] 0 2024-04-06T03:40:44.178 INFO:tasks.workunit.client.0.smithi094.stdout:9/684: mkdir d2/dc/d21/d23/d47/dfa 0 2024-04-06T03:40:44.178 INFO:tasks.workunit.client.0.smithi094.stdout:9/685: stat d2/c4f 0 2024-04-06T03:40:44.180 INFO:tasks.workunit.client.1.smithi136.stdout:1/761: symlink d12/d1b/da7/dcc/l102 0 2024-04-06T03:40:44.180 INFO:tasks.workunit.client.1.smithi136.stdout:1/762: stat d12/d1b/da7/dcc 0 2024-04-06T03:40:44.180 INFO:tasks.workunit.client.1.smithi136.stdout:1/763: fsync d12/d3b/d44/f67 0 2024-04-06T03:40:44.183 INFO:tasks.workunit.client.1.smithi136.stdout:9/779: symlink d17/d19/da3/dc5/l10e 0 2024-04-06T03:40:44.185 INFO:tasks.workunit.client.0.smithi094.stdout:6/696: rename dc/dab/cd6 to dc/d11/d97/cfa 0 2024-04-06T03:40:44.187 INFO:tasks.workunit.client.0.smithi094.stdout:9/686: dread d2/dc/d21/d23/d5b/f89 [0,4194304] 0 2024-04-06T03:40:44.188 INFO:tasks.workunit.client.0.smithi094.stdout:3/724: creat d9/d1e/d6e/f107 x:0 0 0 2024-04-06T03:40:44.189 INFO:tasks.workunit.client.0.smithi094.stdout:5/678: dread db/f23 [0,4194304] 0 2024-04-06T03:40:44.189 INFO:tasks.workunit.client.1.smithi136.stdout:6/693: symlink d2/d7/d26/d30/d6e/d58/d5d/d62/le1 0 2024-04-06T03:40:44.191 INFO:tasks.workunit.client.0.smithi094.stdout:6/697: mkdir dc/dd3/d72/d88/dfb 0 2024-04-06T03:40:44.193 INFO:tasks.workunit.client.1.smithi136.stdout:8/821: unlink d3/de/d18/d7d/de0/le9 0 2024-04-06T03:40:44.194 INFO:tasks.workunit.client.0.smithi094.stdout:9/687: unlink d2/dc/d21/d26/d78/d69/cd2 0 2024-04-06T03:40:44.195 INFO:tasks.workunit.client.1.smithi136.stdout:5/837: getdents d1/dc/d1c/d32 0 2024-04-06T03:40:44.196 INFO:tasks.workunit.client.0.smithi094.stdout:3/725: write d9/f3f [797680,73440] 0 2024-04-06T03:40:44.197 INFO:tasks.workunit.client.0.smithi094.stdout:5/679: truncate db/fd 1185296 0 2024-04-06T03:40:44.198 INFO:tasks.workunit.client.1.smithi136.stdout:7/846: link d5/df/d1d/f7f d5/df/d53/d68/d11c/f128 0 2024-04-06T03:40:44.199 INFO:tasks.workunit.client.0.smithi094.stdout:3/726: dread d9/d10/fd8 [0,4194304] 0 2024-04-06T03:40:44.199 INFO:tasks.workunit.client.1.smithi136.stdout:7/847: dread d5/df/d12/d1b/d58/f7a [0,4194304] 0 2024-04-06T03:40:44.201 INFO:tasks.workunit.client.0.smithi094.stdout:6/698: rename dc/dd3/d4e/fcf to dc/dd3/d72/d88/da3/dc4/ffc 0 2024-04-06T03:40:44.201 INFO:tasks.workunit.client.0.smithi094.stdout:6/699: chown dc/dd3/f1f 8311 1 2024-04-06T03:40:44.207 INFO:tasks.workunit.client.0.smithi094.stdout:9/688: mknod d2/dc/d21/d23/d47/d51/cfb 0 2024-04-06T03:40:44.207 INFO:tasks.workunit.client.1.smithi136.stdout:9/780: truncate d17/d20/f48 319364 0 2024-04-06T03:40:44.208 INFO:tasks.workunit.client.1.smithi136.stdout:6/694: symlink d2/d7/d26/d30/dc4/le2 0 2024-04-06T03:40:44.209 INFO:tasks.workunit.client.0.smithi094.stdout:6/700: dread dc/dd3/f52 [0,4194304] 0 2024-04-06T03:40:44.211 INFO:tasks.workunit.client.0.smithi094.stdout:9/689: write d2/dc/d21/d23/d5b/f89 [1189294,127416] 0 2024-04-06T03:40:44.212 INFO:tasks.workunit.client.1.smithi136.stdout:8/822: mknod d3/da/d26/d35/d3d/dc7/dce/df2/cfc 0 2024-04-06T03:40:44.215 INFO:tasks.workunit.client.1.smithi136.stdout:5/838: getdents d1/dc 0 2024-04-06T03:40:44.215 INFO:tasks.workunit.client.1.smithi136.stdout:5/839: write d1/d98/dbc/fdc [688275,94013] 0 2024-04-06T03:40:44.216 INFO:tasks.workunit.client.0.smithi094.stdout:3/727: dwrite d9/d1e/d6e/d75/dba/d7c/faa [0,4194304] 0 2024-04-06T03:40:44.216 INFO:tasks.workunit.client.0.smithi094.stdout:3/728: write d9/f3f [3856625,105471] 0 2024-04-06T03:40:44.216 INFO:tasks.workunit.client.0.smithi094.stdout:3/729: readlink d9/d1e/d26/d55/ld9 0 2024-04-06T03:40:44.216 INFO:tasks.workunit.client.0.smithi094.stdout:5/680: symlink db/d19/lea 0 2024-04-06T03:40:44.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/681: dread db/d19/d21/f93 [0,4194304] 0 2024-04-06T03:40:44.218 INFO:tasks.workunit.client.1.smithi136.stdout:1/764: dwrite d12/f24 [0,4194304] 0 2024-04-06T03:40:44.218 INFO:tasks.workunit.client.1.smithi136.stdout:1/765: chown d12/d3b/d60/d7e/d37/c5b 15 1 2024-04-06T03:40:44.219 INFO:tasks.workunit.client.0.smithi094.stdout:6/701: getdents dc/d3a/d4a 0 2024-04-06T03:40:44.219 INFO:tasks.workunit.client.0.smithi094.stdout:6/702: chown dc/dd3/d24/f32 14 1 2024-04-06T03:40:44.220 INFO:tasks.workunit.client.0.smithi094.stdout:6/703: creat dc/dd3/d4e/ffd x:0 0 0 2024-04-06T03:40:44.223 INFO:tasks.workunit.client.1.smithi136.stdout:7/848: rename d5/df/d53/fb9 to d5/df/d12/d1b/dbb/f129 0 2024-04-06T03:40:44.225 INFO:tasks.workunit.client.1.smithi136.stdout:9/781: symlink d17/d20/d28/d81/d88/dcb/l10f 0 2024-04-06T03:40:44.227 INFO:tasks.workunit.client.1.smithi136.stdout:6/695: link d2/d7/d26/d30/c43 d2/d7/d26/d68/d9b/ce3 0 2024-04-06T03:40:44.230 INFO:tasks.workunit.client.0.smithi094.stdout:3/730: creat d9/d10/d9b/db7/f108 x:0 0 0 2024-04-06T03:40:44.231 INFO:tasks.workunit.client.1.smithi136.stdout:8/823: dwrite d3/da/fb [8388608,4194304] 0 2024-04-06T03:40:44.231 INFO:tasks.workunit.client.1.smithi136.stdout:1/766: symlink d12/d3b/d74/l103 0 2024-04-06T03:40:44.231 INFO:tasks.workunit.client.1.smithi136.stdout:1/767: fsync d12/d3b/d44/d5f/fdb 0 2024-04-06T03:40:44.232 INFO:tasks.workunit.client.0.smithi094.stdout:5/682: truncate db/d80/f97 4036250 0 2024-04-06T03:40:44.232 INFO:tasks.workunit.client.0.smithi094.stdout:5/683: write db/d19/dc1/dd4/d77/fbb [239389,20070] 0 2024-04-06T03:40:44.234 INFO:tasks.workunit.client.1.smithi136.stdout:9/782: write fa [7817313,99709] 0 2024-04-06T03:40:44.234 INFO:tasks.workunit.client.1.smithi136.stdout:9/783: fsync d17/d20/d25/f31 0 2024-04-06T03:40:44.235 INFO:tasks.workunit.client.1.smithi136.stdout:9/784: write d17/d21/d29/d34/d66/de1/d51/d57/f10b [762482,8361] 0 2024-04-06T03:40:44.235 INFO:tasks.workunit.client.1.smithi136.stdout:9/785: fsync d17/d20/d28/d81/f83 0 2024-04-06T03:40:44.235 INFO:tasks.workunit.client.1.smithi136.stdout:9/786: write d17/d20/d28/f98 [1347520,57282] 0 2024-04-06T03:40:44.235 INFO:tasks.workunit.client.1.smithi136.stdout:1/768: dread d12/d3b/d44/fa5 [0,4194304] 0 2024-04-06T03:40:44.235 INFO:tasks.workunit.client.1.smithi136.stdout:5/840: rename d1/dc/d1c/d32/d8b/de3 to d1/d16/d89/d8a/dfc 0 2024-04-06T03:40:44.237 INFO:tasks.workunit.client.1.smithi136.stdout:7/849: symlink d5/df/d1d/d6f/d4d/l12a 0 2024-04-06T03:40:44.239 INFO:tasks.workunit.client.0.smithi094.stdout:5/684: write db/d19/d2c/f31 [5064366,95952] 0 2024-04-06T03:40:44.239 INFO:tasks.workunit.client.1.smithi136.stdout:8/824: dread d3/de/f56 [0,4194304] 0 2024-04-06T03:40:44.240 INFO:tasks.workunit.client.1.smithi136.stdout:6/696: unlink d2/d25/f3b 0 2024-04-06T03:40:44.241 INFO:tasks.workunit.client.1.smithi136.stdout:9/787: read d17/d21/d32/d3f/fb4 [3273199,46631] 0 2024-04-06T03:40:44.243 INFO:tasks.workunit.client.0.smithi094.stdout:6/704: rename dc/d3a/d4a/d54/dde/d50/d61/lc1 to dc/d3a/df4/df6/d7b/dc6/lfe 0 2024-04-06T03:40:44.245 INFO:tasks.workunit.client.0.smithi094.stdout:3/731: mkdir d9/d1e/d109 0 2024-04-06T03:40:44.246 INFO:tasks.workunit.client.1.smithi136.stdout:1/769: mknod d12/d3b/d44/c104 0 2024-04-06T03:40:44.249 INFO:tasks.workunit.client.1.smithi136.stdout:5/841: unlink d1/dc/d1c/d32/feb 0 2024-04-06T03:40:44.250 INFO:tasks.workunit.client.0.smithi094.stdout:5/685: mkdir db/d80/deb 0 2024-04-06T03:40:44.251 INFO:tasks.workunit.client.1.smithi136.stdout:7/850: symlink d5/df/d1d/d6f/l12b 0 2024-04-06T03:40:44.252 INFO:tasks.workunit.client.0.smithi094.stdout:6/705: symlink dc/d3a/df4/df6/df2/lff 0 2024-04-06T03:40:44.252 INFO:tasks.workunit.client.0.smithi094.stdout:6/706: fdatasync dc/dd3/d72/d88/da3/dc4/ffc 0 2024-04-06T03:40:44.253 INFO:tasks.workunit.client.1.smithi136.stdout:8/825: link d3/d2d/f55 d3/da/d26/d35/d39/d70/d7e/ffd 0 2024-04-06T03:40:44.255 INFO:tasks.workunit.client.0.smithi094.stdout:3/732: rename d9/fc to d9/d10/dbb/de1/f10a 0 2024-04-06T03:40:44.259 INFO:tasks.workunit.client.1.smithi136.stdout:6/697: dread d2/d7/d26/f54 [0,4194304] 0 2024-04-06T03:40:44.259 INFO:tasks.workunit.client.1.smithi136.stdout:6/698: dread - d2/d7/d26/d30/d6e/d58/f7d zero size 2024-04-06T03:40:44.260 INFO:tasks.workunit.client.0.smithi094.stdout:6/707: creat dc/d3a/d4a/d94/dac/f100 x:0 0 0 2024-04-06T03:40:44.260 INFO:tasks.workunit.client.0.smithi094.stdout:5/686: dread db/f6f [0,4194304] 0 2024-04-06T03:40:44.261 INFO:tasks.workunit.client.1.smithi136.stdout:9/788: creat d17/d20/d28/d49/d75/d60/f110 x:0 0 0 2024-04-06T03:40:44.262 INFO:tasks.workunit.client.0.smithi094.stdout:3/733: mkdir d9/de5/d10b 0 2024-04-06T03:40:44.262 INFO:tasks.workunit.client.0.smithi094.stdout:3/734: chown d9/d1e/d6e/d75/dba/c43 44478507 1 2024-04-06T03:40:44.262 INFO:tasks.workunit.client.0.smithi094.stdout:3/735: getdents d9/d1e/d6e/d75/dba/dc5 0 2024-04-06T03:40:44.264 INFO:tasks.workunit.client.1.smithi136.stdout:5/842: dwrite d1/dc/d1c/d32/d70/ff7 [0,4194304] 0 2024-04-06T03:40:44.265 INFO:tasks.workunit.client.1.smithi136.stdout:1/770: rename d12/d1b/d1e/dd0/f70 to d12/d3b/d74/dd6/f105 0 2024-04-06T03:40:44.266 INFO:tasks.workunit.client.0.smithi094.stdout:6/708: unlink dc/l15 0 2024-04-06T03:40:44.266 INFO:tasks.workunit.client.0.smithi094.stdout:6/709: creat dc/d3a/d4a/d54/dde/d50/d61/f101 x:0 0 0 2024-04-06T03:40:44.269 INFO:tasks.workunit.client.1.smithi136.stdout:6/699: dread d2/d7/d26/d30/f50 [0,4194304] 0 2024-04-06T03:40:44.269 INFO:tasks.workunit.client.0.smithi094.stdout:5/687: mkdir db/d19/d24/d28/dec 0 2024-04-06T03:40:44.271 INFO:tasks.workunit.client.1.smithi136.stdout:8/826: mknod d3/de/d18/d7d/de0/cfe 0 2024-04-06T03:40:44.271 INFO:tasks.workunit.client.1.smithi136.stdout:8/827: chown d3/da/d26/d90/dae/fed 8327613 1 2024-04-06T03:40:44.271 INFO:tasks.workunit.client.1.smithi136.stdout:6/700: dread d2/d7/d26/d30/d6e/d3d/f6a [0,4194304] 0 2024-04-06T03:40:44.271 INFO:tasks.workunit.client.1.smithi136.stdout:8/828: fdatasync d3/da/d26/d35/dcb/ff4 0 2024-04-06T03:40:44.271 INFO:tasks.workunit.client.1.smithi136.stdout:6/701: chown d2/d7/d26/d30 88998 1 2024-04-06T03:40:44.272 INFO:tasks.workunit.client.0.smithi094.stdout:3/736: creat d9/d9d/dc1/f10c x:0 0 0 2024-04-06T03:40:44.274 INFO:tasks.workunit.client.1.smithi136.stdout:9/789: link d17/d21/d29/d34/d66/c6b d17/d19/c111 0 2024-04-06T03:40:44.274 INFO:tasks.workunit.client.1.smithi136.stdout:9/790: fsync d17/d20/f95 0 2024-04-06T03:40:44.279 INFO:tasks.workunit.client.0.smithi094.stdout:6/710: unlink dc/dd3/d24/fe5 0 2024-04-06T03:40:44.281 INFO:tasks.workunit.client.0.smithi094.stdout:5/688: mknod db/d19/dc1/dd4/ced 0 2024-04-06T03:40:44.281 INFO:tasks.workunit.client.0.smithi094.stdout:5/689: stat db/c60 0 2024-04-06T03:40:44.281 INFO:tasks.workunit.client.0.smithi094.stdout:5/690: chown db/d19/d2c/fc2 437925633 1 2024-04-06T03:40:44.281 INFO:tasks.workunit.client.0.smithi094.stdout:5/691: creat db/d19/d30/d83/fee x:0 0 0 2024-04-06T03:40:44.284 INFO:tasks.workunit.client.0.smithi094.stdout:3/737: dwrite d9/d10/f16 [4194304,4194304] 0 2024-04-06T03:40:44.285 INFO:tasks.workunit.client.1.smithi136.stdout:5/843: mknod d1/d98/cfd 0 2024-04-06T03:40:44.290 INFO:tasks.workunit.client.1.smithi136.stdout:1/771: link d12/d3b/d44/d5f/db7/fed d12/d53/dc5/f106 0 2024-04-06T03:40:44.290 INFO:tasks.workunit.client.1.smithi136.stdout:1/772: write d12/d1b/d1e/d23/f92 [404692,5427] 0 2024-04-06T03:40:44.291 INFO:tasks.workunit.client.1.smithi136.stdout:1/773: fdatasync d12/d3b/d74/d7a/dea/ff4 0 2024-04-06T03:40:44.292 INFO:tasks.workunit.client.0.smithi094.stdout:5/692: creat db/d19/d2c/d6d/de6/fef x:0 0 0 2024-04-06T03:40:44.292 INFO:tasks.workunit.client.0.smithi094.stdout:5/693: write db/f76 [380256,108030] 0 2024-04-06T03:40:44.293 INFO:tasks.workunit.client.0.smithi094.stdout:6/711: dwrite dc/dd3/f1f [0,4194304] 0 2024-04-06T03:40:44.294 INFO:tasks.workunit.client.1.smithi136.stdout:7/851: rmdir d5/df/d12/d1b/d3a/d8d 39 2024-04-06T03:40:44.295 INFO:tasks.workunit.client.0.smithi094.stdout:5/694: link db/d19/d30/d83/fe1 db/d7c/dd7/ff0 0 2024-04-06T03:40:44.295 INFO:tasks.workunit.client.0.smithi094.stdout:3/738: dwrite d9/d20/d30/f69 [4194304,4194304] 0 2024-04-06T03:40:44.296 INFO:tasks.workunit.client.1.smithi136.stdout:8/829: unlink d3/d2d/dad/cc8 0 2024-04-06T03:40:44.297 INFO:tasks.workunit.client.1.smithi136.stdout:8/830: truncate d3/da/d26/d35/f38 4429790 0 2024-04-06T03:40:44.299 INFO:tasks.workunit.client.0.smithi094.stdout:6/712: creat dc/dd3/d72/d88/f102 x:0 0 0 2024-04-06T03:40:44.300 INFO:tasks.workunit.client.1.smithi136.stdout:6/702: creat d2/d7/d26/d68/ddc/fe4 x:0 0 0 2024-04-06T03:40:44.305 INFO:tasks.workunit.client.0.smithi094.stdout:5/695: dwrite db/d19/d24/f89 [0,4194304] 0 2024-04-06T03:40:44.306 INFO:tasks.workunit.client.0.smithi094.stdout:5/696: chown db/d19/d2c/f2d 1 1 2024-04-06T03:40:44.306 INFO:tasks.workunit.client.1.smithi136.stdout:5/844: creat d1/d16/ffe x:0 0 0 2024-04-06T03:40:44.306 INFO:tasks.workunit.client.1.smithi136.stdout:5/845: chown d1/dc/d10/da3 5753 1 2024-04-06T03:40:44.307 INFO:tasks.workunit.client.0.smithi094.stdout:3/739: rmdir d9/d20/d30 39 2024-04-06T03:40:44.307 INFO:tasks.workunit.client.0.smithi094.stdout:3/740: write d9/d20/d30/f37 [695468,72242] 0 2024-04-06T03:40:44.307 INFO:tasks.workunit.client.0.smithi094.stdout:3/741: write d9/d20/d2d/da9/fe0 [890586,79029] 0 2024-04-06T03:40:44.307 INFO:tasks.workunit.client.0.smithi094.stdout:3/742: chown d9/d1e/d6e/d75/d90/fa0 0 1 2024-04-06T03:40:44.307 INFO:tasks.workunit.client.0.smithi094.stdout:3/743: chown d9/d10/f47 24566 1 2024-04-06T03:40:44.309 INFO:tasks.workunit.client.1.smithi136.stdout:1/774: rename d12/d3b/d74/l82 to d12/d3b/d44/d5f/d5a/de2/l107 0 2024-04-06T03:40:44.310 INFO:tasks.workunit.client.0.smithi094.stdout:6/713: mkdir dc/d3a/d4a/d54/d71/d103 0 2024-04-06T03:40:44.310 INFO:tasks.workunit.client.0.smithi094.stdout:6/714: dread - dc/d3a/d4a/d54/dde/d50/daf/fba zero size 2024-04-06T03:40:44.310 INFO:tasks.workunit.client.0.smithi094.stdout:6/715: truncate dc/d3a/d4a/d54/dde/d50/fbf 703966 0 2024-04-06T03:40:44.312 INFO:tasks.workunit.client.1.smithi136.stdout:7/852: creat d5/df/d12/d1b/d3a/d8d/f12c x:0 0 0 2024-04-06T03:40:44.312 INFO:tasks.workunit.client.1.smithi136.stdout:7/853: chown d5/df/d12/d1b/f30 1361725 1 2024-04-06T03:40:44.314 INFO:tasks.workunit.client.1.smithi136.stdout:8/831: mkdir d3/da/d26/dfb/dff 0 2024-04-06T03:40:44.355 INFO:tasks.workunit.client.1.smithi136.stdout:6/703: mknod d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/d94/ce5 0 2024-04-06T03:40:44.356 INFO:tasks.workunit.client.1.smithi136.stdout:1/775: dwrite d12/d1b/d22/f43 [0,4194304] 0 2024-04-06T03:40:44.357 INFO:tasks.workunit.client.0.smithi094.stdout:5/697: link db/d19/d24/d2e/cd2 db/d7c/d98/cf1 0 2024-04-06T03:40:44.357 INFO:tasks.workunit.client.1.smithi136.stdout:9/791: rmdir d17/d20/d25/d71/dd5 39 2024-04-06T03:40:44.358 INFO:tasks.workunit.client.0.smithi094.stdout:6/716: dwrite dc/dd3/d72/d88/da3/dc4/ffc [0,4194304] 0 2024-04-06T03:40:44.364 INFO:tasks.workunit.client.0.smithi094.stdout:3/744: unlink d9/d10/d9b/lda 0 2024-04-06T03:40:44.366 INFO:tasks.workunit.client.1.smithi136.stdout:5/846: rename d1/dc/d1c/d32/d8b/f73 to d1/d9b/dea/fff 0 2024-04-06T03:40:44.367 INFO:tasks.workunit.client.0.smithi094.stdout:5/698: mknod db/d7c/d98/cf2 0 2024-04-06T03:40:44.369 INFO:tasks.workunit.client.1.smithi136.stdout:7/854: mknod d5/df/c12d 0 2024-04-06T03:40:44.371 INFO:tasks.workunit.client.1.smithi136.stdout:8/832: mknod d3/de/d18/d7d/ddf/d6c/d7f/c100 0 2024-04-06T03:40:44.371 INFO:tasks.workunit.client.1.smithi136.stdout:8/833: write d3/da/d26/d35/d53/dd2/fe3 [403641,20055] 0 2024-04-06T03:40:44.372 INFO:tasks.workunit.client.0.smithi094.stdout:6/717: dwrite dc/dd3/f38 [0,4194304] 0 2024-04-06T03:40:44.372 INFO:tasks.workunit.client.0.smithi094.stdout:6/718: write dc/d3a/d4a/d54/d71/fcb [4538630,63315] 0 2024-04-06T03:40:44.372 INFO:tasks.workunit.client.1.smithi136.stdout:9/792: dwrite d17/d21/d29/d34/d66/de1/d51/d57/fdb [0,4194304] 0 2024-04-06T03:40:44.375 INFO:tasks.workunit.client.0.smithi094.stdout:3/745: symlink d9/d1e/d109/l10d 0 2024-04-06T03:40:44.375 INFO:tasks.workunit.client.0.smithi094.stdout:3/746: fdatasync d9/f19 0 2024-04-06T03:40:44.375 INFO:tasks.workunit.client.0.smithi094.stdout:3/747: fsync d9/d10/d7e/da3/fdc 0 2024-04-06T03:40:44.375 INFO:tasks.workunit.client.0.smithi094.stdout:3/748: dread - d9/d1d/d7d/fbe zero size 2024-04-06T03:40:44.376 INFO:tasks.workunit.client.1.smithi136.stdout:1/776: symlink d12/d3b/d44/d5f/d5a/l108 0 2024-04-06T03:40:44.376 INFO:tasks.workunit.client.0.smithi094.stdout:5/699: truncate db/d19/d30/f8e 708168 0 2024-04-06T03:40:44.379 INFO:tasks.workunit.client.1.smithi136.stdout:5/847: mknod d1/dc/d1c/d32/d70/dc1/c100 0 2024-04-06T03:40:44.383 INFO:tasks.workunit.client.1.smithi136.stdout:7/855: rename d5/df/d12/d1b/f47 to d5/df/d53/d68/d8f/f12e 0 2024-04-06T03:40:44.383 INFO:tasks.workunit.client.1.smithi136.stdout:7/856: creat d5/df/d12/d1b/d3a/d4a/ddf/f12f x:0 0 0 2024-04-06T03:40:44.383 INFO:tasks.workunit.client.1.smithi136.stdout:7/857: chown d5/df/d53/d8a/fe8 4100410 1 2024-04-06T03:40:44.384 INFO:tasks.workunit.client.0.smithi094.stdout:6/719: unlink dc/c21 0 2024-04-06T03:40:44.385 INFO:tasks.workunit.client.1.smithi136.stdout:0/758: sync 2024-04-06T03:40:44.385 INFO:tasks.workunit.client.1.smithi136.stdout:4/665: sync 2024-04-06T03:40:44.385 INFO:tasks.workunit.client.1.smithi136.stdout:3/729: sync 2024-04-06T03:40:44.385 INFO:tasks.workunit.client.1.smithi136.stdout:2/812: sync 2024-04-06T03:40:44.385 INFO:tasks.workunit.client.1.smithi136.stdout:4/666: chown d1d/d31/d3e/d55 198 1 2024-04-06T03:40:44.386 INFO:tasks.workunit.client.1.smithi136.stdout:8/834: creat d3/da/d26/d35/d53/f101 x:0 0 0 2024-04-06T03:40:44.388 INFO:tasks.workunit.client.0.smithi094.stdout:5/700: read db/d19/d24/f9d [2921557,25495] 0 2024-04-06T03:40:44.388 INFO:tasks.workunit.client.0.smithi094.stdout:5/701: write db/d19/dc1/dd4/f43 [823145,102892] 0 2024-04-06T03:40:44.388 INFO:tasks.workunit.client.0.smithi094.stdout:5/702: rename db to db/d7c/df3 22 2024-04-06T03:40:44.390 INFO:tasks.workunit.client.1.smithi136.stdout:9/793: dwrite d17/d21/d29/f5f [0,4194304] 0 2024-04-06T03:40:44.390 INFO:tasks.workunit.client.1.smithi136.stdout:9/794: chown d17/d21/d32/f74 932 1 2024-04-06T03:40:44.390 INFO:tasks.workunit.client.1.smithi136.stdout:9/795: dread - d17/d21/d29/f78 zero size 2024-04-06T03:40:44.391 INFO:tasks.workunit.client.0.smithi094.stdout:6/720: dread dc/d3a/d4a/f56 [0,4194304] 0 2024-04-06T03:40:44.392 INFO:tasks.workunit.client.1.smithi136.stdout:4/667: dread d1d/d2f/fab [0,4194304] 0 2024-04-06T03:40:44.394 INFO:tasks.workunit.client.0.smithi094.stdout:5/703: rename db/d19/d24/d28/l42 to db/d19/dc1/dd4/d77/lf4 0 2024-04-06T03:40:44.397 INFO:tasks.workunit.client.0.smithi094.stdout:3/749: rmdir d9/d20/d2d/da9 39 2024-04-06T03:40:44.400 INFO:tasks.workunit.client.1.smithi136.stdout:4/668: dread d1d/d31/f58 [0,4194304] 0 2024-04-06T03:40:44.403 INFO:tasks.workunit.client.0.smithi094.stdout:5/704: write db/d19/d24/fdb [1686679,25269] 0 2024-04-06T03:40:44.403 INFO:tasks.workunit.client.0.smithi094.stdout:5/705: rename db/d19 to db/d19/dc1/ddf/df5 22 2024-04-06T03:40:44.403 INFO:tasks.workunit.client.0.smithi094.stdout:5/706: stat db/c60 0 2024-04-06T03:40:44.409 INFO:tasks.workunit.client.1.smithi136.stdout:5/848: mknod d1/d16/d89/d8a/df1/c101 0 2024-04-06T03:40:44.414 INFO:tasks.workunit.client.1.smithi136.stdout:1/777: rename d12/d3b/d44/d5f/l65 to d12/d1b/da7/dd3/l109 0 2024-04-06T03:40:44.414 INFO:tasks.workunit.client.1.smithi136.stdout:1/778: truncate d12/d3b/d74/d7a/dea/ff4 14130 0 2024-04-06T03:40:44.416 INFO:tasks.workunit.client.0.smithi094.stdout:3/750: dwrite d9/d20/d76/f5c [0,4194304] 0 2024-04-06T03:40:44.416 INFO:tasks.workunit.client.0.smithi094.stdout:3/751: write d9/d1d/f39 [692018,115277] 0 2024-04-06T03:40:44.416 INFO:tasks.workunit.client.0.smithi094.stdout:3/752: write d9/d10/d7e/f86 [544852,12169] 0 2024-04-06T03:40:44.416 INFO:tasks.workunit.client.0.smithi094.stdout:3/753: write d9/d1e/d6e/d75/dba/d97/fd0 [220800,121141] 0 2024-04-06T03:40:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:44 smithi094 ceph-mon[29713]: pgmap v22: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 221 MiB/s rd, 277 MiB/s wr, 396 op/s 2024-04-06T03:40:44.418 INFO:tasks.workunit.client.1.smithi136.stdout:7/858: creat d5/df/d12/d1b/d3a/d4a/dc4/f130 x:0 0 0 2024-04-06T03:40:44.419 INFO:tasks.workunit.client.0.smithi094.stdout:4/784: write d0/df/d1e/d34/f39 [798109,54706] 0 2024-04-06T03:40:44.422 INFO:tasks.workunit.client.0.smithi094.stdout:5/707: symlink db/d19/dc1/dd4/dcf/lf6 0 2024-04-06T03:40:44.422 INFO:tasks.workunit.client.0.smithi094.stdout:5/708: chown db/d19/d24/d28/d85/db5/lc9 1094 1 2024-04-06T03:40:44.422 INFO:tasks.workunit.client.0.smithi094.stdout:5/709: dread - db/d7c/d98/f9f zero size 2024-04-06T03:40:44.423 INFO:tasks.workunit.client.0.smithi094.stdout:7/643: sync 2024-04-06T03:40:44.423 INFO:tasks.workunit.client.0.smithi094.stdout:1/709: sync 2024-04-06T03:40:44.423 INFO:tasks.workunit.client.0.smithi094.stdout:8/776: sync 2024-04-06T03:40:44.423 INFO:tasks.workunit.client.0.smithi094.stdout:0/702: sync 2024-04-06T03:40:44.424 INFO:tasks.workunit.client.1.smithi136.stdout:3/730: rmdir d0/d4/d12/d19/d55/d3c 39 2024-04-06T03:40:44.424 INFO:tasks.workunit.client.1.smithi136.stdout:3/731: write d0/d4/fa [1248984,112954] 0 2024-04-06T03:40:44.424 INFO:tasks.workunit.client.1.smithi136.stdout:3/732: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/ffd x:0 0 0 2024-04-06T03:40:44.425 INFO:tasks.workunit.client.0.smithi094.stdout:3/754: dread d9/d1e/f9f [4194304,4194304] 0 2024-04-06T03:40:44.426 INFO:tasks.workunit.client.1.smithi136.stdout:4/669: dwrite d1d/f26 [4194304,4194304] 0 2024-04-06T03:40:44.426 INFO:tasks.workunit.client.1.smithi136.stdout:4/670: getdents d1d/d2f/da6/de1 0 2024-04-06T03:40:44.426 INFO:tasks.workunit.client.1.smithi136.stdout:5/849: dwrite d1/dc/d1c/d32/d8b/f7f [0,4194304] 0 2024-04-06T03:40:44.426 INFO:tasks.workunit.client.1.smithi136.stdout:5/850: stat d1/dc/d1c/d32/f38 0 2024-04-06T03:40:44.427 INFO:tasks.workunit.client.1.smithi136.stdout:2/813: creat d3/d4a/d9d/d18/d1d/d3c/f10c x:0 0 0 2024-04-06T03:40:44.427 INFO:tasks.workunit.client.0.smithi094.stdout:4/785: dread d0/df/d13/f36 [0,4194304] 0 2024-04-06T03:40:44.435 INFO:tasks.workunit.client.0.smithi094.stdout:5/710: unlink db/d19/d2c/d6d/f5f 0 2024-04-06T03:40:44.435 INFO:tasks.workunit.client.1.smithi136.stdout:7/859: dwrite d5/df/d12/d1b/d3a/d4a/dc4/f130 [0,4194304] 0 2024-04-06T03:40:44.436 INFO:tasks.workunit.client.1.smithi136.stdout:3/733: dread d0/d4/d12/d3a/d51/da6/fb6 [0,4194304] 0 2024-04-06T03:40:44.436 INFO:tasks.workunit.client.1.smithi136.stdout:3/734: truncate d0/d4/d12/d19/d2c/f7f 36911 0 2024-04-06T03:40:44.437 INFO:tasks.workunit.client.1.smithi136.stdout:8/835: symlink d3/da/d26/d35/dcb/l102 0 2024-04-06T03:40:44.437 INFO:tasks.workunit.client.0.smithi094.stdout:5/711: dread db/d19/d30/d83/fd3 [0,4194304] 0 2024-04-06T03:40:44.439 INFO:tasks.workunit.client.0.smithi094.stdout:5/712: dread db/d19/d2c/d57/fbe [0,4194304] 0 2024-04-06T03:40:44.440 INFO:tasks.workunit.client.0.smithi094.stdout:1/710: unlink d0/d7/d1a/d64/cd7 0 2024-04-06T03:40:44.440 INFO:tasks.workunit.client.0.smithi094.stdout:1/711: chown d0 4 1 2024-04-06T03:40:44.440 INFO:tasks.workunit.client.0.smithi094.stdout:1/712: creat d0/d1/d27/da5/d40/ff3 x:0 0 0 2024-04-06T03:40:44.441 INFO:tasks.workunit.client.0.smithi094.stdout:3/755: dwrite d9/d1e/f9f [0,4194304] 0 2024-04-06T03:40:44.442 INFO:tasks.workunit.client.1.smithi136.stdout:8/836: dread d3/f5e [0,4194304] 0 2024-04-06T03:40:44.442 INFO:tasks.workunit.client.1.smithi136.stdout:8/837: stat d3/de/d18/d7d/ddf/d6c/dab/fd4 0 2024-04-06T03:40:44.445 INFO:tasks.workunit.client.0.smithi094.stdout:7/644: rename d2/d6/f73 to d2/d6/dc/d12/d61/fc7 0 2024-04-06T03:40:44.448 INFO:tasks.workunit.client.0.smithi094.stdout:8/777: link f13 d18/d2c/d37/d60/d70/f10d 0 2024-04-06T03:40:44.450 INFO:tasks.workunit.client.1.smithi136.stdout:7/860: dwrite d5/df/d53/d68/ffb [4194304,4194304] 0 2024-04-06T03:40:44.450 INFO:tasks.workunit.client.1.smithi136.stdout:9/796: mkdir d17/d20/d25/da4/d112 0 2024-04-06T03:40:44.451 INFO:tasks.workunit.client.0.smithi094.stdout:0/703: getdents d1/d9/dd/d18/d5d/d8d 0 2024-04-06T03:40:44.453 INFO:tasks.workunit.client.0.smithi094.stdout:4/786: mknod d0/d77/c108 0 2024-04-06T03:40:44.453 INFO:tasks.workunit.client.0.smithi094.stdout:3/756: dwrite d9/d20/d76/f5c [0,4194304] 0 2024-04-06T03:40:44.454 INFO:tasks.workunit.client.1.smithi136.stdout:1/779: symlink d12/d3b/d74/d94/l10a 0 2024-04-06T03:40:44.456 INFO:tasks.workunit.client.0.smithi094.stdout:5/713: truncate db/f6f 2396443 0 2024-04-06T03:40:44.456 INFO:tasks.workunit.client.1.smithi136.stdout:1/780: write d12/d3b/d60/d7e/d37/f4e [4620193,90463] 0 2024-04-06T03:40:44.456 INFO:tasks.workunit.client.1.smithi136.stdout:1/781: dread - d12/fe0 zero size 2024-04-06T03:40:44.458 INFO:tasks.workunit.client.1.smithi136.stdout:0/759: creat d1/d6/ded/f103 x:0 0 0 2024-04-06T03:40:44.462 INFO:tasks.workunit.client.0.smithi094.stdout:1/713: symlink d0/d1/d27/da5/d3f/de9/lf4 0 2024-04-06T03:40:44.463 INFO:tasks.workunit.client.0.smithi094.stdout:5/714: dread db/d19/d2c/d57/f5c [4194304,4194304] 0 2024-04-06T03:40:44.465 INFO:tasks.workunit.client.1.smithi136.stdout:4/671: creat d1d/d31/d3e/d55/dba/fee x:0 0 0 2024-04-06T03:40:44.466 INFO:tasks.workunit.client.0.smithi094.stdout:7/645: mkdir d2/d6/dc/d12/d29/d35/d57/dc8 0 2024-04-06T03:40:44.469 INFO:tasks.workunit.client.0.smithi094.stdout:8/778: rename d18/d30/f99 to d18/d3c/d49/d91/d94/df3/d100/f10e 0 2024-04-06T03:40:44.469 INFO:tasks.workunit.client.0.smithi094.stdout:8/779: creat d18/d3c/d49/d91/d94/daa/f10f x:0 0 0 2024-04-06T03:40:44.471 INFO:tasks.workunit.client.1.smithi136.stdout:2/814: symlink d3/d15/d19/d4e/l10d 0 2024-04-06T03:40:44.472 INFO:tasks.workunit.client.0.smithi094.stdout:0/704: mknod d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/ced 0 2024-04-06T03:40:44.475 INFO:tasks.workunit.client.0.smithi094.stdout:4/787: creat d0/d4/f109 x:0 0 0 2024-04-06T03:40:44.480 INFO:tasks.workunit.client.0.smithi094.stdout:8/780: dwrite d18/d3c/d49/d91/d94/daa/fe9 [0,4194304] 0 2024-04-06T03:40:44.480 INFO:tasks.workunit.client.0.smithi094.stdout:8/781: read - d18/d3c/d8b/f10b zero size 2024-04-06T03:40:44.488 INFO:tasks.workunit.client.1.smithi136.stdout:3/735: creat d0/d4/d12/d19/d55/d6f/df0/ffe x:0 0 0 2024-04-06T03:40:44.508 INFO:tasks.workunit.client.1.smithi136.stdout:8/838: symlink d3/da/d26/d35/d39/l103 0 2024-04-06T03:40:44.508 INFO:tasks.workunit.client.1.smithi136.stdout:8/839: dread - d3/da/d26/d90/dae/db5/fe5 zero size 2024-04-06T03:40:44.508 INFO:tasks.workunit.client.1.smithi136.stdout:8/840: fsync d3/da/d26/d35/d39/f99 0 2024-04-06T03:40:44.512 INFO:tasks.workunit.client.0.smithi094.stdout:1/714: mknod d0/d1/d27/da5/d40/cf5 0 2024-04-06T03:40:44.512 INFO:tasks.workunit.client.1.smithi136.stdout:6/704: rmdir d2/d7/d26 39 2024-04-06T03:40:44.512 INFO:tasks.workunit.client.1.smithi136.stdout:6/705: chown d2/d7/d26/d30/c42 406368 1 2024-04-06T03:40:44.512 INFO:tasks.workunit.client.1.smithi136.stdout:6/706: readlink d2/d7/l73 0 2024-04-06T03:40:44.514 INFO:tasks.workunit.client.1.smithi136.stdout:7/861: rename d5/fa1 to d5/d6d/f131 0 2024-04-06T03:40:44.516 INFO:tasks.workunit.client.1.smithi136.stdout:1/782: truncate d12/d1b/d1e/d6f/fca 52833 0 2024-04-06T03:40:44.516 INFO:tasks.workunit.client.1.smithi136.stdout:1/783: stat d12/d3b/d44/ffa 0 2024-04-06T03:40:44.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/784: read d12/d3b/d60/d6d/da2/fb1 [116905,27455] 0 2024-04-06T03:40:44.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/785: dread - d12/fe0 zero size 2024-04-06T03:40:44.517 INFO:tasks.workunit.client.0.smithi094.stdout:7/646: truncate d2/d6/dc/dd/f28 1210026 0 2024-04-06T03:40:44.519 INFO:tasks.workunit.client.1.smithi136.stdout:0/760: mknod d1/d6/d2c/d57/d43/dcf/de1/c104 0 2024-04-06T03:40:44.519 INFO:tasks.workunit.client.1.smithi136.stdout:0/761: dread - d1/d6/d2c/d1a/d36/ffd zero size 2024-04-06T03:40:44.521 INFO:tasks.workunit.client.1.smithi136.stdout:4/672: mknod d1d/d31/d3e/d55/d89/dea/cef 0 2024-04-06T03:40:44.521 INFO:tasks.workunit.client.1.smithi136.stdout:4/673: fdatasync d1d/d31/d3e/d55/fe7 0 2024-04-06T03:40:44.521 INFO:tasks.workunit.client.0.smithi094.stdout:1/715: write d0/d1/d27/da5/d3f/f5c [3161285,75029] 0 2024-04-06T03:40:44.522 INFO:tasks.workunit.client.1.smithi136.stdout:2/815: symlink d3/ddc/l10e 0 2024-04-06T03:40:44.524 INFO:tasks.workunit.client.1.smithi136.stdout:7/862: write d5/df/d12/d1b/f10c [1099285,83999] 0 2024-04-06T03:40:44.524 INFO:tasks.workunit.client.1.smithi136.stdout:7/863: fdatasync d5/df/d1d/d6f/d4d/d69/fc8 0 2024-04-06T03:40:44.524 INFO:tasks.workunit.client.1.smithi136.stdout:7/864: truncate d5/df/d53/d68/da3/dd0/df0/ffe 239000 0 2024-04-06T03:40:44.524 INFO:tasks.workunit.client.0.smithi094.stdout:1/716: dread d0/fde [4194304,4194304] 0 2024-04-06T03:40:44.525 INFO:tasks.workunit.client.0.smithi094.stdout:0/705: unlink d1/d9/dd/d18/d5d/d60/f92 0 2024-04-06T03:40:44.527 INFO:tasks.workunit.client.0.smithi094.stdout:4/788: symlink d0/df/d13/d4a/l10a 0 2024-04-06T03:40:44.533 INFO:tasks.workunit.client.0.smithi094.stdout:5/715: dread db/d19/d30/f8e [0,4194304] 0 2024-04-06T03:40:44.533 INFO:tasks.workunit.client.1.smithi136.stdout:8/841: symlink d3/de/d18/d7d/ddf/d6c/d7f/l104 0 2024-04-06T03:40:44.536 INFO:tasks.workunit.client.1.smithi136.stdout:2/816: dwrite d3/d9/fe [0,4194304] 0 2024-04-06T03:40:44.536 INFO:tasks.workunit.client.1.smithi136.stdout:2/817: stat d3/d4a/d9d/d18/cb8 0 2024-04-06T03:40:44.536 INFO:tasks.workunit.client.1.smithi136.stdout:6/707: creat d2/d7/d26/d30/d6e/d58/d5d/fe6 x:0 0 0 2024-04-06T03:40:44.537 INFO:tasks.workunit.client.0.smithi094.stdout:8/782: creat d18/d3c/d49/d91/d94/df3/f110 x:0 0 0 2024-04-06T03:40:44.549 INFO:tasks.workunit.client.1.smithi136.stdout:1/786: creat d12/d1b/d1e/dd0/f10b x:0 0 0 2024-04-06T03:40:44.549 INFO:tasks.workunit.client.1.smithi136.stdout:2/818: dread d3/d15/d19/f1b [0,4194304] 0 2024-04-06T03:40:44.552 INFO:tasks.workunit.client.1.smithi136.stdout:0/762: creat d1/d6/d2c/d57/d43/dcf/de1/f105 x:0 0 0 2024-04-06T03:40:44.552 INFO:tasks.workunit.client.1.smithi136.stdout:0/763: readlink d1/d6/l100 0 2024-04-06T03:40:44.553 INFO:tasks.workunit.client.1.smithi136.stdout:2/819: dread d3/d4a/d9d/d18/d4b/dbd/fc3 [0,4194304] 0 2024-04-06T03:40:44.554 INFO:tasks.workunit.client.1.smithi136.stdout:4/674: mkdir d1d/d2f/d6c/da5/df0 0 2024-04-06T03:40:44.557 INFO:tasks.workunit.client.0.smithi094.stdout:8/783: dwrite d18/d2c/d37/d60/d70/f10d [0,4194304] 0 2024-04-06T03:40:44.558 INFO:tasks.workunit.client.1.smithi136.stdout:8/842: dwrite d3/da/d26/d35/d53/f5b [4194304,4194304] 0 2024-04-06T03:40:44.558 INFO:tasks.workunit.client.0.smithi094.stdout:0/706: dwrite d1/d9/dd/d18/d1e/d22/f97 [0,4194304] 0 2024-04-06T03:40:44.558 INFO:tasks.workunit.client.0.smithi094.stdout:0/707: creat d1/d9/dd/d18/d1e/d86/d96/daa/fee x:0 0 0 2024-04-06T03:40:44.558 INFO:tasks.workunit.client.0.smithi094.stdout:0/708: fsync d1/d9/dd/d18/d5d/d60/d6b/dab/d4e/ddf/f99 0 2024-04-06T03:40:44.561 INFO:tasks.workunit.client.0.smithi094.stdout:5/716: unlink db/d19/d21/f52 0 2024-04-06T03:40:44.563 INFO:tasks.workunit.client.1.smithi136.stdout:7/865: mkdir d5/df/d53/d68/d132 0 2024-04-06T03:40:44.563 INFO:tasks.workunit.client.1.smithi136.stdout:7/866: chown d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/c83 25084 1 2024-04-06T03:40:44.563 INFO:tasks.workunit.client.1.smithi136.stdout:1/787: write d12/d1b/f35 [836618,99518] 0 2024-04-06T03:40:44.563 INFO:tasks.workunit.client.1.smithi136.stdout:1/788: creat d12/d1b/da7/daa/f10c x:0 0 0 2024-04-06T03:40:44.565 INFO:tasks.workunit.client.1.smithi136.stdout:6/708: mkdir d2/d7/d26/d30/d9c/de7 0 2024-04-06T03:40:44.574 INFO:tasks.workunit.client.0.smithi094.stdout:5/717: unlink db/d19/d30/f81 0 2024-04-06T03:40:44.579 INFO:tasks.workunit.client.1.smithi136.stdout:9/797: rename d17/d21/d29/d34 to d17/d21/d32/dc6/d113 0 2024-04-06T03:40:44.579 INFO:tasks.workunit.client.1.smithi136.stdout:9/798: write d17/d19/da3/dc5/fd3 [164583,5448] 0 2024-04-06T03:40:44.580 INFO:tasks.workunit.client.1.smithi136.stdout:1/789: dwrite d12/d1b/da7/db8/fc7 [0,4194304] 0 2024-04-06T03:40:44.580 INFO:tasks.workunit.client.1.smithi136.stdout:6/709: dwrite d2/d7/d26/d30/d6e/f10 [0,4194304] 0 2024-04-06T03:40:44.581 INFO:tasks.workunit.client.1.smithi136.stdout:4/675: creat d1d/d70/ff1 x:0 0 0 2024-04-06T03:40:44.581 INFO:tasks.workunit.client.1.smithi136.stdout:4/676: chown d1d/d2f/d6c/da5/faa 6927 1 2024-04-06T03:40:44.581 INFO:tasks.workunit.client.1.smithi136.stdout:4/677: creat d1d/d2f/d6c/d91/ff2 x:0 0 0 2024-04-06T03:40:44.585 INFO:tasks.workunit.client.0.smithi094.stdout:7/647: rename d2/d6/dc/d12/d29/d35 to d2/d6/dc/dd/df/d95/dc9 0 2024-04-06T03:40:44.585 INFO:tasks.workunit.client.0.smithi094.stdout:7/648: chown d2/d6/dc/dd/d90/lad 269974721 1 2024-04-06T03:40:44.588 INFO:tasks.workunit.client.1.smithi136.stdout:7/867: creat d5/df/d1d/d6f/d127/f133 x:0 0 0 2024-04-06T03:40:44.597 INFO:tasks.workunit.client.0.smithi094.stdout:1/717: rename d0/d1/d27/da5/d40/db0/dcd to d0/d1/d27/da5/d40/db1/d2f/d76/d30/df6 0 2024-04-06T03:40:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:44 smithi136 ceph-mon[36436]: pgmap v22: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 221 MiB/s rd, 277 MiB/s wr, 396 op/s 2024-04-06T03:40:44.600 INFO:tasks.workunit.client.1.smithi136.stdout:0/764: rename d1/d6/d2c/d57/d43/dcf/de1/f105 to d1/d6/d2c/d57/de3/f106 0 2024-04-06T03:40:44.600 INFO:tasks.workunit.client.1.smithi136.stdout:0/765: chown d1/d6/d2c/d57/d43/d6d/d9b/d5c/ff9 325 1 2024-04-06T03:40:44.601 INFO:tasks.workunit.client.0.smithi094.stdout:4/789: rename d0/d15/d3a/d61/d81/f87 to d0/df/f10b 0 2024-04-06T03:40:44.603 INFO:tasks.workunit.client.0.smithi094.stdout:4/790: dread d0/d15/d3a/d61/d81/da0/fc5 [0,4194304] 0 2024-04-06T03:40:44.604 INFO:tasks.workunit.client.0.smithi094.stdout:1/718: link d0/d1/d27/da5/d40/db1/d2f/d3c/da0/fdf d0/d1/d27/da5/d40/d50/ff7 0 2024-04-06T03:40:44.608 INFO:tasks.workunit.client.0.smithi094.stdout:0/709: rename d1/d9/dd/d18/d5d/d60/d6b to d1/d9/dd/d18/d1e/def 0 2024-04-06T03:40:44.609 INFO:tasks.workunit.client.1.smithi136.stdout:1/790: mkdir d12/d1b/d1e/d4d/d10d 0 2024-04-06T03:40:44.609 INFO:tasks.workunit.client.1.smithi136.stdout:1/791: readlink d12/d3b/d60/d6d/da2/le5 0 2024-04-06T03:40:44.609 INFO:tasks.workunit.client.0.smithi094.stdout:3/757: truncate d9/d20/d76/f5c 540944 0 2024-04-06T03:40:44.612 INFO:tasks.workunit.client.0.smithi094.stdout:5/718: rename db/d19/d24/d2e/l96 to db/d19/dc1/dd4/d77/lf7 0 2024-04-06T03:40:44.618 INFO:tasks.workunit.client.0.smithi094.stdout:1/719: dread d0/d1/d27/da5/d40/db1/d2f/d76/d30/fe4 [0,4194304] 0 2024-04-06T03:40:44.618 INFO:tasks.workunit.client.0.smithi094.stdout:1/720: readlink d0/d1/d27/da5/d40/db1/d2f/d4b/lc2 0 2024-04-06T03:40:44.619 INFO:tasks.workunit.client.0.smithi094.stdout:1/721: mknod d0/d1/d27/da5/d40/db1/d2f/d76/d30/cf8 0 2024-04-06T03:40:44.619 INFO:tasks.workunit.client.1.smithi136.stdout:7/868: unlink d5/df/d12/d1b/d3a/d4a/dc4/lca 0 2024-04-06T03:40:44.619 INFO:tasks.workunit.client.1.smithi136.stdout:7/869: chown d5/df/d11/f124 389 1 2024-04-06T03:40:44.620 INFO:tasks.workunit.client.1.smithi136.stdout:7/870: chown d5/df/d12/d1b/d3a/d4a/dc4 653 1 2024-04-06T03:40:44.620 INFO:tasks.workunit.client.0.smithi094.stdout:4/791: dwrite d0/d15/d3a/d3e/d46/f62 [0,4194304] 0 2024-04-06T03:40:44.620 INFO:tasks.workunit.client.0.smithi094.stdout:4/792: stat d0/d15/d3a/l5c 0 2024-04-06T03:40:44.620 INFO:tasks.workunit.client.1.smithi136.stdout:2/820: rename d3/d15/d19/f60 to d3/d9/df3/f10f 0 2024-04-06T03:40:44.620 INFO:tasks.workunit.client.1.smithi136.stdout:0/766: rmdir d1/d6/d2c/d57/d43/d6d 39 2024-04-06T03:40:44.623 INFO:tasks.workunit.client.0.smithi094.stdout:4/793: creat d0/df/dbd/f10c x:0 0 0 2024-04-06T03:40:44.624 INFO:tasks.workunit.client.1.smithi136.stdout:1/792: symlink d12/d3b/d60/d7e/dbe/l10e 0 2024-04-06T03:40:44.625 INFO:tasks.workunit.client.1.smithi136.stdout:1/793: chown d12/d3b/d74/d7a/dea/dd2/lf2 2 1 2024-04-06T03:40:44.625 INFO:tasks.workunit.client.1.smithi136.stdout:1/794: dread - d12/d1b/d1e/dd1/fdf zero size 2024-04-06T03:40:44.627 INFO:tasks.workunit.client.1.smithi136.stdout:0/767: write d1/d6/d2c/d57/f66 [2855091,60729] 0 2024-04-06T03:40:44.628 INFO:tasks.workunit.client.1.smithi136.stdout:6/710: rename d2/d7/d26/lac to d2/d7/d13/d89/le8 0 2024-04-06T03:40:44.631 INFO:tasks.workunit.client.1.smithi136.stdout:2/821: unlink d3/d9/d3b/d70/dc8/f109 0 2024-04-06T03:40:44.631 INFO:tasks.workunit.client.1.smithi136.stdout:2/822: dread - d3/d4a/d9d/d18/d4b/f9c zero size 2024-04-06T03:40:44.631 INFO:tasks.workunit.client.1.smithi136.stdout:2/823: write d3/d15/f50 [2791296,92809] 0 2024-04-06T03:40:44.634 INFO:tasks.workunit.client.1.smithi136.stdout:4/678: write d1d/d31/f3b [216864,59337] 0 2024-04-06T03:40:44.634 INFO:tasks.workunit.client.1.smithi136.stdout:4/679: write d1d/d31/d3e/d56/f61 [5205921,74787] 0 2024-04-06T03:40:44.640 INFO:tasks.workunit.client.1.smithi136.stdout:1/795: link d12/d3b/d74/d7a/fad d12/d3b/d60/d7e/d37/f10f 0 2024-04-06T03:40:44.641 INFO:tasks.workunit.client.1.smithi136.stdout:2/824: unlink d3/d9/c7f 0 2024-04-06T03:40:44.643 INFO:tasks.workunit.client.1.smithi136.stdout:4/680: symlink d1d/d2f/da6/dbf/lf3 0 2024-04-06T03:40:44.645 INFO:tasks.workunit.client.0.smithi094.stdout:1/722: dwrite d0/d7/f46 [0,4194304] 0 2024-04-06T03:40:44.645 INFO:tasks.workunit.client.0.smithi094.stdout:1/723: fsync d0/d1/d27/da5/d40/db1/d2f/f77 0 2024-04-06T03:40:44.646 INFO:tasks.workunit.client.1.smithi136.stdout:0/768: getdents d1/d6/d2c 0 2024-04-06T03:40:44.646 INFO:tasks.workunit.client.1.smithi136.stdout:0/769: chown d1/c5b 16578461 1 2024-04-06T03:40:44.646 INFO:tasks.workunit.client.1.smithi136.stdout:0/770: dread - d1/d6/d2c/d84/ff1 zero size 2024-04-06T03:40:44.649 INFO:tasks.workunit.client.1.smithi136.stdout:1/796: unlink d12/d1b/da7/ca9 0 2024-04-06T03:40:44.649 INFO:tasks.workunit.client.0.smithi094.stdout:4/794: dwrite d0/df/db3/dd8/fed [0,4194304] 0 2024-04-06T03:40:44.652 INFO:tasks.workunit.client.1.smithi136.stdout:0/771: mknod d1/d6/d2c/d57/d43/d6d/d85/da7/de5/c107 0 2024-04-06T03:40:44.656 INFO:tasks.workunit.client.1.smithi136.stdout:4/681: dread d1d/d31/f44 [0,4194304] 0 2024-04-06T03:40:44.656 INFO:tasks.workunit.client.0.smithi094.stdout:4/795: creat d0/d15/d3a/d3e/d49/f10d x:0 0 0 2024-04-06T03:40:44.656 INFO:tasks.workunit.client.0.smithi094.stdout:4/796: write d0/df/d13/d4a/dfa/f105 [246834,114622] 0 2024-04-06T03:40:44.657 INFO:tasks.workunit.client.1.smithi136.stdout:0/772: dread d1/d6/d2c/d57/d43/faa [0,4194304] 0 2024-04-06T03:40:44.658 INFO:tasks.workunit.client.1.smithi136.stdout:4/682: rename d1d/d31/d3e/d56/d7b/fce to d1d/d31/d3e/ff4 0 2024-04-06T03:40:44.658 INFO:tasks.workunit.client.0.smithi094.stdout:4/797: write d0/f8 [5197678,52868] 0 2024-04-06T03:40:44.661 INFO:tasks.workunit.client.1.smithi136.stdout:4/683: creat d1d/d31/d3e/d56/ff5 x:0 0 0 2024-04-06T03:40:44.661 INFO:tasks.workunit.client.1.smithi136.stdout:6/711: dwrite d2/d7/d26/d30/f4c [0,4194304] 0 2024-04-06T03:40:44.662 INFO:tasks.workunit.client.1.smithi136.stdout:6/712: symlink d2/d7/d26/d68/d9b/le9 0 2024-04-06T03:40:44.662 INFO:tasks.workunit.client.1.smithi136.stdout:6/713: truncate d2/d7/d13/d79/dc2/fc9 1040009 0 2024-04-06T03:40:44.664 INFO:tasks.workunit.client.1.smithi136.stdout:2/825: dread d3/d4a/d9d/d18/f74 [0,4194304] 0 2024-04-06T03:40:44.665 INFO:tasks.workunit.client.1.smithi136.stdout:6/714: rmdir d2/d7/d26/d30/d6e/d58/d5d/d62/da4 0 2024-04-06T03:40:44.665 INFO:tasks.workunit.client.1.smithi136.stdout:6/715: write d2/d7/d26/d30/fd4 [1396774,120982] 0 2024-04-06T03:40:44.665 INFO:tasks.workunit.client.0.smithi094.stdout:7/649: dwrite d2/d6/dc/d12/d61/fc7 [0,4194304] 0 2024-04-06T03:40:44.667 INFO:tasks.workunit.client.1.smithi136.stdout:2/826: mkdir d3/d15/d19/d110 0 2024-04-06T03:40:44.674 INFO:tasks.workunit.client.1.smithi136.stdout:4/684: dread d1d/d31/f58 [0,4194304] 0 2024-04-06T03:40:44.674 INFO:tasks.workunit.client.1.smithi136.stdout:6/716: creat d2/d7/d26/d30/dc4/fea x:0 0 0 2024-04-06T03:40:44.675 INFO:tasks.workunit.client.0.smithi094.stdout:7/650: rmdir d2/d6/dc/d12/d29/d3f/db9 39 2024-04-06T03:40:44.676 INFO:tasks.workunit.client.1.smithi136.stdout:2/827: unlink d3/d4a/d9d/d18/d1d/d91/fea 0 2024-04-06T03:40:44.677 INFO:tasks.workunit.client.1.smithi136.stdout:2/828: chown d3/d4a/d9d/d18/d4b/f5d 1429 1 2024-04-06T03:40:44.678 INFO:tasks.workunit.client.0.smithi094.stdout:7/651: creat d2/d6/dc/dd/df/d36/fca x:0 0 0 2024-04-06T03:40:44.678 INFO:tasks.workunit.client.0.smithi094.stdout:7/652: chown d2/d6/dc/d12/d29/d3f/db9 24111 1 2024-04-06T03:40:44.680 INFO:tasks.workunit.client.1.smithi136.stdout:5/851: sync 2024-04-06T03:40:44.680 INFO:tasks.workunit.client.1.smithi136.stdout:5/852: write d1/d16/d17/fd9 [344676,3476] 0 2024-04-06T03:40:44.681 INFO:tasks.workunit.client.0.smithi094.stdout:7/653: truncate d2/d6/f9 407518 0 2024-04-06T03:40:44.682 INFO:tasks.workunit.client.0.smithi094.stdout:7/654: readlink d2/d6/l40 0 2024-04-06T03:40:44.690 INFO:tasks.workunit.client.1.smithi136.stdout:2/829: dread d3/d4a/d9d/d18/d1d/f35 [0,4194304] 0 2024-04-06T03:40:44.690 INFO:tasks.workunit.client.1.smithi136.stdout:2/830: write d3/d15/d19/dac/fb9 [2699298,103682] 0 2024-04-06T03:40:44.691 INFO:tasks.workunit.client.0.smithi094.stdout:0/710: dwrite d1/d9/dd/d18/d1a/f4d [4194304,4194304] 0 2024-04-06T03:40:44.691 INFO:tasks.workunit.client.0.smithi094.stdout:1/724: dwrite d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/f9c [0,4194304] 0 2024-04-06T03:40:44.692 INFO:tasks.workunit.client.0.smithi094.stdout:0/711: write d1/d9/dd/d18/d32/d42/f68 [130269,2286] 0 2024-04-06T03:40:44.692 INFO:tasks.workunit.client.0.smithi094.stdout:0/712: getdents d1/d9/dd/d18/d1a/d44/dc2 0 2024-04-06T03:40:44.692 INFO:tasks.workunit.client.0.smithi094.stdout:7/655: mknod d2/d6/dc/d12/d9a/ccb 0 2024-04-06T03:40:44.692 INFO:tasks.workunit.client.0.smithi094.stdout:4/798: dwrite d0/df/d13/f36 [0,4194304] 0 2024-04-06T03:40:44.692 INFO:tasks.workunit.client.0.smithi094.stdout:4/799: getdents d0/d4/d67/dec 0 2024-04-06T03:40:44.692 INFO:tasks.workunit.client.1.smithi136.stdout:1/797: dwrite d12/d1b/d1e/d6f/fca [0,4194304] 0 2024-04-06T03:40:44.694 INFO:tasks.workunit.client.0.smithi094.stdout:1/725: creat d0/d7/ff9 x:0 0 0 2024-04-06T03:40:44.695 INFO:tasks.workunit.client.1.smithi136.stdout:4/685: dwrite d1d/d70/f94 [0,4194304] 0 2024-04-06T03:40:44.695 INFO:tasks.workunit.client.1.smithi136.stdout:4/686: chown d1d/d31/d3e/d55/d89/c9f 111532898 1 2024-04-06T03:40:44.696 INFO:tasks.workunit.client.0.smithi094.stdout:7/656: mkdir d2/d6/dc/d12/d3a/dcc 0 2024-04-06T03:40:44.698 INFO:tasks.workunit.client.1.smithi136.stdout:6/717: mknod d2/d7/d26/ceb 0 2024-04-06T03:40:44.700 INFO:tasks.workunit.client.1.smithi136.stdout:5/853: dwrite d1/dc/d1c/d32/d70/ff7 [0,4194304] 0 2024-04-06T03:40:44.700 INFO:tasks.workunit.client.1.smithi136.stdout:5/854: dread - d1/d16/d17/db2/fd7 zero size 2024-04-06T03:40:44.700 INFO:tasks.workunit.client.1.smithi136.stdout:9/799: getdents d17/d21/d29 0 2024-04-06T03:40:44.702 INFO:tasks.workunit.client.0.smithi094.stdout:1/726: write d0/d7/d1a/d64/f72 [2743479,55903] 0 2024-04-06T03:40:44.702 INFO:tasks.workunit.client.0.smithi094.stdout:1/727: chown d0/d1/d27/cf1 7513 1 2024-04-06T03:40:44.703 INFO:tasks.workunit.client.0.smithi094.stdout:1/728: creat d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/db9/ffa x:0 0 0 2024-04-06T03:40:44.703 INFO:tasks.workunit.client.1.smithi136.stdout:6/718: dread d2/d7/f2c [0,4194304] 0 2024-04-06T03:40:44.703 INFO:tasks.workunit.client.1.smithi136.stdout:6/719: read d2/d7/d26/d30/d6e/db7/fbd [35193,57715] 0 2024-04-06T03:40:44.704 INFO:tasks.workunit.client.1.smithi136.stdout:2/831: rename d3/d4a/d9d/d18/cb8 to d3/d4a/d9d/d18/d1d/d3c/df7/c111 0 2024-04-06T03:40:44.704 INFO:tasks.workunit.client.1.smithi136.stdout:3/736: sync 2024-04-06T03:40:44.704 INFO:tasks.workunit.client.1.smithi136.stdout:3/737: fdatasync d0/d4/d12/d19/d55/d3c/d46/f72 0 2024-04-06T03:40:44.709 INFO:tasks.workunit.client.0.smithi094.stdout:4/800: dread d0/d4/fd [0,4194304] 0 2024-04-06T03:40:44.709 INFO:tasks.workunit.client.0.smithi094.stdout:4/801: write d0/df/d1e/fa7 [204655,75862] 0 2024-04-06T03:40:44.713 INFO:tasks.workunit.client.0.smithi094.stdout:0/713: dwrite d1/d9/dd/d18/d1a/f7e [0,4194304] 0 2024-04-06T03:40:44.713 INFO:tasks.workunit.client.0.smithi094.stdout:0/714: readlink d1/d9/dd/d18/d1e/def/dab/d4e/l51 0 2024-04-06T03:40:44.714 INFO:tasks.workunit.client.1.smithi136.stdout:4/687: symlink d1d/d31/d8e/lf6 0 2024-04-06T03:40:44.716 INFO:tasks.workunit.client.1.smithi136.stdout:5/855: symlink d1/dc/d1c/d32/l102 0 2024-04-06T03:40:44.716 INFO:tasks.workunit.client.1.smithi136.stdout:5/856: creat d1/d16/d89/d8a/f103 x:0 0 0 2024-04-06T03:40:44.716 INFO:tasks.workunit.client.1.smithi136.stdout:5/857: write d1/dc/d10/d99/fd2 [848837,27295] 0 2024-04-06T03:40:44.716 INFO:tasks.workunit.client.1.smithi136.stdout:5/858: creat d1/d16/d89/d8a/f104 x:0 0 0 2024-04-06T03:40:44.716 INFO:tasks.workunit.client.1.smithi136.stdout:5/859: chown d1/dc/d1c/d32/d8b/d20/l31 35034933 1 2024-04-06T03:40:44.717 INFO:tasks.workunit.client.1.smithi136.stdout:9/800: creat d17/d20/d25/da4/dbb/de4/f114 x:0 0 0 2024-04-06T03:40:44.721 INFO:tasks.workunit.client.1.smithi136.stdout:3/738: mknod d0/d4/d12/d19/d55/d3c/dcb/cff 0 2024-04-06T03:40:44.721 INFO:tasks.workunit.client.1.smithi136.stdout:6/720: rename d2/d7/d26/c71 to d2/d7/d9e/ddf/cec 0 2024-04-06T03:40:44.722 INFO:tasks.workunit.client.1.smithi136.stdout:1/798: getdents d12/d53/dc5 0 2024-04-06T03:40:44.723 INFO:tasks.workunit.client.1.smithi136.stdout:2/832: dwrite d3/d4a/d9d/d8a/d93/d5f/f78 [0,4194304] 0 2024-04-06T03:40:44.731 INFO:tasks.workunit.client.0.smithi094.stdout:1/729: dwrite d0/d4/f9 [4194304,4194304] 0 2024-04-06T03:40:44.733 INFO:tasks.workunit.client.1.smithi136.stdout:3/739: mkdir d0/d4/d12/d3a/d51/da6/d100 0 2024-04-06T03:40:44.733 INFO:tasks.workunit.client.0.smithi094.stdout:4/802: dwrite d0/df/d1e/f24 [4194304,4194304] 0 2024-04-06T03:40:44.739 INFO:tasks.workunit.client.1.smithi136.stdout:6/721: creat d2/d7/d26/d68/fed x:0 0 0 2024-04-06T03:40:44.739 INFO:tasks.workunit.client.0.smithi094.stdout:1/730: dread d0/d7/f90 [0,4194304] 0 2024-04-06T03:40:44.749 INFO:tasks.workunit.client.1.smithi136.stdout:4/688: dwrite d1d/d31/d3e/d56/ff5 [0,4194304] 0 2024-04-06T03:40:44.749 INFO:tasks.workunit.client.1.smithi136.stdout:4/689: fsync d1d/f9d 0 2024-04-06T03:40:44.751 INFO:tasks.workunit.client.0.smithi094.stdout:3/758: rename d9/d20/d76/f5c to d9/d1e/d26/f10e 0 2024-04-06T03:40:44.752 INFO:tasks.workunit.client.1.smithi136.stdout:1/799: dwrite d12/d3b/d74/d7a/dea/d77/fc1 [0,4194304] 0 2024-04-06T03:40:44.752 INFO:tasks.workunit.client.1.smithi136.stdout:1/800: dread - d12/d1b/d1e/dd1/fdf zero size 2024-04-06T03:40:44.754 INFO:tasks.workunit.client.0.smithi094.stdout:1/731: symlink d0/d7/d1a/d64/lfb 0 2024-04-06T03:40:44.757 INFO:tasks.workunit.client.1.smithi136.stdout:5/860: rename d1/d9b/db4/lbd to d1/d98/dbc/l105 0 2024-04-06T03:40:44.760 INFO:tasks.workunit.client.0.smithi094.stdout:5/719: rename db/d19/c3b to db/d19/d24/d28/d85/dd5/cf8 0 2024-04-06T03:40:44.762 INFO:tasks.workunit.client.1.smithi136.stdout:3/740: creat d0/d4/d12/d19/d55/d3c/d5a/f101 x:0 0 0 2024-04-06T03:40:44.764 INFO:tasks.workunit.client.0.smithi094.stdout:0/715: dwrite d1/d9/dd/d18/d1e/d22/fe8 [0,4194304] 0 2024-04-06T03:40:44.765 INFO:tasks.workunit.client.0.smithi094.stdout:3/759: mkdir d9/d1e/d26/d55/d62/db8/d10f 0 2024-04-06T03:40:44.767 INFO:tasks.workunit.client.1.smithi136.stdout:2/833: dwrite d3/d15/d19/f39 [4194304,4194304] 0 2024-04-06T03:40:44.768 INFO:tasks.workunit.client.0.smithi094.stdout:4/803: dwrite d0/d4/d67/f79 [0,4194304] 0 2024-04-06T03:40:44.768 INFO:tasks.workunit.client.0.smithi094.stdout:4/804: stat d0/df/d1e/fa7 0 2024-04-06T03:40:44.769 INFO:tasks.workunit.client.1.smithi136.stdout:6/722: symlink d2/d7/d26/d30/d4f/d75/lee 0 2024-04-06T03:40:44.769 INFO:tasks.workunit.client.1.smithi136.stdout:6/723: dread - d2/d7/d13/d89/fb9 zero size 2024-04-06T03:40:44.769 INFO:tasks.workunit.client.1.smithi136.stdout:9/801: dwrite d17/d19/f35 [0,4194304] 0 2024-04-06T03:40:44.770 INFO:tasks.workunit.client.1.smithi136.stdout:1/801: dwrite d12/d3b/d60/d7e/dbe/fd8 [0,4194304] 0 2024-04-06T03:40:44.772 INFO:tasks.workunit.client.0.smithi094.stdout:1/732: symlink d0/d1/d27/da5/d40/db1/d2f/d76/lfc 0 2024-04-06T03:40:44.776 INFO:tasks.workunit.client.0.smithi094.stdout:7/657: rename d2/d6/dc/dd/df/c1d to d2/d6/dc/d12/d29/ccd 0 2024-04-06T03:40:44.778 INFO:tasks.workunit.client.0.smithi094.stdout:5/720: mknod db/d19/d2c/d57/cf9 0 2024-04-06T03:40:44.784 INFO:tasks.workunit.client.1.smithi136.stdout:5/861: dwrite d1/d9b/dea/ff2 [0,4194304] 0 2024-04-06T03:40:44.784 INFO:tasks.workunit.client.1.smithi136.stdout:5/862: chown d1/dc/l50 41641 1 2024-04-06T03:40:44.784 INFO:tasks.workunit.client.1.smithi136.stdout:1/802: dwrite d12/d3b/d44/d5f/fde [0,4194304] 0 2024-04-06T03:40:44.784 INFO:tasks.workunit.client.1.smithi136.stdout:1/803: chown d12/d1b/d1e/d23 155 1 2024-04-06T03:40:44.785 INFO:tasks.workunit.client.0.smithi094.stdout:1/733: dread d0/d1/d27/da5/d40/db1/d2f/d4b/f8a [0,4194304] 0 2024-04-06T03:40:44.785 INFO:tasks.workunit.client.0.smithi094.stdout:0/716: mknod d1/d9/dd/d18/d32/d42/cf0 0 2024-04-06T03:40:44.785 INFO:tasks.workunit.client.0.smithi094.stdout:0/717: truncate d1/d9/dd/d18/d1e/def/dab/d4e/f9e 638146 0 2024-04-06T03:40:44.785 INFO:tasks.workunit.client.0.smithi094.stdout:0/718: chown d1/d9/dd/d18/d5d/de9 1399 1 2024-04-06T03:40:44.785 INFO:tasks.workunit.client.0.smithi094.stdout:0/719: chown d1/d9/dd/d18/d1a/ld0 763002 1 2024-04-06T03:40:44.785 INFO:tasks.workunit.client.0.smithi094.stdout:0/720: write d1/d9/dd/d18/d1e/d22/fe8 [4984291,2817] 0 2024-04-06T03:40:44.787 INFO:tasks.workunit.client.0.smithi094.stdout:3/760: dwrite d9/d20/d2d/da9/fe0 [0,4194304] 0 2024-04-06T03:40:44.787 INFO:tasks.workunit.client.1.smithi136.stdout:4/690: getdents d1d/d31/d78 0 2024-04-06T03:40:44.790 INFO:tasks.workunit.client.0.smithi094.stdout:4/805: dwrite d0/df/f10b [0,4194304] 0 2024-04-06T03:40:44.798 INFO:tasks.workunit.client.0.smithi094.stdout:7/658: mkdir d2/d6/dc/d12/d29/d3f/dce 0 2024-04-06T03:40:44.798 INFO:tasks.workunit.client.0.smithi094.stdout:7/659: truncate d2/d6/dc/d12/d9a/da0/faf 327790 0 2024-04-06T03:40:44.798 INFO:tasks.workunit.client.0.smithi094.stdout:7/660: chown d2/d6/dc/d12/d9a/da0/faf 1243 1 2024-04-06T03:40:44.799 INFO:tasks.workunit.client.1.smithi136.stdout:3/741: getdents d0/d4/d12/d19/d2e 0 2024-04-06T03:40:44.801 INFO:tasks.workunit.client.0.smithi094.stdout:5/721: mkdir db/d19/d24/d28/d85/dd5/dfa 0 2024-04-06T03:40:44.803 INFO:tasks.workunit.client.0.smithi094.stdout:0/721: mknod d1/d9/dd/d18/d32/d42/cf1 0 2024-04-06T03:40:44.804 INFO:tasks.workunit.client.1.smithi136.stdout:2/834: symlink d3/d15/d19/d43/daa/dec/l112 0 2024-04-06T03:40:44.805 INFO:tasks.workunit.client.1.smithi136.stdout:2/835: chown d3/d15/d19/d43/daa/dec 0 1 2024-04-06T03:40:44.805 INFO:tasks.workunit.client.1.smithi136.stdout:2/836: fdatasync d3/d9b/f86 0 2024-04-06T03:40:44.806 INFO:tasks.workunit.client.0.smithi094.stdout:1/734: dwrite d0/d7/d1a/da8/fbb [0,4194304] 0 2024-04-06T03:40:44.807 INFO:tasks.workunit.client.1.smithi136.stdout:1/804: dwrite d12/d1b/d22/f43 [0,4194304] 0 2024-04-06T03:40:44.807 INFO:tasks.workunit.client.1.smithi136.stdout:6/724: symlink d2/d25/lef 0 2024-04-06T03:40:44.807 INFO:tasks.workunit.client.1.smithi136.stdout:4/691: dwrite d1d/d31/d3e/f71 [0,4194304] 0 2024-04-06T03:40:44.808 INFO:tasks.workunit.client.0.smithi094.stdout:5/722: dread db/d19/d24/fdb [0,4194304] 0 2024-04-06T03:40:44.808 INFO:tasks.workunit.client.0.smithi094.stdout:5/723: fsync db/d19/d24/f37 0 2024-04-06T03:40:44.809 INFO:tasks.workunit.client.1.smithi136.stdout:9/802: symlink d17/d21/d29/de5/d4b/l115 0 2024-04-06T03:40:44.809 INFO:tasks.workunit.client.1.smithi136.stdout:9/803: chown d17/d21/d32/dc6/d113/cad 11705400 1 2024-04-06T03:40:44.814 INFO:tasks.workunit.client.0.smithi094.stdout:1/735: write d0/d7/d1a/da8/fbb [1290806,48873] 0 2024-04-06T03:40:44.814 INFO:tasks.workunit.client.0.smithi094.stdout:1/736: chown d0/d7/d1a/da8/cac 686174231 1 2024-04-06T03:40:44.814 INFO:tasks.workunit.client.0.smithi094.stdout:1/737: write d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/f8d [1872649,93232] 0 2024-04-06T03:40:44.814 INFO:tasks.workunit.client.0.smithi094.stdout:4/806: creat d0/d15/d3a/d61/df8/f10e x:0 0 0 2024-04-06T03:40:44.816 INFO:tasks.workunit.client.1.smithi136.stdout:8/843: sync 2024-04-06T03:40:44.816 INFO:tasks.workunit.client.0.smithi094.stdout:4/807: dread d0/d15/d3a/d61/d81/da0/fc5 [0,4194304] 0 2024-04-06T03:40:44.816 INFO:tasks.workunit.client.0.smithi094.stdout:4/808: creat d0/d77/dc4/f10f x:0 0 0 2024-04-06T03:40:44.818 INFO:tasks.workunit.client.1.smithi136.stdout:5/863: mkdir d1/d16/d89/d8a/df1/d106 0 2024-04-06T03:40:44.819 INFO:tasks.workunit.client.0.smithi094.stdout:5/724: dread db/d7c/d98/fc6 [0,4194304] 0 2024-04-06T03:40:44.822 INFO:tasks.workunit.client.1.smithi136.stdout:1/805: dwrite d12/d3b/d44/fa5 [0,4194304] 0 2024-04-06T03:40:44.822 INFO:tasks.workunit.client.1.smithi136.stdout:1/806: dread - d12/d3b/d60/d7e/ffc zero size 2024-04-06T03:40:44.822 INFO:tasks.workunit.client.1.smithi136.stdout:1/807: creat d12/d3b/d44/f110 x:0 0 0 2024-04-06T03:40:44.826 INFO:tasks.workunit.client.1.smithi136.stdout:3/742: creat d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/f102 x:0 0 0 2024-04-06T03:40:44.827 INFO:tasks.workunit.client.1.smithi136.stdout:2/837: creat d3/ddc/f113 x:0 0 0 2024-04-06T03:40:44.827 INFO:tasks.workunit.client.1.smithi136.stdout:2/838: fsync d3/d15/d19/fc7 0 2024-04-06T03:40:44.831 INFO:tasks.workunit.client.0.smithi094.stdout:0/722: link d1/d9/dd/d18/d5d/d60/cc4 d1/d9/dd/d18/d5d/de9/cf2 0 2024-04-06T03:40:44.834 INFO:tasks.workunit.client.1.smithi136.stdout:3/743: dread d0/d4/f76 [0,4194304] 0 2024-04-06T03:40:44.835 INFO:tasks.workunit.client.1.smithi136.stdout:4/692: unlink d1d/d2f/da6/dbf/lf3 0 2024-04-06T03:40:44.838 INFO:tasks.workunit.client.1.smithi136.stdout:9/804: mkdir d17/d21/d32/dc6/d113/d66/de1/d51/d116 0 2024-04-06T03:40:44.838 INFO:tasks.workunit.client.1.smithi136.stdout:9/805: dread - d17/d21/d32/dc6/d113/d66/de1/d51/f6d zero size 2024-04-06T03:40:44.840 INFO:tasks.workunit.client.1.smithi136.stdout:5/864: dwrite d1/f40 [0,4194304] 0 2024-04-06T03:40:44.840 INFO:tasks.workunit.client.0.smithi094.stdout:1/738: creat d0/d1/dd3/de7/ffd x:0 0 0 2024-04-06T03:40:44.840 INFO:tasks.workunit.client.0.smithi094.stdout:5/725: dwrite db/f11 [4194304,4194304] 0 2024-04-06T03:40:44.840 INFO:tasks.workunit.client.0.smithi094.stdout:5/726: write db/d19/d2c/d57/fbe [1101586,47114] 0 2024-04-06T03:40:44.845 INFO:tasks.workunit.client.1.smithi136.stdout:3/744: dread d0/d4/d12/d19/d55/d3c/d5a/fac [4194304,4194304] 0 2024-04-06T03:40:44.845 INFO:tasks.workunit.client.1.smithi136.stdout:3/745: dread - d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/fe0 zero size 2024-04-06T03:40:44.845 INFO:tasks.workunit.client.0.smithi094.stdout:7/661: getdents d2/d6/dc/d12 0 2024-04-06T03:40:44.846 INFO:tasks.workunit.client.1.smithi136.stdout:2/839: dwrite d3/d4a/d9d/d18/d4b/dbd/fc3 [0,4194304] 0 2024-04-06T03:40:44.847 INFO:tasks.workunit.client.1.smithi136.stdout:8/844: mkdir d3/de/d18/d7d/ddf/d6c/d7f/d105 0 2024-04-06T03:40:44.847 INFO:tasks.workunit.client.1.smithi136.stdout:8/845: stat d3/da/d26/d35/d53/f7c 0 2024-04-06T03:40:44.850 INFO:tasks.workunit.client.0.smithi094.stdout:0/723: creat d1/d9/dd/d18/d78/deb/ff3 x:0 0 0 2024-04-06T03:40:44.850 INFO:tasks.workunit.client.0.smithi094.stdout:0/724: chown d1/d9/dd/d18/d1e/d22/d35/l72 1335499147 1 2024-04-06T03:40:44.850 INFO:tasks.workunit.client.0.smithi094.stdout:0/725: write d1/d9/dd/d18/d1e/def/dab/d4e/fbe [469817,4805] 0 2024-04-06T03:40:44.858 INFO:tasks.workunit.client.1.smithi136.stdout:4/693: creat d1d/d2f/d6c/da5/ff7 x:0 0 0 2024-04-06T03:40:44.858 INFO:tasks.workunit.client.1.smithi136.stdout:4/694: creat d1d/d31/d3e/d56/d7b/ff8 x:0 0 0 2024-04-06T03:40:44.858 INFO:tasks.workunit.client.1.smithi136.stdout:4/695: stat d1d/d70/f6a 0 2024-04-06T03:40:44.860 INFO:tasks.workunit.client.1.smithi136.stdout:9/806: mkdir d17/d21/d29/de5/da5/dce/dd1/d117 0 2024-04-06T03:40:44.861 INFO:tasks.workunit.client.1.smithi136.stdout:0/773: sync 2024-04-06T03:40:44.861 INFO:tasks.workunit.client.1.smithi136.stdout:7/871: sync 2024-04-06T03:40:44.861 INFO:tasks.workunit.client.1.smithi136.stdout:0/774: fsync d1/d6/d2c/d57/d43/d6d/d85/d78/f97 0 2024-04-06T03:40:44.862 INFO:tasks.workunit.client.1.smithi136.stdout:0/775: creat d1/d53/f108 x:0 0 0 2024-04-06T03:40:44.862 INFO:tasks.workunit.client.1.smithi136.stdout:5/865: write d1/d98/dbc/fc5 [627074,14840] 0 2024-04-06T03:40:44.864 INFO:tasks.workunit.client.1.smithi136.stdout:4/696: dread d1d/d70/f64 [0,4194304] 0 2024-04-06T03:40:44.864 INFO:tasks.workunit.client.1.smithi136.stdout:4/697: read d1d/f9d [173621,6537] 0 2024-04-06T03:40:44.870 INFO:tasks.workunit.client.1.smithi136.stdout:9/807: write d17/d21/d29/f5f [2549356,106542] 0 2024-04-06T03:40:44.872 INFO:tasks.workunit.client.1.smithi136.stdout:2/840: creat d3/d15/d19/d43/daa/dce/f114 x:0 0 0 2024-04-06T03:40:44.872 INFO:tasks.workunit.client.0.smithi094.stdout:1/739: mknod d0/d1/d27/da5/db2/dce/cfe 0 2024-04-06T03:40:44.872 INFO:tasks.workunit.client.1.smithi136.stdout:2/841: creat d3/d9/d3b/d70/dc8/f115 x:0 0 0 2024-04-06T03:40:44.873 INFO:tasks.workunit.client.1.smithi136.stdout:8/846: symlink d3/de/d18/d7d/ddf/d6c/dab/l106 0 2024-04-06T03:40:44.878 INFO:tasks.workunit.client.0.smithi094.stdout:5/727: link db/d19/d21/ca5 db/d19/d30/d83/cfb 0 2024-04-06T03:40:44.878 INFO:tasks.workunit.client.0.smithi094.stdout:5/728: chown db/d19/d24/d28/f3f 59700580 1 2024-04-06T03:40:44.878 INFO:tasks.workunit.client.0.smithi094.stdout:5/729: fdatasync db/d19/d24/d28/d85/dd5/fe3 0 2024-04-06T03:40:44.878 INFO:tasks.workunit.client.0.smithi094.stdout:5/730: chown db/d19/d24/f53 36 1 2024-04-06T03:40:44.880 INFO:tasks.workunit.client.0.smithi094.stdout:4/809: mknod d0/c110 0 2024-04-06T03:40:44.883 INFO:tasks.workunit.client.0.smithi094.stdout:7/662: symlink d2/d6/dc/d12/d61/lcf 0 2024-04-06T03:40:44.884 INFO:tasks.workunit.client.0.smithi094.stdout:7/663: truncate d2/d6/dc/d12/d3a/d56/f71 281860 0 2024-04-06T03:40:44.886 INFO:tasks.workunit.client.1.smithi136.stdout:5/866: dwrite d1/dc/d1c/d32/d70/ff7 [4194304,4194304] 0 2024-04-06T03:40:44.889 INFO:tasks.workunit.client.0.smithi094.stdout:0/726: rename d1/d9/dd/d18/d1e/d22/fe8 to d1/d9/dd/d18/d1e/d22/d35/dac/ff4 0 2024-04-06T03:40:44.891 INFO:tasks.workunit.client.1.smithi136.stdout:4/698: mknod d1d/d31/d3e/d55/dba/cf9 0 2024-04-06T03:40:44.897 INFO:tasks.workunit.client.0.smithi094.stdout:4/810: dwrite d0/df/d1e/d34/f88 [0,4194304] 0 2024-04-06T03:40:44.898 INFO:tasks.workunit.client.0.smithi094.stdout:1/740: dwrite d0/d1/d27/da5/d40/db1/d2f/d3c/d48/f89 [0,4194304] 0 2024-04-06T03:40:44.898 INFO:tasks.workunit.client.0.smithi094.stdout:7/664: creat d2/d6/dc/d12/d9a/dbf/fd0 x:0 0 0 2024-04-06T03:40:44.899 INFO:tasks.workunit.client.1.smithi136.stdout:9/808: dwrite d17/d20/d28/fae [0,4194304] 0 2024-04-06T03:40:44.900 INFO:tasks.workunit.client.1.smithi136.stdout:2/842: creat d3/d4a/d9d/d8a/d93/d5f/df0/f116 x:0 0 0 2024-04-06T03:40:44.902 INFO:tasks.workunit.client.0.smithi094.stdout:5/731: getdents db/d7c/d98 0 2024-04-06T03:40:44.903 INFO:tasks.workunit.client.1.smithi136.stdout:5/867: dread d1/d16/d89/daf/fc8 [0,4194304] 0 2024-04-06T03:40:44.903 INFO:tasks.workunit.client.0.smithi094.stdout:0/727: dread d1/d9/dd/d18/d1e/def/dab/d4e/de4/f8e [0,4194304] 0 2024-04-06T03:40:44.906 INFO:tasks.workunit.client.1.smithi136.stdout:8/847: rmdir d3/da/d26/d35/d39/daa 0 2024-04-06T03:40:44.906 INFO:tasks.workunit.client.1.smithi136.stdout:8/848: truncate d3/da/d26/d35/d3d/dc7/dce/df2/fa2 1349802 0 2024-04-06T03:40:44.907 INFO:tasks.workunit.client.1.smithi136.stdout:4/699: dwrite d1d/d31/d3e/d55/d7f/fb6 [0,4194304] 0 2024-04-06T03:40:44.910 INFO:tasks.workunit.client.1.smithi136.stdout:1/808: rename d12/d1b/d1e/dd0 to d12/d1b/d28/deb/d111 0 2024-04-06T03:40:44.910 INFO:tasks.workunit.client.1.smithi136.stdout:1/809: write d12/d3b/d74/dd6/f105 [570557,41532] 0 2024-04-06T03:40:44.912 INFO:tasks.workunit.client.0.smithi094.stdout:5/732: write db/d19/d30/f44 [418808,93062] 0 2024-04-06T03:40:44.912 INFO:tasks.workunit.client.1.smithi136.stdout:7/872: getdents d5/df/d3d 0 2024-04-06T03:40:44.912 INFO:tasks.workunit.client.1.smithi136.stdout:7/873: truncate d5/df/d11/f89 864998 0 2024-04-06T03:40:44.912 INFO:tasks.workunit.client.1.smithi136.stdout:7/874: truncate d5/df/d1d/d6f/d4d/d69/fed 1025955 0 2024-04-06T03:40:44.916 INFO:tasks.workunit.client.0.smithi094.stdout:4/811: write d0/d15/d3a/d3e/d49/db2/fc2 [3857429,45208] 0 2024-04-06T03:40:44.919 INFO:tasks.workunit.client.0.smithi094.stdout:7/665: link d2/d6/dc/dd/df/c6e d2/d6/dc/dd/cd1 0 2024-04-06T03:40:44.919 INFO:tasks.workunit.client.0.smithi094.stdout:7/666: chown d2/d6/dc/dd/df/d95/dc9/f54 159 1 2024-04-06T03:40:44.921 INFO:tasks.workunit.client.0.smithi094.stdout:0/728: symlink d1/d9/dd/d18/d1e/def/dab/d4e/de4/lf5 0 2024-04-06T03:40:44.921 INFO:tasks.workunit.client.0.smithi094.stdout:0/729: truncate d1/d9/dd/d18/d1e/d86/fdd 459926 0 2024-04-06T03:40:44.922 INFO:tasks.workunit.client.0.smithi094.stdout:5/733: creat db/d19/d24/d28/d85/dd5/dfa/ffc x:0 0 0 2024-04-06T03:40:44.925 INFO:tasks.workunit.client.1.smithi136.stdout:2/843: dwrite d3/d4a/d9d/d18/d1d/d3c/fa0 [0,4194304] 0 2024-04-06T03:40:44.926 INFO:tasks.workunit.client.1.smithi136.stdout:9/809: dwrite d17/d20/d25/da4/dbb/fc4 [0,4194304] 0 2024-04-06T03:40:44.926 INFO:tasks.workunit.client.1.smithi136.stdout:9/810: fsync d17/d21/d32/f74 0 2024-04-06T03:40:44.926 INFO:tasks.workunit.client.0.smithi094.stdout:1/741: rmdir d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38 39 2024-04-06T03:40:44.931 INFO:tasks.workunit.client.1.smithi136.stdout:8/849: link d3/de/d18/d7d/de0/cfe d3/de/d18/d7d/ddf/d6c/d7f/c107 0 2024-04-06T03:40:44.931 INFO:tasks.workunit.client.1.smithi136.stdout:8/850: truncate d3/da/f13 1520491 0 2024-04-06T03:40:44.932 INFO:tasks.workunit.client.0.smithi094.stdout:7/667: mkdir d2/d6/dc/dd/dd2 0 2024-04-06T03:40:44.934 INFO:tasks.workunit.client.0.smithi094.stdout:0/730: mkdir d1/d9/dd/d18/d1e/def/d67/df6 0 2024-04-06T03:40:44.935 INFO:tasks.workunit.client.0.smithi094.stdout:0/731: stat d1/d9/dd/d18/d32/d42 0 2024-04-06T03:40:44.936 INFO:tasks.workunit.client.0.smithi094.stdout:5/734: mkdir db/d19/d2c/d6d/de6/dfd 0 2024-04-06T03:40:44.938 INFO:tasks.workunit.client.1.smithi136.stdout:4/700: link d1d/d2f/da6/dbf/fe4 d1d/d2f/da6/ffa 0 2024-04-06T03:40:44.938 INFO:tasks.workunit.client.1.smithi136.stdout:4/701: truncate d1d/d31/d3e/d56/d7b/f75 2179290 0 2024-04-06T03:40:44.938 INFO:tasks.workunit.client.1.smithi136.stdout:4/702: fsync d1d/d2f/f57 0 2024-04-06T03:40:44.939 INFO:tasks.workunit.client.0.smithi094.stdout:4/812: rename d0/d15/d3a/c9c to d0/df/d1e/c111 0 2024-04-06T03:40:44.945 INFO:tasks.workunit.client.0.smithi094.stdout:7/668: getdents d2/d6/dc/d12/d29/d3f 0 2024-04-06T03:40:44.945 INFO:tasks.workunit.client.0.smithi094.stdout:7/669: fsync d2/d6/dc/f74 0 2024-04-06T03:40:44.946 INFO:tasks.workunit.client.0.smithi094.stdout:1/742: dwrite d0/d7/d1a/d74/fe6 [0,4194304] 0 2024-04-06T03:40:44.947 INFO:tasks.workunit.client.1.smithi136.stdout:0/776: rename d1/d6/d2c/d1a/f81 to d1/d6/d2c/d57/d43/d6d/d85/d78/f109 0 2024-04-06T03:40:44.947 INFO:tasks.workunit.client.1.smithi136.stdout:3/746: write d0/d4/d12/d19/d2c/f7f [79432,120878] 0 2024-04-06T03:40:44.947 INFO:tasks.workunit.client.1.smithi136.stdout:3/747: fdatasync d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 0 2024-04-06T03:40:44.947 INFO:tasks.workunit.client.1.smithi136.stdout:3/748: read d0/d4/d12/d19/d55/d3c/f58 [733549,34798] 0 2024-04-06T03:40:44.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/732: symlink d1/d9/dd/d18/d32/d42/lf7 0 2024-04-06T03:40:44.949 INFO:tasks.workunit.client.0.smithi094.stdout:5/735: creat db/d7c/ffe x:0 0 0 2024-04-06T03:40:44.950 INFO:tasks.workunit.client.0.smithi094.stdout:5/736: readlink db/d19/d24/d28/d85/db5/lc9 0 2024-04-06T03:40:44.951 INFO:tasks.workunit.client.1.smithi136.stdout:1/810: unlink d12/d3b/d44/d5f/d5a/la6 0 2024-04-06T03:40:44.952 INFO:tasks.workunit.client.0.smithi094.stdout:4/813: creat d0/de8/f112 x:0 0 0 2024-04-06T03:40:44.954 INFO:tasks.workunit.client.1.smithi136.stdout:7/875: creat d5/df/d1d/d6f/d4d/d69/f134 x:0 0 0 2024-04-06T03:40:44.957 INFO:tasks.workunit.client.0.smithi094.stdout:7/670: rename d2/d6/dc/d12/d61/fc7 to d2/d6/dc/d12/d9a/fd3 0 2024-04-06T03:40:44.959 INFO:tasks.workunit.client.1.smithi136.stdout:1/811: dread d12/d3b/d44/d5f/fde [0,4194304] 0 2024-04-06T03:40:44.959 INFO:tasks.workunit.client.1.smithi136.stdout:7/876: dread d5/df/d53/d8a/fbc [0,4194304] 0 2024-04-06T03:40:44.960 INFO:tasks.workunit.client.0.smithi094.stdout:2/721: sync 2024-04-06T03:40:44.960 INFO:tasks.workunit.client.1.smithi136.stdout:2/844: symlink d3/d15/d19/dac/l117 0 2024-04-06T03:40:44.960 INFO:tasks.workunit.client.0.smithi094.stdout:5/737: dread db/fae [0,4194304] 0 2024-04-06T03:40:44.960 INFO:tasks.workunit.client.0.smithi094.stdout:5/738: fdatasync db/d19/d30/d83/fe5 0 2024-04-06T03:40:44.966 INFO:tasks.workunit.client.1.smithi136.stdout:7/877: write d5/df/d12/f122 [671346,89344] 0 2024-04-06T03:40:44.966 INFO:tasks.workunit.client.1.smithi136.stdout:7/878: write d5/df/d12/dd6/f115 [239012,60043] 0 2024-04-06T03:40:44.969 INFO:tasks.workunit.client.1.smithi136.stdout:9/811: unlink d17/d20/d28/f98 0 2024-04-06T03:40:44.969 INFO:tasks.workunit.client.1.smithi136.stdout:9/812: write d17/d20/d28/d49/d75/d60/f110 [612004,103507] 0 2024-04-06T03:40:44.970 INFO:tasks.workunit.client.1.smithi136.stdout:3/749: dwrite d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 [0,4194304] 0 2024-04-06T03:40:44.972 INFO:tasks.workunit.client.1.smithi136.stdout:8/851: creat d3/da/d26/d90/dd5/f108 x:0 0 0 2024-04-06T03:40:44.974 INFO:tasks.workunit.client.1.smithi136.stdout:4/703: creat d1d/d31/d3e/dd4/ffb x:0 0 0 2024-04-06T03:40:44.976 INFO:tasks.workunit.client.0.smithi094.stdout:4/814: dwrite d0/d15/fa2 [0,4194304] 0 2024-04-06T03:40:44.987 INFO:tasks.workunit.client.1.smithi136.stdout:0/777: creat d1/d6/d2c/d57/d43/d6d/d9b/f10a x:0 0 0 2024-04-06T03:40:44.987 INFO:tasks.workunit.client.1.smithi136.stdout:0/778: write d1/d6/d2c/d57/d43/f50 [1006029,127384] 0 2024-04-06T03:40:44.987 INFO:tasks.workunit.client.1.smithi136.stdout:0/779: chown d1/d6/d2c/d57/d62/cbb 203 1 2024-04-06T03:40:44.987 INFO:tasks.workunit.client.1.smithi136.stdout:0/780: chown d1/d6/d2c/d57/d43/d6d/d9b/d5c/f91 13623831 1 2024-04-06T03:40:44.990 INFO:tasks.workunit.client.0.smithi094.stdout:1/743: mkdir d0/d1/d27/da5/d40/db1/d2f/d76/dff 0 2024-04-06T03:40:44.991 INFO:tasks.workunit.client.1.smithi136.stdout:1/812: link d12/d1b/d28/deb/d111/f9e d12/d1b/da7/daa/f112 0 2024-04-06T03:40:44.991 INFO:tasks.workunit.client.1.smithi136.stdout:1/813: readlink d12/d3b/d60/d6d/ld7 0 2024-04-06T03:40:44.992 INFO:tasks.workunit.client.0.smithi094.stdout:0/733: symlink d1/d9/dd/d18/d1e/d22/d35/lf8 0 2024-04-06T03:40:44.992 INFO:tasks.workunit.client.0.smithi094.stdout:0/734: read d1/d9/dd/d18/d1e/d22/f37 [482183,22260] 0 2024-04-06T03:40:44.993 INFO:tasks.workunit.client.1.smithi136.stdout:7/879: dwrite d5/df/d11/fe6 [0,4194304] 0 2024-04-06T03:40:44.993 INFO:tasks.workunit.client.1.smithi136.stdout:7/880: chown d5/df/d3d/f44 154509 1 2024-04-06T03:40:44.993 INFO:tasks.workunit.client.0.smithi094.stdout:4/815: dwrite d0/df/d1e/d34/f39 [0,4194304] 0 2024-04-06T03:40:44.994 INFO:tasks.workunit.client.1.smithi136.stdout:2/845: creat d3/d15/d19/d43/d54/d10b/f118 x:0 0 0 2024-04-06T03:40:44.994 INFO:tasks.workunit.client.1.smithi136.stdout:2/846: write d3/d9/d3b/fd7 [4789243,59587] 0 2024-04-06T03:40:44.996 INFO:tasks.workunit.client.0.smithi094.stdout:7/671: rmdir d2/d6/dc/dd/df/d95/dc9/d57/dc8 0 2024-04-06T03:40:45.000 INFO:tasks.workunit.client.0.smithi094.stdout:2/722: creat d0/dd/da4/ff5 x:0 0 0 2024-04-06T03:40:45.002 INFO:tasks.workunit.client.1.smithi136.stdout:9/813: link d17/d19/da3/dc5/fd3 d17/d19/da3/dc5/deb/f118 0 2024-04-06T03:40:45.004 INFO:tasks.workunit.client.1.smithi136.stdout:4/704: rename d1d/d31/d3e/d55/d89/db3/fca to d1d/d31/d3e/d56/ffc 0 2024-04-06T03:40:45.007 INFO:tasks.workunit.client.0.smithi094.stdout:5/739: rename db/d19/d24/d28/d85/db5/lc9 to db/d19/dc1/dd4/lff 0 2024-04-06T03:40:45.007 INFO:tasks.workunit.client.0.smithi094.stdout:5/740: chown db/d7c/dd7 449481230 1 2024-04-06T03:40:45.008 INFO:tasks.workunit.client.1.smithi136.stdout:6/725: sync 2024-04-06T03:40:45.014 INFO:tasks.workunit.client.0.smithi094.stdout:1/744: rmdir d0/d1/d27/da5/d40/db1/d2f/d3c/d48 39 2024-04-06T03:40:45.015 INFO:tasks.workunit.client.1.smithi136.stdout:8/852: dwrite d3/d2d/f67 [0,4194304] 0 2024-04-06T03:40:45.016 INFO:tasks.workunit.client.1.smithi136.stdout:8/853: write d3/da/da0/fc1 [354721,80965] 0 2024-04-06T03:40:45.016 INFO:tasks.workunit.client.1.smithi136.stdout:0/781: dwrite d1/d53/f108 [0,4194304] 0 2024-04-06T03:40:45.016 INFO:tasks.workunit.client.0.smithi094.stdout:0/735: stat d1/d9/l19 0 2024-04-06T03:40:45.018 INFO:tasks.workunit.client.0.smithi094.stdout:5/741: write db/fa8 [6182116,1406] 0 2024-04-06T03:40:45.019 INFO:tasks.workunit.client.0.smithi094.stdout:5/742: chown db/d19/d24/d28/d85/db5 87320949 1 2024-04-06T03:40:45.021 INFO:tasks.workunit.client.1.smithi136.stdout:3/750: dwrite d0/d4/d12/d19/d55/d6f/ddb/fe7 [0,4194304] 0 2024-04-06T03:40:45.021 INFO:tasks.workunit.client.1.smithi136.stdout:6/726: dread d2/d7/d26/d30/d6e/d58/d5d/f65 [0,4194304] 0 2024-04-06T03:40:45.021 INFO:tasks.workunit.client.1.smithi136.stdout:3/751: chown d0/d4/d12/d19/d55/d3c 2 1 2024-04-06T03:40:45.021 INFO:tasks.workunit.client.1.smithi136.stdout:3/752: fdatasync d0/d4/d12/d19/d2c/fe8 0 2024-04-06T03:40:45.021 INFO:tasks.workunit.client.1.smithi136.stdout:3/753: chown d0/d4/d12/d19/d55/d3c/d5a/fe3 138 1 2024-04-06T03:40:45.021 INFO:tasks.workunit.client.1.smithi136.stdout:2/847: symlink d3/d4a/d9d/d18/d1d/d3c/l119 0 2024-04-06T03:40:45.022 INFO:tasks.workunit.client.1.smithi136.stdout:1/814: dwrite d12/f24 [0,4194304] 0 2024-04-06T03:40:45.022 INFO:tasks.workunit.client.1.smithi136.stdout:1/815: fdatasync d12/d53/f95 0 2024-04-06T03:40:45.030 INFO:tasks.workunit.client.1.smithi136.stdout:9/814: rename d17/d20/d25/d71/dd5/fff to d17/d19/da3/d101/f119 0 2024-04-06T03:40:45.030 INFO:tasks.workunit.client.1.smithi136.stdout:9/815: chown d17/d21/d32/dc6/d113/dbd 50942285 1 2024-04-06T03:40:45.030 INFO:tasks.workunit.client.1.smithi136.stdout:9/816: chown d17/d19/da3/dc5/deb/f4e 827 1 2024-04-06T03:40:45.030 INFO:tasks.workunit.client.1.smithi136.stdout:9/817: write d17/d21/d32/d3f/f44 [1835155,5921] 0 2024-04-06T03:40:45.030 INFO:tasks.workunit.client.1.smithi136.stdout:9/818: fsync d17/d20/d28/d81/d88/dcb/ff6 0 2024-04-06T03:40:45.030 INFO:tasks.workunit.client.1.smithi136.stdout:9/819: fdatasync d17/d20/d25/da4/dbb/fc4 0 2024-04-06T03:40:45.030 INFO:tasks.workunit.client.1.smithi136.stdout:9/820: write d17/d20/fba [1367050,71702] 0 2024-04-06T03:40:45.030 INFO:tasks.workunit.client.0.smithi094.stdout:5/743: dread db/fa8 [4194304,4194304] 0 2024-04-06T03:40:45.031 INFO:tasks.workunit.client.1.smithi136.stdout:4/705: unlink d1d/d2f/d6c/da5/fcd 0 2024-04-06T03:40:45.032 INFO:tasks.workunit.client.1.smithi136.stdout:4/706: creat d1d/d2f/d6c/d91/ffd x:0 0 0 2024-04-06T03:40:45.034 INFO:tasks.workunit.client.0.smithi094.stdout:4/816: creat d0/df/d1e/f113 x:0 0 0 2024-04-06T03:40:45.034 INFO:tasks.workunit.client.0.smithi094.stdout:4/817: creat d0/df/db3/dd8/f114 x:0 0 0 2024-04-06T03:40:45.035 INFO:tasks.workunit.client.0.smithi094.stdout:4/818: dread d0/df/d1e/fa7 [0,4194304] 0 2024-04-06T03:40:45.036 INFO:tasks.workunit.client.0.smithi094.stdout:2/723: creat d0/d8/ff6 x:0 0 0 2024-04-06T03:40:45.040 INFO:tasks.workunit.client.0.smithi094.stdout:5/744: write db/d19/d2c/d57/f5c [3188165,73137] 0 2024-04-06T03:40:45.040 INFO:tasks.workunit.client.0.smithi094.stdout:7/672: rename d2/d6/dc/d12/d3a/d56 to d2/d6/dc/d12/d9a/da0/dd4 0 2024-04-06T03:40:45.042 INFO:tasks.workunit.client.0.smithi094.stdout:0/736: creat d1/d9/dd/d18/d1e/def/d67/df6/ff9 x:0 0 0 2024-04-06T03:40:45.042 INFO:tasks.workunit.client.1.smithi136.stdout:8/854: dwrite d3/da/d26/d90/dae/fda [0,4194304] 0 2024-04-06T03:40:45.045 INFO:tasks.workunit.client.1.smithi136.stdout:4/707: dread d1d/d2f/f97 [0,4194304] 0 2024-04-06T03:40:45.045 INFO:tasks.workunit.client.1.smithi136.stdout:4/708: stat d1d/d31/d3e/dd4/ffb 0 2024-04-06T03:40:45.050 INFO:tasks.workunit.client.1.smithi136.stdout:0/782: mknod d1/d6/d2c/d57/d43/dcf/de1/c10b 0 2024-04-06T03:40:45.053 INFO:tasks.workunit.client.0.smithi094.stdout:1/745: rename d0/d1/d27/da5/d3f/c56 to d0/d1/d27/da5/d40/db1/d2f/d3c/d48/dcb/c100 0 2024-04-06T03:40:45.053 INFO:tasks.workunit.client.0.smithi094.stdout:5/745: write db/d19/d24/d2e/fb3 [498560,100495] 0 2024-04-06T03:40:45.055 INFO:tasks.workunit.client.0.smithi094.stdout:7/673: mknod d2/d6/dc/d12/cd5 0 2024-04-06T03:40:45.056 INFO:tasks.workunit.client.0.smithi094.stdout:2/724: dwrite d0/d8/d1d/f38 [0,4194304] 0 2024-04-06T03:40:45.058 INFO:tasks.workunit.client.0.smithi094.stdout:0/737: write d1/d9/dd/d18/d1e/d22/d35/dac/ff4 [279188,48466] 0 2024-04-06T03:40:45.061 INFO:tasks.workunit.client.0.smithi094.stdout:5/746: mknod db/d19/d21/c100 0 2024-04-06T03:40:45.064 INFO:tasks.workunit.client.0.smithi094.stdout:7/674: symlink d2/d6/dc/d12/d61/ld6 0 2024-04-06T03:40:45.065 INFO:tasks.workunit.client.0.smithi094.stdout:9/690: sync 2024-04-06T03:40:45.070 INFO:tasks.workunit.client.1.smithi136.stdout:6/727: rmdir d2/d7/d26/d30/d9c/de7 0 2024-04-06T03:40:45.071 INFO:tasks.workunit.client.0.smithi094.stdout:2/725: symlink d0/dd/lf7 0 2024-04-06T03:40:45.073 INFO:tasks.workunit.client.1.smithi136.stdout:3/754: creat d0/d4/d12/d19/d55/d6f/d82/f103 x:0 0 0 2024-04-06T03:40:45.074 INFO:tasks.workunit.client.1.smithi136.stdout:2/848: unlink d3/d9/fe 0 2024-04-06T03:40:45.075 INFO:tasks.workunit.client.0.smithi094.stdout:0/738: creat d1/d9/dd/d18/d1a/d44/dc2/ffa x:0 0 0 2024-04-06T03:40:45.077 INFO:tasks.workunit.client.0.smithi094.stdout:1/746: rmdir d0/d7/d1a/d74 39 2024-04-06T03:40:45.077 INFO:tasks.workunit.client.0.smithi094.stdout:1/747: fsync d0/d1/d27/da5/d40/db1/d2f/d3c/da0/fdf 0 2024-04-06T03:40:45.077 INFO:tasks.workunit.client.0.smithi094.stdout:1/748: stat d0/d1/d27/da5/d40/db1/dc3/cd8 0 2024-04-06T03:40:45.080 INFO:tasks.workunit.client.0.smithi094.stdout:5/747: mknod db/d19/dc1/dd4/dcf/c101 0 2024-04-06T03:40:45.080 INFO:tasks.workunit.client.0.smithi094.stdout:5/748: stat db/d19/dc1/dd4/d77/f6a 0 2024-04-06T03:40:45.080 INFO:tasks.workunit.client.0.smithi094.stdout:5/749: dread - db/d19/d30/d83/fe4 zero size 2024-04-06T03:40:45.081 INFO:tasks.workunit.client.1.smithi136.stdout:9/821: getdents d17/d21/d29/de5/da5 0 2024-04-06T03:40:45.082 INFO:tasks.workunit.client.0.smithi094.stdout:7/675: rename d2/d6/dc/dd/df/d36/d79/lab to d2/d6/dc/dd/df/d36/ld7 0 2024-04-06T03:40:45.087 INFO:tasks.workunit.client.0.smithi094.stdout:0/739: symlink d1/d9/dd/d18/d1e/def/d67/df6/lfb 0 2024-04-06T03:40:45.090 INFO:tasks.workunit.client.0.smithi094.stdout:5/750: mkdir db/d19/d2c/d6d/de6/d102 0 2024-04-06T03:40:45.091 INFO:tasks.workunit.client.1.smithi136.stdout:8/855: symlink d3/de/d18/d7d/ddf/d6c/dab/l109 0 2024-04-06T03:40:45.095 INFO:tasks.workunit.client.0.smithi094.stdout:2/726: rename d0/d8/d27/d2b/d8e/fac to d0/dd/d19/d31/d67/d98/ddc/ff8 0 2024-04-06T03:40:45.095 INFO:tasks.workunit.client.0.smithi094.stdout:2/727: readlink d0/dd/d1b/d72/l9c 0 2024-04-06T03:40:45.097 INFO:tasks.workunit.client.1.smithi136.stdout:4/709: unlink d1d/d2f/d6c/d91/fc8 0 2024-04-06T03:40:45.099 INFO:tasks.workunit.client.0.smithi094.stdout:7/676: symlink d2/d6/dc/dd/df/d95/dc9/da4/db7/ld8 0 2024-04-06T03:40:45.102 INFO:tasks.workunit.client.0.smithi094.stdout:9/691: getdents d2/dc/d46 0 2024-04-06T03:40:45.106 INFO:tasks.workunit.client.0.smithi094.stdout:0/740: mknod d1/d9/dd/d18/d1e/def/d67/cfc 0 2024-04-06T03:40:45.108 INFO:tasks.workunit.client.1.smithi136.stdout:6/728: rename d2/d7/d26/d30/d6e/d58/d5d/d62/f64 to d2/d93/dab/ff0 0 2024-04-06T03:40:45.109 INFO:tasks.workunit.client.0.smithi094.stdout:7/677: dread d2/d6/dac/f70 [4194304,4194304] 0 2024-04-06T03:40:45.109 INFO:tasks.workunit.client.0.smithi094.stdout:7/678: fsync d2/d6/dc/dd/f28 0 2024-04-06T03:40:45.110 INFO:tasks.workunit.client.0.smithi094.stdout:5/751: symlink db/d19/d24/d28/dec/l103 0 2024-04-06T03:40:45.111 INFO:tasks.workunit.client.1.smithi136.stdout:3/755: getdents d0/d4/d12/d19/d55/d6f/d82/d5d/d6a 0 2024-04-06T03:40:45.111 INFO:tasks.workunit.client.1.smithi136.stdout:3/756: fdatasync d0/f13 0 2024-04-06T03:40:45.111 INFO:tasks.workunit.client.1.smithi136.stdout:3/757: write d0/d4/d12/d2d/fbe [1815565,93851] 0 2024-04-06T03:40:45.112 INFO:tasks.workunit.client.1.smithi136.stdout:2/849: creat d3/d9/d3b/d87/db1/f11a x:0 0 0 2024-04-06T03:40:45.114 INFO:tasks.workunit.client.0.smithi094.stdout:1/749: rename d0/d7/d2d/l22 to d0/d7/l101 0 2024-04-06T03:40:45.118 INFO:tasks.workunit.client.0.smithi094.stdout:0/741: write d1/d9/dd/d18/d32/f9c [6692025,66451] 0 2024-04-06T03:40:45.119 INFO:tasks.workunit.client.0.smithi094.stdout:0/742: write d1/d9/dd/d18/d1e/f8c [27571,57371] 0 2024-04-06T03:40:45.119 INFO:tasks.workunit.client.1.smithi136.stdout:2/850: write d3/d4a/ff2 [811914,95329] 0 2024-04-06T03:40:45.119 INFO:tasks.workunit.client.1.smithi136.stdout:2/851: readlink d3/d4a/d9d/d18/d1d/d91/lbc 0 2024-04-06T03:40:45.121 INFO:tasks.workunit.client.0.smithi094.stdout:2/728: unlink d0/dd/d1b/d72/fef 0 2024-04-06T03:40:45.122 INFO:tasks.workunit.client.1.smithi136.stdout:4/710: mknod d1d/d31/d3e/d55/dba/cfe 0 2024-04-06T03:40:45.123 INFO:tasks.workunit.client.0.smithi094.stdout:9/692: creat d2/ffc x:0 0 0 2024-04-06T03:40:45.124 INFO:tasks.workunit.client.0.smithi094.stdout:1/750: dread d0/d1/d27/da5/d40/db1/d2f/d4b/f91 [0,4194304] 0 2024-04-06T03:40:45.124 INFO:tasks.workunit.client.1.smithi136.stdout:8/856: dwrite d3/f2a [0,4194304] 0 2024-04-06T03:40:45.124 INFO:tasks.workunit.client.1.smithi136.stdout:3/758: write d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 [1408376,117177] 0 2024-04-06T03:40:45.124 INFO:tasks.workunit.client.1.smithi136.stdout:3/759: creat d0/d4/d12/d19/d55/d3c/d46/f104 x:0 0 0 2024-04-06T03:40:45.124 INFO:tasks.workunit.client.1.smithi136.stdout:3/760: chown d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/ffd 2 1 2024-04-06T03:40:45.125 INFO:tasks.workunit.client.1.smithi136.stdout:3/761: chown d0/d4/d12/d3a/l68 0 1 2024-04-06T03:40:45.125 INFO:tasks.workunit.client.1.smithi136.stdout:3/762: readlink d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/la1 0 2024-04-06T03:40:45.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/743: dread d1/d9/dd/d18/d1a/f54 [0,4194304] 0 2024-04-06T03:40:45.126 INFO:tasks.workunit.client.0.smithi094.stdout:0/744: fsync d1/d9/dd/d18/d1e/def/dab/d4e/ddf/f99 0 2024-04-06T03:40:45.128 INFO:tasks.workunit.client.1.smithi136.stdout:9/822: link d17/d19/da3/dc5/deb/d9e/dfb/f107 d17/d21/d32/dc6/d113/f11a 0 2024-04-06T03:40:45.130 INFO:tasks.workunit.client.0.smithi094.stdout:7/679: symlink d2/d6/dc/d12/d9a/da0/dd4/db1/ld9 0 2024-04-06T03:40:45.130 INFO:tasks.workunit.client.1.smithi136.stdout:6/729: dwrite d2/d7/d26/d30/d6e/f2f [0,4194304] 0 2024-04-06T03:40:45.132 INFO:tasks.workunit.client.1.smithi136.stdout:2/852: symlink d3/d15/l11b 0 2024-04-06T03:40:45.132 INFO:tasks.workunit.client.1.smithi136.stdout:2/853: write d3/d15/fe7 [957976,14522] 0 2024-04-06T03:40:45.132 INFO:tasks.workunit.client.1.smithi136.stdout:2/854: readlink d3/d4a/d9d/d18/d1d/d3c/leb 0 2024-04-06T03:40:45.133 INFO:tasks.workunit.client.0.smithi094.stdout:7/680: write d2/d6/dc/d12/d9a/da0/faf [89628,21876] 0 2024-04-06T03:40:45.133 INFO:tasks.workunit.client.0.smithi094.stdout:5/752: creat db/d7c/dd7/f104 x:0 0 0 2024-04-06T03:40:45.133 INFO:tasks.workunit.client.0.smithi094.stdout:5/753: write db/d19/d24/d28/f8c [1167571,81676] 0 2024-04-06T03:40:45.135 INFO:tasks.workunit.client.0.smithi094.stdout:2/729: creat d0/d18/ff9 x:0 0 0 2024-04-06T03:40:45.138 INFO:tasks.workunit.client.0.smithi094.stdout:1/751: dread d0/d1/d27/da5/d3f/f5c [0,4194304] 0 2024-04-06T03:40:45.139 INFO:tasks.workunit.client.0.smithi094.stdout:0/745: dwrite d1/d9/dd/d18/d1e/d86/fc5 [0,4194304] 0 2024-04-06T03:40:45.139 INFO:tasks.workunit.client.0.smithi094.stdout:9/693: unlink d2/dc/d21/ld1 0 2024-04-06T03:40:45.142 INFO:tasks.workunit.client.0.smithi094.stdout:7/681: rename d2/d6/dc/l13 to d2/d6/dc/d12/d61/lda 0 2024-04-06T03:40:45.144 INFO:tasks.workunit.client.1.smithi136.stdout:8/857: symlink d3/da/d26/d90/dae/l10a 0 2024-04-06T03:40:45.146 INFO:tasks.workunit.client.0.smithi094.stdout:5/754: mknod db/d19/d30/d83/c105 0 2024-04-06T03:40:45.150 INFO:tasks.workunit.client.1.smithi136.stdout:3/763: link d0/d4/d12/d19/d2e/d3b/f5f d0/f105 0 2024-04-06T03:40:45.151 INFO:tasks.workunit.client.0.smithi094.stdout:1/752: symlink d0/d1/d27/da5/db2/dce/l102 0 2024-04-06T03:40:45.151 INFO:tasks.workunit.client.0.smithi094.stdout:1/753: chown d0/d1/ca4 21469289 1 2024-04-06T03:40:45.152 INFO:tasks.workunit.client.1.smithi136.stdout:9/823: creat d17/d21/d29/de5/f11b x:0 0 0 2024-04-06T03:40:45.154 INFO:tasks.workunit.client.0.smithi094.stdout:9/694: mkdir d2/dc/d21/d26/d8a/de9/dfd 0 2024-04-06T03:40:45.154 INFO:tasks.workunit.client.0.smithi094.stdout:9/695: readlink d2/d2c/d33/lc3 0 2024-04-06T03:40:45.154 INFO:tasks.workunit.client.0.smithi094.stdout:4/819: dwrite d0/df/d1e/f7f [0,4194304] 0 2024-04-06T03:40:45.154 INFO:tasks.workunit.client.0.smithi094.stdout:4/820: write d0/d15/d3a/d3e/d46/f72 [463785,32818] 0 2024-04-06T03:40:45.154 INFO:tasks.workunit.client.0.smithi094.stdout:4/821: read - d0/d15/d3a/d61/d81/fc0 zero size 2024-04-06T03:40:45.155 INFO:tasks.workunit.client.0.smithi094.stdout:4/822: fdatasync d0/d15/d3a/d3e/d49/f10d 0 2024-04-06T03:40:45.155 INFO:tasks.workunit.client.0.smithi094.stdout:4/823: stat d0/df/db3/dd8/f114 0 2024-04-06T03:40:45.155 INFO:tasks.workunit.client.0.smithi094.stdout:4/824: write d0/d4/fd [3524658,76832] 0 2024-04-06T03:40:45.159 INFO:tasks.workunit.client.1.smithi136.stdout:6/730: mknod d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/cf1 0 2024-04-06T03:40:45.161 INFO:tasks.workunit.client.0.smithi094.stdout:7/682: symlink d2/d6/dc/dd/d90/ldb 0 2024-04-06T03:40:45.161 INFO:tasks.workunit.client.0.smithi094.stdout:7/683: readlink d2/d6/dc/dd/df/d95/dc9/da4/db7/ld8 0 2024-04-06T03:40:45.162 INFO:tasks.workunit.client.0.smithi094.stdout:5/755: write db/d19/d24/f34 [5064489,100226] 0 2024-04-06T03:40:45.162 INFO:tasks.workunit.client.0.smithi094.stdout:5/756: creat db/d19/d2c/f106 x:0 0 0 2024-04-06T03:40:45.162 INFO:tasks.workunit.client.0.smithi094.stdout:5/757: dread - db/d19/d2c/fc2 zero size 2024-04-06T03:40:45.162 INFO:tasks.workunit.client.0.smithi094.stdout:5/758: chown db/d19/d24/f37 38130107 1 2024-04-06T03:40:45.163 INFO:tasks.workunit.client.0.smithi094.stdout:1/754: symlink d0/d1/d8f/l103 0 2024-04-06T03:40:45.164 INFO:tasks.workunit.client.1.smithi136.stdout:2/855: symlink d3/d15/d19/dac/l11c 0 2024-04-06T03:40:45.164 INFO:tasks.workunit.client.1.smithi136.stdout:2/856: creat d3/d15/d19/d43/daa/dce/f11d x:0 0 0 2024-04-06T03:40:45.164 INFO:tasks.workunit.client.1.smithi136.stdout:2/857: creat d3/d4a/d9d/d8a/d93/d5f/df0/f11e x:0 0 0 2024-04-06T03:40:45.166 INFO:tasks.workunit.client.1.smithi136.stdout:8/858: creat d3/de/d18/d7d/ddf/d6c/f10b x:0 0 0 2024-04-06T03:40:45.169 INFO:tasks.workunit.client.1.smithi136.stdout:6/731: mknod d2/d93/dab/cf2 0 2024-04-06T03:40:45.170 INFO:tasks.workunit.client.1.smithi136.stdout:9/824: dread d17/d20/d28/d49/d75/f91 [0,4194304] 0 2024-04-06T03:40:45.174 INFO:tasks.workunit.client.1.smithi136.stdout:2/858: mkdir d3/d15/d19/d11f 0 2024-04-06T03:40:45.175 INFO:tasks.workunit.client.1.smithi136.stdout:5/868: sync 2024-04-06T03:40:45.176 INFO:tasks.workunit.client.0.smithi094.stdout:3/761: sync 2024-04-06T03:40:45.176 INFO:tasks.workunit.client.0.smithi094.stdout:8/784: sync 2024-04-06T03:40:45.176 INFO:tasks.workunit.client.0.smithi094.stdout:6/721: sync 2024-04-06T03:40:45.178 INFO:tasks.workunit.client.1.smithi136.stdout:8/859: link d3/da/d26/d35/d53/f5b d3/da/da0/f10c 0 2024-04-06T03:40:45.181 INFO:tasks.workunit.client.0.smithi094.stdout:7/684: symlink d2/d6/d93/ldc 0 2024-04-06T03:40:45.182 INFO:tasks.workunit.client.1.smithi136.stdout:6/732: creat d2/d7/d13/d79/ff3 x:0 0 0 2024-04-06T03:40:45.184 INFO:tasks.workunit.client.1.smithi136.stdout:3/764: dwrite d0/d4/d12/d19/d55/d3c/f58 [0,4194304] 0 2024-04-06T03:40:45.184 INFO:tasks.workunit.client.1.smithi136.stdout:3/765: stat d0/d4/d12/d19/d55/f24 0 2024-04-06T03:40:45.186 INFO:tasks.workunit.client.0.smithi094.stdout:7/685: write d2/d6/dc/dd/f2f [1383426,110332] 0 2024-04-06T03:40:45.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/746: rename d1/d9/dd/d18/d1e/d22/d35 to d1/d9/dd/d18/d5d/d60/dfd 0 2024-04-06T03:40:45.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/747: creat d1/d9/dd/d18/d78/ffe x:0 0 0 2024-04-06T03:40:45.190 INFO:tasks.workunit.client.0.smithi094.stdout:0/748: chown d1/d9/dd/d18/d1a/f25 3326487 1 2024-04-06T03:40:45.191 INFO:tasks.workunit.client.0.smithi094.stdout:2/730: getdents d0/d8/d27/d2b/d8e 0 2024-04-06T03:40:45.193 INFO:tasks.workunit.client.1.smithi136.stdout:5/869: rename d1/d16/d4b/l5a to d1/d9b/l107 0 2024-04-06T03:40:45.195 INFO:tasks.workunit.client.0.smithi094.stdout:8/785: mkdir d18/d2c/d37/d60/da3/d111 0 2024-04-06T03:40:45.196 INFO:tasks.workunit.client.0.smithi094.stdout:8/786: fsync d18/d2c/d37/d60/fde 0 2024-04-06T03:40:45.196 INFO:tasks.workunit.client.0.smithi094.stdout:8/787: chown f14 1039440520 1 2024-04-06T03:40:45.199 INFO:tasks.workunit.client.0.smithi094.stdout:3/762: dwrite d9/d1e/d26/d55/f65 [0,4194304] 0 2024-04-06T03:40:45.199 INFO:tasks.workunit.client.0.smithi094.stdout:4/825: dwrite d0/d4/f76 [0,4194304] 0 2024-04-06T03:40:45.199 INFO:tasks.workunit.client.0.smithi094.stdout:4/826: dread - d0/df/d1e/d34/db5/ffe zero size 2024-04-06T03:40:45.200 INFO:tasks.workunit.client.1.smithi136.stdout:2/859: dwrite d3/d4a/d9d/d18/d1d/d3c/f10c [0,4194304] 0 2024-04-06T03:40:45.200 INFO:tasks.workunit.client.1.smithi136.stdout:2/860: fdatasync d3/d15/d19/d43/d54/d10b/f118 0 2024-04-06T03:40:45.200 INFO:tasks.workunit.client.0.smithi094.stdout:9/696: dwrite d2/dc/d21/d23/d5b/ff0 [0,4194304] 0 2024-04-06T03:40:45.201 INFO:tasks.workunit.client.0.smithi094.stdout:9/697: write d2/dc/d80/fac [38216,9848] 0 2024-04-06T03:40:45.202 INFO:tasks.workunit.client.1.smithi136.stdout:8/860: mkdir d3/da/d26/d35/dcb/d10d 0 2024-04-06T03:40:45.204 INFO:tasks.workunit.client.1.smithi136.stdout:6/733: write d2/d7/d26/d30/d6e/f35 [1585138,22427] 0 2024-04-06T03:40:45.204 INFO:tasks.workunit.client.1.smithi136.stdout:6/734: dread - d2/d7/d26/d30/d4f/fcb zero size 2024-04-06T03:40:45.205 INFO:tasks.workunit.client.0.smithi094.stdout:7/686: mkdir d2/d6/dac/ddd 0 2024-04-06T03:40:45.207 INFO:tasks.workunit.client.0.smithi094.stdout:1/755: getdents d0/d1/dd3/de7 0 2024-04-06T03:40:45.208 INFO:tasks.workunit.client.1.smithi136.stdout:5/870: dwrite d1/dc/f91 [0,4194304] 0 2024-04-06T03:40:45.208 INFO:tasks.workunit.client.1.smithi136.stdout:5/871: chown d1/d16/d17/c6f 12093990 1 2024-04-06T03:40:45.210 INFO:tasks.workunit.client.0.smithi094.stdout:5/759: rename db/d19/dc1/dd4/lff to db/d80/l107 0 2024-04-06T03:40:45.214 INFO:tasks.workunit.client.0.smithi094.stdout:2/731: mknod d0/dd/d19/dbe/cfa 0 2024-04-06T03:40:45.217 INFO:tasks.workunit.client.0.smithi094.stdout:3/763: unlink d9/d9d/fe3 0 2024-04-06T03:40:45.217 INFO:tasks.workunit.client.0.smithi094.stdout:3/764: chown d9/d20/f4e 0 1 2024-04-06T03:40:45.218 INFO:tasks.workunit.client.1.smithi136.stdout:2/861: dwrite d3/d4a/d9d/d8a/d93/f67 [0,4194304] 0 2024-04-06T03:40:45.220 INFO:tasks.workunit.client.0.smithi094.stdout:8/788: dwrite d18/d2c/d37/fa2 [4194304,4194304] 0 2024-04-06T03:40:45.223 INFO:tasks.workunit.client.0.smithi094.stdout:6/722: creat dc/d3a/d4a/d94/f104 x:0 0 0 2024-04-06T03:40:45.223 INFO:tasks.workunit.client.0.smithi094.stdout:6/723: chown dc/c6e 14527739 1 2024-04-06T03:40:45.225 INFO:tasks.workunit.client.1.smithi136.stdout:8/861: creat d3/f10e x:0 0 0 2024-04-06T03:40:45.225 INFO:tasks.workunit.client.1.smithi136.stdout:8/862: creat d3/da/d26/d90/dae/db5/f10f x:0 0 0 2024-04-06T03:40:45.226 INFO:tasks.workunit.client.1.smithi136.stdout:7/881: sync 2024-04-06T03:40:45.227 INFO:tasks.workunit.client.0.smithi094.stdout:9/698: unlink d2/dc/da6/ce7 0 2024-04-06T03:40:45.229 INFO:tasks.workunit.client.0.smithi094.stdout:1/756: mknod d0/d1/d27/da5/d3f/c104 0 2024-04-06T03:40:45.229 INFO:tasks.workunit.client.0.smithi094.stdout:1/757: truncate d0/d1/d27/da5/d40/ff3 276925 0 2024-04-06T03:40:45.233 INFO:tasks.workunit.client.0.smithi094.stdout:0/749: rename d1/d9/dd/d18/d1a/f4d to d1/d9/dd/d18/d32/d42/fff 0 2024-04-06T03:40:45.235 INFO:tasks.workunit.client.1.smithi136.stdout:6/735: link d2/d7/d26/d30/d6e/c34 d2/d7/d26/d30/d4f/cf4 0 2024-04-06T03:40:45.235 INFO:tasks.workunit.client.1.smithi136.stdout:6/736: fsync d2/d7/d26/d30/d6e/d58/f8e 0 2024-04-06T03:40:45.237 INFO:tasks.workunit.client.0.smithi094.stdout:5/760: creat db/d19/d24/d28/f108 x:0 0 0 2024-04-06T03:40:45.239 INFO:tasks.workunit.client.0.smithi094.stdout:2/732: getdents d0/dd/d1b/d72/d8f 0 2024-04-06T03:40:45.239 INFO:tasks.workunit.client.1.smithi136.stdout:7/882: dwrite d5/df/d53/d68/fb0 [0,4194304] 0 2024-04-06T03:40:45.240 INFO:tasks.workunit.client.0.smithi094.stdout:4/827: link d0/d15/d3a/d61/d81/da0/cd5 d0/c115 0 2024-04-06T03:40:45.242 INFO:tasks.workunit.client.1.smithi136.stdout:5/872: mkdir d1/dc/d1c/d32/d8b/db0/d108 0 2024-04-06T03:40:45.242 INFO:tasks.workunit.client.1.smithi136.stdout:5/873: fdatasync d1/d16/d4b/f95 0 2024-04-06T03:40:45.243 INFO:tasks.workunit.client.0.smithi094.stdout:6/724: stat c6 0 2024-04-06T03:40:45.246 INFO:tasks.workunit.client.1.smithi136.stdout:7/883: read d5/df/d1d/d54/fa4 [11862,4705] 0 2024-04-06T03:40:45.246 INFO:tasks.workunit.client.1.smithi136.stdout:8/863: rmdir d3/da/d26/d35/d3d/dc7 39 2024-04-06T03:40:45.246 INFO:tasks.workunit.client.1.smithi136.stdout:8/864: chown d3/da/d26/d35/d39/d70 1 1 2024-04-06T03:40:45.247 INFO:tasks.workunit.client.0.smithi094.stdout:9/699: creat d2/dc/ffe x:0 0 0 2024-04-06T03:40:45.247 INFO:tasks.workunit.client.0.smithi094.stdout:9/700: write d2/d2c/d99/fb4 [990174,97462] 0 2024-04-06T03:40:45.247 INFO:tasks.workunit.client.0.smithi094.stdout:9/701: fdatasync d2/dc/d46/f4e 0 2024-04-06T03:40:45.247 INFO:tasks.workunit.client.0.smithi094.stdout:9/702: chown d2/d2c/ld7 8 1 2024-04-06T03:40:45.248 INFO:tasks.workunit.client.1.smithi136.stdout:6/737: creat d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/ff5 x:0 0 0 2024-04-06T03:40:45.255 INFO:tasks.workunit.client.0.smithi094.stdout:6/725: write dc/d3a/df4/df6/fa5 [2092039,111344] 0 2024-04-06T03:40:45.256 INFO:tasks.workunit.client.1.smithi136.stdout:5/874: mkdir d1/dc/d1c/d109 0 2024-04-06T03:40:45.257 INFO:tasks.workunit.client.0.smithi094.stdout:1/758: mknod d0/d1/d27/da5/d40/db1/d2f/d76/c105 0 2024-04-06T03:40:45.259 INFO:tasks.workunit.client.0.smithi094.stdout:9/703: read d2/dc/d21/d23/d2f/f91 [91605,101425] 0 2024-04-06T03:40:45.261 INFO:tasks.workunit.client.0.smithi094.stdout:7/687: rename d2/d6/dc/dd/df/d95/dc9/da4 to d2/d6/dc/d12/d3a/dcc/dde 0 2024-04-06T03:40:45.261 INFO:tasks.workunit.client.0.smithi094.stdout:7/688: chown d2/d6/dc/d12/d61/ld6 45294599 1 2024-04-06T03:40:45.265 INFO:tasks.workunit.client.0.smithi094.stdout:5/761: symlink db/d7c/d8b/l109 0 2024-04-06T03:40:45.267 INFO:tasks.workunit.client.1.smithi136.stdout:7/884: rename d5/df/d53/d68/da3/dd0 to d5/df/d53/d68/d135 0 2024-04-06T03:40:45.268 INFO:tasks.workunit.client.0.smithi094.stdout:2/733: mkdir d0/d8/d27/d2b/d8e/dfb 0 2024-04-06T03:40:45.269 INFO:tasks.workunit.client.1.smithi136.stdout:9/825: dwrite f16 [0,4194304] 0 2024-04-06T03:40:45.269 INFO:tasks.workunit.client.1.smithi136.stdout:1/816: sync 2024-04-06T03:40:45.269 INFO:tasks.workunit.client.1.smithi136.stdout:0/783: sync 2024-04-06T03:40:45.269 INFO:tasks.workunit.client.1.smithi136.stdout:8/865: rmdir d3/da/d26/d35/d53/dd2 39 2024-04-06T03:40:45.269 INFO:tasks.workunit.client.1.smithi136.stdout:0/784: write d1/d6/d2c/d57/f66 [4917743,64385] 0 2024-04-06T03:40:45.270 INFO:tasks.workunit.client.0.smithi094.stdout:4/828: symlink d0/d15/d3a/d61/l116 0 2024-04-06T03:40:45.271 INFO:tasks.workunit.client.0.smithi094.stdout:4/829: stat d0/d4/d67/dcb/le4 0 2024-04-06T03:40:45.272 INFO:tasks.workunit.client.1.smithi136.stdout:5/875: rename d1/d16/d89/d8a/dfc/lf0 to d1/d16/d4b/l10a 0 2024-04-06T03:40:45.274 INFO:tasks.workunit.client.1.smithi136.stdout:7/885: creat d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/f136 x:0 0 0 2024-04-06T03:40:45.274 INFO:tasks.workunit.client.1.smithi136.stdout:7/886: chown d5/df/f10 414148 1 2024-04-06T03:40:45.275 INFO:tasks.workunit.client.0.smithi094.stdout:1/759: link d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/db9/fea d0/d1/d27/da5/d40/d50/f106 0 2024-04-06T03:40:45.276 INFO:tasks.workunit.client.1.smithi136.stdout:9/826: read d17/d19/da3/dc5/fec [134987,33679] 0 2024-04-06T03:40:45.276 INFO:tasks.workunit.client.1.smithi136.stdout:8/866: mknod d3/da/d26/d35/dcb/d10d/c110 0 2024-04-06T03:40:45.277 INFO:tasks.workunit.client.0.smithi094.stdout:9/704: mknod d2/dc/d21/d26/d78/cff 0 2024-04-06T03:40:45.278 INFO:tasks.workunit.client.1.smithi136.stdout:1/817: dread d12/d1b/f86 [0,4194304] 0 2024-04-06T03:40:45.279 INFO:tasks.workunit.client.1.smithi136.stdout:0/785: rename d1/d6/d2c/d57/d43/d6d/d85/da7/fb4 to d1/d4e/f10c 0 2024-04-06T03:40:45.279 INFO:tasks.workunit.client.1.smithi136.stdout:0/786: chown d1/d6/d2c/d57/d43/d6d/d85/ce4 2463 1 2024-04-06T03:40:45.280 INFO:tasks.workunit.client.0.smithi094.stdout:0/750: rename d1/d9/dd/d18/d32/d42/f45 to d1/d9/dd/d18/d5d/de9/f100 0 2024-04-06T03:40:45.281 INFO:tasks.workunit.client.1.smithi136.stdout:5/876: symlink d1/d16/d17/db2/l10b 0 2024-04-06T03:40:45.281 INFO:tasks.workunit.client.0.smithi094.stdout:7/689: symlink d2/d6/dc/d12/d9a/dbd/ldf 0 2024-04-06T03:40:45.281 INFO:tasks.workunit.client.0.smithi094.stdout:7/690: stat d2/d6/dc/dd/df/d36/fca 0 2024-04-06T03:40:45.283 INFO:tasks.workunit.client.1.smithi136.stdout:7/887: mknod d5/df/d12/d1b/dbb/dbe/c137 0 2024-04-06T03:40:45.283 INFO:tasks.workunit.client.1.smithi136.stdout:7/888: write d5/df/d12/d1b/d3a/d4a/ddf/f12f [960794,55295] 0 2024-04-06T03:40:45.283 INFO:tasks.workunit.client.1.smithi136.stdout:7/889: chown d5/df/d53/d68/d135/df0/ffe 18 1 2024-04-06T03:40:45.286 INFO:tasks.workunit.client.1.smithi136.stdout:9/827: getdents d17/d21/d32/dc6/d113/d66/de1/d51 0 2024-04-06T03:40:45.286 INFO:tasks.workunit.client.1.smithi136.stdout:7/890: dread d5/d6d/f107 [0,4194304] 0 2024-04-06T03:40:45.286 INFO:tasks.workunit.client.1.smithi136.stdout:7/891: readlink d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/l65 0 2024-04-06T03:40:45.287 INFO:tasks.workunit.client.0.smithi094.stdout:5/762: mknod db/d19/d24/d28/dec/c10a 0 2024-04-06T03:40:45.287 INFO:tasks.workunit.client.0.smithi094.stdout:4/830: symlink d0/d15/d3a/d61/d81/l117 0 2024-04-06T03:40:45.287 INFO:tasks.workunit.client.1.smithi136.stdout:1/818: symlink d12/d3b/d44/d5f/d5a/l113 0 2024-04-06T03:40:45.291 INFO:tasks.workunit.client.0.smithi094.stdout:1/760: rename d0/d7/d2d/l63 to d0/d1/d27/da5/d3f/de9/l107 0 2024-04-06T03:40:45.293 INFO:tasks.workunit.client.1.smithi136.stdout:0/787: mkdir d1/d6/d2c/d57/d43/d9e/d10d 0 2024-04-06T03:40:45.293 INFO:tasks.workunit.client.1.smithi136.stdout:0/788: readlink d1/d6/d2c/d1a/ld0 0 2024-04-06T03:40:45.293 INFO:tasks.workunit.client.1.smithi136.stdout:0/789: dread - d1/d6/d2c/d57/d43/d9e/f102 zero size 2024-04-06T03:40:45.295 INFO:tasks.workunit.client.0.smithi094.stdout:0/751: creat d1/d9/dd/d18/d5d/d60/dbc/dda/f101 x:0 0 0 2024-04-06T03:40:45.296 INFO:tasks.workunit.client.1.smithi136.stdout:9/828: rmdir d17/d21/d29/de5/da5 39 2024-04-06T03:40:45.299 INFO:tasks.workunit.client.1.smithi136.stdout:7/892: rename d5/df/d1d/d54/d5c/dc7/f102 to d5/df/d1d/d54/d5c/f138 0 2024-04-06T03:40:45.299 INFO:tasks.workunit.client.1.smithi136.stdout:7/893: truncate d5/d6d/f119 824503 0 2024-04-06T03:40:45.300 INFO:tasks.workunit.client.0.smithi094.stdout:5/763: stat db/d19/d24/d28/d85/l91 0 2024-04-06T03:40:45.300 INFO:tasks.workunit.client.0.smithi094.stdout:9/705: dwrite d2/dc/d21/d26/d78/d69/da8/f7f [0,4194304] 0 2024-04-06T03:40:45.302 INFO:tasks.workunit.client.1.smithi136.stdout:1/819: symlink d12/d3b/d60/d6d/da2/l114 0 2024-04-06T03:40:45.303 INFO:tasks.workunit.client.1.smithi136.stdout:0/790: mknod d1/d6/d2c/c10e 0 2024-04-06T03:40:45.306 INFO:tasks.workunit.client.1.smithi136.stdout:9/829: creat d17/d21/d29/de5/da5/dce/dd1/d117/f11c x:0 0 0 2024-04-06T03:40:45.307 INFO:tasks.workunit.client.1.smithi136.stdout:5/877: write d1/d16/d4b/f57 [4048389,43382] 0 2024-04-06T03:40:45.308 INFO:tasks.workunit.client.1.smithi136.stdout:4/711: sync 2024-04-06T03:40:45.309 INFO:tasks.workunit.client.1.smithi136.stdout:4/712: write d1d/fbb [723845,25193] 0 2024-04-06T03:40:45.309 INFO:tasks.workunit.client.1.smithi136.stdout:4/713: chown d1d/d2f/da6/de1 219734 1 2024-04-06T03:40:45.309 INFO:tasks.workunit.client.1.smithi136.stdout:4/714: creat d1d/d2f/d6c/da5/fff x:0 0 0 2024-04-06T03:40:45.309 INFO:tasks.workunit.client.1.smithi136.stdout:6/738: dwrite d2/d7/d26/d30/d6e/d58/f7d [0,4194304] 0 2024-04-06T03:40:45.309 INFO:tasks.workunit.client.1.smithi136.stdout:7/894: unlink d5/df/d12/d1b/d3a/c90 0 2024-04-06T03:40:45.309 INFO:tasks.workunit.client.0.smithi094.stdout:3/765: dwrite d9/d20/d30/f4d [4194304,4194304] 0 2024-04-06T03:40:45.309 INFO:tasks.workunit.client.0.smithi094.stdout:1/761: truncate d0/d7/f52 7565912 0 2024-04-06T03:40:45.310 INFO:tasks.workunit.client.1.smithi136.stdout:0/791: stat d1/d6/c7b 0 2024-04-06T03:40:45.310 INFO:tasks.workunit.client.1.smithi136.stdout:0/792: write d1/d6/d2c/d57/d43/ff2 [3919358,99963] 0 2024-04-06T03:40:45.312 INFO:tasks.workunit.client.0.smithi094.stdout:4/831: dwrite d0/d15/fd7 [0,4194304] 0 2024-04-06T03:40:45.313 INFO:tasks.workunit.client.0.smithi094.stdout:7/691: dwrite d2/d6/dc/d12/d29/f89 [0,4194304] 0 2024-04-06T03:40:45.314 INFO:tasks.workunit.client.1.smithi136.stdout:8/867: dwrite d3/da/d26/d35/d53/d86/f87 [0,4194304] 0 2024-04-06T03:40:45.314 INFO:tasks.workunit.client.1.smithi136.stdout:8/868: truncate d3/de/d18/d6d/f7b 2045253 0 2024-04-06T03:40:45.315 INFO:tasks.workunit.client.0.smithi094.stdout:0/752: creat d1/d9/dd/d18/d32/f102 x:0 0 0 2024-04-06T03:40:45.318 INFO:tasks.workunit.client.0.smithi094.stdout:1/762: dread d0/f19 [0,4194304] 0 2024-04-06T03:40:45.319 INFO:tasks.workunit.client.1.smithi136.stdout:9/830: symlink d17/d21/d29/de5/da5/df5/l11d 0 2024-04-06T03:40:45.319 INFO:tasks.workunit.client.1.smithi136.stdout:9/831: write d17/d21/fc9 [605573,32004] 0 2024-04-06T03:40:45.322 INFO:tasks.workunit.client.1.smithi136.stdout:1/820: dwrite d12/d1b/d28/f91 [0,4194304] 0 2024-04-06T03:40:45.322 INFO:tasks.workunit.client.1.smithi136.stdout:1/821: dread - d12/d3b/d44/d5f/fdb zero size 2024-04-06T03:40:45.324 INFO:tasks.workunit.client.1.smithi136.stdout:0/793: dread d1/d6/d2c/d1a/fe7 [0,4194304] 0 2024-04-06T03:40:45.325 INFO:tasks.workunit.client.1.smithi136.stdout:5/878: dwrite d1/d16/d4b/f80 [0,4194304] 0 2024-04-06T03:40:45.326 INFO:tasks.workunit.client.0.smithi094.stdout:9/706: link d2/dc/d21/d23/d5b/ff0 d2/dc/da6/f100 0 2024-04-06T03:40:45.328 INFO:tasks.workunit.client.1.smithi136.stdout:4/715: unlink d1d/d31/d3e/d56/d7b/fdf 0 2024-04-06T03:40:45.328 INFO:tasks.workunit.client.1.smithi136.stdout:4/716: readlink d1d/d2f/lad 0 2024-04-06T03:40:45.328 INFO:tasks.workunit.client.1.smithi136.stdout:8/869: dread d3/d2d/dad/ff3 [4194304,4194304] 0 2024-04-06T03:40:45.330 INFO:tasks.workunit.client.0.smithi094.stdout:3/766: link d9/d1e/d6e/l100 d9/d1e/d26/d55/d62/db8/d10f/l110 0 2024-04-06T03:40:45.330 INFO:tasks.workunit.client.0.smithi094.stdout:3/767: truncate d9/d20/d76/f83 1026259 0 2024-04-06T03:40:45.330 INFO:tasks.workunit.client.0.smithi094.stdout:3/768: chown d9/d1e/d6e/d75/dba/d3a/ccf 3700 1 2024-04-06T03:40:45.337 INFO:tasks.workunit.client.0.smithi094.stdout:4/832: chown d0/l5f 968840 1 2024-04-06T03:40:45.344 INFO:tasks.workunit.client.1.smithi136.stdout:7/895: unlink d5/d6d/f131 0 2024-04-06T03:40:45.344 INFO:tasks.workunit.client.1.smithi136.stdout:7/896: chown d5/f2e 230544 1 2024-04-06T03:40:45.344 INFO:tasks.workunit.client.0.smithi094.stdout:0/753: mknod d1/d9/dd/d18/d1e/def/dab/d4e/de4/c103 0 2024-04-06T03:40:45.344 INFO:tasks.workunit.client.0.smithi094.stdout:0/754: dread - d1/d9/dd/d18/d1a/d44/f80 zero size 2024-04-06T03:40:45.348 INFO:tasks.workunit.client.0.smithi094.stdout:5/764: rmdir db/d19/d24/d28/d85/dd5/dfa 39 2024-04-06T03:40:45.350 INFO:tasks.workunit.client.0.smithi094.stdout:9/707: symlink d2/dc/d21/dc8/l101 0 2024-04-06T03:40:45.352 INFO:tasks.workunit.client.1.smithi136.stdout:9/832: rename d17/d20/d28/d49/d75/f91 to d17/d20/d25/d3c/d55/d8a/f11e 0 2024-04-06T03:40:45.355 INFO:tasks.workunit.client.0.smithi094.stdout:3/769: rename d9/d1e/d6e/d75/dba/d3a/daf/cb6 to d9/d20/d76/c111 0 2024-04-06T03:40:45.356 INFO:tasks.workunit.client.0.smithi094.stdout:5/765: dread db/d19/d2c/f31 [0,4194304] 0 2024-04-06T03:40:45.357 INFO:tasks.workunit.client.0.smithi094.stdout:4/833: mknod d0/d4/d67/c118 0 2024-04-06T03:40:45.363 INFO:tasks.workunit.client.1.smithi136.stdout:0/794: symlink d1/d6/d2c/d57/d43/l10f 0 2024-04-06T03:40:45.365 INFO:tasks.workunit.client.0.smithi094.stdout:1/763: mkdir d0/d7/d1a/d108 0 2024-04-06T03:40:45.367 INFO:tasks.workunit.client.0.smithi094.stdout:9/708: mknod d2/d2c/d99/c102 0 2024-04-06T03:40:45.367 INFO:tasks.workunit.client.0.smithi094.stdout:9/709: write d2/d2c/d33/d64/dc6/fdb [35310,64520] 0 2024-04-06T03:40:45.369 INFO:tasks.workunit.client.0.smithi094.stdout:5/766: mkdir db/d80/d10b 0 2024-04-06T03:40:45.371 INFO:tasks.workunit.client.0.smithi094.stdout:3/770: dwrite d9/d1e/d26/d55/f65 [0,4194304] 0 2024-04-06T03:40:45.371 INFO:tasks.workunit.client.0.smithi094.stdout:3/771: readlink d9/d10/dbb/de1/leb 0 2024-04-06T03:40:45.373 INFO:tasks.workunit.client.1.smithi136.stdout:5/879: link d1/d16/d4b/d7b/ff6 d1/dc/d1c/f10c 0 2024-04-06T03:40:45.373 INFO:tasks.workunit.client.0.smithi094.stdout:1/764: truncate d0/f93 4609268 0 2024-04-06T03:40:45.375 INFO:tasks.workunit.client.1.smithi136.stdout:4/717: symlink d1d/d2f/d6c/da5/l100 0 2024-04-06T03:40:45.376 INFO:tasks.workunit.client.0.smithi094.stdout:4/834: dwrite d0/d15/d3a/d61/d81/da0/fdd [0,4194304] 0 2024-04-06T03:40:45.376 INFO:tasks.workunit.client.0.smithi094.stdout:9/710: mkdir d2/dc/d80/ddd/d6b/d103 0 2024-04-06T03:40:45.377 INFO:tasks.workunit.client.0.smithi094.stdout:9/711: dread - d2/dc/d10/ff4 zero size 2024-04-06T03:40:45.377 INFO:tasks.workunit.client.0.smithi094.stdout:9/712: chown d2/dc/d21/d26/cb9 13 1 2024-04-06T03:40:45.377 INFO:tasks.workunit.client.0.smithi094.stdout:9/713: creat d2/dc/f104 x:0 0 0 2024-04-06T03:40:45.378 INFO:tasks.workunit.client.1.smithi136.stdout:7/897: mknod d5/df/d53/d68/da3/dec/c139 0 2024-04-06T03:40:45.378 INFO:tasks.workunit.client.1.smithi136.stdout:7/898: chown d5/df/d53/l101 8299761 1 2024-04-06T03:40:45.378 INFO:tasks.workunit.client.0.smithi094.stdout:5/767: creat db/d19/d24/d28/dec/f10c x:0 0 0 2024-04-06T03:40:45.381 INFO:tasks.workunit.client.1.smithi136.stdout:8/870: rename d3/da/d26/d35/d39/f65 to d3/da/d26/d35/d53/dd2/f111 0 2024-04-06T03:40:45.381 INFO:tasks.workunit.client.0.smithi094.stdout:0/755: getdents d1/d9/dd/d18/d32 0 2024-04-06T03:40:45.385 INFO:tasks.workunit.client.1.smithi136.stdout:9/833: rmdir d17/d21/d29 39 2024-04-06T03:40:45.386 INFO:tasks.workunit.client.1.smithi136.stdout:9/834: creat d17/d21/d29/de5/f11f x:0 0 0 2024-04-06T03:40:45.386 INFO:tasks.workunit.client.1.smithi136.stdout:9/835: chown d17/d19/da3/dc5/deb/d9e 97236 1 2024-04-06T03:40:45.387 INFO:tasks.workunit.client.1.smithi136.stdout:7/899: write d5/f82 [3432919,10928] 0 2024-04-06T03:40:45.387 INFO:tasks.workunit.client.1.smithi136.stdout:7/900: chown d5/df/d53/d68/d8f/fc1 42258 1 2024-04-06T03:40:45.387 INFO:tasks.workunit.client.0.smithi094.stdout:3/772: mkdir d9/d112 0 2024-04-06T03:40:45.388 INFO:tasks.workunit.client.0.smithi094.stdout:1/765: rmdir d0/d1/d27/da5 39 2024-04-06T03:40:45.389 INFO:tasks.workunit.client.1.smithi136.stdout:1/822: rmdir d12/d53 39 2024-04-06T03:40:45.389 INFO:tasks.workunit.client.1.smithi136.stdout:1/823: stat d12/d3b/d44/d5f/fde 0 2024-04-06T03:40:45.391 INFO:tasks.workunit.client.0.smithi094.stdout:5/768: dwrite db/d19/d24/f34 [4194304,4194304] 0 2024-04-06T03:40:45.393 INFO:tasks.workunit.client.0.smithi094.stdout:4/835: truncate d0/d15/d3a/d3e/d46/fa3 1306052 0 2024-04-06T03:40:45.394 INFO:tasks.workunit.client.1.smithi136.stdout:5/880: mkdir d1/d16/d10d 0 2024-04-06T03:40:45.396 INFO:tasks.workunit.client.1.smithi136.stdout:4/718: rename d1d/d31/d3e/d55/d89/laf to d1d/l101 0 2024-04-06T03:40:45.398 INFO:tasks.workunit.client.0.smithi094.stdout:9/714: rename d2/dc/d21/d23/d2f/f57 to d2/dc/d10/d6a/f105 0 2024-04-06T03:40:45.399 INFO:tasks.workunit.client.0.smithi094.stdout:9/715: chown d2/dc/d10/d6a/f105 4947 1 2024-04-06T03:40:45.399 INFO:tasks.workunit.client.0.smithi094.stdout:9/716: chown d2/dc/d21/d23/db0/fce 503438775 1 2024-04-06T03:40:45.399 INFO:tasks.workunit.client.1.smithi136.stdout:3/766: sync 2024-04-06T03:40:45.399 INFO:tasks.workunit.client.1.smithi136.stdout:3/767: getdents d0/d4/d12/d19/d55/d6f/d82/d5d/dbd 0 2024-04-06T03:40:45.399 INFO:tasks.workunit.client.1.smithi136.stdout:3/768: dread - d0/d4/d12/d19/d55/f9f zero size 2024-04-06T03:40:45.399 INFO:tasks.workunit.client.1.smithi136.stdout:8/871: mknod d3/d2d/dad/c112 0 2024-04-06T03:40:45.402 INFO:tasks.workunit.client.0.smithi094.stdout:3/773: mkdir d9/d1e/d26/d54/d113 0 2024-04-06T03:40:45.402 INFO:tasks.workunit.client.0.smithi094.stdout:3/774: creat d9/d1d/d7d/f114 x:0 0 0 2024-04-06T03:40:45.405 INFO:tasks.workunit.client.1.smithi136.stdout:9/836: creat d17/d20/d28/f120 x:0 0 0 2024-04-06T03:40:45.406 INFO:tasks.workunit.client.0.smithi094.stdout:5/769: mkdir db/d19/d2c/d6d/de6/dfd/d10d 0 2024-04-06T03:40:45.406 INFO:tasks.workunit.client.0.smithi094.stdout:5/770: fdatasync db/d19/d30/d83/fd3 0 2024-04-06T03:40:45.406 INFO:tasks.workunit.client.0.smithi094.stdout:5/771: fsync db/d19/d24/d2e/fb3 0 2024-04-06T03:40:45.406 INFO:tasks.workunit.client.0.smithi094.stdout:5/772: chown db/d7c/d8b/l109 0 1 2024-04-06T03:40:45.407 INFO:tasks.workunit.client.1.smithi136.stdout:3/769: dread d0/fb [0,4194304] 0 2024-04-06T03:40:45.407 INFO:tasks.workunit.client.1.smithi136.stdout:3/770: creat d0/d4/d12/d19/d55/d6f/ddb/f106 x:0 0 0 2024-04-06T03:40:45.407 INFO:tasks.workunit.client.0.smithi094.stdout:4/836: mkdir d0/d4/d67/d9f/d119 0 2024-04-06T03:40:45.409 INFO:tasks.workunit.client.1.smithi136.stdout:7/901: mknod d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/c13a 0 2024-04-06T03:40:45.412 INFO:tasks.workunit.client.1.smithi136.stdout:1/824: creat d12/d3b/d74/f115 x:0 0 0 2024-04-06T03:40:45.414 INFO:tasks.workunit.client.0.smithi094.stdout:7/692: write d2/d6/dc/d12/d29/f89 [4187957,112347] 0 2024-04-06T03:40:45.416 INFO:tasks.workunit.client.0.smithi094.stdout:1/766: rename d0/d1/d27/da5/f9f to d0/d1/d27/da5/d40/db1/d2f/d76/daf/f109 0 2024-04-06T03:40:45.418 INFO:tasks.workunit.client.1.smithi136.stdout:0/795: mknod d1/d6/c110 0 2024-04-06T03:40:45.421 INFO:tasks.workunit.client.1.smithi136.stdout:5/881: unlink d1/d9b/d9c/da9/cb7 0 2024-04-06T03:40:45.424 INFO:tasks.workunit.client.0.smithi094.stdout:7/693: dread d2/d6/dc/d12/d3a/fa2 [0,4194304] 0 2024-04-06T03:40:45.424 INFO:tasks.workunit.client.1.smithi136.stdout:8/872: link d3/da/l25 d3/da/d26/d35/d3d/dc7/l113 0 2024-04-06T03:40:45.425 INFO:tasks.workunit.client.1.smithi136.stdout:2/862: sync 2024-04-06T03:40:45.426 INFO:tasks.workunit.client.1.smithi136.stdout:9/837: symlink d17/d21/d32/dc6/d113/d66/de1/d51/l121 0 2024-04-06T03:40:45.427 INFO:tasks.workunit.client.0.smithi094.stdout:1/767: mknod d0/d7/d2d/c10a 0 2024-04-06T03:40:45.429 INFO:tasks.workunit.client.0.smithi094.stdout:7/694: creat d2/d6/dac/ddd/fe0 x:0 0 0 2024-04-06T03:40:45.430 INFO:tasks.workunit.client.1.smithi136.stdout:4/719: dwrite d1d/d2f/d6c/d91/ff2 [0,4194304] 0 2024-04-06T03:40:45.430 INFO:tasks.workunit.client.1.smithi136.stdout:3/771: getdents d0/d4/d12/d3a/d69 0 2024-04-06T03:40:45.432 INFO:tasks.workunit.client.0.smithi094.stdout:1/768: mknod d0/d1/d27/da5/d40/c10b 0 2024-04-06T03:40:45.434 INFO:tasks.workunit.client.0.smithi094.stdout:3/775: dwrite d9/d20/d30/f37 [0,4194304] 0 2024-04-06T03:40:45.435 INFO:tasks.workunit.client.0.smithi094.stdout:4/837: dwrite d0/de8/f112 [0,4194304] 0 2024-04-06T03:40:45.435 INFO:tasks.workunit.client.0.smithi094.stdout:5/773: dwrite db/f12 [0,4194304] 0 2024-04-06T03:40:45.435 INFO:tasks.workunit.client.0.smithi094.stdout:5/774: fdatasync db/d19/d30/d83/fa1 0 2024-04-06T03:40:45.437 INFO:tasks.workunit.client.1.smithi136.stdout:7/902: link d5/df/d3d/f8e d5/df/d53/d68/d135/d106/d11a/f13b 0 2024-04-06T03:40:45.438 INFO:tasks.workunit.client.1.smithi136.stdout:1/825: symlink d12/d1b/d1e/l116 0 2024-04-06T03:40:45.438 INFO:tasks.workunit.client.1.smithi136.stdout:0/796: mknod d1/d6/d2c/d57/d43/d6d/d9b/d5c/d96/c111 0 2024-04-06T03:40:45.438 INFO:tasks.workunit.client.1.smithi136.stdout:6/739: sync 2024-04-06T03:40:45.441 INFO:tasks.workunit.client.1.smithi136.stdout:5/882: mkdir d1/d16/d17/db2/d10e 0 2024-04-06T03:40:45.441 INFO:tasks.workunit.client.1.smithi136.stdout:5/883: stat d1/d16/d89/d8a/f103 0 2024-04-06T03:40:45.441 INFO:tasks.workunit.client.0.smithi094.stdout:1/769: write d0/d1/d27/da5/d40/fdc [2415384,6273] 0 2024-04-06T03:40:45.442 INFO:tasks.workunit.client.1.smithi136.stdout:7/903: dread d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/f5e [0,4194304] 0 2024-04-06T03:40:45.443 INFO:tasks.workunit.client.1.smithi136.stdout:2/863: truncate d3/d9/d3b/d87/ff6 579362 0 2024-04-06T03:40:45.446 INFO:tasks.workunit.client.1.smithi136.stdout:9/838: creat d17/d21/d32/dc6/d113/dbd/f122 x:0 0 0 2024-04-06T03:40:45.447 INFO:tasks.workunit.client.0.smithi094.stdout:4/838: mkdir d0/d77/dc4/d11a 0 2024-04-06T03:40:45.448 INFO:tasks.workunit.client.0.smithi094.stdout:4/839: write d0/df/d57/f86 [353394,51263] 0 2024-04-06T03:40:45.449 INFO:tasks.workunit.client.0.smithi094.stdout:4/840: stat d0/d4/d67/c118 0 2024-04-06T03:40:45.449 INFO:tasks.workunit.client.0.smithi094.stdout:3/776: dwrite d9/d1e/d6e/d75/dba/d3a/fa8 [0,4194304] 0 2024-04-06T03:40:45.449 INFO:tasks.workunit.client.0.smithi094.stdout:3/777: creat d9/d20/d2d/f115 x:0 0 0 2024-04-06T03:40:45.451 INFO:tasks.workunit.client.0.smithi094.stdout:1/770: mkdir d0/d1/d27/da5/d40/db1/d2f/d3c/d48/dcb/d10c 0 2024-04-06T03:40:45.451 INFO:tasks.workunit.client.0.smithi094.stdout:1/771: write d0/d7/ff9 [239628,30034] 0 2024-04-06T03:40:45.451 INFO:tasks.workunit.client.0.smithi094.stdout:1/772: creat d0/d1/dd3/f10d x:0 0 0 2024-04-06T03:40:45.451 INFO:tasks.workunit.client.0.smithi094.stdout:1/773: write d0/d7/d1a/fd1 [92556,85661] 0 2024-04-06T03:40:45.452 INFO:tasks.workunit.client.0.smithi094.stdout:4/841: mkdir d0/d4/d67/d9f/d11b 0 2024-04-06T03:40:45.453 INFO:tasks.workunit.client.1.smithi136.stdout:4/720: mkdir d1d/d31/d3e/d56/d102 0 2024-04-06T03:40:45.456 INFO:tasks.workunit.client.1.smithi136.stdout:3/772: symlink d0/d4/d12/d2d/l107 0 2024-04-06T03:40:45.460 INFO:tasks.workunit.client.1.smithi136.stdout:1/826: unlink d12/d3b/d74/d7a/dea/d77/fdc 0 2024-04-06T03:40:45.464 INFO:tasks.workunit.client.1.smithi136.stdout:0/797: mkdir d1/d6/d2c/d57/d43/d6d/d85/d112 0 2024-04-06T03:40:45.464 INFO:tasks.workunit.client.1.smithi136.stdout:3/773: read d0/d4/d12/d19/d55/d3c/d5a/d79/d98/fda [122161,101082] 0 2024-04-06T03:40:45.464 INFO:tasks.workunit.client.1.smithi136.stdout:3/774: truncate d0/d4/d12/d19/d55/d3c/d5a/d79/ff1 577691 0 2024-04-06T03:40:45.464 INFO:tasks.workunit.client.1.smithi136.stdout:1/827: dread d12/d3b/f4c [0,4194304] 0 2024-04-06T03:40:45.465 INFO:tasks.workunit.client.1.smithi136.stdout:1/828: creat d12/d3b/d60/d7e/dbe/f117 x:0 0 0 2024-04-06T03:40:45.465 INFO:tasks.workunit.client.1.smithi136.stdout:1/829: chown d12/d3b/d60/d7e/dbe 5355788 1 2024-04-06T03:40:45.465 INFO:tasks.workunit.client.0.smithi094.stdout:1/774: truncate d0/d7/f78 722745 0 2024-04-06T03:40:45.465 INFO:tasks.workunit.client.0.smithi094.stdout:1/775: chown d0/d7/c92 81 1 2024-04-06T03:40:45.465 INFO:tasks.workunit.client.0.smithi094.stdout:1/776: chown d0/d1/d27/da5/d3f/c43 0 1 2024-04-06T03:40:45.465 INFO:tasks.workunit.client.0.smithi094.stdout:1/777: chown d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/l82 160 1 2024-04-06T03:40:45.467 INFO:tasks.workunit.client.1.smithi136.stdout:1/830: dread d12/d53/dc5/f106 [0,4194304] 0 2024-04-06T03:40:45.467 INFO:tasks.workunit.client.1.smithi136.stdout:1/831: dread - d12/d3b/d74/d7a/dea/d77/d96/f97 zero size 2024-04-06T03:40:45.469 INFO:tasks.workunit.client.1.smithi136.stdout:6/740: rename d2/l8 to d2/d7/d26/d30/d6e/d58/d5d/lf6 0 2024-04-06T03:40:45.469 INFO:tasks.workunit.client.0.smithi094.stdout:4/842: mkdir d0/df/db3/d11c 0 2024-04-06T03:40:45.469 INFO:tasks.workunit.client.0.smithi094.stdout:4/843: creat d0/d77/f11d x:0 0 0 2024-04-06T03:40:45.471 INFO:tasks.workunit.client.1.smithi136.stdout:5/884: link d1/dc/d1c/d32/d8b/d20/l26 d1/d16/d89/d8a/l10f 0 2024-04-06T03:40:45.471 INFO:tasks.workunit.client.1.smithi136.stdout:5/885: write d1/d9b/d9c/fcd [822622,81651] 0 2024-04-06T03:40:45.471 INFO:tasks.workunit.client.1.smithi136.stdout:5/886: read - d1/d16/ffe zero size 2024-04-06T03:40:45.472 INFO:tasks.workunit.client.0.smithi094.stdout:3/778: dwrite d9/d1d/f6a [0,4194304] 0 2024-04-06T03:40:45.473 INFO:tasks.workunit.client.1.smithi136.stdout:7/904: creat d5/f13c x:0 0 0 2024-04-06T03:40:45.474 INFO:tasks.workunit.client.1.smithi136.stdout:7/905: fsync d5/df/d53/d68/d135/d106/f117 0 2024-04-06T03:40:45.474 INFO:tasks.workunit.client.1.smithi136.stdout:4/721: dwrite d1d/d2f/d6c/d91/fc4 [0,4194304] 0 2024-04-06T03:40:45.477 INFO:tasks.workunit.client.1.smithi136.stdout:8/873: rename d3/da/d26/d35/d53/fa9 to d3/da/d26/d35/d53/f114 0 2024-04-06T03:40:45.478 INFO:tasks.workunit.client.1.smithi136.stdout:8/874: write d3/da/d26/d35/d3d/f64 [5510520,11669] 0 2024-04-06T03:40:45.479 INFO:tasks.workunit.client.0.smithi094.stdout:4/844: dread d0/d4/f6 [0,4194304] 0 2024-04-06T03:40:45.479 INFO:tasks.workunit.client.0.smithi094.stdout:4/845: read d0/df/d13/d4a/f8b [735894,10171] 0 2024-04-06T03:40:45.480 INFO:tasks.workunit.client.1.smithi136.stdout:2/864: readlink d3/d9/d3b/l55 0 2024-04-06T03:40:45.480 INFO:tasks.workunit.client.1.smithi136.stdout:2/865: fsync d3/d15/d19/d43/daa/dce/f114 0 2024-04-06T03:40:45.483 INFO:tasks.workunit.client.1.smithi136.stdout:9/839: symlink d17/d21/d32/dc6/d113/d66/de1/l123 0 2024-04-06T03:40:45.483 INFO:tasks.workunit.client.0.smithi094.stdout:1/778: dwrite d0/d1/d27/da5/d40/d50/f106 [0,4194304] 0 2024-04-06T03:40:45.484 INFO:tasks.workunit.client.0.smithi094.stdout:3/779: rename d9/d1e/d26/d55/cc6 to d9/d1e/d26/c116 0 2024-04-06T03:40:45.485 INFO:tasks.workunit.client.1.smithi136.stdout:4/722: write d1d/d31/d3e/d56/d7b/fd5 [157975,80467] 0 2024-04-06T03:40:45.486 INFO:tasks.workunit.client.1.smithi136.stdout:3/775: symlink d0/d4/d12/d3a/d51/da6/l108 0 2024-04-06T03:40:45.486 INFO:tasks.workunit.client.1.smithi136.stdout:3/776: chown d0/d4/d12/d19/d55/d3c/d5a/ca3 39 1 2024-04-06T03:40:45.486 INFO:tasks.workunit.client.1.smithi136.stdout:3/777: getdents d0/d4/d12/d19/d55/d6f/d82/d5d/dbd 0 2024-04-06T03:40:45.488 INFO:tasks.workunit.client.0.smithi094.stdout:4/846: creat d0/df/d13/deb/f11e x:0 0 0 2024-04-06T03:40:45.491 INFO:tasks.workunit.client.1.smithi136.stdout:1/832: creat d12/d3b/d74/d7a/f118 x:0 0 0 2024-04-06T03:40:45.493 INFO:tasks.workunit.client.1.smithi136.stdout:0/798: rename d1/d6/d2c/l6f to d1/d6/d2c/d57/d43/d6d/l113 0 2024-04-06T03:40:45.493 INFO:tasks.workunit.client.1.smithi136.stdout:0/799: fdatasync d1/d6/d2c/d57/d43/d6d/d85/d86/fba 0 2024-04-06T03:40:45.493 INFO:tasks.workunit.client.1.smithi136.stdout:0/800: stat d1/d6/d2c/d1a/d36/f76 0 2024-04-06T03:40:45.494 INFO:tasks.workunit.client.0.smithi094.stdout:1/779: creat d0/d1/d27/da5/d40/db1/d2f/f10e x:0 0 0 2024-04-06T03:40:45.498 INFO:tasks.workunit.client.0.smithi094.stdout:1/780: unlink d0/d1/d27/da5/d40/f7a 0 2024-04-06T03:40:45.501 INFO:tasks.workunit.client.0.smithi094.stdout:1/781: symlink d0/d1/d27/da5/d40/db1/dc3/l10f 0 2024-04-06T03:40:45.503 INFO:tasks.workunit.client.1.smithi136.stdout:8/875: mkdir d3/da/da0/d115 0 2024-04-06T03:40:45.503 INFO:tasks.workunit.client.1.smithi136.stdout:8/876: dread - d3/da/d26/d35/d3d/dc7/dce/fd6 zero size 2024-04-06T03:40:45.504 INFO:tasks.workunit.client.1.smithi136.stdout:2/866: symlink d3/d4a/d9d/d18/df8/l120 0 2024-04-06T03:40:45.504 INFO:tasks.workunit.client.0.smithi094.stdout:4/847: dwrite d0/d23/fd1 [0,4194304] 0 2024-04-06T03:40:45.510 INFO:tasks.workunit.client.1.smithi136.stdout:9/840: unlink d17/d21/fc9 0 2024-04-06T03:40:45.510 INFO:tasks.workunit.client.1.smithi136.stdout:9/841: creat d17/d20/d25/da4/dbb/f124 x:0 0 0 2024-04-06T03:40:45.511 INFO:tasks.workunit.client.0.smithi094.stdout:4/848: mknod d0/df/d1e/d34/c11f 0 2024-04-06T03:40:45.513 INFO:tasks.workunit.client.1.smithi136.stdout:4/723: truncate d1d/d31/d3e/d55/dba/fd7 5180399 0 2024-04-06T03:40:45.513 INFO:tasks.workunit.client.1.smithi136.stdout:4/724: chown f7 2885 1 2024-04-06T03:40:45.515 INFO:tasks.workunit.client.1.smithi136.stdout:3/778: mkdir d0/d4/d12/dee/d109 0 2024-04-06T03:40:45.515 INFO:tasks.workunit.client.1.smithi136.stdout:3/779: write d0/d4/d12/d19/d55/f24 [2589373,52281] 0 2024-04-06T03:40:45.516 INFO:tasks.workunit.client.0.smithi094.stdout:1/782: dwrite d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/fb7 [0,4194304] 0 2024-04-06T03:40:45.516 INFO:tasks.workunit.client.0.smithi094.stdout:1/783: write d0/d1/d27/da5/d40/d50/ff7 [873135,81804] 0 2024-04-06T03:40:45.516 INFO:tasks.workunit.client.1.smithi136.stdout:1/833: symlink d12/d1b/d1e/d9f/l119 0 2024-04-06T03:40:45.517 INFO:tasks.workunit.client.0.smithi094.stdout:4/849: getdents d0/d77 0 2024-04-06T03:40:45.517 INFO:tasks.workunit.client.0.smithi094.stdout:4/850: chown d0/df/d13/f36 0 1 2024-04-06T03:40:45.518 INFO:tasks.workunit.client.1.smithi136.stdout:3/780: dread d0/d4/d12/d2d/f33 [0,4194304] 0 2024-04-06T03:40:45.519 INFO:tasks.workunit.client.1.smithi136.stdout:0/801: unlink d1/d6/d2c/d1a/d36/d59/fb9 0 2024-04-06T03:40:45.520 INFO:tasks.workunit.client.1.smithi136.stdout:6/741: rmdir d2/d25 39 2024-04-06T03:40:45.520 INFO:tasks.workunit.client.1.smithi136.stdout:6/742: chown d2/d7/d26/d30/d6e/d58/d5d/fd0 30546242 1 2024-04-06T03:40:45.522 INFO:tasks.workunit.client.1.smithi136.stdout:5/887: getdents d1/d9b/dea 0 2024-04-06T03:40:45.524 INFO:tasks.workunit.client.0.smithi094.stdout:1/784: rename d0/d7/d1a/d64/f72 to d0/d1/dd3/de7/f110 0 2024-04-06T03:40:45.524 INFO:tasks.workunit.client.0.smithi094.stdout:1/785: write d0/d1/d27/da5/d40/db1/d2f/d4b/f91 [3520010,111048] 0 2024-04-06T03:40:45.525 INFO:tasks.workunit.client.1.smithi136.stdout:8/877: mknod d3/da/d26/d35/d3d/dc7/dce/df2/c116 0 2024-04-06T03:40:45.525 INFO:tasks.workunit.client.1.smithi136.stdout:8/878: write d3/da/d26/d35/d3d/dc7/dce/fe4 [998496,127945] 0 2024-04-06T03:40:45.525 INFO:tasks.workunit.client.1.smithi136.stdout:8/879: chown d3/de/d18/d7d/ddf/d6c/d7f/d105 1150306736 1 2024-04-06T03:40:45.526 INFO:tasks.workunit.client.0.smithi094.stdout:4/851: truncate d0/d15/d3a/d3e/d46/f62 2742408 0 2024-04-06T03:40:45.527 INFO:tasks.workunit.client.1.smithi136.stdout:2/867: mknod d3/d4a/c121 0 2024-04-06T03:40:45.528 INFO:tasks.workunit.client.1.smithi136.stdout:2/868: chown d3/d4a/d9d/d18/d4b/f65 17194077 1 2024-04-06T03:40:45.529 INFO:tasks.workunit.client.0.smithi094.stdout:4/852: getdents d0/df/dbd/d107 0 2024-04-06T03:40:45.530 INFO:tasks.workunit.client.1.smithi136.stdout:4/725: truncate d1d/d70/f5b 1938616 0 2024-04-06T03:40:45.530 INFO:tasks.workunit.client.1.smithi136.stdout:4/726: dread - d1d/d31/d3e/d56/d7b/ff8 zero size 2024-04-06T03:40:45.533 INFO:tasks.workunit.client.0.smithi094.stdout:1/786: dread d0/d1/d27/da5/f53 [0,4194304] 0 2024-04-06T03:40:45.534 INFO:tasks.workunit.client.1.smithi136.stdout:8/880: dread d3/d2d/d5f/fb4 [0,4194304] 0 2024-04-06T03:40:45.538 INFO:tasks.workunit.client.1.smithi136.stdout:3/781: creat d0/d4/d12/d19/d2c/dc9/f10a x:0 0 0 2024-04-06T03:40:45.539 INFO:tasks.workunit.client.0.smithi094.stdout:4/853: dread d0/df/d1e/f24 [4194304,4194304] 0 2024-04-06T03:40:45.539 INFO:tasks.workunit.client.1.smithi136.stdout:6/743: rmdir d2/d7/d26/d30/d4f 39 2024-04-06T03:40:45.540 INFO:tasks.workunit.client.1.smithi136.stdout:5/888: mknod d1/dc/c110 0 2024-04-06T03:40:45.540 INFO:tasks.workunit.client.1.smithi136.stdout:5/889: chown d1/dc/d10/d1e/f97 2906684 1 2024-04-06T03:40:45.542 INFO:tasks.workunit.client.1.smithi136.stdout:2/869: creat d3/d4a/d77/f122 x:0 0 0 2024-04-06T03:40:45.542 INFO:tasks.workunit.client.1.smithi136.stdout:2/870: readlink d3/d4a/d9d/d18/d1d/d91/lbc 0 2024-04-06T03:40:45.542 INFO:tasks.workunit.client.1.smithi136.stdout:2/871: creat d3/d15/d19/d43/daa/dce/f123 x:0 0 0 2024-04-06T03:40:45.542 INFO:tasks.workunit.client.1.smithi136.stdout:9/842: mknod d17/d21/d32/dc6/d113/c125 0 2024-04-06T03:40:45.543 INFO:tasks.workunit.client.1.smithi136.stdout:2/872: dread d3/d4a/d77/fcd [0,4194304] 0 2024-04-06T03:40:45.543 INFO:tasks.workunit.client.1.smithi136.stdout:2/873: dread - d3/d4a/d9d/d18/d4b/f9c zero size 2024-04-06T03:40:45.543 INFO:tasks.workunit.client.1.smithi136.stdout:2/874: readlink d3/d15/d19/d43/daa/lbf 0 2024-04-06T03:40:45.544 INFO:tasks.workunit.client.1.smithi136.stdout:4/727: mknod d1d/d31/d3e/d55/d89/c103 0 2024-04-06T03:40:45.544 INFO:tasks.workunit.client.0.smithi094.stdout:3/780: dwrite d9/fd [4194304,4194304] 0 2024-04-06T03:40:45.545 INFO:tasks.workunit.client.0.smithi094.stdout:3/781: chown d9/d1e/d6e/d75/dba/d7c/faa 11255592 1 2024-04-06T03:40:45.547 INFO:tasks.workunit.client.1.smithi136.stdout:0/802: dread d1/d6/fc3 [0,4194304] 0 2024-04-06T03:40:45.550 INFO:tasks.workunit.client.1.smithi136.stdout:0/803: dread d1/d6/d2c/d1a/d36/fc2 [0,4194304] 0 2024-04-06T03:40:45.550 INFO:tasks.workunit.client.1.smithi136.stdout:0/804: stat d1/d4e/f75 0 2024-04-06T03:40:45.550 INFO:tasks.workunit.client.0.smithi094.stdout:1/787: dwrite d0/d1/d27/da5/f53 [4194304,4194304] 0 2024-04-06T03:40:45.551 INFO:tasks.workunit.client.0.smithi094.stdout:3/782: link d9/d1e/d6e/d75/dba/d7c/da2/lcb d9/d1e/d26/d6c/d8e/df8/l117 0 2024-04-06T03:40:45.552 INFO:tasks.workunit.client.1.smithi136.stdout:8/881: unlink d3/de/d18/d7d/fb9 0 2024-04-06T03:40:45.554 INFO:tasks.workunit.client.1.smithi136.stdout:3/782: stat d0/d4/d12/d19/d55/d3c/d46/l7c 0 2024-04-06T03:40:45.554 INFO:tasks.workunit.client.1.smithi136.stdout:3/783: chown d0/d4/d12/d19/d55/d3c/d5a/d79/d98 1312099024 1 2024-04-06T03:40:45.555 INFO:tasks.workunit.client.0.smithi094.stdout:4/854: dwrite d0/d23/f6e [0,4194304] 0 2024-04-06T03:40:45.557 INFO:tasks.workunit.client.1.smithi136.stdout:6/744: rmdir d2/d7/d26/d30/d6e/d58/d5d/d62/d53 39 2024-04-06T03:40:45.557 INFO:tasks.workunit.client.1.smithi136.stdout:6/745: creat d2/d7/d26/d30/dc4/ff7 x:0 0 0 2024-04-06T03:40:45.559 INFO:tasks.workunit.client.0.smithi094.stdout:1/788: getdents d0/d1/d27/da5/d40/db1/d2f/d4b 0 2024-04-06T03:40:45.559 INFO:tasks.workunit.client.0.smithi094.stdout:1/789: fdatasync d0/d7/f46 0 2024-04-06T03:40:45.561 INFO:tasks.workunit.client.1.smithi136.stdout:5/890: creat d1/d16/d89/daf/f111 x:0 0 0 2024-04-06T03:40:45.562 INFO:tasks.workunit.client.0.smithi094.stdout:3/783: symlink d9/l118 0 2024-04-06T03:40:45.562 INFO:tasks.workunit.client.0.smithi094.stdout:3/784: write d9/d10/f81 [163369,70580] 0 2024-04-06T03:40:45.569 INFO:tasks.workunit.client.1.smithi136.stdout:1/834: rename d12/d1b/d1e to d12/d3b/d74/d7a/dea/d11a 0 2024-04-06T03:40:45.570 INFO:tasks.workunit.client.0.smithi094.stdout:4/855: mkdir d0/d77/d120 0 2024-04-06T03:40:45.570 INFO:tasks.workunit.client.0.smithi094.stdout:4/856: chown d0/d77/dc4/fe2 9 1 2024-04-06T03:40:45.572 INFO:tasks.workunit.client.1.smithi136.stdout:2/875: mknod d3/d15/d19/d43/daa/dec/c124 0 2024-04-06T03:40:45.572 INFO:tasks.workunit.client.1.smithi136.stdout:2/876: readlink d3/d4a/d9d/d18/d1d/d3c/le6 0 2024-04-06T03:40:45.573 INFO:tasks.workunit.client.0.smithi094.stdout:3/785: rmdir d9/d1e/d26/d6c/d8e/df8 39 2024-04-06T03:40:45.573 INFO:tasks.workunit.client.0.smithi094.stdout:3/786: creat d9/d1e/d6e/d75/d90/f119 x:0 0 0 2024-04-06T03:40:45.574 INFO:tasks.workunit.client.1.smithi136.stdout:2/877: write d3/d4a/d9d/d8a/d93/d5f/f82 [752157,97830] 0 2024-04-06T03:40:45.574 INFO:tasks.workunit.client.1.smithi136.stdout:2/878: fsync d3/d15/d19/d43/f94 0 2024-04-06T03:40:45.574 INFO:tasks.workunit.client.1.smithi136.stdout:4/728: creat d1d/dd0/f104 x:0 0 0 2024-04-06T03:40:45.574 INFO:tasks.workunit.client.1.smithi136.stdout:4/729: write d1d/d31/d3e/d55/f5c [3375311,28884] 0 2024-04-06T03:40:45.574 INFO:tasks.workunit.client.1.smithi136.stdout:4/730: write d1d/d31/d3e/dd4/fd1 [1941745,33593] 0 2024-04-06T03:40:45.574 INFO:tasks.workunit.client.0.smithi094.stdout:4/857: mknod d0/df/d57/c121 0 2024-04-06T03:40:45.576 INFO:tasks.workunit.client.1.smithi136.stdout:0/805: symlink d1/d6/d2c/d57/d43/d6d/d9b/l114 0 2024-04-06T03:40:45.578 INFO:tasks.workunit.client.1.smithi136.stdout:3/784: dwrite d0/d4/f22 [4194304,4194304] 0 2024-04-06T03:40:45.578 INFO:tasks.workunit.client.1.smithi136.stdout:3/785: write d0/d4/d12/d2d/d37/f9d [63557,18373] 0 2024-04-06T03:40:45.578 INFO:tasks.workunit.client.0.smithi094.stdout:1/790: dwrite d0/d1/d27/da5/d40/db1/dc3/d5f/d70/fe8 [0,4194304] 0 2024-04-06T03:40:45.578 INFO:tasks.workunit.client.1.smithi136.stdout:3/786: read d0/d4/fa [494976,14203] 0 2024-04-06T03:40:45.582 INFO:tasks.workunit.client.1.smithi136.stdout:3/787: dread d0/d4/d12/d19/d2c/f7f [0,4194304] 0 2024-04-06T03:40:45.582 INFO:tasks.workunit.client.1.smithi136.stdout:3/788: write d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/fb2 [4589979,108971] 0 2024-04-06T03:40:45.582 INFO:tasks.workunit.client.1.smithi136.stdout:3/789: chown d0/d4/d12/d3a 14603 1 2024-04-06T03:40:45.583 INFO:tasks.workunit.client.0.smithi094.stdout:3/787: rename d9/d1d/d7d/f9c to d9/d1e/d6e/d75/dba/d7c/da2/df2/f11a 0 2024-04-06T03:40:45.585 INFO:tasks.workunit.client.1.smithi136.stdout:6/746: link d2/d7/d26/d30/d6e/d58/cd2 d2/d7/d26/d30/d6e/db7/cf8 0 2024-04-06T03:40:45.586 INFO:tasks.workunit.client.1.smithi136.stdout:0/806: dread d1/d6/d2c/f87 [0,4194304] 0 2024-04-06T03:40:45.587 INFO:tasks.workunit.client.0.smithi094.stdout:4/858: creat d0/d15/d3a/d3e/f122 x:0 0 0 2024-04-06T03:40:45.587 INFO:tasks.workunit.client.0.smithi094.stdout:4/859: write d0/d77/f80 [1926076,115742] 0 2024-04-06T03:40:45.589 INFO:tasks.workunit.client.1.smithi136.stdout:5/891: readlink d1/l33 0 2024-04-06T03:40:45.593 INFO:tasks.workunit.client.1.smithi136.stdout:9/843: rename f13 to d17/d21/d29/de5/f126 0 2024-04-06T03:40:45.595 INFO:tasks.workunit.client.1.smithi136.stdout:6/747: dread d2/d7/d26/d30/d6e/d58/f7d [0,4194304] 0 2024-04-06T03:40:45.596 INFO:tasks.workunit.client.1.smithi136.stdout:2/879: getdents d3/d9/d3b 0 2024-04-06T03:40:45.598 INFO:tasks.workunit.client.0.smithi094.stdout:3/788: link d9/d1e/d26/d55/d62/c7f d9/de5/d10b/c11b 0 2024-04-06T03:40:45.607 INFO:tasks.workunit.client.0.smithi094.stdout:3/789: mknod d9/d1d/d7d/dd7/c11c 0 2024-04-06T03:40:45.610 INFO:tasks.workunit.client.1.smithi136.stdout:3/790: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/df2/f10b x:0 0 0 2024-04-06T03:40:45.610 INFO:tasks.workunit.client.1.smithi136.stdout:3/791: read - d0/d4/d12/d19/d55/d3c/d5a/f101 zero size 2024-04-06T03:40:45.610 INFO:tasks.workunit.client.1.smithi136.stdout:3/792: chown d0/d4/d12/dc3/dd3 0 1 2024-04-06T03:40:45.610 INFO:tasks.workunit.client.1.smithi136.stdout:3/793: fsync d0/d4/d12/d19/d55/d3c/d5a/d79/ff1 0 2024-04-06T03:40:45.611 INFO:tasks.workunit.client.1.smithi136.stdout:5/892: dwrite d1/d16/d4b/f80 [4194304,4194304] 0 2024-04-06T03:40:45.611 INFO:tasks.workunit.client.1.smithi136.stdout:5/893: creat d1/dc/d10/f112 x:0 0 0 2024-04-06T03:40:45.611 INFO:tasks.workunit.client.1.smithi136.stdout:1/835: getdents d12/d3b/d60/d7e/dbe 0 2024-04-06T03:40:45.612 INFO:tasks.workunit.client.1.smithi136.stdout:2/880: symlink d3/d9/d3b/d70/l125 0 2024-04-06T03:40:45.613 INFO:tasks.workunit.client.1.smithi136.stdout:0/807: dwrite d1/d6/d2c/f3d [4194304,4194304] 0 2024-04-06T03:40:45.613 INFO:tasks.workunit.client.1.smithi136.stdout:0/808: write d1/d6/d2c/d1a/f94 [492427,79101] 0 2024-04-06T03:40:45.614 INFO:tasks.workunit.client.1.smithi136.stdout:9/844: dwrite d17/d19/f61 [0,4194304] 0 2024-04-06T03:40:45.614 INFO:tasks.workunit.client.1.smithi136.stdout:3/794: mkdir d0/d4/d12/d19/d55/d3c/d5a/d79/d98/d10c 0 2024-04-06T03:40:45.614 INFO:tasks.workunit.client.1.smithi136.stdout:7/906: sync 2024-04-06T03:40:45.614 INFO:tasks.workunit.client.1.smithi136.stdout:9/845: read d17/d21/d32/d3f/f44 [270977,130210] 0 2024-04-06T03:40:45.615 INFO:tasks.workunit.client.1.smithi136.stdout:9/846: creat d17/d20/d25/da4/dbb/f127 x:0 0 0 2024-04-06T03:40:45.616 INFO:tasks.workunit.client.1.smithi136.stdout:3/795: dread d0/d4/d12/d2d/d37/f6d [0,4194304] 0 2024-04-06T03:40:45.619 INFO:tasks.workunit.client.1.smithi136.stdout:1/836: read d12/d3b/d44/fa5 [2720911,62603] 0 2024-04-06T03:40:45.619 INFO:tasks.workunit.client.1.smithi136.stdout:5/894: mkdir d1/dc/d1c/d32/d8b/d20/d66/d113 0 2024-04-06T03:40:45.619 INFO:tasks.workunit.client.1.smithi136.stdout:1/837: fdatasync d12/d3b/d74/d7a/dea/d11a/d9f/fa8 0 2024-04-06T03:40:45.619 INFO:tasks.workunit.client.1.smithi136.stdout:2/881: write d3/d15/d19/fc7 [4973633,25885] 0 2024-04-06T03:40:45.623 INFO:tasks.workunit.client.1.smithi136.stdout:7/907: write d5/df/d12/d1b/d3a/d4a/dc4/f130 [3774758,44709] 0 2024-04-06T03:40:45.623 INFO:tasks.workunit.client.1.smithi136.stdout:7/908: dread - d5/df/d1d/d6f/d4d/d69/f94 zero size 2024-04-06T03:40:45.628 INFO:tasks.workunit.client.1.smithi136.stdout:0/809: getdents d1/d6/d2c/d57/d43 0 2024-04-06T03:40:45.628 INFO:tasks.workunit.client.1.smithi136.stdout:8/882: read d3/de/d60/f98 [3255678,130445] 0 2024-04-06T03:40:45.629 INFO:tasks.workunit.client.0.smithi094.stdout:4/860: dread d0/d15/f1a [0,4194304] 0 2024-04-06T03:40:45.630 INFO:tasks.workunit.client.1.smithi136.stdout:9/847: truncate d17/d21/f7c 2630085 0 2024-04-06T03:40:45.632 INFO:tasks.workunit.client.1.smithi136.stdout:3/796: getdents d0/d4/d12/d19/d55/d3c/d5a/d79 0 2024-04-06T03:40:45.633 INFO:tasks.workunit.client.1.smithi136.stdout:5/895: mkdir d1/dc/d10/da3/d114 0 2024-04-06T03:40:45.635 INFO:tasks.workunit.client.1.smithi136.stdout:2/882: creat d3/d15/d19/dac/f126 x:0 0 0 2024-04-06T03:40:45.635 INFO:tasks.workunit.client.1.smithi136.stdout:7/909: dread d5/df/d53/f39 [0,4194304] 0 2024-04-06T03:40:45.639 INFO:tasks.workunit.client.0.smithi094.stdout:4/861: write d0/d4/f76 [1235577,116720] 0 2024-04-06T03:40:45.640 INFO:tasks.workunit.client.0.smithi094.stdout:1/791: dwrite d0/d4/fc [0,4194304] 0 2024-04-06T03:40:45.641 INFO:tasks.workunit.client.0.smithi094.stdout:4/862: read d0/f8 [4793304,67253] 0 2024-04-06T03:40:45.641 INFO:tasks.workunit.client.0.smithi094.stdout:3/790: dwrite d9/d1e/d6e/d75/dba/f38 [0,4194304] 0 2024-04-06T03:40:45.646 INFO:tasks.workunit.client.1.smithi136.stdout:8/883: dwrite d3/de/f61 [0,4194304] 0 2024-04-06T03:40:45.646 INFO:tasks.workunit.client.1.smithi136.stdout:8/884: truncate d3/da/d26/d35/d53/f101 803934 0 2024-04-06T03:40:45.646 INFO:tasks.workunit.client.1.smithi136.stdout:1/838: dread d12/f2c [0,4194304] 0 2024-04-06T03:40:45.646 INFO:tasks.workunit.client.1.smithi136.stdout:1/839: fsync d12/d3b/d74/d7a/dea/d11a/d6f/fca 0 2024-04-06T03:40:45.648 INFO:tasks.workunit.client.0.smithi094.stdout:1/792: dread d0/d1/d27/da5/d40/db1/d2f/d76/d7d/fae [0,4194304] 0 2024-04-06T03:40:45.648 INFO:tasks.workunit.client.0.smithi094.stdout:1/793: write d0/d4/fc0 [785781,46564] 0 2024-04-06T03:40:45.649 INFO:tasks.workunit.client.1.smithi136.stdout:0/810: dread d1/d6/f1f [0,4194304] 0 2024-04-06T03:40:45.650 INFO:tasks.workunit.client.0.smithi094.stdout:4/863: dread d0/d23/f32 [0,4194304] 0 2024-04-06T03:40:45.651 INFO:tasks.workunit.client.0.smithi094.stdout:1/794: mknod d0/d1/dd3/c111 0 2024-04-06T03:40:45.651 INFO:tasks.workunit.client.0.smithi094.stdout:1/795: read d0/d1/d27/da5/d40/db1/faa [1173213,13394] 0 2024-04-06T03:40:45.651 INFO:tasks.workunit.client.1.smithi136.stdout:5/896: read d1/f4 [27407,25632] 0 2024-04-06T03:40:45.652 INFO:tasks.workunit.client.1.smithi136.stdout:9/848: dread d17/d21/d32/dc6/d113/d66/de1/f8c [0,4194304] 0 2024-04-06T03:40:45.652 INFO:tasks.workunit.client.1.smithi136.stdout:9/849: truncate d17/d21/d29/de5/da5/db8/fd0 1742898 0 2024-04-06T03:40:45.653 INFO:tasks.workunit.client.1.smithi136.stdout:9/850: dread - d17/d20/d25/da4/dbb/de4/f114 zero size 2024-04-06T03:40:45.653 INFO:tasks.workunit.client.0.smithi094.stdout:1/796: read d0/d7/d2d/f61 [201131,36545] 0 2024-04-06T03:40:45.653 INFO:tasks.workunit.client.0.smithi094.stdout:1/797: chown d0/d7/d1a/f9b 392040372 1 2024-04-06T03:40:45.654 INFO:tasks.workunit.client.1.smithi136.stdout:5/897: dread d1/d98/dbc/fdc [0,4194304] 0 2024-04-06T03:40:45.654 INFO:tasks.workunit.client.1.smithi136.stdout:5/898: fsync d1/dc/d1c/f56 0 2024-04-06T03:40:45.654 INFO:tasks.workunit.client.0.smithi094.stdout:1/798: getdents d0/d1/d27/da5/d40/db1/dc3/d5f 0 2024-04-06T03:40:45.660 INFO:tasks.workunit.client.0.smithi094.stdout:3/791: dwrite d9/d1d/d7d/fb0 [0,4194304] 0 2024-04-06T03:40:45.660 INFO:tasks.workunit.client.0.smithi094.stdout:3/792: fdatasync d9/d1d/f36 0 2024-04-06T03:40:45.660 INFO:tasks.workunit.client.1.smithi136.stdout:3/797: rename d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/l85 to d0/d4/d12/d19/d55/d6f/ddb/df9/l10d 0 2024-04-06T03:40:45.662 INFO:tasks.workunit.client.1.smithi136.stdout:2/883: mkdir d3/d127 0 2024-04-06T03:40:45.662 INFO:tasks.workunit.client.1.smithi136.stdout:2/884: fdatasync d3/d9/d3b/fd7 0 2024-04-06T03:40:45.664 INFO:tasks.workunit.client.1.smithi136.stdout:7/910: unlink d5/df/d1d/d54/d5c/f110 0 2024-04-06T03:40:45.664 INFO:tasks.workunit.client.1.smithi136.stdout:7/911: write d5/df/d1d/d54/d5c/fdc [66060,74925] 0 2024-04-06T03:40:45.664 INFO:tasks.workunit.client.1.smithi136.stdout:7/912: readlink d5/df/d53/l101 0 2024-04-06T03:40:45.670 INFO:tasks.workunit.client.1.smithi136.stdout:8/885: creat d3/da/d26/d35/d3d/dc0/f117 x:0 0 0 2024-04-06T03:40:45.670 INFO:tasks.workunit.client.1.smithi136.stdout:8/886: fdatasync d3/da/d26/d35/dcb/ff4 0 2024-04-06T03:40:45.672 INFO:tasks.workunit.client.0.smithi094.stdout:1/799: dwrite d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/f8d [0,4194304] 0 2024-04-06T03:40:45.676 INFO:tasks.workunit.client.0.smithi094.stdout:3/793: unlink d9/d10/d9b/db7/f108 0 2024-04-06T03:40:45.678 INFO:tasks.workunit.client.1.smithi136.stdout:5/899: unlink d1/dc/d1c/d32/d70/cf3 0 2024-04-06T03:40:45.679 INFO:tasks.workunit.client.1.smithi136.stdout:7/913: dwrite d5/df/d12/d1b/f10c [4194304,4194304] 0 2024-04-06T03:40:45.681 INFO:tasks.workunit.client.1.smithi136.stdout:3/798: dwrite d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/fe0 [0,4194304] 0 2024-04-06T03:40:45.681 INFO:tasks.workunit.client.1.smithi136.stdout:3/799: chown d0/d4/d12/d19/d55/d6f/df0 11435 1 2024-04-06T03:40:45.687 INFO:tasks.workunit.client.1.smithi136.stdout:9/851: rename d17/l6f to d17/d20/d25/d3c/d55/l128 0 2024-04-06T03:40:45.687 INFO:tasks.workunit.client.1.smithi136.stdout:9/852: write d17/d19/da3/dc5/deb/f4e [204505,25965] 0 2024-04-06T03:40:45.687 INFO:tasks.workunit.client.1.smithi136.stdout:9/853: dread - d17/d21/d32/dc6/d113/f11a zero size 2024-04-06T03:40:45.687 INFO:tasks.workunit.client.1.smithi136.stdout:9/854: dread - d17/d20/d28/d81/d88/dcb/ff6 zero size 2024-04-06T03:40:45.688 INFO:tasks.workunit.client.1.smithi136.stdout:0/811: dwrite d1/d6/f45 [0,4194304] 0 2024-04-06T03:40:45.689 INFO:tasks.workunit.client.1.smithi136.stdout:2/885: link d3/d4a/lf4 d3/d15/d19/d4e/l128 0 2024-04-06T03:40:45.689 INFO:tasks.workunit.client.1.smithi136.stdout:2/886: chown d3/c26 1 1 2024-04-06T03:40:45.690 INFO:tasks.workunit.client.0.smithi094.stdout:4/864: rename d0/d15/d3a/d3e to d0/d77/d123 0 2024-04-06T03:40:45.690 INFO:tasks.workunit.client.0.smithi094.stdout:4/865: chown d0/df/l30 20045 1 2024-04-06T03:40:45.691 INFO:tasks.workunit.client.1.smithi136.stdout:5/900: write d1/d16/d4b/f80 [801706,76348] 0 2024-04-06T03:40:45.693 INFO:tasks.workunit.client.1.smithi136.stdout:7/914: dwrite d5/df/d1d/d54/d5c/ff8 [0,4194304] 0 2024-04-06T03:40:45.693 INFO:tasks.workunit.client.1.smithi136.stdout:7/915: write d5/df/d12/f123 [1608672,33801] 0 2024-04-06T03:40:45.693 INFO:tasks.workunit.client.1.smithi136.stdout:7/916: chown d5/c35 2606512 1 2024-04-06T03:40:45.693 INFO:tasks.workunit.client.1.smithi136.stdout:7/917: write d5/df/d12/f123 [1249930,17266] 0 2024-04-06T03:40:45.696 INFO:tasks.workunit.client.1.smithi136.stdout:8/887: mkdir d3/de/d60/d118 0 2024-04-06T03:40:45.698 INFO:tasks.workunit.client.1.smithi136.stdout:1/840: getdents d12/d3b/d60/d7e 0 2024-04-06T03:40:45.700 INFO:tasks.workunit.client.0.smithi094.stdout:1/800: creat d0/d1/f112 x:0 0 0 2024-04-06T03:40:45.700 INFO:tasks.workunit.client.0.smithi094.stdout:1/801: truncate d0/d1/d27/da5/d40/db1/d2f/f54 9129953 0 2024-04-06T03:40:45.700 INFO:tasks.workunit.client.0.smithi094.stdout:1/802: write d0/d1/d27/f2e [915491,84375] 0 2024-04-06T03:40:45.702 INFO:tasks.workunit.client.1.smithi136.stdout:7/918: dread d5/df/d12/d1b/d3a/d4a/d70/ddb/faf [0,4194304] 0 2024-04-06T03:40:45.702 INFO:tasks.workunit.client.1.smithi136.stdout:7/919: chown d5/df/d53/d68/d135/d106 425 1 2024-04-06T03:40:45.702 INFO:tasks.workunit.client.1.smithi136.stdout:4/731: sync 2024-04-06T03:40:45.702 INFO:tasks.workunit.client.1.smithi136.stdout:6/748: sync 2024-04-06T03:40:45.703 INFO:tasks.workunit.client.1.smithi136.stdout:0/812: dwrite d1/d6/d2c/d84/fd7 [0,4194304] 0 2024-04-06T03:40:45.707 INFO:tasks.workunit.client.1.smithi136.stdout:9/855: dread d17/d21/d32/d3f/f5b [0,4194304] 0 2024-04-06T03:40:45.710 INFO:tasks.workunit.client.1.smithi136.stdout:3/800: rename d0/d4/d12/d19/d2c/l8d to d0/d4/d12/d19/d55/d6f/d82/d5d/dbd/l10e 0 2024-04-06T03:40:45.712 INFO:tasks.workunit.client.1.smithi136.stdout:2/887: mknod d3/d15/d19/d43/d54/c129 0 2024-04-06T03:40:45.714 INFO:tasks.workunit.client.1.smithi136.stdout:4/732: dwrite d1d/d31/d3e/d55/fe3 [0,4194304] 0 2024-04-06T03:40:45.719 INFO:tasks.workunit.client.1.smithi136.stdout:8/888: mkdir d3/da/d26/d35/d39/df5/d119 0 2024-04-06T03:40:45.721 INFO:tasks.workunit.client.1.smithi136.stdout:1/841: symlink d12/d3b/d60/d7e/dbe/l11b 0 2024-04-06T03:40:45.725 INFO:tasks.workunit.client.1.smithi136.stdout:7/920: getdents d5/df 0 2024-04-06T03:40:45.725 INFO:tasks.workunit.client.1.smithi136.stdout:7/921: write d5/df/d53/f43 [1693356,71200] 0 2024-04-06T03:40:45.725 INFO:tasks.workunit.client.1.smithi136.stdout:7/922: fsync d5/df/d1d/d54/d5c/dc7/fd8 0 2024-04-06T03:40:45.725 INFO:tasks.workunit.client.1.smithi136.stdout:2/888: dwrite d3/d9b/fcb [0,4194304] 0 2024-04-06T03:40:45.726 INFO:tasks.workunit.client.0.smithi094.stdout:3/794: dread d9/d1e/d6e/d75/dba/d7c/da2/fc9 [0,4194304] 0 2024-04-06T03:40:45.726 INFO:tasks.workunit.client.1.smithi136.stdout:6/749: symlink d2/d7/d13/lf9 0 2024-04-06T03:40:45.727 INFO:tasks.workunit.client.0.smithi094.stdout:4/866: symlink d0/df/d91/l124 0 2024-04-06T03:40:45.727 INFO:tasks.workunit.client.0.smithi094.stdout:4/867: readlink d0/df/d1e/l7e 0 2024-04-06T03:40:45.729 INFO:tasks.workunit.client.0.smithi094.stdout:1/803: rename d0/d4 to d0/d1/d113 0 2024-04-06T03:40:45.731 INFO:tasks.workunit.client.1.smithi136.stdout:0/813: creat d1/d6/d2c/d57/d43/d6d/d85/f115 x:0 0 0 2024-04-06T03:40:45.733 INFO:tasks.workunit.client.1.smithi136.stdout:9/856: mknod d17/d21/d32/dc6/d113/d66/de1/d51/d57/c129 0 2024-04-06T03:40:45.734 INFO:tasks.workunit.client.0.smithi094.stdout:3/795: creat d9/d1e/d6e/d75/dba/d97/f11d x:0 0 0 2024-04-06T03:40:45.734 INFO:tasks.workunit.client.1.smithi136.stdout:0/814: write d1/d6/d2c/f87 [3486298,116110] 0 2024-04-06T03:40:45.735 INFO:tasks.workunit.client.0.smithi094.stdout:4/868: mkdir d0/d4/d67/dec/d125 0 2024-04-06T03:40:45.737 INFO:tasks.workunit.client.1.smithi136.stdout:3/801: mkdir d0/d4/d12/d3a/d51/d8e/d10f 0 2024-04-06T03:40:45.738 INFO:tasks.workunit.client.0.smithi094.stdout:1/804: getdents d0/d1/d27/da5/d3f/de9 0 2024-04-06T03:40:45.741 INFO:tasks.workunit.client.0.smithi094.stdout:4/869: write d0/d15/f8f [404340,111699] 0 2024-04-06T03:40:45.741 INFO:tasks.workunit.client.0.smithi094.stdout:3/796: mknod d9/d112/c11e 0 2024-04-06T03:40:45.741 INFO:tasks.workunit.client.0.smithi094.stdout:3/797: fdatasync d9/d1e/d6e/d75/dba/d7c/da2/fc9 0 2024-04-06T03:40:45.742 INFO:tasks.workunit.client.0.smithi094.stdout:1/805: rename d0/d1/fbc to d0/d1/d27/da5/d3f/f114 0 2024-04-06T03:40:45.743 INFO:tasks.workunit.client.0.smithi094.stdout:4/870: mknod d0/d4/d67/c126 0 2024-04-06T03:40:45.744 INFO:tasks.workunit.client.1.smithi136.stdout:5/901: rmdir d1/dc/d1c 39 2024-04-06T03:40:45.745 INFO:tasks.workunit.client.1.smithi136.stdout:5/902: dread d1/d16/d4b/f8e [0,4194304] 0 2024-04-06T03:40:45.748 INFO:tasks.workunit.client.0.smithi094.stdout:3/798: unlink d9/d1e/d6e/d75/dba/d7c/da2/ced 0 2024-04-06T03:40:45.750 INFO:tasks.workunit.client.0.smithi094.stdout:4/871: symlink d0/d77/d123/d49/l127 0 2024-04-06T03:40:45.751 INFO:tasks.workunit.client.1.smithi136.stdout:4/733: symlink d1d/d2f/l105 0 2024-04-06T03:40:45.751 INFO:tasks.workunit.client.1.smithi136.stdout:1/842: rename d12/d1b/d28/deb to d12/d1b/d28/d11c 0 2024-04-06T03:40:45.753 INFO:tasks.workunit.client.0.smithi094.stdout:4/872: symlink d0/df/d1e/d34/db6/l128 0 2024-04-06T03:40:45.753 INFO:tasks.workunit.client.0.smithi094.stdout:4/873: chown d0/d4/f7c 4079 1 2024-04-06T03:40:45.756 INFO:tasks.workunit.client.1.smithi136.stdout:5/903: dread d1/dc/d1c/d32/d8b/d20/d66/f9e [0,4194304] 0 2024-04-06T03:40:45.756 INFO:tasks.workunit.client.0.smithi094.stdout:4/874: rename d0/d77/d123/d50 to d0/d4/d129 0 2024-04-06T03:40:45.761 INFO:tasks.workunit.client.1.smithi136.stdout:7/923: symlink d5/df/d1d/d54/d5c/l13d 0 2024-04-06T03:40:45.764 INFO:tasks.workunit.client.1.smithi136.stdout:1/843: write d12/f24 [2786192,53484] 0 2024-04-06T03:40:45.767 INFO:tasks.workunit.client.1.smithi136.stdout:2/889: mkdir d3/d9/d3b/d70/d12a 0 2024-04-06T03:40:45.768 INFO:tasks.workunit.client.1.smithi136.stdout:6/750: rmdir d2/d7/d13 39 2024-04-06T03:40:45.769 INFO:tasks.workunit.client.0.smithi094.stdout:1/806: dwrite d0/d7/d1a/d74/fe6 [4194304,4194304] 0 2024-04-06T03:40:45.770 INFO:tasks.workunit.client.1.smithi136.stdout:9/857: link d17/d21/d32/dc6/d113/d66/de1/l123 d17/d21/d32/dc6/d113/dbd/l12a 0 2024-04-06T03:40:45.771 INFO:tasks.workunit.client.1.smithi136.stdout:9/858: stat d17/d20/d25/d71/dd5/fed 0 2024-04-06T03:40:45.771 INFO:tasks.workunit.client.1.smithi136.stdout:3/802: symlink d0/d4/d12/d19/d55/d6f/df0/l110 0 2024-04-06T03:40:45.772 INFO:tasks.workunit.client.1.smithi136.stdout:3/803: write d0/d4/f45 [956537,10627] 0 2024-04-06T03:40:45.773 INFO:tasks.workunit.client.1.smithi136.stdout:5/904: mkdir d1/dc/d1c/d109/d115 0 2024-04-06T03:40:45.773 INFO:tasks.workunit.client.0.smithi094.stdout:3/799: dwrite d9/d10/fd8 [0,4194304] 0 2024-04-06T03:40:45.775 INFO:tasks.workunit.client.0.smithi094.stdout:4/875: dwrite d0/f8 [4194304,4194304] 0 2024-04-06T03:40:45.776 INFO:tasks.workunit.client.1.smithi136.stdout:7/924: dread d5/f52 [0,4194304] 0 2024-04-06T03:40:45.776 INFO:tasks.workunit.client.1.smithi136.stdout:7/925: getdents d5/df 0 2024-04-06T03:40:45.776 INFO:tasks.workunit.client.1.smithi136.stdout:1/844: getdents d12/d3b/d74/d7a/dea 0 2024-04-06T03:40:45.778 INFO:tasks.workunit.client.1.smithi136.stdout:4/734: dwrite d1d/dd0/f104 [0,4194304] 0 2024-04-06T03:40:45.780 INFO:tasks.workunit.client.1.smithi136.stdout:2/890: rename d3/d4a/d9d/d18/d1d/d91 to d3/d127/d12b 0 2024-04-06T03:40:45.780 INFO:tasks.workunit.client.1.smithi136.stdout:2/891: write d3/d15/f50 [2896716,4085] 0 2024-04-06T03:40:45.782 INFO:tasks.workunit.client.1.smithi136.stdout:6/751: symlink d2/lfa 0 2024-04-06T03:40:45.784 INFO:tasks.workunit.client.0.smithi094.stdout:3/800: creat d9/d10/d7e/da3/f11f x:0 0 0 2024-04-06T03:40:45.784 INFO:tasks.workunit.client.1.smithi136.stdout:2/892: dread d3/d4a/d77/fcd [0,4194304] 0 2024-04-06T03:40:45.785 INFO:tasks.workunit.client.1.smithi136.stdout:9/859: rmdir d17/d21/d32/dc6/d113/d66/de1/d51/d57 39 2024-04-06T03:40:45.785 INFO:tasks.workunit.client.1.smithi136.stdout:9/860: fsync d17/d21/d32/dc6/d113/f11a 0 2024-04-06T03:40:45.787 INFO:tasks.workunit.client.1.smithi136.stdout:4/735: dread d1d/d31/d3e/d56/f81 [0,4194304] 0 2024-04-06T03:40:45.787 INFO:tasks.workunit.client.1.smithi136.stdout:4/736: dread - d1d/d31/d3e/d55/d7f/f83 zero size 2024-04-06T03:40:45.788 INFO:tasks.workunit.client.1.smithi136.stdout:3/804: mkdir d0/d4/d12/d19/d2c/dc9/d111 0 2024-04-06T03:40:45.790 INFO:tasks.workunit.client.0.smithi094.stdout:4/876: mkdir d0/d4/d67/dec/d12a 0 2024-04-06T03:40:45.792 INFO:tasks.workunit.client.0.smithi094.stdout:1/807: dwrite d0/d7/f78 [0,4194304] 0 2024-04-06T03:40:45.792 INFO:tasks.workunit.client.0.smithi094.stdout:1/808: fdatasync d0/d1/d27/da5/d40/fc7 0 2024-04-06T03:40:45.793 INFO:tasks.workunit.client.0.smithi094.stdout:3/801: link d9/d1e/d6e/d75/dba/d3a/c61 d9/d10/dd4/c120 0 2024-04-06T03:40:45.794 INFO:tasks.workunit.client.1.smithi136.stdout:1/845: dwrite d12/d53/f66 [0,4194304] 0 2024-04-06T03:40:45.794 INFO:tasks.workunit.client.1.smithi136.stdout:1/846: fdatasync d12/d1b/da7/ff6 0 2024-04-06T03:40:45.794 INFO:tasks.workunit.client.1.smithi136.stdout:7/926: link d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/f136 d5/df/d1d/d6f/d4d/d69/f13e 0 2024-04-06T03:40:45.796 INFO:tasks.workunit.client.0.smithi094.stdout:4/877: mknod d0/d15/d3a/d61/d81/c12b 0 2024-04-06T03:40:45.796 INFO:tasks.workunit.client.1.smithi136.stdout:9/861: write d17/d20/d25/f26 [3500299,46627] 0 2024-04-06T03:40:45.802 INFO:tasks.workunit.client.1.smithi136.stdout:6/752: creat d2/d7/d26/d30/d4f/d75/ffb x:0 0 0 2024-04-06T03:40:45.803 INFO:tasks.workunit.client.0.smithi094.stdout:1/809: mkdir d0/d1/d27/da5/d40/db1/d2f/d76/dff/d115 0 2024-04-06T03:40:45.803 INFO:tasks.workunit.client.1.smithi136.stdout:1/847: dread d12/d1b/da7/db8/fc7 [0,4194304] 0 2024-04-06T03:40:45.804 INFO:tasks.workunit.client.1.smithi136.stdout:2/893: rename d3/d4a/d9d/d18/d4b/dbd to d3/d15/d19/d43/daa/dec/d12c 0 2024-04-06T03:40:45.806 INFO:tasks.workunit.client.0.smithi094.stdout:3/802: creat d9/d1e/d26/d6c/f121 x:0 0 0 2024-04-06T03:40:45.806 INFO:tasks.workunit.client.0.smithi094.stdout:3/803: truncate d9/d1e/d6e/d75/dba/d7c/da2/df2/f11a 1948294 0 2024-04-06T03:40:45.806 INFO:tasks.workunit.client.0.smithi094.stdout:3/804: chown d9/d1e/d6e/d75/dba/d7c/da2/lb4 15272848 1 2024-04-06T03:40:45.815 INFO:tasks.workunit.client.1.smithi136.stdout:9/862: mknod d17/d20/c12b 0 2024-04-06T03:40:45.818 INFO:tasks.workunit.client.1.smithi136.stdout:6/753: mkdir d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/d94/dfc 0 2024-04-06T03:40:45.820 INFO:tasks.workunit.client.1.smithi136.stdout:3/805: getdents d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/df2 0 2024-04-06T03:40:45.820 INFO:tasks.workunit.client.1.smithi136.stdout:3/806: chown d0/d4/d12/d19/d55/d3c/d5a/d79/fa4 107056194 1 2024-04-06T03:40:45.820 INFO:tasks.workunit.client.1.smithi136.stdout:1/848: dwrite d12/d3b/d44/d5f/fde [4194304,4194304] 0 2024-04-06T03:40:45.821 INFO:tasks.workunit.client.0.smithi094.stdout:1/810: rename d0/d1/d27/da5/d40/db1/f6f to d0/d1/f116 0 2024-04-06T03:40:45.822 INFO:tasks.workunit.client.1.smithi136.stdout:7/927: rmdir d5/df/d53/d68/da3 39 2024-04-06T03:40:45.824 INFO:tasks.workunit.client.0.smithi094.stdout:3/805: unlink d9/d1d/f36 0 2024-04-06T03:40:45.824 INFO:tasks.workunit.client.0.smithi094.stdout:1/811: dread d0/d7/ff9 [0,4194304] 0 2024-04-06T03:40:45.824 INFO:tasks.workunit.client.1.smithi136.stdout:9/863: dread d17/d20/d25/da4/dbb/fc4 [0,4194304] 0 2024-04-06T03:40:45.826 INFO:tasks.workunit.client.1.smithi136.stdout:3/807: rename d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/fb2 to d0/d4/d12/d19/d55/f112 0 2024-04-06T03:40:45.827 INFO:tasks.workunit.client.0.smithi094.stdout:4/878: getdents d0/d77/dc4 0 2024-04-06T03:40:45.831 INFO:tasks.workunit.client.1.smithi136.stdout:8/889: sync 2024-04-06T03:40:45.832 INFO:tasks.workunit.client.1.smithi136.stdout:1/849: link d12/d1b/c90 d12/d3b/d60/d7e/d37/dbb/c11d 0 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:8/789: sync 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:2/734: sync 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:9/717: sync 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:7/695: sync 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:7/696: stat d2/d6/dc/d12/d9a/da0/dd4/f71 0 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:8/790: write d18/d2c/f47 [4591768,70312] 0 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:9/718: write d2/dc/ffe [778289,39328] 0 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:5/775: sync 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:0/756: sync 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:6/726: sync 2024-04-06T03:40:45.833 INFO:tasks.workunit.client.0.smithi094.stdout:6/727: fsync dc/d3a/df4/df6/f4d 0 2024-04-06T03:40:45.834 INFO:tasks.workunit.client.0.smithi094.stdout:4/879: write d0/d77/d123/ff2 [1982605,37538] 0 2024-04-06T03:40:45.838 INFO:tasks.workunit.client.1.smithi136.stdout:9/864: link d17/d20/lf4 d17/d20/d25/da4/dbb/de4/l12c 0 2024-04-06T03:40:45.841 INFO:tasks.workunit.client.0.smithi094.stdout:9/719: unlink d2/f30 0 2024-04-06T03:40:45.841 INFO:tasks.workunit.client.0.smithi094.stdout:9/720: readlink d2/d2c/ld7 0 2024-04-06T03:40:45.841 INFO:tasks.workunit.client.0.smithi094.stdout:9/721: fsync d2/ffc 0 2024-04-06T03:40:45.843 INFO:tasks.workunit.client.1.smithi136.stdout:3/808: rename d0/d4/d12/d2d/c38 to d0/d4/d12/d19/d2e/c113 0 2024-04-06T03:40:45.844 INFO:tasks.workunit.client.0.smithi094.stdout:7/697: symlink d2/d6/dc/d12/d29/daa/le1 0 2024-04-06T03:40:45.844 INFO:tasks.workunit.client.0.smithi094.stdout:7/698: stat d2/d6/dc/d12/d29/d3f/db9/cc5 0 2024-04-06T03:40:45.846 INFO:tasks.workunit.client.0.smithi094.stdout:1/812: dread d0/d1/d27/da5/d3f/f5c [0,4194304] 0 2024-04-06T03:40:45.849 INFO:tasks.workunit.client.0.smithi094.stdout:3/806: write d9/d20/d2d/f67 [2060483,68513] 0 2024-04-06T03:40:45.849 INFO:tasks.workunit.client.0.smithi094.stdout:3/807: creat d9/d1e/d26/d6c/f122 x:0 0 0 2024-04-06T03:40:45.849 INFO:tasks.workunit.client.0.smithi094.stdout:3/808: chown d9/d9d/dc1 338 1 2024-04-06T03:40:45.850 INFO:tasks.workunit.client.1.smithi136.stdout:9/865: getdents d17/d20/d28/d81/d88 0 2024-04-06T03:40:45.850 INFO:tasks.workunit.client.1.smithi136.stdout:6/754: dwrite d2/d7/d26/d30/d4f/fcb [0,4194304] 0 2024-04-06T03:40:45.850 INFO:tasks.workunit.client.1.smithi136.stdout:7/928: dwrite d5/df/d53/d68/fb0 [4194304,4194304] 0 2024-04-06T03:40:45.850 INFO:tasks.workunit.client.0.smithi094.stdout:2/735: rename d0/d8/d4f/l6b to d0/dd/d1b/d72/da8/dd2/lfc 0 2024-04-06T03:40:45.851 INFO:tasks.workunit.client.1.smithi136.stdout:3/809: rename d0/d4/fa to d0/d4/d12/d19/d55/d3c/d5a/d79/d98/dc6/f114 0 2024-04-06T03:40:45.853 INFO:tasks.workunit.client.0.smithi094.stdout:7/699: dread d2/d6/f24 [0,4194304] 0 2024-04-06T03:40:45.854 INFO:tasks.workunit.client.1.smithi136.stdout:8/890: dwrite d3/da/d26/d35/d53/dd2/fe3 [0,4194304] 0 2024-04-06T03:40:45.856 INFO:tasks.workunit.client.0.smithi094.stdout:0/757: truncate d1/d9/dd/d18/d5d/d60/dfd/f52 863596 0 2024-04-06T03:40:45.856 INFO:tasks.workunit.client.0.smithi094.stdout:8/791: dwrite d18/f22 [4194304,4194304] 0 2024-04-06T03:40:45.857 INFO:tasks.workunit.client.1.smithi136.stdout:9/866: dread d17/d20/d25/f26 [0,4194304] 0 2024-04-06T03:40:45.857 INFO:tasks.workunit.client.1.smithi136.stdout:7/929: dread d5/df/d53/f43 [0,4194304] 0 2024-04-06T03:40:45.858 INFO:tasks.workunit.client.0.smithi094.stdout:6/728: dwrite dc/d11/d97/fa1 [0,4194304] 0 2024-04-06T03:40:45.858 INFO:tasks.workunit.client.0.smithi094.stdout:9/722: dwrite d2/d2c/d99/faf [0,4194304] 0 2024-04-06T03:40:45.860 INFO:tasks.workunit.client.0.smithi094.stdout:3/809: mknod d9/d1e/d26/d6c/c123 0 2024-04-06T03:40:45.869 INFO:tasks.workunit.client.0.smithi094.stdout:2/736: unlink d0/d8/d1d/d77/fcf 0 2024-04-06T03:40:45.870 INFO:tasks.workunit.client.0.smithi094.stdout:2/737: write d0/d8/fb3 [4764040,66986] 0 2024-04-06T03:40:45.871 INFO:tasks.workunit.client.1.smithi136.stdout:6/755: link d2/d92/fc1 d2/d7/d13/d89/ffd 0 2024-04-06T03:40:45.872 INFO:tasks.workunit.client.1.smithi136.stdout:9/867: dread d17/d20/d25/d71/dd5/fd8 [0,4194304] 0 2024-04-06T03:40:45.873 INFO:tasks.workunit.client.0.smithi094.stdout:5/776: rename db/d7c/dd7/f104 to db/d19/d24/d2e/f10e 0 2024-04-06T03:40:45.877 INFO:tasks.workunit.client.1.smithi136.stdout:0/815: sync 2024-04-06T03:40:45.877 INFO:tasks.workunit.client.0.smithi094.stdout:0/758: creat d1/d9/dd/d18/d1e/d86/d96/dc0/f104 x:0 0 0 2024-04-06T03:40:45.877 INFO:tasks.workunit.client.0.smithi094.stdout:9/723: dwrite d2/dc/d80/ddd/d6b/f73 [0,4194304] 0 2024-04-06T03:40:45.880 INFO:tasks.workunit.client.0.smithi094.stdout:2/738: dread d0/d8/f11 [0,4194304] 0 2024-04-06T03:40:45.880 INFO:tasks.workunit.client.0.smithi094.stdout:6/729: dwrite dc/d3a/d4a/d54/dde/f9e [0,4194304] 0 2024-04-06T03:40:45.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/792: chown d18/d3c/d49/c5a 2043698 1 2024-04-06T03:40:45.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/793: stat d18/d2c/d37/d60/da3/d111 0 2024-04-06T03:40:45.882 INFO:tasks.workunit.client.0.smithi094.stdout:8/794: write d18/d2c/fac [171504,28716] 0 2024-04-06T03:40:45.883 INFO:tasks.workunit.client.1.smithi136.stdout:8/891: creat d3/da/d26/d35/dcb/d10d/f11a x:0 0 0 2024-04-06T03:40:45.887 INFO:tasks.workunit.client.1.smithi136.stdout:7/930: truncate d5/f2e 3418132 0 2024-04-06T03:40:45.887 INFO:tasks.workunit.client.1.smithi136.stdout:7/931: write d5/df/d53/d8a/f11d [822853,14794] 0 2024-04-06T03:40:45.890 INFO:tasks.workunit.client.0.smithi094.stdout:3/810: creat d9/d20/d2d/da9/f124 x:0 0 0 2024-04-06T03:40:45.892 INFO:tasks.workunit.client.0.smithi094.stdout:4/880: rename d0/df/d1e/d34/ff0 to d0/df/db3/f12c 0 2024-04-06T03:40:45.893 INFO:tasks.workunit.client.1.smithi136.stdout:9/868: creat d17/f12d x:0 0 0 2024-04-06T03:40:45.893 INFO:tasks.workunit.client.1.smithi136.stdout:9/869: chown d17/d21/d29/de5/f11f 45262 1 2024-04-06T03:40:45.894 INFO:tasks.workunit.client.0.smithi094.stdout:5/777: unlink db/fd 0 2024-04-06T03:40:45.900 INFO:tasks.workunit.client.1.smithi136.stdout:8/892: mkdir d3/da/d26/d35/d3d/dc0/d11b 0 2024-04-06T03:40:45.900 INFO:tasks.workunit.client.0.smithi094.stdout:9/724: symlink d2/dc/d21/d26/d78/d69/da8/l106 0 2024-04-06T03:40:45.901 INFO:tasks.workunit.client.0.smithi094.stdout:9/725: write d2/dc/d21/d23/fe1 [448731,72546] 0 2024-04-06T03:40:45.901 INFO:tasks.workunit.client.1.smithi136.stdout:0/816: dwrite d1/d6/d2c/d57/d43/fa2 [0,4194304] 0 2024-04-06T03:40:45.901 INFO:tasks.workunit.client.1.smithi136.stdout:7/932: mkdir d5/d13f 0 2024-04-06T03:40:45.902 INFO:tasks.workunit.client.0.smithi094.stdout:6/730: mknod dc/d3a/d4a/d54/d71/d103/c105 0 2024-04-06T03:40:45.902 INFO:tasks.workunit.client.0.smithi094.stdout:6/731: write dc/d3a/df4/df6/fe2 [959269,68070] 0 2024-04-06T03:40:45.902 INFO:tasks.workunit.client.0.smithi094.stdout:6/732: readlink dc/d11/l37 0 2024-04-06T03:40:45.903 INFO:tasks.workunit.client.1.smithi136.stdout:6/756: rmdir d2/d7/d26/d30/d6e/d58/d5d/d62/d8d 39 2024-04-06T03:40:45.903 INFO:tasks.workunit.client.1.smithi136.stdout:6/757: dread - d2/d7/d26/d30/d6e/d58/d5d/d62/fb0 zero size 2024-04-06T03:40:45.904 INFO:tasks.workunit.client.0.smithi094.stdout:8/795: getdents d18/d34/d52 0 2024-04-06T03:40:45.906 INFO:tasks.workunit.client.0.smithi094.stdout:4/881: write d0/d77/d123/d46/fce [2927328,87433] 0 2024-04-06T03:40:45.907 INFO:tasks.workunit.client.1.smithi136.stdout:9/870: getdents d17/d21/d32/dc6/d113/d66/de1/d51/d57/de3 0 2024-04-06T03:40:45.908 INFO:tasks.workunit.client.0.smithi094.stdout:1/813: rename d0/d7/f78 to d0/d1/dd3/f117 0 2024-04-06T03:40:45.908 INFO:tasks.workunit.client.0.smithi094.stdout:1/814: dread - d0/d1/d27/da5/fc4 zero size 2024-04-06T03:40:45.910 INFO:tasks.workunit.client.1.smithi136.stdout:3/810: rename d0/d4/d12/d19/d2c to d0/d4/d12/d19/d115 0 2024-04-06T03:40:45.910 INFO:tasks.workunit.client.0.smithi094.stdout:5/778: creat db/d19/d2c/d6d/de6/d102/f10f x:0 0 0 2024-04-06T03:40:45.911 INFO:tasks.workunit.client.1.smithi136.stdout:8/893: getdents d3/da/d26 0 2024-04-06T03:40:45.912 INFO:tasks.workunit.client.0.smithi094.stdout:0/759: getdents d1/d9/dd/d18/d78 0 2024-04-06T03:40:45.914 INFO:tasks.workunit.client.0.smithi094.stdout:9/726: symlink d2/d2c/l107 0 2024-04-06T03:40:45.917 INFO:tasks.workunit.client.0.smithi094.stdout:6/733: link dc/d3a/d4a/d94/dac/fb8 dc/f106 0 2024-04-06T03:40:45.919 INFO:tasks.workunit.client.0.smithi094.stdout:0/760: read d1/d9/dd/d18/d5d/de9/f100 [1364214,46644] 0 2024-04-06T03:40:45.919 INFO:tasks.workunit.client.1.smithi136.stdout:3/811: dread d0/d4/d12/d19/f29 [0,4194304] 0 2024-04-06T03:40:45.919 INFO:tasks.workunit.client.1.smithi136.stdout:3/812: chown d0/d4/d12/d19/d55/d6f/d82/fb4 103743655 1 2024-04-06T03:40:45.920 INFO:tasks.workunit.client.1.smithi136.stdout:3/813: write d0/d4/d12/d19/d55/d3c/d46/f72 [4823671,72451] 0 2024-04-06T03:40:45.920 INFO:tasks.workunit.client.1.smithi136.stdout:0/817: mkdir d1/d6/d2c/d57/d43/d6d/d9b/d116 0 2024-04-06T03:40:45.920 INFO:tasks.workunit.client.1.smithi136.stdout:3/814: dread d0/d4/d12/d19/d55/d6f/fb1 [0,4194304] 0 2024-04-06T03:40:45.920 INFO:tasks.workunit.client.1.smithi136.stdout:3/815: stat d0/d4/d12/d3a 0 2024-04-06T03:40:45.921 INFO:tasks.workunit.client.1.smithi136.stdout:4/737: sync 2024-04-06T03:40:45.922 INFO:tasks.workunit.client.1.smithi136.stdout:2/894: sync 2024-04-06T03:40:45.922 INFO:tasks.workunit.client.1.smithi136.stdout:1/850: sync 2024-04-06T03:40:45.922 INFO:tasks.workunit.client.1.smithi136.stdout:5/905: sync 2024-04-06T03:40:45.922 INFO:tasks.workunit.client.1.smithi136.stdout:5/906: chown d1/d16/d4b/l77 74725 1 2024-04-06T03:40:45.922 INFO:tasks.workunit.client.1.smithi136.stdout:4/738: write d1d/d31/d78/feb [408665,22188] 0 2024-04-06T03:40:45.923 INFO:tasks.workunit.client.0.smithi094.stdout:8/796: mknod d18/d3c/dc4/dbc/c112 0 2024-04-06T03:40:45.923 INFO:tasks.workunit.client.0.smithi094.stdout:8/797: readlink d18/d2c/d37/d60/da3/db1/lb4 0 2024-04-06T03:40:45.923 INFO:tasks.workunit.client.0.smithi094.stdout:8/798: dread - d18/d2c/f6e zero size 2024-04-06T03:40:45.923 INFO:tasks.workunit.client.0.smithi094.stdout:8/799: dread - d18/d3c/d49/d91/d94/daa/ffd zero size 2024-04-06T03:40:45.925 INFO:tasks.workunit.client.1.smithi136.stdout:7/933: unlink d5/df/d1d/f7f 0 2024-04-06T03:40:45.925 INFO:tasks.workunit.client.1.smithi136.stdout:8/894: dwrite d3/da/d26/d35/d53/dd2/f111 [0,4194304] 0 2024-04-06T03:40:45.928 INFO:tasks.workunit.client.1.smithi136.stdout:6/758: rename d2/d7/d26/ceb to d2/d7/d13/d89/cfe 0 2024-04-06T03:40:45.929 INFO:tasks.workunit.client.0.smithi094.stdout:4/882: mknod d0/d4/d67/d9f/d11b/c12d 0 2024-04-06T03:40:45.932 INFO:tasks.workunit.client.0.smithi094.stdout:7/700: rename d2/d6/dc/dd/f92 to d2/d6/dc/d12/d9a/da0/dd4/fe2 0 2024-04-06T03:40:45.932 INFO:tasks.workunit.client.0.smithi094.stdout:7/701: fsync d2/fb 0 2024-04-06T03:40:45.932 INFO:tasks.workunit.client.1.smithi136.stdout:3/816: mkdir d0/d4/d12/d2d/d37/d116 0 2024-04-06T03:40:45.934 INFO:tasks.workunit.client.0.smithi094.stdout:1/815: mknod d0/d1/d27/da5/d40/db1/d2f/d3c/da0/c118 0 2024-04-06T03:40:45.936 INFO:tasks.workunit.client.0.smithi094.stdout:5/779: truncate db/d19/d2c/f31 5580743 0 2024-04-06T03:40:45.936 INFO:tasks.workunit.client.0.smithi094.stdout:5/780: chown db/d19/d24/d28/d85 2205 1 2024-04-06T03:40:45.936 INFO:tasks.workunit.client.1.smithi136.stdout:2/895: link d3/d4a/d9d/l5b d3/d15/d19/d110/l12d 0 2024-04-06T03:40:45.938 INFO:tasks.workunit.client.0.smithi094.stdout:9/727: unlink d2/dc/d46/lc4 0 2024-04-06T03:40:45.939 INFO:tasks.workunit.client.1.smithi136.stdout:1/851: mkdir d12/d3b/d74/d7a/dea/d11a/d6f/d11e 0 2024-04-06T03:40:45.940 INFO:tasks.workunit.client.0.smithi094.stdout:6/734: creat dc/dd3/f107 x:0 0 0 2024-04-06T03:40:45.943 INFO:tasks.workunit.client.0.smithi094.stdout:8/800: chown d18/d3c/d49/d80/faf 193 1 2024-04-06T03:40:45.944 INFO:tasks.workunit.client.1.smithi136.stdout:7/934: stat d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/l65 0 2024-04-06T03:40:45.944 INFO:tasks.workunit.client.1.smithi136.stdout:7/935: write d5/df/d3d/f97 [592457,90443] 0 2024-04-06T03:40:45.944 INFO:tasks.workunit.client.1.smithi136.stdout:7/936: chown d5/f82 1172080783 1 2024-04-06T03:40:45.947 INFO:tasks.workunit.client.1.smithi136.stdout:5/907: dwrite d1/d16/d89/daf/f111 [0,4194304] 0 2024-04-06T03:40:45.947 INFO:tasks.workunit.client.0.smithi094.stdout:4/883: dwrite d0/d23/dc1/fcd [0,4194304] 0 2024-04-06T03:40:45.948 INFO:tasks.workunit.client.0.smithi094.stdout:4/884: truncate d0/d77/d123/d46/f95 4530235 0 2024-04-06T03:40:45.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/761: dwrite d1/d9/dd/d18/d32/f5f [0,4194304] 0 2024-04-06T03:40:45.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/762: truncate d1/d9/dd/fde 520825 0 2024-04-06T03:40:45.951 INFO:tasks.workunit.client.1.smithi136.stdout:1/852: dread d12/d3b/d60/d7e/d37/f4e [0,4194304] 0 2024-04-06T03:40:45.953 INFO:tasks.workunit.client.1.smithi136.stdout:8/895: mkdir d3/da/d26/d90/dae/db5/d11c 0 2024-04-06T03:40:45.953 INFO:tasks.workunit.client.1.smithi136.stdout:8/896: read - d3/de/d60/fe6 zero size 2024-04-06T03:40:45.956 INFO:tasks.workunit.client.1.smithi136.stdout:4/739: dread d1d/d31/f40 [0,4194304] 0 2024-04-06T03:40:45.956 INFO:tasks.workunit.client.1.smithi136.stdout:4/740: write d1d/d31/d78/fbd [435289,44731] 0 2024-04-06T03:40:45.956 INFO:tasks.workunit.client.1.smithi136.stdout:4/741: fsync d1d/d31/d3e/d55/d7f/fb6 0 2024-04-06T03:40:45.956 INFO:tasks.workunit.client.0.smithi094.stdout:2/739: rename d0/d8/d1d/d77/fcd to d0/dd/d19/d31/dcc/ffd 0 2024-04-06T03:40:45.957 INFO:tasks.workunit.client.1.smithi136.stdout:9/871: rename d17/d20/d25/d71/fa2 to d17/d19/da3/dc5/deb/d9e/dfb/f12e 0 2024-04-06T03:40:45.957 INFO:tasks.workunit.client.1.smithi136.stdout:9/872: truncate d17/d21/d32/dc6/d113/f11a 467705 0 2024-04-06T03:40:45.960 INFO:tasks.workunit.client.0.smithi094.stdout:7/702: creat d2/d6/dc/d12/d9a/dbd/fe3 x:0 0 0 2024-04-06T03:40:45.960 INFO:tasks.workunit.client.1.smithi136.stdout:6/759: mknod d2/d7/d13/d79/cff 0 2024-04-06T03:40:45.962 INFO:tasks.workunit.client.0.smithi094.stdout:1/816: symlink d0/d1/d27/da5/d40/db1/d2f/d3c/d48/l119 0 2024-04-06T03:40:45.962 INFO:tasks.workunit.client.0.smithi094.stdout:1/817: fsync d0/d1/d27/da5/f3b 0 2024-04-06T03:40:45.962 INFO:tasks.workunit.client.0.smithi094.stdout:1/818: truncate d0/d1/d27/da5/d40/db1/d2f/d4b/f9e 756911 0 2024-04-06T03:40:45.962 INFO:tasks.workunit.client.0.smithi094.stdout:0/763: dwrite d1/d9/dd/d18/d5d/d60/dfd/f3c [0,4194304] 0 2024-04-06T03:40:45.963 INFO:tasks.workunit.client.1.smithi136.stdout:3/817: mknod d0/d4/d12/d19/d55/d6f/d82/d5d/dbd/c117 0 2024-04-06T03:40:45.965 INFO:tasks.workunit.client.0.smithi094.stdout:5/781: mkdir db/d110 0 2024-04-06T03:40:45.968 INFO:tasks.workunit.client.0.smithi094.stdout:9/728: creat d2/dc/d80/ddd/d6b/f108 x:0 0 0 2024-04-06T03:40:45.968 INFO:tasks.workunit.client.0.smithi094.stdout:9/729: write d2/fe [3710592,50219] 0 2024-04-06T03:40:45.969 INFO:tasks.workunit.client.0.smithi094.stdout:8/801: rmdir d18/d3c/d76/dd9 39 2024-04-06T03:40:45.973 INFO:tasks.workunit.client.0.smithi094.stdout:1/819: dwrite d0/d1/d27/da5/d40/db1/d2f/d3c/d48/fee [0,4194304] 0 2024-04-06T03:40:45.975 INFO:tasks.workunit.client.0.smithi094.stdout:1/820: dread d0/d1/d27/da5/d40/d50/ff7 [0,4194304] 0 2024-04-06T03:40:45.976 INFO:tasks.workunit.client.1.smithi136.stdout:7/937: getdents d5/df/d1d/d6f/d4d 0 2024-04-06T03:40:45.985 INFO:tasks.workunit.client.1.smithi136.stdout:5/908: creat d1/dc/d1c/d32/d70/f116 x:0 0 0 2024-04-06T03:40:45.985 INFO:tasks.workunit.client.1.smithi136.stdout:5/909: getdents d1/dc/d10/da3/d114 0 2024-04-06T03:40:45.987 INFO:tasks.workunit.client.0.smithi094.stdout:3/811: rename d9/d20/d30/d88/ff1 to d9/d1e/d26/d6c/f125 0 2024-04-06T03:40:45.987 INFO:tasks.workunit.client.0.smithi094.stdout:3/812: write d9/d1e/d6e/d75/dba/d97/f11d [332908,5548] 0 2024-04-06T03:40:45.989 INFO:tasks.workunit.client.1.smithi136.stdout:1/853: mkdir d12/d3b/d74/dd6/d11f 0 2024-04-06T03:40:45.989 INFO:tasks.workunit.client.1.smithi136.stdout:1/854: dread - d12/d3b/d44/d5f/db7/fdd zero size 2024-04-06T03:40:45.989 INFO:tasks.workunit.client.1.smithi136.stdout:1/855: chown d12/d3b/d74/d94 91 1 2024-04-06T03:40:45.989 INFO:tasks.workunit.client.1.smithi136.stdout:1/856: fsync d12/d3b/d74/d7a/dea/d77/fc1 0 2024-04-06T03:40:45.991 INFO:tasks.workunit.client.0.smithi094.stdout:2/740: creat d0/d8/d4f/dab/ffe x:0 0 0 2024-04-06T03:40:45.992 INFO:tasks.workunit.client.1.smithi136.stdout:8/897: rmdir d3/da/d26/d35/d3d/dc7/dce/df2 39 2024-04-06T03:40:45.992 INFO:tasks.workunit.client.1.smithi136.stdout:8/898: getdents d3/de/d60/de1 0 2024-04-06T03:40:45.992 INFO:tasks.workunit.client.1.smithi136.stdout:8/899: dread - d3/da/d26/d35/dcb/d10d/f11a zero size 2024-04-06T03:40:45.994 INFO:tasks.workunit.client.0.smithi094.stdout:7/703: getdents d2/d6/dc/d12/d3a 0 2024-04-06T03:40:46.004 INFO:tasks.workunit.client.1.smithi136.stdout:7/938: dwrite d5/df/d11/f2b [0,4194304] 0 2024-04-06T03:40:46.004 INFO:tasks.workunit.client.1.smithi136.stdout:0/818: rename d1/d6/d2c/d1a/l32 to d1/d6/d2c/d57/d43/d6d/d9b/d116/l117 0 2024-04-06T03:40:46.005 INFO:tasks.workunit.client.1.smithi136.stdout:0/819: fsync d1/d6/d2c/d57/d43/d6d/d9b/d5c/f6a 0 2024-04-06T03:40:46.005 INFO:tasks.workunit.client.1.smithi136.stdout:0/820: chown d1/d6/d2c/d1a/d36/d59/f38 23 1 2024-04-06T03:40:46.006 INFO:tasks.workunit.client.0.smithi094.stdout:0/764: mkdir d1/d9/dd/d18/d5d/d60/dbc/d105 0 2024-04-06T03:40:46.008 INFO:tasks.workunit.client.1.smithi136.stdout:9/873: creat d17/d20/d25/d71/dd5/f12f x:0 0 0 2024-04-06T03:40:46.009 INFO:tasks.workunit.client.0.smithi094.stdout:5/782: link db/d19/d2c/l95 db/d19/d21/l111 0 2024-04-06T03:40:46.011 INFO:tasks.workunit.client.1.smithi136.stdout:9/874: dread fb [0,4194304] 0 2024-04-06T03:40:46.011 INFO:tasks.workunit.client.1.smithi136.stdout:9/875: write d17/d21/d29/de5/da5/dce/dd1/fda [175337,64120] 0 2024-04-06T03:40:46.012 INFO:tasks.workunit.client.1.smithi136.stdout:5/910: dwrite d1/d9b/dea/ff2 [0,4194304] 0 2024-04-06T03:40:46.012 INFO:tasks.workunit.client.1.smithi136.stdout:5/911: creat d1/d16/f117 x:0 0 0 2024-04-06T03:40:46.012 INFO:tasks.workunit.client.0.smithi094.stdout:2/741: dwrite d0/d8/d1d/d77/fa1 [0,4194304] 0 2024-04-06T03:40:46.012 INFO:tasks.workunit.client.0.smithi094.stdout:9/730: symlink d2/dc/d21/d26/d78/d69/da8/l109 0 2024-04-06T03:40:46.013 INFO:tasks.workunit.client.1.smithi136.stdout:6/760: creat d2/d7/d13/f100 x:0 0 0 2024-04-06T03:40:46.016 INFO:tasks.workunit.client.0.smithi094.stdout:8/802: mknod d18/d5c/c113 0 2024-04-06T03:40:46.021 INFO:tasks.workunit.client.0.smithi094.stdout:4/885: rmdir d0/df/dbd 39 2024-04-06T03:40:46.021 INFO:tasks.workunit.client.0.smithi094.stdout:4/886: truncate d0/d77/d123/d46/fa3 2026568 0 2024-04-06T03:40:46.021 INFO:tasks.workunit.client.0.smithi094.stdout:9/731: write d2/dc/d21/d26/f3c [2146022,109407] 0 2024-04-06T03:40:46.022 INFO:tasks.workunit.client.0.smithi094.stdout:9/732: truncate d2/dc/d10/ff4 764454 0 2024-04-06T03:40:46.022 INFO:tasks.workunit.client.1.smithi136.stdout:1/857: getdents d12/d3b/d74/d7a/dea/d77/d96 0 2024-04-06T03:40:46.023 INFO:tasks.workunit.client.0.smithi094.stdout:1/821: creat d0/d1/d8f/f11a x:0 0 0 2024-04-06T03:40:46.023 INFO:tasks.workunit.client.0.smithi094.stdout:1/822: chown d0/d1/d27/da5/d40/db1/d2f/d76/d30/df6 1833 1 2024-04-06T03:40:46.024 INFO:tasks.workunit.client.1.smithi136.stdout:8/900: creat d3/de/f11d x:0 0 0 2024-04-06T03:40:46.026 INFO:tasks.workunit.client.1.smithi136.stdout:9/876: dwrite d17/d19/da3/dc5/deb/fb1 [4194304,4194304] 0 2024-04-06T03:40:46.027 INFO:tasks.workunit.client.0.smithi094.stdout:6/735: rename dc/dd3/d72/f77 to dc/d3a/df4/df6/df2/f108 0 2024-04-06T03:40:46.028 INFO:tasks.workunit.client.0.smithi094.stdout:6/736: dread dc/d3a/df4/df6/fc2 [0,4194304] 0 2024-04-06T03:40:46.028 INFO:tasks.workunit.client.0.smithi094.stdout:6/737: fdatasync dc/dd3/f107 0 2024-04-06T03:40:46.029 INFO:tasks.workunit.client.0.smithi094.stdout:8/803: dwrite d18/d3c/d76/f5b [0,4194304] 0 2024-04-06T03:40:46.032 INFO:tasks.workunit.client.1.smithi136.stdout:7/939: rmdir d5/df/d12/d1b/d3a/d4a/d70/ddb 39 2024-04-06T03:40:46.032 INFO:tasks.workunit.client.1.smithi136.stdout:7/940: chown d5/df/d12/d1b/d3a/fc3 2339068 1 2024-04-06T03:40:46.035 INFO:tasks.workunit.client.1.smithi136.stdout:4/742: rename d1d/d31/d3e/d56/cda to d1d/d2f/d6c/da5/c106 0 2024-04-06T03:40:46.036 INFO:tasks.workunit.client.1.smithi136.stdout:0/821: symlink d1/d6/d2c/d57/d43/d6d/d85/d78/l118 0 2024-04-06T03:40:46.036 INFO:tasks.workunit.client.1.smithi136.stdout:0/822: creat d1/d6/d2c/d84/f119 x:0 0 0 2024-04-06T03:40:46.050 INFO:tasks.workunit.client.0.smithi094.stdout:0/765: creat d1/d9/dd/d18/d1e/def/f106 x:0 0 0 2024-04-06T03:40:46.054 INFO:tasks.workunit.client.1.smithi136.stdout:5/912: symlink d1/dc/d1c/d32/d70/dc1/l118 0 2024-04-06T03:40:46.054 INFO:tasks.workunit.client.0.smithi094.stdout:5/783: mknod db/d19/d2c/d57/c112 0 2024-04-06T03:40:46.057 INFO:tasks.workunit.client.1.smithi136.stdout:5/913: dread d1/f96 [4194304,4194304] 0 2024-04-06T03:40:46.058 INFO:tasks.workunit.client.1.smithi136.stdout:6/761: creat d2/d93/dab/f101 x:0 0 0 2024-04-06T03:40:46.058 INFO:tasks.workunit.client.1.smithi136.stdout:8/901: rmdir d3/da/d26/d35/d53/d86/db0 39 2024-04-06T03:40:46.062 INFO:tasks.workunit.client.0.smithi094.stdout:2/742: mknod d0/d8/d4f/dab/cff 0 2024-04-06T03:40:46.064 INFO:tasks.workunit.client.1.smithi136.stdout:9/877: mkdir d17/d19/da3/dc5/d130 0 2024-04-06T03:40:46.071 INFO:tasks.workunit.client.0.smithi094.stdout:9/733: creat d2/dc/d46/f10a x:0 0 0 2024-04-06T03:40:46.081 INFO:tasks.workunit.client.0.smithi094.stdout:9/734: dread d2/dc/d21/d23/d2f/f5f [4194304,4194304] 0 2024-04-06T03:40:46.081 INFO:tasks.workunit.client.0.smithi094.stdout:6/738: rename dc/d3a/d4a/d54/dde/d50/d61/ff5 to dc/d3a/d4a/d94/f109 0 2024-04-06T03:40:46.084 INFO:tasks.workunit.client.1.smithi136.stdout:0/823: fsync d1/d6/fa1 0 2024-04-06T03:40:46.084 INFO:tasks.workunit.client.1.smithi136.stdout:0/824: truncate d1/d6/d2c/d1a/d36/f5d 880237 0 2024-04-06T03:40:46.085 INFO:tasks.workunit.client.0.smithi094.stdout:6/739: dread dc/f28 [0,4194304] 0 2024-04-06T03:40:46.086 INFO:tasks.workunit.client.0.smithi094.stdout:0/766: mknod d1/d9/dd/d18/d1e/d86/d96/dc0/c107 0 2024-04-06T03:40:46.088 INFO:tasks.workunit.client.1.smithi136.stdout:6/762: link d2/d7/d26/d30/d6e/d58/d5d/d62/d53/l7b d2/d7/d26/d30/d6e/d58/d5d/l102 0 2024-04-06T03:40:46.089 INFO:tasks.workunit.client.1.smithi136.stdout:4/743: dwrite d1d/d31/f48 [0,4194304] 0 2024-04-06T03:40:46.089 INFO:tasks.workunit.client.1.smithi136.stdout:4/744: creat d1d/d31/d3e/d55/d7f/f107 x:0 0 0 2024-04-06T03:40:46.089 INFO:tasks.workunit.client.1.smithi136.stdout:4/745: stat d1d/d31/d3e/d55/d89/c103 0 2024-04-06T03:40:46.089 INFO:tasks.workunit.client.1.smithi136.stdout:4/746: readlink d1d/d31/d3e/d55/lb1 0 2024-04-06T03:40:46.089 INFO:tasks.workunit.client.0.smithi094.stdout:9/735: read d2/dc/d21/d26/d78/d69/da8/f3a [4141623,1463] 0 2024-04-06T03:40:46.090 INFO:tasks.workunit.client.0.smithi094.stdout:5/784: creat db/d110/f113 x:0 0 0 2024-04-06T03:40:46.090 INFO:tasks.workunit.client.0.smithi094.stdout:5/785: dread - db/d19/d24/d28/f108 zero size 2024-04-06T03:40:46.091 INFO:tasks.workunit.client.1.smithi136.stdout:8/902: mkdir d3/da/d26/d35/d39/d70/d11e 0 2024-04-06T03:40:46.094 INFO:tasks.workunit.client.0.smithi094.stdout:2/743: getdents d0 0 2024-04-06T03:40:46.094 INFO:tasks.workunit.client.0.smithi094.stdout:2/744: chown d0/dd/d19/c9f 2 1 2024-04-06T03:40:46.094 INFO:tasks.workunit.client.0.smithi094.stdout:2/745: dread - d0/dd/da4/fcb zero size 2024-04-06T03:40:46.094 INFO:tasks.workunit.client.0.smithi094.stdout:2/746: chown d0/d8/d1d/cd8 10577 1 2024-04-06T03:40:46.094 INFO:tasks.workunit.client.0.smithi094.stdout:2/747: creat d0/dd/d3d/f100 x:0 0 0 2024-04-06T03:40:46.096 INFO:tasks.workunit.client.1.smithi136.stdout:9/878: truncate d17/d20/d28/d49/d75/d60/f7d 5028766 0 2024-04-06T03:40:46.098 INFO:tasks.workunit.client.0.smithi094.stdout:1/823: rmdir d0/d1/dd3 39 2024-04-06T03:40:46.099 INFO:tasks.workunit.client.1.smithi136.stdout:9/879: dread d17/d21/d32/f74 [0,4194304] 0 2024-04-06T03:40:46.101 INFO:tasks.workunit.client.1.smithi136.stdout:1/858: rename d12/d3b/d60/d6d to d12/d1b/d120 0 2024-04-06T03:40:46.102 INFO:tasks.workunit.client.1.smithi136.stdout:1/859: readlink d12/d3b/d44/d5f/d5a/l108 0 2024-04-06T03:40:46.103 INFO:tasks.workunit.client.0.smithi094.stdout:8/804: rename d18/d5c/de3/d10a to d18/d3c/d49/d91/d114 0 2024-04-06T03:40:46.106 INFO:tasks.workunit.client.1.smithi136.stdout:6/763: dread d2/d7/d26/d30/d6e/f69 [0,4194304] 0 2024-04-06T03:40:46.106 INFO:tasks.workunit.client.1.smithi136.stdout:0/825: mkdir d1/d6/d2c/d57/d43/d6d/d85/d11a 0 2024-04-06T03:40:46.107 INFO:tasks.workunit.client.0.smithi094.stdout:9/736: dwrite d2/dc/d21/d26/d78/d69/da8/d96/fa0 [4194304,4194304] 0 2024-04-06T03:40:46.107 INFO:tasks.workunit.client.0.smithi094.stdout:9/737: chown d2/dc/d21/d23/d47/d51/fd4 49636 1 2024-04-06T03:40:46.107 INFO:tasks.workunit.client.0.smithi094.stdout:9/738: write d2/ff7 [836087,95186] 0 2024-04-06T03:40:46.107 INFO:tasks.workunit.client.0.smithi094.stdout:9/739: fdatasync d2/dc/d21/f9c 0 2024-04-06T03:40:46.107 INFO:tasks.workunit.client.0.smithi094.stdout:8/805: dread d18/d2c/d37/d60/da3/db8/dc3/fd4 [0,4194304] 0 2024-04-06T03:40:46.108 INFO:tasks.workunit.client.1.smithi136.stdout:1/860: dread d12/d3b/d8e/fbc [0,4194304] 0 2024-04-06T03:40:46.108 INFO:tasks.workunit.client.1.smithi136.stdout:1/861: write d12/d1b/d28/f91 [4395975,10106] 0 2024-04-06T03:40:46.109 INFO:tasks.workunit.client.0.smithi094.stdout:6/740: unlink dc/dd3/d24/f25 0 2024-04-06T03:40:46.109 INFO:tasks.workunit.client.0.smithi094.stdout:6/741: write dc/d3a/d4a/d94/f109 [603989,59840] 0 2024-04-06T03:40:46.109 INFO:tasks.workunit.client.0.smithi094.stdout:9/740: dread d2/dc/d46/f60 [0,4194304] 0 2024-04-06T03:40:46.109 INFO:tasks.workunit.client.0.smithi094.stdout:0/767: mknod d1/d9/dd/d18/d1e/d22/d3e/c108 0 2024-04-06T03:40:46.109 INFO:tasks.workunit.client.0.smithi094.stdout:0/768: chown d1/d9/dd/l6d 25347437 1 2024-04-06T03:40:46.109 INFO:tasks.workunit.client.0.smithi094.stdout:0/769: fsync d1/d9/dd/d18/d1e/d86/d96/dc0/f104 0 2024-04-06T03:40:46.109 INFO:tasks.workunit.client.0.smithi094.stdout:0/770: write d1/d9/fd4 [851769,33598] 0 2024-04-06T03:40:46.115 INFO:tasks.workunit.client.0.smithi094.stdout:1/824: rename d0/d7/d1a/lba to d0/d7/d1a/da8/l11b 0 2024-04-06T03:40:46.116 INFO:tasks.workunit.client.1.smithi136.stdout:8/903: mkdir d3/da/d26/d35/d3d/dc0/d11f 0 2024-04-06T03:40:46.118 INFO:tasks.workunit.client.0.smithi094.stdout:8/806: mkdir d18/d2c/d37/d115 0 2024-04-06T03:40:46.120 INFO:tasks.workunit.client.1.smithi136.stdout:9/880: creat d17/d21/d29/de5/d4b/f131 x:0 0 0 2024-04-06T03:40:46.120 INFO:tasks.workunit.client.1.smithi136.stdout:9/881: fsync d17/d21/d32/fc7 0 2024-04-06T03:40:46.121 INFO:tasks.workunit.client.0.smithi094.stdout:9/741: mknod d2/d2c/d33/d64/dc6/c10b 0 2024-04-06T03:40:46.124 INFO:tasks.workunit.client.0.smithi094.stdout:0/771: mkdir d1/d9/dd/d18/d78/d109 0 2024-04-06T03:40:46.124 INFO:tasks.workunit.client.1.smithi136.stdout:0/826: dwrite d1/d6/d2c/d57/d43/f50 [0,4194304] 0 2024-04-06T03:40:46.126 INFO:tasks.workunit.client.1.smithi136.stdout:1/862: unlink d12/d3b/d74/d94/l10a 0 2024-04-06T03:40:46.126 INFO:tasks.workunit.client.1.smithi136.stdout:1/863: chown d12/d1b 784 1 2024-04-06T03:40:46.126 INFO:tasks.workunit.client.0.smithi094.stdout:8/807: dread fd [0,4194304] 0 2024-04-06T03:40:46.126 INFO:tasks.workunit.client.0.smithi094.stdout:8/808: dread - d18/d2c/ffc zero size 2024-04-06T03:40:46.127 INFO:tasks.workunit.client.0.smithi094.stdout:1/825: mknod d0/d1/d27/da5/d40/db1/d2f/d4b/c11c 0 2024-04-06T03:40:46.127 INFO:tasks.workunit.client.0.smithi094.stdout:1/826: fsync d0/d7/d1a/fd1 0 2024-04-06T03:40:46.127 INFO:tasks.workunit.client.0.smithi094.stdout:1/827: write d0/d1/d27/da5/d40/db1/d2f/d4b/f8a [4966757,88230] 0 2024-04-06T03:40:46.127 INFO:tasks.workunit.client.0.smithi094.stdout:1/828: write d0/d7/d1a/da8/fbb [2179060,47367] 0 2024-04-06T03:40:46.128 INFO:tasks.workunit.client.0.smithi094.stdout:8/809: truncate d18/d30/f40 2503325 0 2024-04-06T03:40:46.129 INFO:tasks.workunit.client.0.smithi094.stdout:9/742: write d2/dc/d21/d26/d78/d69/da8/d96/fa0 [5913921,37362] 0 2024-04-06T03:40:46.129 INFO:tasks.workunit.client.0.smithi094.stdout:9/743: chown d2/dc/d21/d23/d5b/de2/cca 39 1 2024-04-06T03:40:46.131 INFO:tasks.workunit.client.0.smithi094.stdout:9/744: dread d2/dc/d21/d23/f7c [0,4194304] 0 2024-04-06T03:40:46.131 INFO:tasks.workunit.client.0.smithi094.stdout:9/745: chown d2/dc/d21/d23/d47/lc2 2 1 2024-04-06T03:40:46.131 INFO:tasks.workunit.client.1.smithi136.stdout:8/904: dread d3/d2d/d5f/ff0 [0,4194304] 0 2024-04-06T03:40:46.135 INFO:tasks.workunit.client.0.smithi094.stdout:0/772: write d1/d9/dd/d18/d1e/d22/f6a [3033702,42420] 0 2024-04-06T03:40:46.136 INFO:tasks.workunit.client.1.smithi136.stdout:3/818: sync 2024-04-06T03:40:46.136 INFO:tasks.workunit.client.0.smithi094.stdout:8/810: truncate f17 3559617 0 2024-04-06T03:40:46.136 INFO:tasks.workunit.client.0.smithi094.stdout:8/811: readlink d18/d2c/l105 0 2024-04-06T03:40:46.137 INFO:tasks.workunit.client.0.smithi094.stdout:9/746: symlink d2/dc/d80/ddd/d6b/d103/l10c 0 2024-04-06T03:40:46.139 INFO:tasks.workunit.client.0.smithi094.stdout:8/812: unlink d18/d2c/cab 0 2024-04-06T03:40:46.140 INFO:tasks.workunit.client.0.smithi094.stdout:7/704: dwrite d2/d6/f24 [0,4194304] 0 2024-04-06T03:40:46.140 INFO:tasks.workunit.client.1.smithi136.stdout:0/827: mknod d1/d6/d2c/d57/d43/dcf/de1/c11b 0 2024-04-06T03:40:46.141 INFO:tasks.workunit.client.1.smithi136.stdout:2/896: sync 2024-04-06T03:40:46.142 INFO:tasks.workunit.client.1.smithi136.stdout:1/864: symlink d12/d53/dc5/l121 0 2024-04-06T03:40:46.143 INFO:tasks.workunit.client.1.smithi136.stdout:0/828: dread d1/f18 [0,4194304] 0 2024-04-06T03:40:46.143 INFO:tasks.workunit.client.1.smithi136.stdout:8/905: creat d3/de/d60/d118/f120 x:0 0 0 2024-04-06T03:40:46.143 INFO:tasks.workunit.client.1.smithi136.stdout:8/906: getdents d3/da/d26/d90/dae/db5/d11c 0 2024-04-06T03:40:46.143 INFO:tasks.workunit.client.1.smithi136.stdout:8/907: truncate d3/f5e 2163766 0 2024-04-06T03:40:46.144 INFO:tasks.workunit.client.0.smithi094.stdout:0/773: write d1/d9/fc [8307037,105196] 0 2024-04-06T03:40:46.145 INFO:tasks.workunit.client.1.smithi136.stdout:5/914: rename d1/d9b/d9c/da9 to d1/dc/d1c/d32/d119 0 2024-04-06T03:40:46.145 INFO:tasks.workunit.client.0.smithi094.stdout:7/705: mknod d2/d6/dc/d12/ce4 0 2024-04-06T03:40:46.147 INFO:tasks.workunit.client.1.smithi136.stdout:3/819: mkdir d0/d4/d12/d19/d55/d3c/d53/de6/d118 0 2024-04-06T03:40:46.148 INFO:tasks.workunit.client.0.smithi094.stdout:8/813: link d18/d3c/d76/f5b d18/f116 0 2024-04-06T03:40:46.148 INFO:tasks.workunit.client.0.smithi094.stdout:8/814: creat d18/d3c/d8b/f117 x:0 0 0 2024-04-06T03:40:46.151 INFO:tasks.workunit.client.1.smithi136.stdout:1/865: truncate d12/d3b/f84 2627016 0 2024-04-06T03:40:46.153 INFO:tasks.workunit.client.1.smithi136.stdout:0/829: creat d1/d6/d2c/d57/d43/d6d/d85/da7/f11c x:0 0 0 2024-04-06T03:40:46.154 INFO:tasks.workunit.client.0.smithi094.stdout:8/815: link d18/d3c/d49/d91/ff5 d18/d2c/dda/f118 0 2024-04-06T03:40:46.156 INFO:tasks.workunit.client.1.smithi136.stdout:9/882: rename d17/d20/d25/l2b to d17/d21/d29/de5/da5/db8/de0/l132 0 2024-04-06T03:40:46.156 INFO:tasks.workunit.client.1.smithi136.stdout:9/883: write d17/d21/d32/fc7 [191275,64685] 0 2024-04-06T03:40:46.158 INFO:tasks.workunit.client.1.smithi136.stdout:3/820: unlink d0/d4/d12/d3a/d51/da6/l108 0 2024-04-06T03:40:46.160 INFO:tasks.workunit.client.1.smithi136.stdout:1/866: truncate d12/d3b/d74/fba 2810842 0 2024-04-06T03:40:46.161 INFO:tasks.workunit.client.1.smithi136.stdout:0/830: creat d1/d6/d2c/d57/d43/d6d/d85/da7/de5/f11d x:0 0 0 2024-04-06T03:40:46.161 INFO:tasks.workunit.client.1.smithi136.stdout:9/884: rmdir d17/d21/d29 39 2024-04-06T03:40:46.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/897: read d3/d15/d19/d43/d54/f8c [3202823,32023] 0 2024-04-06T03:40:46.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/898: chown d3/d4a/d77/fcd 239832 1 2024-04-06T03:40:46.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/899: fsync d3/d15/d19/d43/d54/d10b/f118 0 2024-04-06T03:40:46.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/900: fdatasync d3/d9/d3b/d70/dc8/f115 0 2024-04-06T03:40:46.164 INFO:tasks.workunit.client.1.smithi136.stdout:0/831: mkdir d1/d6/d2c/d57/d43/d6d/d85/da7/de5/dec/d11e 0 2024-04-06T03:40:46.164 INFO:tasks.workunit.client.0.smithi094.stdout:7/706: dwrite d2/d6/dc/dd/df/f66 [0,4194304] 0 2024-04-06T03:40:46.164 INFO:tasks.workunit.client.0.smithi094.stdout:7/707: fsync d2/d6/dc/dd/df/d95/dc9/f3d 0 2024-04-06T03:40:46.165 INFO:tasks.workunit.client.1.smithi136.stdout:9/885: creat d17/d20/d28/d49/f133 x:0 0 0 2024-04-06T03:40:46.165 INFO:tasks.workunit.client.1.smithi136.stdout:2/901: getdents d3/d4a/d9d 0 2024-04-06T03:40:46.167 INFO:tasks.workunit.client.1.smithi136.stdout:0/832: creat d1/d6/d2c/d57/d43/d9e/d10d/f11f x:0 0 0 2024-04-06T03:40:46.167 INFO:tasks.workunit.client.1.smithi136.stdout:0/833: readlink d1/d6/d2c/l79 0 2024-04-06T03:40:46.167 INFO:tasks.workunit.client.1.smithi136.stdout:0/834: readlink d1/d6/d2c/d57/d43/lcc 0 2024-04-06T03:40:46.167 INFO:tasks.workunit.client.1.smithi136.stdout:0/835: write d1/d6/d2c/d1a/d36/d59/f31 [2856071,129613] 0 2024-04-06T03:40:46.168 INFO:tasks.workunit.client.1.smithi136.stdout:2/902: dread d3/d4a/d9d/fb6 [0,4194304] 0 2024-04-06T03:40:46.168 INFO:tasks.workunit.client.1.smithi136.stdout:2/903: chown d3/d15 1 1 2024-04-06T03:40:46.169 INFO:tasks.workunit.client.1.smithi136.stdout:9/886: creat d17/d19/da3/dc5/deb/d9e/f134 x:0 0 0 2024-04-06T03:40:46.172 INFO:tasks.workunit.client.0.smithi094.stdout:7/708: rename d2/d6/dc/dd/df/d95/dc9/d57/ca1 to d2/d6/dc/dd/df/ce5 0 2024-04-06T03:40:46.172 INFO:tasks.workunit.client.0.smithi094.stdout:7/709: readlink d2/d6/dc/dd/d90/ldb 0 2024-04-06T03:40:46.172 INFO:tasks.workunit.client.0.smithi094.stdout:7/710: truncate d2/d6/dc/dd/df/fb8 1018010 0 2024-04-06T03:40:46.172 INFO:tasks.workunit.client.0.smithi094.stdout:7/711: creat d2/d6/dc/dd/df/d36/fe6 x:0 0 0 2024-04-06T03:40:46.173 INFO:tasks.workunit.client.0.smithi094.stdout:8/816: dread d18/d2c/f2e [0,4194304] 0 2024-04-06T03:40:46.175 INFO:tasks.workunit.client.0.smithi094.stdout:8/817: mkdir d18/d2c/d119 0 2024-04-06T03:40:46.177 INFO:tasks.workunit.client.1.smithi136.stdout:2/904: read d3/d4a/d9d/d8a/d93/d5f/f78 [3768647,65630] 0 2024-04-06T03:40:46.177 INFO:tasks.workunit.client.1.smithi136.stdout:2/905: dread - d3/ddc/f113 zero size 2024-04-06T03:40:46.177 INFO:tasks.workunit.client.1.smithi136.stdout:2/906: write d3/d9/d3b/d87/db1/f11a [438724,13249] 0 2024-04-06T03:40:46.177 INFO:tasks.workunit.client.1.smithi136.stdout:2/907: write d3/ddc/f113 [545968,107708] 0 2024-04-06T03:40:46.177 INFO:tasks.workunit.client.1.smithi136.stdout:2/908: dread - d3/d15/d19/d43/daa/dec/fee zero size 2024-04-06T03:40:46.178 INFO:tasks.workunit.client.0.smithi094.stdout:8/818: rmdir d18/d3c/d49/d91/d94/daa/ddc 39 2024-04-06T03:40:46.192 INFO:tasks.workunit.client.0.smithi094.stdout:8/819: write d18/d2c/dda/f118 [1567955,88105] 0 2024-04-06T03:40:46.193 INFO:tasks.workunit.client.0.smithi094.stdout:8/820: mkdir d18/d3c/d11a 0 2024-04-06T03:40:46.193 INFO:tasks.workunit.client.0.smithi094.stdout:8/821: getdents d18/d3c/d49/d91/d114 0 2024-04-06T03:40:46.193 INFO:tasks.workunit.client.0.smithi094.stdout:8/822: truncate d18/d2c/ffc 790921 0 2024-04-06T03:40:46.193 INFO:tasks.workunit.client.0.smithi094.stdout:8/823: readlink d18/d2c/d37/d60/da3/db1/lc6 0 2024-04-06T03:40:46.195 INFO:tasks.workunit.client.1.smithi136.stdout:0/836: dwrite d1/fc9 [4194304,4194304] 0 2024-04-06T03:40:46.197 INFO:tasks.workunit.client.1.smithi136.stdout:2/909: dwrite d3/d15/d19/d4e/fae [0,4194304] 0 2024-04-06T03:40:46.198 INFO:tasks.workunit.client.1.smithi136.stdout:0/837: mknod d1/d6/d2c/d57/d43/d6d/d9b/d116/c120 0 2024-04-06T03:40:46.198 INFO:tasks.workunit.client.1.smithi136.stdout:0/838: write d1/d6/d2c/d57/d43/d6d/d85/f7f [629718,5117] 0 2024-04-06T03:40:46.198 INFO:tasks.workunit.client.0.smithi094.stdout:7/712: dwrite d2/d6/dc/d12/d9a/da0/dd4/f9c [0,4194304] 0 2024-04-06T03:40:46.198 INFO:tasks.workunit.client.0.smithi094.stdout:7/713: write d2/d6/dc/dd/df/d95/dc9/f8d [1244033,14721] 0 2024-04-06T03:40:46.203 INFO:tasks.workunit.client.0.smithi094.stdout:2/748: dwrite d0/d8/da0/fd9 [0,4194304] 0 2024-04-06T03:40:46.204 INFO:tasks.workunit.client.0.smithi094.stdout:7/714: mknod d2/d6/dc/dd/d90/ce7 0 2024-04-06T03:40:46.213 INFO:tasks.workunit.client.0.smithi094.stdout:2/749: rmdir d0/d8/d27/d2b/d8e/dfb 0 2024-04-06T03:40:46.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/786: dwrite db/d19/dc1/dd4/d77/fbb [0,4194304] 0 2024-04-06T03:40:46.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/787: truncate db/d7c/d98/f9f 222510 0 2024-04-06T03:40:46.217 INFO:tasks.workunit.client.0.smithi094.stdout:5/788: dread - db/d19/d30/d83/fee zero size 2024-04-06T03:40:46.219 INFO:tasks.workunit.client.0.smithi094.stdout:2/750: unlink d0/dd/d1b/c51 0 2024-04-06T03:40:46.219 INFO:tasks.workunit.client.0.smithi094.stdout:7/715: rename d2/d6/dc/dd/df/d36/fca to d2/d6/dc/d12/d3a/fe8 0 2024-04-06T03:40:46.225 INFO:tasks.workunit.client.0.smithi094.stdout:2/751: unlink d0/dd/d19/d31/f42 0 2024-04-06T03:40:46.227 INFO:tasks.workunit.client.0.smithi094.stdout:2/752: creat d0/d8/da0/ded/f101 x:0 0 0 2024-04-06T03:40:46.230 INFO:tasks.workunit.client.0.smithi094.stdout:5/789: write db/d19/d24/f89 [1311077,16024] 0 2024-04-06T03:40:46.230 INFO:tasks.workunit.client.0.smithi094.stdout:8/824: dwrite d18/d3c/dc4/dbc/f93 [0,4194304] 0 2024-04-06T03:40:46.230 INFO:tasks.workunit.client.0.smithi094.stdout:1/829: write d0/d7/f26 [4828021,99286] 0 2024-04-06T03:40:46.231 INFO:tasks.workunit.client.0.smithi094.stdout:6/742: dwrite dc/d3a/df4/df6/df2/f108 [0,4194304] 0 2024-04-06T03:40:46.231 INFO:tasks.workunit.client.0.smithi094.stdout:9/747: fsync d2/ff7 0 2024-04-06T03:40:46.231 INFO:tasks.workunit.client.0.smithi094.stdout:9/748: creat d2/dc/d21/d23/f10d x:0 0 0 2024-04-06T03:40:46.231 INFO:tasks.workunit.client.0.smithi094.stdout:9/749: write d2/dc/d21/fb5 [738828,84646] 0 2024-04-06T03:40:46.232 INFO:tasks.workunit.client.1.smithi136.stdout:8/908: dwrite d3/de/d18/f74 [0,4194304] 0 2024-04-06T03:40:46.235 INFO:tasks.workunit.client.0.smithi094.stdout:5/790: mknod db/d80/c114 0 2024-04-06T03:40:46.235 INFO:tasks.workunit.client.1.smithi136.stdout:3/821: dwrite d0/d4/d12/d19/d55/d3c/faa [0,4194304] 0 2024-04-06T03:40:46.237 INFO:tasks.workunit.client.1.smithi136.stdout:1/867: dwrite d12/d3b/d44/d5f/db7/fdd [0,4194304] 0 2024-04-06T03:40:46.240 INFO:tasks.workunit.client.0.smithi094.stdout:0/774: dwrite d1/d9/dd/d18/d5d/de9/f100 [0,4194304] 0 2024-04-06T03:40:46.243 INFO:tasks.workunit.client.0.smithi094.stdout:7/716: dwrite d2/f11 [0,4194304] 0 2024-04-06T03:40:46.243 INFO:tasks.workunit.client.1.smithi136.stdout:9/887: dwrite d17/d20/d28/d49/d75/d60/f110 [0,4194304] 0 2024-04-06T03:40:46.243 INFO:tasks.workunit.client.1.smithi136.stdout:0/839: dwrite d1/d6/d2c/d57/d43/d6d/d9b/d5c/f91 [0,4194304] 0 2024-04-06T03:40:46.244 INFO:tasks.workunit.client.1.smithi136.stdout:9/888: write d17/d19/da3/dc5/fd3 [90979,31403] 0 2024-04-06T03:40:46.244 INFO:tasks.workunit.client.1.smithi136.stdout:9/889: write d17/d21/d29/de5/da5/dce/dd1/fda [62776,80728] 0 2024-04-06T03:40:46.245 INFO:tasks.workunit.client.0.smithi094.stdout:8/825: creat d18/d3c/d76/f11b x:0 0 0 2024-04-06T03:40:46.248 INFO:tasks.workunit.client.1.smithi136.stdout:8/909: getdents d3/da/da0 0 2024-04-06T03:40:46.248 INFO:tasks.workunit.client.1.smithi136.stdout:8/910: chown d3/de/d18/d7d/ddf/d6c/d7f/fbf 12392 1 2024-04-06T03:40:46.248 INFO:tasks.workunit.client.1.smithi136.stdout:8/911: truncate d3/de/d18/d7d/ddf/d6c/f10b 334036 0 2024-04-06T03:40:46.248 INFO:tasks.workunit.client.1.smithi136.stdout:8/912: chown d3/f77 3 1 2024-04-06T03:40:46.251 INFO:tasks.workunit.client.1.smithi136.stdout:3/822: read d0/d4/d12/d19/d115/fe8 [749774,23863] 0 2024-04-06T03:40:46.251 INFO:tasks.workunit.client.0.smithi094.stdout:1/830: rename d0/d1/d27/da5/d40/db1/dc3/d5f/d70/fe8 to d0/d1/d27/da5/d40/db1/d2f/d76/f11d 0 2024-04-06T03:40:46.251 INFO:tasks.workunit.client.0.smithi094.stdout:1/831: fdatasync d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/f9c 0 2024-04-06T03:40:46.253 INFO:tasks.workunit.client.1.smithi136.stdout:1/868: dwrite d12/d3b/d74/dd6/f100 [0,4194304] 0 2024-04-06T03:40:46.259 INFO:tasks.workunit.client.1.smithi136.stdout:4/747: rmdir d1d/d31/d3e/d55/d7f 39 2024-04-06T03:40:46.260 INFO:tasks.workunit.client.1.smithi136.stdout:3/823: dread d0/d4/d12/d3a/f4b [0,4194304] 0 2024-04-06T03:40:46.260 INFO:tasks.workunit.client.1.smithi136.stdout:3/824: stat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97 0 2024-04-06T03:40:46.261 INFO:tasks.workunit.client.1.smithi136.stdout:3/825: chown d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/df2/f10b 109735705 1 2024-04-06T03:40:46.261 INFO:tasks.workunit.client.0.smithi094.stdout:8/826: write d18/d3c/dc4/dbc/fc1 [1757576,82839] 0 2024-04-06T03:40:46.261 INFO:tasks.workunit.client.0.smithi094.stdout:7/717: dwrite d2/d6/dc/d12/d9a/dbd/fc4 [0,4194304] 0 2024-04-06T03:40:46.262 INFO:tasks.workunit.client.1.smithi136.stdout:7/941: sync 2024-04-06T03:40:46.270 INFO:tasks.workunit.client.1.smithi136.stdout:7/942: dread d5/f2e [0,4194304] 0 2024-04-06T03:40:46.270 INFO:tasks.workunit.client.1.smithi136.stdout:7/943: write d5/df/d1d/d6f/d4d/d69/f134 [749487,14447] 0 2024-04-06T03:40:46.270 INFO:tasks.workunit.client.1.smithi136.stdout:1/869: dwrite d12/d3b/d74/d7a/f8d [0,4194304] 0 2024-04-06T03:40:46.271 INFO:tasks.workunit.client.0.smithi094.stdout:5/791: truncate db/d19/dc1/dd4/d77/f82 2169324 0 2024-04-06T03:40:46.271 INFO:tasks.workunit.client.0.smithi094.stdout:5/792: creat db/d19/d2c/d6d/f115 x:0 0 0 2024-04-06T03:40:46.273 INFO:tasks.workunit.client.1.smithi136.stdout:0/840: link d1/d6/d2c/d1a/f4b d1/d6/d2c/d57/d43/d6d/f121 0 2024-04-06T03:40:46.273 INFO:tasks.workunit.client.1.smithi136.stdout:0/841: readlink d1/d6/d2c/d1a/d36/d59/l24 0 2024-04-06T03:40:46.276 INFO:tasks.workunit.client.0.smithi094.stdout:0/775: truncate d1/d9/dd/d18/d1e/d86/fc5 2107559 0 2024-04-06T03:40:46.276 INFO:tasks.workunit.client.0.smithi094.stdout:0/776: dread - d1/d9/dd/d18/d78/fea zero size 2024-04-06T03:40:46.277 INFO:tasks.workunit.client.1.smithi136.stdout:9/890: mknod d17/d21/d32/dc6/d113/d66/de1/d51/d116/c135 0 2024-04-06T03:40:46.277 INFO:tasks.workunit.client.1.smithi136.stdout:9/891: write d17/d20/d28/f9c [374140,115744] 0 2024-04-06T03:40:46.277 INFO:tasks.workunit.client.1.smithi136.stdout:9/892: stat d17/d20/d25/da4/dbb 0 2024-04-06T03:40:46.279 INFO:tasks.workunit.client.1.smithi136.stdout:7/944: write d5/df/d53/d68/ffb [1047670,88458] 0 2024-04-06T03:40:46.279 INFO:tasks.workunit.client.1.smithi136.stdout:7/945: chown d5/df/d3d/f3f 781853 1 2024-04-06T03:40:46.280 INFO:tasks.workunit.client.1.smithi136.stdout:8/913: rename d3/da/d26/d35/d53/dd2/f111 to d3/da/d26/d35/d53/d86/db0/f121 0 2024-04-06T03:40:46.284 INFO:tasks.workunit.client.1.smithi136.stdout:5/915: getdents d1/d9b/d9c 0 2024-04-06T03:40:46.286 INFO:tasks.workunit.client.1.smithi136.stdout:4/748: symlink d1d/dd0/l108 0 2024-04-06T03:40:46.289 INFO:tasks.workunit.client.1.smithi136.stdout:3/826: rmdir d0/d4/d12/d19/d2e/d3b/d49 39 2024-04-06T03:40:46.293 INFO:tasks.workunit.client.1.smithi136.stdout:4/749: write d1d/d31/d3e/d56/f81 [2006083,107895] 0 2024-04-06T03:40:46.293 INFO:tasks.workunit.client.1.smithi136.stdout:4/750: stat d1d/d31/d3e/d55/d7f/f83 0 2024-04-06T03:40:46.294 INFO:tasks.workunit.client.1.smithi136.stdout:5/916: dread d1/d16/d4b/f4f [0,4194304] 0 2024-04-06T03:40:46.294 INFO:tasks.workunit.client.1.smithi136.stdout:5/917: stat d1/dc/d10 0 2024-04-06T03:40:46.294 INFO:tasks.workunit.client.1.smithi136.stdout:5/918: write d1/d16/d89/d8a/f104 [368647,38329] 0 2024-04-06T03:40:46.294 INFO:tasks.workunit.client.1.smithi136.stdout:5/919: fsync d1/ffb 0 2024-04-06T03:40:46.294 INFO:tasks.workunit.client.1.smithi136.stdout:8/914: dwrite d3/de/d18/faf [0,4194304] 0 2024-04-06T03:40:46.295 INFO:tasks.workunit.client.1.smithi136.stdout:1/870: mknod d12/d3b/d60/d7e/dbe/c122 0 2024-04-06T03:40:46.299 INFO:tasks.workunit.client.1.smithi136.stdout:0/842: stat d1/d6/d2c/d1a/cee 0 2024-04-06T03:40:46.304 INFO:tasks.workunit.client.1.smithi136.stdout:9/893: creat d17/d21/d29/de5/da5/db8/f136 x:0 0 0 2024-04-06T03:40:46.306 INFO:tasks.workunit.client.1.smithi136.stdout:7/946: mknod d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/d71/c140 0 2024-04-06T03:40:46.309 INFO:tasks.workunit.client.1.smithi136.stdout:4/751: dwrite d1d/d2f/da6/ffa [0,4194304] 0 2024-04-06T03:40:46.314 INFO:tasks.workunit.client.0.smithi094.stdout:9/750: rename d2/dc/d80/ddd/l6f to d2/dc/d80/db7/l10e 0 2024-04-06T03:40:46.318 INFO:tasks.workunit.client.1.smithi136.stdout:5/920: stat d1/dc/d1c/d32/d8b/l2d 0 2024-04-06T03:40:46.319 INFO:tasks.workunit.client.0.smithi094.stdout:9/751: dread d2/dc/f68 [0,4194304] 0 2024-04-06T03:40:46.319 INFO:tasks.workunit.client.0.smithi094.stdout:9/752: stat d2/ffc 0 2024-04-06T03:40:46.319 INFO:tasks.workunit.client.0.smithi094.stdout:6/743: getdents dc/dd3/d72/d88/da3/dc4 0 2024-04-06T03:40:46.319 INFO:tasks.workunit.client.0.smithi094.stdout:6/744: readlink dc/dab/lc9 0 2024-04-06T03:40:46.327 INFO:tasks.workunit.client.1.smithi136.stdout:7/947: dwrite d5/fd [0,4194304] 0 2024-04-06T03:40:46.327 INFO:tasks.workunit.client.1.smithi136.stdout:7/948: stat d5/df/d53/d68/la0 0 2024-04-06T03:40:46.327 INFO:tasks.workunit.client.0.smithi094.stdout:8/827: unlink d18/d2c/f9b 0 2024-04-06T03:40:46.328 INFO:tasks.workunit.client.0.smithi094.stdout:8/828: chown lc 3297092 1 2024-04-06T03:40:46.329 INFO:tasks.workunit.client.1.smithi136.stdout:1/871: symlink d12/d3b/d74/d7a/dea/d11a/d9f/l123 0 2024-04-06T03:40:46.330 INFO:tasks.workunit.client.0.smithi094.stdout:9/753: dread d2/d2c/fa5 [0,4194304] 0 2024-04-06T03:40:46.330 INFO:tasks.workunit.client.0.smithi094.stdout:9/754: chown d2/dc/d21/d23/d2f/f91 2062380722 1 2024-04-06T03:40:46.330 INFO:tasks.workunit.client.0.smithi094.stdout:9/755: write d2/dc/d21/fb5 [1230733,118349] 0 2024-04-06T03:40:46.331 INFO:tasks.workunit.client.0.smithi094.stdout:9/756: fdatasync d2/dc/d10/fb2 0 2024-04-06T03:40:46.331 INFO:tasks.workunit.client.0.smithi094.stdout:9/757: chown d2/dc/d10/d6a/l88 1622 1 2024-04-06T03:40:46.331 INFO:tasks.workunit.client.0.smithi094.stdout:7/718: mknod d2/d6/dac/ddd/ce9 0 2024-04-06T03:40:46.334 INFO:tasks.workunit.client.1.smithi136.stdout:0/843: rename d1/d6/d2c/d1a/cca to d1/d6/d2c/d57/d43/d9e/c122 0 2024-04-06T03:40:46.337 INFO:tasks.workunit.client.0.smithi094.stdout:5/793: symlink db/d19/d21/l116 0 2024-04-06T03:40:46.337 INFO:tasks.workunit.client.0.smithi094.stdout:5/794: readlink db/d19/d24/d2e/l18 0 2024-04-06T03:40:46.338 INFO:tasks.workunit.client.0.smithi094.stdout:2/753: rmdir d0/d8/da0/ded 39 2024-04-06T03:40:46.339 INFO:tasks.workunit.client.0.smithi094.stdout:0/777: truncate d1/d9/dd/d18/f31 722108 0 2024-04-06T03:40:46.342 INFO:tasks.workunit.client.0.smithi094.stdout:1/832: getdents d0/d1/d27/da5/d40/db1/d2f 0 2024-04-06T03:40:46.342 INFO:tasks.workunit.client.0.smithi094.stdout:1/833: creat d0/d1/d27/da5/d40/f11e x:0 0 0 2024-04-06T03:40:46.343 INFO:tasks.workunit.client.0.smithi094.stdout:1/834: fsync d0/d1/f112 0 2024-04-06T03:40:46.343 INFO:tasks.workunit.client.0.smithi094.stdout:1/835: chown d0/d1/l15 3 1 2024-04-06T03:40:46.343 INFO:tasks.workunit.client.1.smithi136.stdout:4/752: mkdir d1d/d2f/d6c/da5/dc1/d109 0 2024-04-06T03:40:46.344 INFO:tasks.workunit.client.1.smithi136.stdout:4/753: write d1d/d31/d3e/d55/fb8 [609225,114638] 0 2024-04-06T03:40:46.344 INFO:tasks.workunit.client.1.smithi136.stdout:4/754: fdatasync d1d/fbb 0 2024-04-06T03:40:46.346 INFO:tasks.workunit.client.1.smithi136.stdout:5/921: creat d1/dc/d1c/d32/d8b/d20/d7c/f11a x:0 0 0 2024-04-06T03:40:46.346 INFO:tasks.workunit.client.1.smithi136.stdout:5/922: write d1/d16/ffe [467055,106002] 0 2024-04-06T03:40:46.346 INFO:tasks.workunit.client.1.smithi136.stdout:5/923: fsync d1/d9b/ff8 0 2024-04-06T03:40:46.347 INFO:tasks.workunit.client.0.smithi094.stdout:8/829: rename d18/d3c/d49/d9f/fb3 to d18/d2c/d37/d115/f11c 0 2024-04-06T03:40:46.349 INFO:tasks.workunit.client.1.smithi136.stdout:7/949: unlink d5/df/d53/d68/da3/cad 0 2024-04-06T03:40:46.351 INFO:tasks.workunit.client.0.smithi094.stdout:9/758: creat d2/dc/d21/d26/d78/d69/f10f x:0 0 0 2024-04-06T03:40:46.351 INFO:tasks.workunit.client.0.smithi094.stdout:9/759: dread - d2/dc/d21/d23/f10d zero size 2024-04-06T03:40:46.352 INFO:tasks.workunit.client.1.smithi136.stdout:1/872: mknod d12/d1b/d28/c124 0 2024-04-06T03:40:46.352 INFO:tasks.workunit.client.1.smithi136.stdout:1/873: chown d12/d3b/d74/d7a/dea 290 1 2024-04-06T03:40:46.354 INFO:tasks.workunit.client.1.smithi136.stdout:0/844: truncate d1/d6/d2c/d57/d43/d6d/f121 7583806 0 2024-04-06T03:40:46.356 INFO:tasks.workunit.client.1.smithi136.stdout:4/755: dwrite d1d/d2f/d6c/f98 [0,4194304] 0 2024-04-06T03:40:46.358 INFO:tasks.workunit.client.1.smithi136.stdout:6/764: sync 2024-04-06T03:40:46.359 INFO:tasks.workunit.client.0.smithi094.stdout:2/754: chown d0/dd/d1b/l6d 31654218 1 2024-04-06T03:40:46.360 INFO:tasks.workunit.client.1.smithi136.stdout:1/874: rmdir d12/d1b/d28 39 2024-04-06T03:40:46.360 INFO:tasks.workunit.client.1.smithi136.stdout:1/875: write d12/d3b/d44/d5f/f30 [4493601,74776] 0 2024-04-06T03:40:46.363 INFO:tasks.workunit.client.0.smithi094.stdout:0/778: getdents d1/d9/dd/d18/d1e/def/dab/d4e 0 2024-04-06T03:40:46.364 INFO:tasks.workunit.client.1.smithi136.stdout:0/845: mknod d1/d6/d2c/d57/d43/d6d/d85/d11a/c123 0 2024-04-06T03:40:46.364 INFO:tasks.workunit.client.1.smithi136.stdout:0/846: write d1/d6/d2c/d57/d43/d6d/d85/da7/f11c [243122,22104] 0 2024-04-06T03:40:46.364 INFO:tasks.workunit.client.1.smithi136.stdout:0/847: creat d1/d6/d2c/d57/d43/d6d/d85/d78/da4/f124 x:0 0 0 2024-04-06T03:40:46.366 INFO:tasks.workunit.client.1.smithi136.stdout:4/756: dread d1d/d2f/d6c/f98 [0,4194304] 0 2024-04-06T03:40:46.366 INFO:tasks.workunit.client.1.smithi136.stdout:4/757: readlink d1d/d2f/da6/dbf/ld3 0 2024-04-06T03:40:46.366 INFO:tasks.workunit.client.1.smithi136.stdout:4/758: write d1d/d31/d3e/d56/f81 [2914546,36113] 0 2024-04-06T03:40:46.366 INFO:tasks.workunit.client.1.smithi136.stdout:7/950: dwrite d5/df/d12/d1b/d58/f7a [0,4194304] 0 2024-04-06T03:40:46.367 INFO:tasks.workunit.client.0.smithi094.stdout:9/760: dwrite d2/dc/ffe [0,4194304] 0 2024-04-06T03:40:46.367 INFO:tasks.workunit.client.1.smithi136.stdout:0/848: dread d1/d6/d2c/d57/d43/d6b/f80 [0,4194304] 0 2024-04-06T03:40:46.367 INFO:tasks.workunit.client.0.smithi094.stdout:1/836: mkdir d0/d7/d2d/d11f 0 2024-04-06T03:40:46.369 INFO:tasks.workunit.client.0.smithi094.stdout:9/761: dread d2/dc/d10/ff4 [0,4194304] 0 2024-04-06T03:40:46.369 INFO:tasks.workunit.client.0.smithi094.stdout:9/762: fsync d2/dc/d80/ddd/fa7 0 2024-04-06T03:40:46.372 INFO:tasks.workunit.client.1.smithi136.stdout:1/876: symlink d12/d1b/d120/da2/l125 0 2024-04-06T03:40:46.372 INFO:tasks.workunit.client.1.smithi136.stdout:6/765: write d2/d7/d26/d30/d6e/f5e [1520277,36878] 0 2024-04-06T03:40:46.372 INFO:tasks.workunit.client.1.smithi136.stdout:1/877: chown d12/d3b/d74/d7a/dea/d77/fc1 579 1 2024-04-06T03:40:46.376 INFO:tasks.workunit.client.1.smithi136.stdout:7/951: write f1 [2070774,56787] 0 2024-04-06T03:40:46.379 INFO:tasks.workunit.client.0.smithi094.stdout:1/837: read d0/d1/d27/da5/d40/db1/d2f/f54 [7314133,112786] 0 2024-04-06T03:40:46.379 INFO:tasks.workunit.client.0.smithi094.stdout:2/755: dwrite d0/dd/d19/d31/d67/d98/fb7 [0,4194304] 0 2024-04-06T03:40:46.383 INFO:tasks.workunit.client.1.smithi136.stdout:6/766: dread d2/d7/d26/d30/fd4 [0,4194304] 0 2024-04-06T03:40:46.383 INFO:tasks.workunit.client.1.smithi136.stdout:4/759: mkdir d1d/d2f/da6/de1/d10a 0 2024-04-06T03:40:46.385 INFO:tasks.workunit.client.1.smithi136.stdout:0/849: creat d1/d6/d2c/d57/d43/d6d/d85/d11a/f125 x:0 0 0 2024-04-06T03:40:46.386 INFO:tasks.workunit.client.1.smithi136.stdout:7/952: read d5/df/d11/fe6 [3315096,55998] 0 2024-04-06T03:40:46.387 INFO:tasks.workunit.client.1.smithi136.stdout:1/878: mknod d12/d3b/d44/d5f/c126 0 2024-04-06T03:40:46.387 INFO:tasks.workunit.client.1.smithi136.stdout:7/953: read d5/df/d12/d1b/d3a/d4a/f86 [605178,16728] 0 2024-04-06T03:40:46.393 INFO:tasks.workunit.client.0.smithi094.stdout:6/745: rmdir dc 39 2024-04-06T03:40:46.393 INFO:tasks.workunit.client.0.smithi094.stdout:6/746: dread - dc/d3a/d4a/f92 zero size 2024-04-06T03:40:46.395 INFO:tasks.workunit.client.1.smithi136.stdout:6/767: link d2/d7/lc d2/d7/d26/d30/d6e/d58/d5d/d62/l103 0 2024-04-06T03:40:46.397 INFO:tasks.workunit.client.1.smithi136.stdout:0/850: truncate d1/f54 549901 0 2024-04-06T03:40:46.397 INFO:tasks.workunit.client.1.smithi136.stdout:0/851: write d1/d6/d2c/d1a/d36/fdb [754358,82763] 0 2024-04-06T03:40:46.397 INFO:tasks.workunit.client.1.smithi136.stdout:0/852: creat d1/d6/d2c/d57/d43/d6d/d85/da7/f126 x:0 0 0 2024-04-06T03:40:46.397 INFO:tasks.workunit.client.1.smithi136.stdout:0/853: dread - d1/d6/d2c/d1a/d36/d59/f38 zero size 2024-04-06T03:40:46.398 INFO:tasks.workunit.client.1.smithi136.stdout:1/879: creat d12/d3b/d60/d7e/d37/dbb/f127 x:0 0 0 2024-04-06T03:40:46.399 INFO:tasks.workunit.client.0.smithi094.stdout:6/747: dread dc/d3a/d4a/f56 [0,4194304] 0 2024-04-06T03:40:46.400 INFO:tasks.workunit.client.0.smithi094.stdout:6/748: fsync dc/d3a/d4a/d54/dde/d50/daf/fdd 0 2024-04-06T03:40:46.400 INFO:tasks.workunit.client.0.smithi094.stdout:5/795: rename db/d19/dc1/dd4/d77/c7b to db/d19/d21/dce/c117 0 2024-04-06T03:40:46.401 INFO:tasks.workunit.client.1.smithi136.stdout:7/954: creat d5/df/d12/d1b/dbb/dbe/f141 x:0 0 0 2024-04-06T03:40:46.402 INFO:tasks.workunit.client.0.smithi094.stdout:8/830: creat d18/d3c/d49/d91/d94/da8/f11d x:0 0 0 2024-04-06T03:40:46.404 INFO:tasks.workunit.client.0.smithi094.stdout:0/779: mknod d1/d9/dd/d18/d1e/def/d67/d76/c10a 0 2024-04-06T03:40:46.406 INFO:tasks.workunit.client.0.smithi094.stdout:0/780: dread d1/d9/f91 [0,4194304] 0 2024-04-06T03:40:46.408 INFO:tasks.workunit.client.1.smithi136.stdout:7/955: write d5/df/f62 [1690825,46060] 0 2024-04-06T03:40:46.409 INFO:tasks.workunit.client.1.smithi136.stdout:0/854: mkdir d1/d6/ded/d127 0 2024-04-06T03:40:46.411 INFO:tasks.workunit.client.0.smithi094.stdout:9/763: mknod d2/d2c/c110 0 2024-04-06T03:40:46.411 INFO:tasks.workunit.client.0.smithi094.stdout:8/831: dread d18/d34/f58 [0,4194304] 0 2024-04-06T03:40:46.412 INFO:tasks.workunit.client.1.smithi136.stdout:1/880: mkdir d12/d1b/da7/db8/d128 0 2024-04-06T03:40:46.412 INFO:tasks.workunit.client.1.smithi136.stdout:1/881: getdents d12/d3b/d60/d7e 0 2024-04-06T03:40:46.412 INFO:tasks.workunit.client.1.smithi136.stdout:1/882: write d12/d1b/d120/da2/fb1 [1435778,105254] 0 2024-04-06T03:40:46.414 INFO:tasks.workunit.client.1.smithi136.stdout:6/768: rename d2/d7/d26/d30/d4f to d2/d104 0 2024-04-06T03:40:46.415 INFO:tasks.workunit.client.1.smithi136.stdout:6/769: chown d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/d94/ce5 661263 1 2024-04-06T03:40:46.415 INFO:tasks.workunit.client.1.smithi136.stdout:6/770: fsync d2/d7/f5a 0 2024-04-06T03:40:46.415 INFO:tasks.workunit.client.0.smithi094.stdout:1/838: mknod d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/c120 0 2024-04-06T03:40:46.422 INFO:tasks.workunit.client.0.smithi094.stdout:2/756: mkdir d0/d8/d102 0 2024-04-06T03:40:46.422 INFO:tasks.workunit.client.0.smithi094.stdout:2/757: write d0/dd/d3d/fde [903769,24127] 0 2024-04-06T03:40:46.423 INFO:tasks.workunit.client.1.smithi136.stdout:7/956: creat d5/df/f142 x:0 0 0 2024-04-06T03:40:46.426 INFO:tasks.workunit.client.0.smithi094.stdout:6/749: creat dc/dd3/d72/d88/da3/dc4/f10a x:0 0 0 2024-04-06T03:40:46.428 INFO:tasks.workunit.client.1.smithi136.stdout:1/883: creat d12/d3b/d74/d7a/dea/d11a/dd1/f129 x:0 0 0 2024-04-06T03:40:46.430 INFO:tasks.workunit.client.0.smithi094.stdout:5/796: rename db/d19/d30/d83/fe4 to db/d19/dc1/dd4/f118 0 2024-04-06T03:40:46.430 INFO:tasks.workunit.client.0.smithi094.stdout:5/797: stat db/d19/d2c/d57/c59 0 2024-04-06T03:40:46.432 INFO:tasks.workunit.client.1.smithi136.stdout:6/771: rename d2/d7/d26/d30/d6e/d3d/f6a to d2/d7/d26/d30/d6e/d3d/f105 0 2024-04-06T03:40:46.433 INFO:tasks.workunit.client.0.smithi094.stdout:0/781: symlink d1/d9/dd/d18/d1e/l10b 0 2024-04-06T03:40:46.433 INFO:tasks.workunit.client.0.smithi094.stdout:0/782: fsync d1/f2c 0 2024-04-06T03:40:46.433 INFO:tasks.workunit.client.0.smithi094.stdout:0/783: fdatasync d1/d9/dd/d18/d1e/d86/d96/daa/fca 0 2024-04-06T03:40:46.436 INFO:tasks.workunit.client.0.smithi094.stdout:8/832: link d18/d30/c4c d18/d2c/d37/d60/da3/db8/dd8/c11e 0 2024-04-06T03:40:46.437 INFO:tasks.workunit.client.0.smithi094.stdout:8/833: truncate d18/f25 1354736 0 2024-04-06T03:40:46.438 INFO:tasks.workunit.client.0.smithi094.stdout:5/798: dread db/d19/d30/f44 [0,4194304] 0 2024-04-06T03:40:46.438 INFO:tasks.workunit.client.0.smithi094.stdout:1/839: symlink d0/d1/d27/da5/d40/db1/d2f/d76/daf/l121 0 2024-04-06T03:40:46.439 INFO:tasks.workunit.client.1.smithi136.stdout:0/855: rmdir d1/d4e 39 2024-04-06T03:40:46.439 INFO:tasks.workunit.client.1.smithi136.stdout:0/856: creat d1/d6/d2c/d84/f128 x:0 0 0 2024-04-06T03:40:46.440 INFO:tasks.workunit.client.1.smithi136.stdout:2/910: sync 2024-04-06T03:40:46.440 INFO:tasks.workunit.client.1.smithi136.stdout:1/884: dwrite d12/d3b/d60/d7e/d37/f4e [4194304,4194304] 0 2024-04-06T03:40:46.441 INFO:tasks.workunit.client.1.smithi136.stdout:6/772: mknod d2/d104/d75/c106 0 2024-04-06T03:40:46.443 INFO:tasks.workunit.client.0.smithi094.stdout:6/750: creat dc/dd3/d72/d88/da3/f10b x:0 0 0 2024-04-06T03:40:46.445 INFO:tasks.workunit.client.1.smithi136.stdout:7/957: getdents d5/df/d53/d68/da3 0 2024-04-06T03:40:46.445 INFO:tasks.workunit.client.1.smithi136.stdout:7/958: dread - d5/df/d12/d1b/d3a/d4a/ddf/f104 zero size 2024-04-06T03:40:46.445 INFO:tasks.workunit.client.1.smithi136.stdout:7/959: readlink d5/df/d53/d8a/l10e 0 2024-04-06T03:40:46.445 INFO:tasks.workunit.client.1.smithi136.stdout:7/960: truncate d5/df/d12/d1b/d3a/d4a/dc4/f130 4372424 0 2024-04-06T03:40:46.448 INFO:tasks.workunit.client.0.smithi094.stdout:0/784: getdents d1/d9/dd/d18/d1e/def/d67 0 2024-04-06T03:40:46.449 INFO:tasks.workunit.client.1.smithi136.stdout:1/885: dread d12/d3b/d74/fba [0,4194304] 0 2024-04-06T03:40:46.455 INFO:tasks.workunit.client.0.smithi094.stdout:5/799: rmdir db/d80/d10b 0 2024-04-06T03:40:46.456 INFO:tasks.workunit.client.1.smithi136.stdout:2/911: mkdir d3/d127/d12e 0 2024-04-06T03:40:46.457 INFO:tasks.workunit.client.0.smithi094.stdout:1/840: rmdir d0/d7/d2d 39 2024-04-06T03:40:46.457 INFO:tasks.workunit.client.1.smithi136.stdout:2/912: dread d3/d15/fed [0,4194304] 0 2024-04-06T03:40:46.459 INFO:tasks.workunit.client.1.smithi136.stdout:6/773: rename d2/d7/d26/d30/d6e/d58/d5d/d62/d53/l7b to d2/l107 0 2024-04-06T03:40:46.459 INFO:tasks.workunit.client.1.smithi136.stdout:6/774: fsync d2/d7/f2c 0 2024-04-06T03:40:46.459 INFO:tasks.workunit.client.1.smithi136.stdout:6/775: creat d2/d7/d13/d79/dc2/ddd/f108 x:0 0 0 2024-04-06T03:40:46.461 INFO:tasks.workunit.client.1.smithi136.stdout:7/961: truncate d5/df/d3d/f44 1359655 0 2024-04-06T03:40:46.462 INFO:tasks.workunit.client.0.smithi094.stdout:5/800: truncate db/d19/d24/f37 791533 0 2024-04-06T03:40:46.466 INFO:tasks.workunit.client.1.smithi136.stdout:1/886: creat d12/d1b/d28/d11c/d111/f12a x:0 0 0 2024-04-06T03:40:46.466 INFO:tasks.workunit.client.0.smithi094.stdout:0/785: dwrite d1/d9/dd/d18/d5d/d60/dfd/dac/fe7 [0,4194304] 0 2024-04-06T03:40:46.466 INFO:tasks.workunit.client.0.smithi094.stdout:6/751: dwrite dc/dd3/d72/d87/f8a [0,4194304] 0 2024-04-06T03:40:46.466 INFO:tasks.workunit.client.1.smithi136.stdout:2/913: creat d3/d9/d3b/d70/f12f x:0 0 0 2024-04-06T03:40:46.468 INFO:tasks.workunit.client.0.smithi094.stdout:5/801: mkdir db/d19/d2c/d6d/de6/d119 0 2024-04-06T03:40:46.470 INFO:tasks.workunit.client.1.smithi136.stdout:7/962: write d5/df/d53/f3c [9621905,34896] 0 2024-04-06T03:40:46.470 INFO:tasks.workunit.client.1.smithi136.stdout:0/857: dwrite d1/d6/d2c/d57/d43/d6d/d85/d86/fbc [0,4194304] 0 2024-04-06T03:40:46.472 INFO:tasks.workunit.client.1.smithi136.stdout:6/776: truncate d2/d93/dab/ff0 4097664 0 2024-04-06T03:40:46.479 INFO:tasks.workunit.client.1.smithi136.stdout:1/887: rename d12/d3b/d60/d7e/d37/f8b to d12/d3b/d74/d7a/dea/dd2/f12b 0 2024-04-06T03:40:46.480 INFO:tasks.workunit.client.0.smithi094.stdout:1/841: dwrite d0/d7/d2d/f61 [0,4194304] 0 2024-04-06T03:40:46.481 INFO:tasks.workunit.client.0.smithi094.stdout:1/842: write d0/d1/d27/da5/d40/d50/ff7 [1740841,59808] 0 2024-04-06T03:40:46.481 INFO:tasks.workunit.client.0.smithi094.stdout:0/786: dwrite d1/d9/dd/d18/d1e/def/dab/d4e/fbe [0,4194304] 0 2024-04-06T03:40:46.482 INFO:tasks.workunit.client.1.smithi136.stdout:2/914: creat d3/d15/d19/d11f/f130 x:0 0 0 2024-04-06T03:40:46.482 INFO:tasks.workunit.client.1.smithi136.stdout:2/915: write d3/d15/d19/dac/f126 [609647,108692] 0 2024-04-06T03:40:46.483 INFO:tasks.workunit.client.1.smithi136.stdout:2/916: readlink d3/d4a/d9d/d18/l6e 0 2024-04-06T03:40:46.483 INFO:tasks.workunit.client.0.smithi094.stdout:0/787: dread d1/d9/dd/d18/d1e/def/fb6 [0,4194304] 0 2024-04-06T03:40:46.483 INFO:tasks.workunit.client.0.smithi094.stdout:5/802: truncate db/d7c/d98/f9f 63645 0 2024-04-06T03:40:46.483 INFO:tasks.workunit.client.0.smithi094.stdout:5/803: write db/d19/d2c/d6d/fcb [794239,15703] 0 2024-04-06T03:40:46.485 INFO:tasks.workunit.client.0.smithi094.stdout:6/752: truncate dc/dd3/d72/d88/f102 564726 0 2024-04-06T03:40:46.489 INFO:tasks.workunit.client.0.smithi094.stdout:9/764: rename d2/d2c to d2/dc/d46/d111 0 2024-04-06T03:40:46.491 INFO:tasks.workunit.client.0.smithi094.stdout:1/843: symlink d0/d1/d27/da5/d3f/d8c/l122 0 2024-04-06T03:40:46.494 INFO:tasks.workunit.client.1.smithi136.stdout:3/827: sync 2024-04-06T03:40:46.494 INFO:tasks.workunit.client.1.smithi136.stdout:3/828: fdatasync d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 0 2024-04-06T03:40:46.494 INFO:tasks.workunit.client.1.smithi136.stdout:3/829: fsync d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/fd5 0 2024-04-06T03:40:46.494 INFO:tasks.workunit.client.1.smithi136.stdout:3/830: chown d0/f13 971510393 1 2024-04-06T03:40:46.495 INFO:tasks.workunit.client.1.smithi136.stdout:7/963: mkdir d5/df/d53/d68/da3/dec/d143 0 2024-04-06T03:40:46.498 INFO:tasks.workunit.client.0.smithi094.stdout:9/765: write d2/dc/da6/f100 [3059259,22593] 0 2024-04-06T03:40:46.501 INFO:tasks.workunit.client.1.smithi136.stdout:0/858: link d1/d6/d2c/d1a/f1d d1/d6/d2c/d57/d43/d6d/d85/da7/de5/dec/f129 0 2024-04-06T03:40:46.501 INFO:tasks.workunit.client.1.smithi136.stdout:0/859: chown d1/d6/d2c/c10e 1 1 2024-04-06T03:40:46.505 INFO:tasks.workunit.client.0.smithi094.stdout:0/788: creat d1/d9/dd/d18/d1a/d44/dc2/f10c x:0 0 0 2024-04-06T03:40:46.506 INFO:tasks.workunit.client.1.smithi136.stdout:1/888: rmdir d12/d1b 39 2024-04-06T03:40:46.507 INFO:tasks.workunit.client.0.smithi094.stdout:5/804: unlink db/d19/d2c/d6d/de6/d102/f10f 0 2024-04-06T03:40:46.508 INFO:tasks.workunit.client.0.smithi094.stdout:6/753: mknod dc/dd3/d4e/c10c 0 2024-04-06T03:40:46.511 INFO:tasks.workunit.client.0.smithi094.stdout:2/758: rename d0/dd/d1b/d72/d8f/f93 to d0/dd/d19/d31/f103 0 2024-04-06T03:40:46.513 INFO:tasks.workunit.client.0.smithi094.stdout:1/844: symlink d0/d1/d27/da5/d40/db1/dc3/d5f/l123 0 2024-04-06T03:40:46.515 INFO:tasks.workunit.client.1.smithi136.stdout:3/831: symlink d0/d4/d12/d2d/d37/d116/l119 0 2024-04-06T03:40:46.520 INFO:tasks.workunit.client.1.smithi136.stdout:0/860: mkdir d1/d6/d2c/d1a/d36/d12a 0 2024-04-06T03:40:46.523 INFO:tasks.workunit.client.1.smithi136.stdout:1/889: mknod d12/d3b/d74/dd6/c12c 0 2024-04-06T03:40:46.525 INFO:tasks.workunit.client.1.smithi136.stdout:2/917: getdents d3/d15/d19/d4e 0 2024-04-06T03:40:46.525 INFO:tasks.workunit.client.1.smithi136.stdout:2/918: chown d3/d4a/c33 58118 1 2024-04-06T03:40:46.527 INFO:tasks.workunit.client.0.smithi094.stdout:0/789: mknod d1/d9/dd/d18/d1e/def/d67/df6/c10d 0 2024-04-06T03:40:46.531 INFO:tasks.workunit.client.0.smithi094.stdout:6/754: fsync fa 0 2024-04-06T03:40:46.536 INFO:tasks.workunit.client.0.smithi094.stdout:8/834: rename d18/d2c/d37/d60/fa4 to d18/d2c/d37/d60/da3/f11f 0 2024-04-06T03:40:46.537 INFO:tasks.workunit.client.0.smithi094.stdout:2/759: mknod d0/dd/d19/d31/dcc/c104 0 2024-04-06T03:40:46.537 INFO:tasks.workunit.client.0.smithi094.stdout:2/760: dread - d0/d8/da0/ded/f101 zero size 2024-04-06T03:40:46.537 INFO:tasks.workunit.client.0.smithi094.stdout:2/761: truncate d0/dd/d1b/d72/fca 576483 0 2024-04-06T03:40:46.538 INFO:tasks.workunit.client.0.smithi094.stdout:9/766: mkdir d2/d112 0 2024-04-06T03:40:46.541 INFO:tasks.workunit.client.1.smithi136.stdout:3/832: dwrite d0/d4/d12/d19/d55/d6f/d82/d5d/d61/fdf [0,4194304] 0 2024-04-06T03:40:46.541 INFO:tasks.workunit.client.1.smithi136.stdout:3/833: creat d0/d4/d12/d19/d55/d6f/ddb/f11a x:0 0 0 2024-04-06T03:40:46.542 INFO:tasks.workunit.client.0.smithi094.stdout:5/805: rmdir db/d19/d21 39 2024-04-06T03:40:46.543 INFO:tasks.workunit.client.1.smithi136.stdout:8/915: sync 2024-04-06T03:40:46.543 INFO:tasks.workunit.client.1.smithi136.stdout:9/894: sync 2024-04-06T03:40:46.543 INFO:tasks.workunit.client.0.smithi094.stdout:6/755: symlink dc/d3a/d4a/d54/dde/l10d 0 2024-04-06T03:40:46.544 INFO:tasks.workunit.client.1.smithi136.stdout:9/895: read d17/d19/fcc [49457,96782] 0 2024-04-06T03:40:46.544 INFO:tasks.workunit.client.1.smithi136.stdout:9/896: chown d17/d21/d32/dc6/d113/d46/d9a/ffd 72714453 1 2024-04-06T03:40:46.544 INFO:tasks.workunit.client.1.smithi136.stdout:9/897: write d17/d20/d25/f100 [141538,60448] 0 2024-04-06T03:40:46.544 INFO:tasks.workunit.client.1.smithi136.stdout:9/898: fsync d17/d21/d32/dc6/d113/d66/de1/f8c 0 2024-04-06T03:40:46.545 INFO:tasks.workunit.client.1.smithi136.stdout:9/899: chown d17/d20/d25/da4 520274 1 2024-04-06T03:40:46.548 INFO:tasks.workunit.client.0.smithi094.stdout:1/845: rename d0/d1/d27/da5/d40/db1/d2f/d3c/d48/c9d to d0/d1/d27/da5/d40/db1/d2f/d76/d30/df6/c124 0 2024-04-06T03:40:46.551 INFO:tasks.workunit.client.0.smithi094.stdout:2/762: unlink d0/dd/f8b 0 2024-04-06T03:40:46.551 INFO:tasks.workunit.client.0.smithi094.stdout:2/763: fdatasync d0/dd/d1b/d72/fe3 0 2024-04-06T03:40:46.552 INFO:tasks.workunit.client.0.smithi094.stdout:4/887: sync 2024-04-06T03:40:46.552 INFO:tasks.workunit.client.0.smithi094.stdout:3/813: sync 2024-04-06T03:40:46.552 INFO:tasks.workunit.client.1.smithi136.stdout:0/861: dwrite d1/d6/d2c/d57/d43/d6d/d85/d78/da4/fe9 [0,4194304] 0 2024-04-06T03:40:46.552 INFO:tasks.workunit.client.1.smithi136.stdout:0/862: write d1/d6/d2c/d1a/d36/f76 [1689750,58045] 0 2024-04-06T03:40:46.553 INFO:tasks.workunit.client.0.smithi094.stdout:5/806: getdents db/d80 0 2024-04-06T03:40:46.556 INFO:tasks.workunit.client.0.smithi094.stdout:6/756: creat dc/d3a/d4a/d54/dde/d50/daf/f10e x:0 0 0 2024-04-06T03:40:46.559 INFO:tasks.workunit.client.1.smithi136.stdout:4/760: sync 2024-04-06T03:40:46.559 INFO:tasks.workunit.client.1.smithi136.stdout:5/924: sync 2024-04-06T03:40:46.562 INFO:tasks.workunit.client.0.smithi094.stdout:0/790: dwrite d1/d9/dd/d18/d1e/d86/d96/dc0/f104 [0,4194304] 0 2024-04-06T03:40:46.566 INFO:tasks.workunit.client.0.smithi094.stdout:7/719: sync 2024-04-06T03:40:46.567 INFO:tasks.workunit.client.0.smithi094.stdout:5/807: creat db/d7c/d98/f11a x:0 0 0 2024-04-06T03:40:46.567 INFO:tasks.workunit.client.1.smithi136.stdout:9/900: dwrite d17/d20/d25/da4/dbb/fc4 [0,4194304] 0 2024-04-06T03:40:46.570 INFO:tasks.workunit.client.0.smithi094.stdout:8/835: dwrite d18/d3c/d76/f8c [0,4194304] 0 2024-04-06T03:40:46.571 INFO:tasks.workunit.client.0.smithi094.stdout:8/836: chown d18/d2c 36710979 1 2024-04-06T03:40:46.571 INFO:tasks.workunit.client.0.smithi094.stdout:8/837: creat d18/d3c/d49/d9f/d109/f120 x:0 0 0 2024-04-06T03:40:46.572 INFO:tasks.workunit.client.0.smithi094.stdout:8/838: creat d18/d3c/d76/f121 x:0 0 0 2024-04-06T03:40:46.572 INFO:tasks.workunit.client.0.smithi094.stdout:1/846: dread d0/d1/d27/da5/d40/db1/f55 [0,4194304] 0 2024-04-06T03:40:46.572 INFO:tasks.workunit.client.0.smithi094.stdout:1/847: stat d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/c120 0 2024-04-06T03:40:46.573 INFO:tasks.workunit.client.1.smithi136.stdout:7/964: rmdir d5/df 39 2024-04-06T03:40:46.578 INFO:tasks.workunit.client.0.smithi094.stdout:0/791: creat d1/d9/dd/d18/d5d/f10e x:0 0 0 2024-04-06T03:40:46.578 INFO:tasks.workunit.client.0.smithi094.stdout:6/757: dread dc/d3a/df4/df6/fa5 [0,4194304] 0 2024-04-06T03:40:46.580 INFO:tasks.workunit.client.1.smithi136.stdout:1/890: getdents d12/d3b/d74/d7a/dea/d11a/d4d 0 2024-04-06T03:40:46.581 INFO:tasks.workunit.client.0.smithi094.stdout:2/764: dwrite d0/dd/d1b/d72/fca [0,4194304] 0 2024-04-06T03:40:46.581 INFO:tasks.workunit.client.0.smithi094.stdout:3/814: read d9/d10/f2b [4185382,26851] 0 2024-04-06T03:40:46.581 INFO:tasks.workunit.client.0.smithi094.stdout:4/888: getdents d0/df/d13 0 2024-04-06T03:40:46.581 INFO:tasks.workunit.client.0.smithi094.stdout:4/889: stat d0/d77/d123/c3f 0 2024-04-06T03:40:46.581 INFO:tasks.workunit.client.1.smithi136.stdout:5/925: dwrite d1/d16/d4b/f80 [8388608,4194304] 0 2024-04-06T03:40:46.581 INFO:tasks.workunit.client.1.smithi136.stdout:5/926: chown d1/d16/d89/d8a/f103 633399 1 2024-04-06T03:40:46.583 INFO:tasks.workunit.client.1.smithi136.stdout:2/919: symlink d3/d4a/d9d/d8a/d93/d5f/df0/l131 0 2024-04-06T03:40:46.587 INFO:tasks.workunit.client.0.smithi094.stdout:8/839: read d18/d2c/d37/d60/da3/f11f [1870684,18879] 0 2024-04-06T03:40:46.587 INFO:tasks.workunit.client.0.smithi094.stdout:5/808: dread db/d19/d2c/d57/fbe [0,4194304] 0 2024-04-06T03:40:46.587 INFO:tasks.workunit.client.0.smithi094.stdout:8/840: stat d18/d3c/d49 0 2024-04-06T03:40:46.587 INFO:tasks.workunit.client.0.smithi094.stdout:7/720: symlink d2/d6/dc/d12/d61/lea 0 2024-04-06T03:40:46.593 INFO:tasks.workunit.client.0.smithi094.stdout:3/815: dread d9/d20/d30/f4d [4194304,4194304] 0 2024-04-06T03:40:46.599 INFO:tasks.workunit.client.1.smithi136.stdout:3/834: getdents d0/d4/d12/d19 0 2024-04-06T03:40:46.599 INFO:tasks.workunit.client.1.smithi136.stdout:3/835: readlink d0/d4/d12/d19/d55/d6f/ddb/lef 0 2024-04-06T03:40:46.599 INFO:tasks.workunit.client.1.smithi136.stdout:3/836: write d0/d4/d12/dc3/fc8 [783907,94318] 0 2024-04-06T03:40:46.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:46 smithi136 ceph-mon[36436]: pgmap v23: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 221 MiB/s rd, 277 MiB/s wr, 396 op/s 2024-04-06T03:40:46.600 INFO:tasks.workunit.client.0.smithi094.stdout:1/848: mknod d0/d1/d8f/c125 0 2024-04-06T03:40:46.601 INFO:tasks.workunit.client.0.smithi094.stdout:4/890: dwrite d0/df/d1e/f7f [0,4194304] 0 2024-04-06T03:40:46.601 INFO:tasks.workunit.client.0.smithi094.stdout:4/891: fsync d0/d4/d67/f79 0 2024-04-06T03:40:46.602 INFO:tasks.workunit.client.1.smithi136.stdout:7/965: dwrite f1 [0,4194304] 0 2024-04-06T03:40:46.602 INFO:tasks.workunit.client.1.smithi136.stdout:8/916: rename d3/de/d18/faf to d3/de/d18/d7d/ddf/d6c/dab/f122 0 2024-04-06T03:40:46.602 INFO:tasks.workunit.client.1.smithi136.stdout:9/901: rename d17 to d17/d19/da3/dc5/d130/d137 22 2024-04-06T03:40:46.602 INFO:tasks.workunit.client.0.smithi094.stdout:0/792: mknod d1/d9/dd/c10f 0 2024-04-06T03:40:46.603 INFO:tasks.workunit.client.0.smithi094.stdout:0/793: truncate d1/d9/dd/d18/d1e/def/dab/d4e/ddf/f99 507393 0 2024-04-06T03:40:46.603 INFO:tasks.workunit.client.0.smithi094.stdout:0/794: chown d1/d9/dd/d18/d78/l7f 7 1 2024-04-06T03:40:46.603 INFO:tasks.workunit.client.1.smithi136.stdout:8/917: dread d3/da/da0/fc1 [0,4194304] 0 2024-04-06T03:40:46.604 INFO:tasks.workunit.client.0.smithi094.stdout:2/765: dwrite d0/dd/d1b/d72/fe3 [0,4194304] 0 2024-04-06T03:40:46.610 INFO:tasks.workunit.client.0.smithi094.stdout:8/841: creat d18/d3c/f122 x:0 0 0 2024-04-06T03:40:46.610 INFO:tasks.workunit.client.0.smithi094.stdout:5/809: dwrite db/fc [0,4194304] 0 2024-04-06T03:40:46.611 INFO:tasks.workunit.client.0.smithi094.stdout:7/721: truncate d2/d6/dc/f2b 1728513 0 2024-04-06T03:40:46.613 INFO:tasks.workunit.client.0.smithi094.stdout:1/849: truncate d0/d1/d27/da5/d40/db1/d2f/d76/d30/d38/f4d 855947 0 2024-04-06T03:40:46.614 INFO:tasks.workunit.client.0.smithi094.stdout:2/766: dread d0/dd/d19/d31/d67/d98/ddc/ff8 [0,4194304] 0 2024-04-06T03:40:46.614 INFO:tasks.workunit.client.1.smithi136.stdout:0/863: symlink d1/d6/d2c/d1a/d9c/l12b 0 2024-04-06T03:40:46.616 INFO:tasks.workunit.client.0.smithi094.stdout:2/767: dread d0/dd/d3d/d49/feb [0,4194304] 0 2024-04-06T03:40:46.618 INFO:tasks.workunit.client.1.smithi136.stdout:4/761: link d1d/f25 d1d/d2f/d6c/da5/dc1/d109/f10b 0 2024-04-06T03:40:46.619 INFO:tasks.workunit.client.0.smithi094.stdout:4/892: mkdir d0/d4/d67/df9/d12e 0 2024-04-06T03:40:46.623 INFO:tasks.workunit.client.0.smithi094.stdout:0/795: unlink d1/d9/dd/d18/d1e/d86/d96/dc0/c107 0 2024-04-06T03:40:46.623 INFO:tasks.workunit.client.0.smithi094.stdout:0/796: chown d1/d9/f90 1 1 2024-04-06T03:40:46.624 INFO:tasks.workunit.client.1.smithi136.stdout:1/891: rmdir d12/d3b/d60/d7e 39 2024-04-06T03:40:46.624 INFO:tasks.workunit.client.1.smithi136.stdout:6/777: sync 2024-04-06T03:40:46.625 INFO:tasks.workunit.client.0.smithi094.stdout:2/768: write d0/dd/d19/d31/f7d [8212747,67957] 0 2024-04-06T03:40:46.625 INFO:tasks.workunit.client.0.smithi094.stdout:2/769: fdatasync d0/d8/d1d/f79 0 2024-04-06T03:40:46.626 INFO:tasks.workunit.client.0.smithi094.stdout:6/758: rename dc/dd3/d72/d88/f96 to dc/f10f 0 2024-04-06T03:40:46.626 INFO:tasks.workunit.client.0.smithi094.stdout:3/816: rename d9/d1e to d9/d1e/d6e/d75/dba/d7c/da2/df2/d126 22 2024-04-06T03:40:46.626 INFO:tasks.workunit.client.0.smithi094.stdout:3/817: fdatasync d9/d20/d2d/f115 0 2024-04-06T03:40:46.628 INFO:tasks.workunit.client.1.smithi136.stdout:5/927: creat d1/dc/d1c/d32/d8b/d20/f11b x:0 0 0 2024-04-06T03:40:46.628 INFO:tasks.workunit.client.1.smithi136.stdout:5/928: creat d1/d16/d17/f11c x:0 0 0 2024-04-06T03:40:46.629 INFO:tasks.workunit.client.0.smithi094.stdout:4/893: dread d0/df/db3/dd8/fed [0,4194304] 0 2024-04-06T03:40:46.629 INFO:tasks.workunit.client.0.smithi094.stdout:1/850: dwrite d0/d7/d1a/da8/fbb [0,4194304] 0 2024-04-06T03:40:46.631 INFO:tasks.workunit.client.1.smithi136.stdout:2/920: symlink d3/d4a/d9d/d18/d4b/dc0/dd1/l132 0 2024-04-06T03:40:46.631 INFO:tasks.workunit.client.1.smithi136.stdout:2/921: dread - d3/d4a/d9d/d8a/d93/d5f/df0/f11e zero size 2024-04-06T03:40:46.633 INFO:tasks.workunit.client.1.smithi136.stdout:3/837: symlink d0/d4/d12/d3a/d51/l11b 0 2024-04-06T03:40:46.633 INFO:tasks.workunit.client.1.smithi136.stdout:3/838: write d0/d4/d12/d19/d55/d3c/d5a/d79/ff1 [1002,131017] 0 2024-04-06T03:40:46.635 INFO:tasks.workunit.client.0.smithi094.stdout:4/894: dread d0/d15/d3a/d61/d81/fba [0,4194304] 0 2024-04-06T03:40:46.635 INFO:tasks.workunit.client.0.smithi094.stdout:8/842: creat d18/d3c/dc4/dbc/f123 x:0 0 0 2024-04-06T03:40:46.635 INFO:tasks.workunit.client.0.smithi094.stdout:8/843: chown d18/d2c/d37/d60/da3/lcb 8 1 2024-04-06T03:40:46.635 INFO:tasks.workunit.client.0.smithi094.stdout:8/844: fsync d18/d3c/d49/d91/d94/daa/ddc/ff6 0 2024-04-06T03:40:46.636 INFO:tasks.workunit.client.0.smithi094.stdout:8/845: chown d18/ff9 1011788 1 2024-04-06T03:40:46.636 INFO:tasks.workunit.client.0.smithi094.stdout:8/846: dread - d18/d3c/dc4/db9/fdf zero size 2024-04-06T03:40:46.637 INFO:tasks.workunit.client.0.smithi094.stdout:8/847: dread d18/d3c/d49/db5/fc2 [0,4194304] 0 2024-04-06T03:40:46.639 INFO:tasks.workunit.client.0.smithi094.stdout:5/810: symlink db/d19/d24/d28/d85/dd5/l11b 0 2024-04-06T03:40:46.639 INFO:tasks.workunit.client.0.smithi094.stdout:5/811: creat db/d19/d2c/d6d/de6/f11c x:0 0 0 2024-04-06T03:40:46.641 INFO:tasks.workunit.client.1.smithi136.stdout:3/839: dread d0/d4/d12/d19/d55/d6f/f9e [0,4194304] 0 2024-04-06T03:40:46.641 INFO:tasks.workunit.client.1.smithi136.stdout:7/966: creat d5/df/d12/d1b/d3a/d4a/ddf/f144 x:0 0 0 2024-04-06T03:40:46.644 INFO:tasks.workunit.client.1.smithi136.stdout:9/902: mknod d17/d21/c138 0 2024-04-06T03:40:46.644 INFO:tasks.workunit.client.1.smithi136.stdout:9/903: chown d17/d21/d29/f92 0 1 2024-04-06T03:40:46.644 INFO:tasks.workunit.client.1.smithi136.stdout:9/904: truncate d17/d20/d28/d49/f96 19302 0 2024-04-06T03:40:46.646 INFO:tasks.workunit.client.1.smithi136.stdout:8/918: rename d3/da/d26/d35/d53/d86/l9c to d3/de/d18/d7d/l123 0 2024-04-06T03:40:46.649 INFO:tasks.workunit.client.0.smithi094.stdout:7/722: mknod d2/d6/dc/d12/d29/d3f/dce/ceb 0 2024-04-06T03:40:46.649 INFO:tasks.workunit.client.0.smithi094.stdout:7/723: chown d2/d6/dc/d12/d9a/dbd/fc4 111085 1 2024-04-06T03:40:46.649 INFO:tasks.workunit.client.0.smithi094.stdout:7/724: write d2/d6/dac/ddd/fe0 [587171,44217] 0 2024-04-06T03:40:46.652 INFO:tasks.workunit.client.1.smithi136.stdout:2/922: dwrite d3/d9/d3b/d87/db1/f11a [0,4194304] 0 2024-04-06T03:40:46.652 INFO:tasks.workunit.client.1.smithi136.stdout:3/840: write d0/d4/d12/d19/d55/d3c/d5a/f89 [16370,25425] 0 2024-04-06T03:40:46.652 INFO:tasks.workunit.client.1.smithi136.stdout:2/923: fsync d3/d4a/d9d/d18/d1d/d3c/f51 0 2024-04-06T03:40:46.653 INFO:tasks.workunit.client.0.smithi094.stdout:0/797: link d1/d9/dd/d18/d1e/def/dab/d4e/de4/l39 d1/d9/dd/d18/d32/d42/l110 0 2024-04-06T03:40:46.655 INFO:tasks.workunit.client.1.smithi136.stdout:3/841: dread d0/d4/d12/d19/d115/f90 [0,4194304] 0 2024-04-06T03:40:46.655 INFO:tasks.workunit.client.1.smithi136.stdout:7/967: dwrite d5/df/d3d/f8e [0,4194304] 0 2024-04-06T03:40:46.655 INFO:tasks.workunit.client.1.smithi136.stdout:3/842: write d0/d4/f76 [4259251,76986] 0 2024-04-06T03:40:46.655 INFO:tasks.workunit.client.1.smithi136.stdout:3/843: fdatasync d0/d4/d12/d2d/f39 0 2024-04-06T03:40:46.655 INFO:tasks.workunit.client.1.smithi136.stdout:3/844: read - d0/d4/d12/d19/d55/d6f/d82/f103 zero size 2024-04-06T03:40:46.655 INFO:tasks.workunit.client.1.smithi136.stdout:3/845: fsync d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 0 2024-04-06T03:40:46.658 INFO:tasks.workunit.client.1.smithi136.stdout:1/892: rmdir d12/d1b/d28 39 2024-04-06T03:40:46.659 INFO:tasks.workunit.client.0.smithi094.stdout:2/770: creat d0/dd/d1b/f105 x:0 0 0 2024-04-06T03:40:46.660 INFO:tasks.workunit.client.1.smithi136.stdout:6/778: creat d2/d104/f109 x:0 0 0 2024-04-06T03:40:46.661 INFO:tasks.workunit.client.0.smithi094.stdout:6/759: creat dc/dd3/d72/d88/f110 x:0 0 0 2024-04-06T03:40:46.664 INFO:tasks.workunit.client.0.smithi094.stdout:3/818: rename d9/d10/c2e to d9/d1e/d26/c127 0 2024-04-06T03:40:46.665 INFO:tasks.workunit.client.1.smithi136.stdout:5/929: mknod d1/dc/d1c/d32/d8b/d20/d7c/c11d 0 2024-04-06T03:40:46.666 INFO:tasks.workunit.client.1.smithi136.stdout:5/930: readlink d1/dc/l72 0 2024-04-06T03:40:46.666 INFO:tasks.workunit.client.1.smithi136.stdout:2/924: dwrite d3/d15/d19/d4e/fba [4194304,4194304] 0 2024-04-06T03:40:46.667 INFO:tasks.workunit.client.1.smithi136.stdout:2/925: creat d3/d9/d3b/d70/f133 x:0 0 0 2024-04-06T03:40:46.667 INFO:tasks.workunit.client.1.smithi136.stdout:2/926: write d3/d15/fd5 [3072843,34891] 0 2024-04-06T03:40:46.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:46 smithi094 ceph-mon[29713]: pgmap v23: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 221 MiB/s rd, 277 MiB/s wr, 396 op/s 2024-04-06T03:40:46.669 INFO:tasks.workunit.client.1.smithi136.stdout:9/905: mknod d17/d21/d29/de5/da5/dce/d102/c139 0 2024-04-06T03:40:46.670 INFO:tasks.workunit.client.0.smithi094.stdout:4/895: mknod d0/d15/d3a/c12f 0 2024-04-06T03:40:46.671 INFO:tasks.workunit.client.1.smithi136.stdout:8/919: truncate d3/d2d/d5f/fb4 2885154 0 2024-04-06T03:40:46.674 INFO:tasks.workunit.client.1.smithi136.stdout:0/864: getdents d1/d6/d2c/d57/d43/d6b 0 2024-04-06T03:40:46.674 INFO:tasks.workunit.client.1.smithi136.stdout:0/865: truncate d1/d53/ff7 4358520 0 2024-04-06T03:40:46.674 INFO:tasks.workunit.client.1.smithi136.stdout:0/866: chown d1/d6/d2c/fc0 53725170 1 2024-04-06T03:40:46.675 INFO:tasks.workunit.client.0.smithi094.stdout:2/771: dwrite d0/dd/d3d/fde [0,4194304] 0 2024-04-06T03:40:46.677 INFO:tasks.workunit.client.0.smithi094.stdout:5/812: truncate db/d19/d24/d2e/f36 3153211 0 2024-04-06T03:40:46.680 INFO:tasks.workunit.client.0.smithi094.stdout:7/725: truncate d2/d6/dc/dd/df/fa9 1772316 0 2024-04-06T03:40:46.680 INFO:tasks.workunit.client.0.smithi094.stdout:7/726: write d2/d6/dc/d12/d29/d3f/f5e [69593,86585] 0 2024-04-06T03:40:46.680 INFO:tasks.workunit.client.0.smithi094.stdout:7/727: fsync d2/d6/f8f 0 2024-04-06T03:40:46.681 INFO:tasks.workunit.client.0.smithi094.stdout:0/798: mkdir d1/d9/dd/d18/d1e/def/dab/d111 0 2024-04-06T03:40:46.681 INFO:tasks.workunit.client.0.smithi094.stdout:0/799: dread - d1/d9/dd/d18/d1e/d86/d96/daa/fee zero size 2024-04-06T03:40:46.684 INFO:tasks.workunit.client.0.smithi094.stdout:6/760: creat dc/d3a/df4/df6/df2/f111 x:0 0 0 2024-04-06T03:40:46.684 INFO:tasks.workunit.client.0.smithi094.stdout:6/761: write dc/d3a/d4a/d54/dde/f57 [4269043,1225] 0 2024-04-06T03:40:46.684 INFO:tasks.workunit.client.1.smithi136.stdout:0/867: write d1/f7 [2373924,52163] 0 2024-04-06T03:40:46.685 INFO:tasks.workunit.client.1.smithi136.stdout:0/868: truncate d1/d4e/f55 1034253 0 2024-04-06T03:40:46.685 INFO:tasks.workunit.client.1.smithi136.stdout:0/869: dread - d1/d6/ded/f103 zero size 2024-04-06T03:40:46.685 INFO:tasks.workunit.client.1.smithi136.stdout:0/870: write d1/d6/d2c/d57/d43/ff2 [1198500,43176] 0 2024-04-06T03:40:46.686 INFO:tasks.workunit.client.0.smithi094.stdout:4/896: symlink d0/df/dbd/d107/l130 0 2024-04-06T03:40:46.686 INFO:tasks.workunit.client.0.smithi094.stdout:4/897: chown d0/l5f 1 1 2024-04-06T03:40:46.687 INFO:tasks.workunit.client.0.smithi094.stdout:3/819: dwrite d9/d10/d7e/da3/f11f [0,4194304] 0 2024-04-06T03:40:46.689 INFO:tasks.workunit.client.0.smithi094.stdout:5/813: write db/fc [3196254,52281] 0 2024-04-06T03:40:46.689 INFO:tasks.workunit.client.1.smithi136.stdout:7/968: rename d5/df/d53/d68/da3/fe5 to d5/df/d12/d1b/d3a/d4a/ddf/f145 0 2024-04-06T03:40:46.690 INFO:tasks.workunit.client.1.smithi136.stdout:9/906: dwrite d17/d20/d28/d49/d75/f9d [0,4194304] 0 2024-04-06T03:40:46.690 INFO:tasks.workunit.client.0.smithi094.stdout:8/848: rmdir d18/d3c/d49/d9f/d109 39 2024-04-06T03:40:46.691 INFO:tasks.workunit.client.1.smithi136.stdout:3/846: truncate d0/d4/d12/d19/d115/fe8 3270046 0 2024-04-06T03:40:46.693 INFO:tasks.workunit.client.1.smithi136.stdout:0/871: dread d1/d6/d2c/d1a/fe7 [0,4194304] 0 2024-04-06T03:40:46.694 INFO:tasks.workunit.client.1.smithi136.stdout:4/762: getdents d1d/d31/d78/dcb 0 2024-04-06T03:40:46.694 INFO:tasks.workunit.client.1.smithi136.stdout:4/763: chown d1d/d31/d3e/c3d 167 1 2024-04-06T03:40:46.694 INFO:tasks.workunit.client.1.smithi136.stdout:4/764: write d1d/d31/d3e/d55/d89/db3/fc5 [888695,87378] 0 2024-04-06T03:40:46.697 INFO:tasks.workunit.client.1.smithi136.stdout:1/893: creat d12/d3b/d60/d7e/f12d x:0 0 0 2024-04-06T03:40:46.697 INFO:tasks.workunit.client.1.smithi136.stdout:1/894: truncate d12/d3b/d44/d5f/d5a/fae 4355707 0 2024-04-06T03:40:46.698 INFO:tasks.workunit.client.0.smithi094.stdout:2/772: creat d0/dd/d1b/d72/d8f/f106 x:0 0 0 2024-04-06T03:40:46.699 INFO:tasks.workunit.client.1.smithi136.stdout:7/969: write d5/df/d1d/d6f/d4d/d69/fc8 [1694097,25542] 0 2024-04-06T03:40:46.699 INFO:tasks.workunit.client.1.smithi136.stdout:7/970: write d5/df/d1d/d54/fa4 [1477483,93326] 0 2024-04-06T03:40:46.699 INFO:tasks.workunit.client.1.smithi136.stdout:7/971: chown d5/df/d1d/d6f/d4d/d69/f7b 1076230 1 2024-04-06T03:40:46.699 INFO:tasks.workunit.client.1.smithi136.stdout:7/972: chown d5/df/d12/d1b/d3a/d8d 9031491 1 2024-04-06T03:40:46.701 INFO:tasks.workunit.client.1.smithi136.stdout:6/779: creat d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/f10a x:0 0 0 2024-04-06T03:40:46.704 INFO:tasks.workunit.client.1.smithi136.stdout:5/931: mkdir d1/d98/dc0/d11e 0 2024-04-06T03:40:46.705 INFO:tasks.workunit.client.1.smithi136.stdout:0/872: read d1/d6/d2c/d57/d43/fa2 [68331,42732] 0 2024-04-06T03:40:46.708 INFO:tasks.workunit.client.0.smithi094.stdout:3/820: dread d9/d1e/d6e/d75/dba/d3a/ffa [0,4194304] 0 2024-04-06T03:40:46.708 INFO:tasks.workunit.client.0.smithi094.stdout:6/762: dwrite dc/fed [0,4194304] 0 2024-04-06T03:40:46.709 INFO:tasks.workunit.client.0.smithi094.stdout:7/728: creat d2/fec x:0 0 0 2024-04-06T03:40:46.709 INFO:tasks.workunit.client.0.smithi094.stdout:7/729: creat d2/d6/dc/d12/d29/fed x:0 0 0 2024-04-06T03:40:46.712 INFO:tasks.workunit.client.0.smithi094.stdout:0/800: mkdir d1/d9/dd/d18/d78/deb/d112 0 2024-04-06T03:40:46.715 INFO:tasks.workunit.client.0.smithi094.stdout:4/898: symlink d0/df/dbd/l131 0 2024-04-06T03:40:46.715 INFO:tasks.workunit.client.0.smithi094.stdout:4/899: chown d0/d15/d3a/d61/d81/l117 4746188 1 2024-04-06T03:40:46.718 INFO:tasks.workunit.client.1.smithi136.stdout:7/973: dwrite d5/df/d12/d1b/f30 [0,4194304] 0 2024-04-06T03:40:46.718 INFO:tasks.workunit.client.1.smithi136.stdout:8/920: creat d3/da/d26/dfb/dff/f124 x:0 0 0 2024-04-06T03:40:46.718 INFO:tasks.workunit.client.0.smithi094.stdout:2/773: dwrite d0/dd/d19/d31/d67/d98/ddc/ff8 [4194304,4194304] 0 2024-04-06T03:40:46.726 INFO:tasks.workunit.client.1.smithi136.stdout:2/927: rename d3/d15/fe7 to d3/ddc/f134 0 2024-04-06T03:40:46.727 INFO:tasks.workunit.client.1.smithi136.stdout:8/921: dread d3/da/d26/d35/d39/d70/f9e [0,4194304] 0 2024-04-06T03:40:46.728 INFO:tasks.workunit.client.0.smithi094.stdout:2/774: dread d0/dd/d19/d31/d67/d98/fb7 [0,4194304] 0 2024-04-06T03:40:46.728 INFO:tasks.workunit.client.0.smithi094.stdout:2/775: fdatasync d0/f57 0 2024-04-06T03:40:46.729 INFO:tasks.workunit.client.1.smithi136.stdout:9/907: link cf d17/d19/da3/dc5/d130/c13a 0 2024-04-06T03:40:46.731 INFO:tasks.workunit.client.1.smithi136.stdout:3/847: mknod d0/d4/d12/d19/d55/d3c/d46/c11c 0 2024-04-06T03:40:46.734 INFO:tasks.workunit.client.1.smithi136.stdout:3/848: dread d0/d4/d12/f5c [0,4194304] 0 2024-04-06T03:40:46.737 INFO:tasks.workunit.client.1.smithi136.stdout:1/895: truncate d12/d1b/d28/d11c/d111/f9e 1739231 0 2024-04-06T03:40:46.739 INFO:tasks.workunit.client.1.smithi136.stdout:5/932: truncate d1/d98/dc6/fe4 1006207 0 2024-04-06T03:40:46.739 INFO:tasks.workunit.client.1.smithi136.stdout:8/922: write d3/da/da0/f10c [3086088,58011] 0 2024-04-06T03:40:46.743 INFO:tasks.workunit.client.0.smithi094.stdout:2/776: dwrite d0/dd/d1b/d72/d8f/f106 [0,4194304] 0 2024-04-06T03:40:46.743 INFO:tasks.workunit.client.0.smithi094.stdout:2/777: write d0/dd/d3d/fde [4363137,130122] 0 2024-04-06T03:40:46.744 INFO:tasks.workunit.client.1.smithi136.stdout:0/873: link d1/d6/d2c/d84/ff1 d1/d6/d2c/d1a/d36/d59/dff/f12c 0 2024-04-06T03:40:46.744 INFO:tasks.workunit.client.1.smithi136.stdout:1/896: write d12/d3b/f84 [1001137,126413] 0 2024-04-06T03:40:46.748 INFO:tasks.workunit.client.1.smithi136.stdout:3/849: write d0/d4/d12/d19/d55/d3c/d5a/d79/fa4 [200981,120342] 0 2024-04-06T03:40:46.748 INFO:tasks.workunit.client.1.smithi136.stdout:3/850: chown d0/d4 0 1 2024-04-06T03:40:46.748 INFO:tasks.workunit.client.1.smithi136.stdout:3/851: stat d0/d4/d12/d19/d55/d3c/d5a/f101 0 2024-04-06T03:40:46.755 INFO:tasks.workunit.client.0.smithi094.stdout:7/730: creat d2/d6/dc/d12/d29/d3f/fee x:0 0 0 2024-04-06T03:40:46.755 INFO:tasks.workunit.client.0.smithi094.stdout:7/731: chown d2/d6/dc/dd/l1b 6421548 1 2024-04-06T03:40:46.758 INFO:tasks.workunit.client.0.smithi094.stdout:4/900: creat d0/d4/d67/df9/d12e/f132 x:0 0 0 2024-04-06T03:40:46.758 INFO:tasks.workunit.client.0.smithi094.stdout:4/901: fdatasync d0/df/d1e/f59 0 2024-04-06T03:40:46.758 INFO:tasks.workunit.client.0.smithi094.stdout:4/902: fdatasync d0/f8 0 2024-04-06T03:40:46.762 INFO:tasks.workunit.client.1.smithi136.stdout:8/923: dwrite d3/da/f41 [0,4194304] 0 2024-04-06T03:40:46.765 INFO:tasks.workunit.client.1.smithi136.stdout:6/780: rename d2/d7/d13/d79/dc2/ld1 to d2/d104/l10b 0 2024-04-06T03:40:46.765 INFO:tasks.workunit.client.1.smithi136.stdout:6/781: stat d2/d7/d26/d30/dc4/fda 0 2024-04-06T03:40:46.767 INFO:tasks.workunit.client.1.smithi136.stdout:5/933: dwrite d1/dc/fc9 [0,4194304] 0 2024-04-06T03:40:46.767 INFO:tasks.workunit.client.1.smithi136.stdout:5/934: truncate d1/d16/d89/d8a/f103 220421 0 2024-04-06T03:40:46.767 INFO:tasks.workunit.client.1.smithi136.stdout:5/935: write d1/d16/d17/fd9 [541509,50804] 0 2024-04-06T03:40:46.767 INFO:tasks.workunit.client.1.smithi136.stdout:5/936: stat d1/dc/f91 0 2024-04-06T03:40:46.767 INFO:tasks.workunit.client.1.smithi136.stdout:5/937: chown d1/dc/d1c/d32/d8b/d20/d7c/c11d 1546834 1 2024-04-06T03:40:46.767 INFO:tasks.workunit.client.0.smithi094.stdout:5/814: rmdir db/d19/d24/d28/d85/dd5 39 2024-04-06T03:40:46.769 INFO:tasks.workunit.client.1.smithi136.stdout:2/928: creat d3/d4a/d9d/d18/df8/f135 x:0 0 0 2024-04-06T03:40:46.769 INFO:tasks.workunit.client.0.smithi094.stdout:4/903: dread d0/df/d13/f37 [0,4194304] 0 2024-04-06T03:40:46.769 INFO:tasks.workunit.client.0.smithi094.stdout:4/904: write d0/d15/d3a/d61/d81/f106 [977364,22411] 0 2024-04-06T03:40:46.769 INFO:tasks.workunit.client.0.smithi094.stdout:4/905: creat d0/df/db3/dd8/f133 x:0 0 0 2024-04-06T03:40:46.770 INFO:tasks.workunit.client.0.smithi094.stdout:6/763: dwrite f7 [0,4194304] 0 2024-04-06T03:40:46.771 INFO:tasks.workunit.client.0.smithi094.stdout:6/764: read - dc/dd3/d72/f7c zero size 2024-04-06T03:40:46.771 INFO:tasks.workunit.client.0.smithi094.stdout:6/765: dread - dc/d3a/d4a/d94/dac/f100 zero size 2024-04-06T03:40:46.771 INFO:tasks.workunit.client.0.smithi094.stdout:6/766: write dc/d3a/d4a/d54/fb5 [142174,12351] 0 2024-04-06T03:40:46.771 INFO:tasks.workunit.client.0.smithi094.stdout:8/849: unlink d18/d3c/d49/f53 0 2024-04-06T03:40:46.773 INFO:tasks.workunit.client.1.smithi136.stdout:1/897: dwrite d12/d3b/d74/d7a/dea/dd2/f12b [0,4194304] 0 2024-04-06T03:40:46.773 INFO:tasks.workunit.client.0.smithi094.stdout:7/732: dwrite d2/d6/dc/d12/d3a/fa2 [0,4194304] 0 2024-04-06T03:40:46.774 INFO:tasks.workunit.client.0.smithi094.stdout:7/733: chown d2/d6/c91 0 1 2024-04-06T03:40:46.778 INFO:tasks.workunit.client.0.smithi094.stdout:2/778: mknod d0/dd/d19/d31/dcc/c107 0 2024-04-06T03:40:46.778 INFO:tasks.workunit.client.0.smithi094.stdout:2/779: write d0/d8/ff6 [56390,60839] 0 2024-04-06T03:40:46.778 INFO:tasks.workunit.client.0.smithi094.stdout:2/780: write d0/d8/d27/d2b/f2d [4558662,75962] 0 2024-04-06T03:40:46.778 INFO:tasks.workunit.client.0.smithi094.stdout:2/781: chown d0/dd/d19/d31/f8c 20 1 2024-04-06T03:40:46.781 INFO:tasks.workunit.client.0.smithi094.stdout:7/734: dread d2/d6/dc/dd/df/d95/dc9/f3d [0,4194304] 0 2024-04-06T03:40:46.781 INFO:tasks.workunit.client.0.smithi094.stdout:7/735: stat d2/d6/dc/dd/df/d95/dc9/f3d 0 2024-04-06T03:40:46.782 INFO:tasks.workunit.client.0.smithi094.stdout:7/736: stat d2/d6/dc/dd/f3e 0 2024-04-06T03:40:46.782 INFO:tasks.workunit.client.0.smithi094.stdout:7/737: write d2/d6/dc/dd/f28 [475584,51054] 0 2024-04-06T03:40:46.790 INFO:tasks.workunit.client.0.smithi094.stdout:3/821: rename d9/d1e/d6e/d75 to d9/d20/d2d/d128 0 2024-04-06T03:40:46.792 INFO:tasks.workunit.client.1.smithi136.stdout:9/908: unlink d17/d21/d32/dc6/d113/c125 0 2024-04-06T03:40:46.793 INFO:tasks.workunit.client.0.smithi094.stdout:5/815: mkdir db/d19/dc1/dd4/d11d 0 2024-04-06T03:40:46.794 INFO:tasks.workunit.client.1.smithi136.stdout:4/765: getdents d1d/dd0 0 2024-04-06T03:40:46.797 INFO:tasks.workunit.client.0.smithi094.stdout:4/906: mkdir d0/d4/d67/dcb/d134 0 2024-04-06T03:40:46.797 INFO:tasks.workunit.client.0.smithi094.stdout:4/907: fdatasync d0/d4/d4b/ff7 0 2024-04-06T03:40:46.798 INFO:tasks.workunit.client.0.smithi094.stdout:4/908: fsync d0/d4/d4b/ff7 0 2024-04-06T03:40:46.798 INFO:tasks.workunit.client.1.smithi136.stdout:1/898: dwrite d12/d3b/d60/d7e/dbe/f117 [0,4194304] 0 2024-04-06T03:40:46.798 INFO:tasks.workunit.client.1.smithi136.stdout:1/899: chown d12/d3b/d74/d7a/dea/d77/d96/ccf 40 1 2024-04-06T03:40:46.804 INFO:tasks.workunit.client.1.smithi136.stdout:0/874: creat d1/d4e/f12d x:0 0 0 2024-04-06T03:40:46.804 INFO:tasks.workunit.client.1.smithi136.stdout:0/875: fdatasync d1/d53/ff7 0 2024-04-06T03:40:46.809 INFO:tasks.workunit.client.0.smithi094.stdout:3/822: write d9/d10/d7e/da3/f11f [1668054,84972] 0 2024-04-06T03:40:46.809 INFO:tasks.workunit.client.0.smithi094.stdout:3/823: chown d9/d1e/d109 13105533 1 2024-04-06T03:40:46.809 INFO:tasks.workunit.client.0.smithi094.stdout:8/850: rmdir d18/d34/d52 39 2024-04-06T03:40:46.812 INFO:tasks.workunit.client.1.smithi136.stdout:3/852: mkdir d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d11d 0 2024-04-06T03:40:46.813 INFO:tasks.workunit.client.0.smithi094.stdout:2/782: symlink d0/d8/d27/d2b/d8e/l108 0 2024-04-06T03:40:46.814 INFO:tasks.workunit.client.1.smithi136.stdout:8/924: symlink d3/de/d18/d7d/ddf/d6c/d7f/d105/l125 0 2024-04-06T03:40:46.815 INFO:tasks.workunit.client.1.smithi136.stdout:9/909: dwrite d17/d20/d28/d49/d75/d60/f110 [0,4194304] 0 2024-04-06T03:40:46.815 INFO:tasks.workunit.client.1.smithi136.stdout:0/876: read d1/d6/d2c/d1a/f39 [5366473,55188] 0 2024-04-06T03:40:46.815 INFO:tasks.workunit.client.1.smithi136.stdout:1/900: dwrite d12/d3b/d74/fcb [0,4194304] 0 2024-04-06T03:40:46.816 INFO:tasks.workunit.client.0.smithi094.stdout:7/738: creat d2/d6/dc/dd/df/d95/dc9/fef x:0 0 0 2024-04-06T03:40:46.816 INFO:tasks.workunit.client.0.smithi094.stdout:2/783: dread d0/d8/f33 [0,4194304] 0 2024-04-06T03:40:46.818 INFO:tasks.workunit.client.0.smithi094.stdout:4/909: dwrite d0/f8 [0,4194304] 0 2024-04-06T03:40:46.818 INFO:tasks.workunit.client.0.smithi094.stdout:0/801: rename d1/d9/dd/d18/d78/lb0 to d1/d9/dd/d18/d5d/d60/dbc/l113 0 2024-04-06T03:40:46.822 INFO:tasks.workunit.client.0.smithi094.stdout:6/767: unlink dc/dd3/d72/d88/l9b 0 2024-04-06T03:40:46.823 INFO:tasks.workunit.client.0.smithi094.stdout:7/739: dread d2/d6/dac/ddd/fe0 [0,4194304] 0 2024-04-06T03:40:46.823 INFO:tasks.workunit.client.0.smithi094.stdout:7/740: chown d2/d6/dc/d12/d29/daa/le1 186 1 2024-04-06T03:40:46.825 INFO:tasks.workunit.client.0.smithi094.stdout:3/824: unlink d9/d20/da6/fe7 0 2024-04-06T03:40:46.825 INFO:tasks.workunit.client.1.smithi136.stdout:8/925: dread d3/f10 [0,4194304] 0 2024-04-06T03:40:46.828 INFO:tasks.workunit.client.0.smithi094.stdout:8/851: creat d18/d2c/d119/f124 x:0 0 0 2024-04-06T03:40:46.828 INFO:tasks.workunit.client.0.smithi094.stdout:8/852: chown d18/d2c/d37 3681164 1 2024-04-06T03:40:46.830 INFO:tasks.workunit.client.0.smithi094.stdout:6/768: write dc/d3a/df4/df6/df2/f108 [3973881,28607] 0 2024-04-06T03:40:46.830 INFO:tasks.workunit.client.0.smithi094.stdout:6/769: fsync dc/d3a/fda 0 2024-04-06T03:40:46.831 INFO:tasks.workunit.client.1.smithi136.stdout:2/929: symlink d3/d4a/d9d/d8a/d93/d5f/l136 0 2024-04-06T03:40:46.832 INFO:tasks.workunit.client.0.smithi094.stdout:7/741: dread d2/d6/dc/d12/d9a/fd3 [0,4194304] 0 2024-04-06T03:40:46.832 INFO:tasks.workunit.client.0.smithi094.stdout:3/825: dread d9/d20/d2d/d128/dba/f38 [0,4194304] 0 2024-04-06T03:40:46.834 INFO:tasks.workunit.client.0.smithi094.stdout:4/910: truncate d0/f22 358157 0 2024-04-06T03:40:46.837 INFO:tasks.workunit.client.0.smithi094.stdout:0/802: truncate d1/d9/dd/d18/d32/f9c 7322203 0 2024-04-06T03:40:46.840 INFO:tasks.workunit.client.1.smithi136.stdout:4/766: rename d1d/d31/d3e/d56/d7b/c9c to d1d/d2f/c10c 0 2024-04-06T03:40:46.840 INFO:tasks.workunit.client.0.smithi094.stdout:3/826: dread d9/d1e/d26/d55/f65 [0,4194304] 0 2024-04-06T03:40:46.841 INFO:tasks.workunit.client.0.smithi094.stdout:5/816: getdents db/d19/d2c/d6d/de6/d102 0 2024-04-06T03:40:46.841 INFO:tasks.workunit.client.0.smithi094.stdout:5/817: chown db/d19/d24/d28/d85/lab 242 1 2024-04-06T03:40:46.843 INFO:tasks.workunit.client.0.smithi094.stdout:8/853: mkdir d18/d3c/d49/d91/d114/d125 0 2024-04-06T03:40:46.843 INFO:tasks.workunit.client.0.smithi094.stdout:8/854: chown d18/d3c/d49/d91/d94/daa/f107 585258 1 2024-04-06T03:40:46.850 INFO:tasks.workunit.client.0.smithi094.stdout:2/784: mknod d0/d18/c109 0 2024-04-06T03:40:46.850 INFO:tasks.workunit.client.0.smithi094.stdout:3/827: read d9/d20/d30/f69 [2634892,62002] 0 2024-04-06T03:40:46.854 INFO:tasks.workunit.client.0.smithi094.stdout:4/911: dwrite d0/d4/fd [0,4194304] 0 2024-04-06T03:40:46.855 INFO:tasks.workunit.client.0.smithi094.stdout:7/742: symlink d2/d6/dc/d12/d9a/da0/dd4/lf0 0 2024-04-06T03:40:46.855 INFO:tasks.workunit.client.0.smithi094.stdout:7/743: dread - d2/d6/dc/dd/d90/fa6 zero size 2024-04-06T03:40:46.859 INFO:tasks.workunit.client.0.smithi094.stdout:0/803: unlink d1/d9/l98 0 2024-04-06T03:40:46.862 INFO:tasks.workunit.client.1.smithi136.stdout:9/910: truncate d17/d21/d29/de5/da5/f10d 340304 0 2024-04-06T03:40:46.862 INFO:tasks.workunit.client.1.smithi136.stdout:9/911: dread - d17/d21/d29/de5/da5/db8/f136 zero size 2024-04-06T03:40:46.867 INFO:tasks.workunit.client.0.smithi094.stdout:4/912: dread d0/df/d13/f8c [0,4194304] 0 2024-04-06T03:40:46.868 INFO:tasks.workunit.client.0.smithi094.stdout:6/770: mkdir dc/dd3/d72/d88/dfb/d112 0 2024-04-06T03:40:46.870 INFO:tasks.workunit.client.1.smithi136.stdout:7/974: sync 2024-04-06T03:40:46.870 INFO:tasks.workunit.client.1.smithi136.stdout:7/975: chown d5/df/l92 853 1 2024-04-06T03:40:46.870 INFO:tasks.workunit.client.1.smithi136.stdout:7/976: dread - d5/df/d12/d1b/dbb/dbe/f141 zero size 2024-04-06T03:40:46.870 INFO:tasks.workunit.client.1.smithi136.stdout:1/901: mknod d12/c12e 0 2024-04-06T03:40:46.872 INFO:tasks.workunit.client.1.smithi136.stdout:6/782: getdents d2/d7/d13/d79/dc2/ddd 0 2024-04-06T03:40:46.874 INFO:tasks.workunit.client.0.smithi094.stdout:2/785: dwrite d0/d8/d27/fb4 [4194304,4194304] 0 2024-04-06T03:40:46.877 INFO:tasks.workunit.client.1.smithi136.stdout:5/938: mkdir d1/d98/dbc/d11f 0 2024-04-06T03:40:46.877 INFO:tasks.workunit.client.1.smithi136.stdout:5/939: fsync d1/d98/dbc/fdc 0 2024-04-06T03:40:46.877 INFO:tasks.workunit.client.1.smithi136.stdout:5/940: truncate d1/d16/d17/fd9 1598201 0 2024-04-06T03:40:46.877 INFO:tasks.workunit.client.1.smithi136.stdout:5/941: chown d1/d98 11269862 1 2024-04-06T03:40:46.880 INFO:tasks.workunit.client.1.smithi136.stdout:3/853: rename d0/d4/d12/d19/d55/d3c/d5a/d79/d98/d10c to d0/d4/d12/d19/d2e/d3b/d11e 0 2024-04-06T03:40:46.882 INFO:tasks.workunit.client.1.smithi136.stdout:7/977: dwrite d5/df/d12/d1b/d3a/d8d/f9e [0,4194304] 0 2024-04-06T03:40:46.882 INFO:tasks.workunit.client.1.smithi136.stdout:7/978: dread - d5/df/d12/d1b/d3a/d4a/ddf/f144 zero size 2024-04-06T03:40:46.882 INFO:tasks.workunit.client.1.smithi136.stdout:3/854: dread d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/f9b [0,4194304] 0 2024-04-06T03:40:46.882 INFO:tasks.workunit.client.1.smithi136.stdout:3/855: fdatasync d0/d4/d12/d19/d55/d3c/faa 0 2024-04-06T03:40:46.883 INFO:tasks.workunit.client.1.smithi136.stdout:4/767: mkdir d1d/dd0/d10d 0 2024-04-06T03:40:46.886 INFO:tasks.workunit.client.1.smithi136.stdout:9/912: unlink d17/d20/d25/d3c/d55/lb0 0 2024-04-06T03:40:46.892 INFO:tasks.workunit.client.1.smithi136.stdout:4/768: dread d1d/d31/d3e/d56/f5e [0,4194304] 0 2024-04-06T03:40:46.892 INFO:tasks.workunit.client.1.smithi136.stdout:9/913: dread d17/d19/fcc [0,4194304] 0 2024-04-06T03:40:46.893 INFO:tasks.workunit.client.1.smithi136.stdout:6/783: rmdir d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/d94/dfc 0 2024-04-06T03:40:46.893 INFO:tasks.workunit.client.1.smithi136.stdout:6/784: dread - d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/f10a zero size 2024-04-06T03:40:46.894 INFO:tasks.workunit.client.1.smithi136.stdout:5/942: truncate d1/d16/f63 2830770 0 2024-04-06T03:40:46.894 INFO:tasks.workunit.client.1.smithi136.stdout:5/943: chown d1/dc/d1c/d32/d8b/d20/lb5 4545 1 2024-04-06T03:40:46.899 INFO:tasks.workunit.client.0.smithi094.stdout:0/804: unlink d1/d9/dd/d18/d1e/def/dab/d4e/de4/l21 0 2024-04-06T03:40:46.902 INFO:tasks.workunit.client.1.smithi136.stdout:0/877: rename d1/d6/d2c/d84/ff1 to d1/d6/d2c/d57/d43/d6d/d85/da7/de5/dec/f12e 0 2024-04-06T03:40:46.907 INFO:tasks.workunit.client.1.smithi136.stdout:7/979: unlink d5/df/d1d/fc6 0 2024-04-06T03:40:46.908 INFO:tasks.workunit.client.1.smithi136.stdout:7/980: read d5/df/d1d/d54/fa4 [971482,16911] 0 2024-04-06T03:40:46.908 INFO:tasks.workunit.client.1.smithi136.stdout:7/981: dread - d5/df/d11/f124 zero size 2024-04-06T03:40:46.909 INFO:tasks.workunit.client.0.smithi094.stdout:5/818: rename db/d19/d24 to db/d19/d21/d11e 0 2024-04-06T03:40:46.911 INFO:tasks.workunit.client.1.smithi136.stdout:1/902: getdents d12/d3b/d8e 0 2024-04-06T03:40:46.913 INFO:tasks.workunit.client.0.smithi094.stdout:4/913: mkdir d0/df/d1e/d34/d135 0 2024-04-06T03:40:46.913 INFO:tasks.workunit.client.0.smithi094.stdout:4/914: truncate d0/df/d13/f5a 542003 0 2024-04-06T03:40:46.914 INFO:tasks.workunit.client.0.smithi094.stdout:4/915: write d0/df/d1e/d34/f88 [4687767,4321] 0 2024-04-06T03:40:46.914 INFO:tasks.workunit.client.1.smithi136.stdout:4/769: dwrite d1d/d2f/d6c/da5/dc1/d109/f10b [4194304,4194304] 0 2024-04-06T03:40:46.914 INFO:tasks.workunit.client.1.smithi136.stdout:4/770: fdatasync d1d/d31/d3e/d55/fe7 0 2024-04-06T03:40:46.917 INFO:tasks.workunit.client.1.smithi136.stdout:9/914: dwrite d17/d21/d32/d3f/f44 [0,4194304] 0 2024-04-06T03:40:46.917 INFO:tasks.workunit.client.1.smithi136.stdout:6/785: creat d2/d7/d26/d68/ddc/f10c x:0 0 0 2024-04-06T03:40:46.920 INFO:tasks.workunit.client.0.smithi094.stdout:0/805: dwrite d1/d9/dd/d18/d32/f9c [0,4194304] 0 2024-04-06T03:40:46.920 INFO:tasks.workunit.client.0.smithi094.stdout:0/806: write d1/d9/dd/d18/d5d/d60/dfd/dac/fe7 [4784512,108971] 0 2024-04-06T03:40:46.920 INFO:tasks.workunit.client.0.smithi094.stdout:0/807: chown d1/d9/dd/d18/d1a/f54 148994963 1 2024-04-06T03:40:46.922 INFO:tasks.workunit.client.0.smithi094.stdout:2/786: creat d0/f10a x:0 0 0 2024-04-06T03:40:46.922 INFO:tasks.workunit.client.1.smithi136.stdout:5/944: getdents d1/dc/d1c/d32/d70/dc1 0 2024-04-06T03:40:46.922 INFO:tasks.workunit.client.1.smithi136.stdout:3/856: dwrite d0/d4/d12/d19/d55/d6f/d82/fb4 [0,4194304] 0 2024-04-06T03:40:46.925 INFO:tasks.workunit.client.1.smithi136.stdout:2/930: rename d3/c26 to d3/d4a/d9d/d18/c137 0 2024-04-06T03:40:46.925 INFO:tasks.workunit.client.1.smithi136.stdout:3/857: dread d0/d4/d12/f5c [0,4194304] 0 2024-04-06T03:40:46.932 INFO:tasks.workunit.client.1.smithi136.stdout:7/982: unlink d5/df/d12/d1b/d3a/d4a/d70/ddb/f72 0 2024-04-06T03:40:46.932 INFO:tasks.workunit.client.1.smithi136.stdout:7/983: chown d5/df/d53/d68 231580 1 2024-04-06T03:40:46.932 INFO:tasks.workunit.client.1.smithi136.stdout:7/984: chown d5/df/d1d/d6f/d127 261353871 1 2024-04-06T03:40:46.936 INFO:tasks.workunit.client.0.smithi094.stdout:5/819: dwrite db/d7c/d98/f9f [0,4194304] 0 2024-04-06T03:40:46.936 INFO:tasks.workunit.client.1.smithi136.stdout:4/771: dwrite d1d/d2f/d6c/da5/ff7 [0,4194304] 0 2024-04-06T03:40:46.936 INFO:tasks.workunit.client.1.smithi136.stdout:4/772: dread - d1d/fed zero size 2024-04-06T03:40:46.936 INFO:tasks.workunit.client.1.smithi136.stdout:1/903: dwrite d12/d1b/da7/db8/fc7 [0,4194304] 0 2024-04-06T03:40:46.937 INFO:tasks.workunit.client.1.smithi136.stdout:5/945: dwrite d1/dc/d1c/d32/fc7 [0,4194304] 0 2024-04-06T03:40:46.942 INFO:tasks.workunit.client.0.smithi094.stdout:8/855: rename d18/d3c/dc4/dbc/f123 to d18/d34/f126 0 2024-04-06T03:40:46.942 INFO:tasks.workunit.client.0.smithi094.stdout:8/856: fsync d18/d3c/d76/dd9/f6d 0 2024-04-06T03:40:46.944 INFO:tasks.workunit.client.1.smithi136.stdout:9/915: truncate d17/d20/d28/f89 7917379 0 2024-04-06T03:40:46.945 INFO:tasks.workunit.client.1.smithi136.stdout:9/916: dread d17/d21/d29/de5/da5/f10d [0,4194304] 0 2024-04-06T03:40:46.947 INFO:tasks.workunit.client.0.smithi094.stdout:4/916: creat d0/d4/d67/dcb/d134/f136 x:0 0 0 2024-04-06T03:40:46.950 INFO:tasks.workunit.client.0.smithi094.stdout:6/771: truncate dc/d3a/f5e 2825615 0 2024-04-06T03:40:46.952 INFO:tasks.workunit.client.0.smithi094.stdout:6/772: write dc/d3a/d4a/f51 [868942,87736] 0 2024-04-06T03:40:46.952 INFO:tasks.workunit.client.0.smithi094.stdout:6/773: write fa [3557589,128404] 0 2024-04-06T03:40:46.952 INFO:tasks.workunit.client.0.smithi094.stdout:6/774: dread - dc/d3a/d4a/f92 zero size 2024-04-06T03:40:46.953 INFO:tasks.workunit.client.1.smithi136.stdout:5/946: dwrite d1/dc/f5b [0,4194304] 0 2024-04-06T03:40:46.953 INFO:tasks.workunit.client.0.smithi094.stdout:0/808: link d1/d9/dd/d18/d1e/d86/ccb d1/d9/dd/d18/d1e/d86/d96/daa/c114 0 2024-04-06T03:40:46.955 INFO:tasks.workunit.client.0.smithi094.stdout:0/809: dread d1/d9/fd4 [0,4194304] 0 2024-04-06T03:40:46.957 INFO:tasks.workunit.client.0.smithi094.stdout:2/787: mkdir d0/d8/da0/d10b 0 2024-04-06T03:40:46.957 INFO:tasks.workunit.client.0.smithi094.stdout:4/917: write d0/d15/fa2 [3493692,54295] 0 2024-04-06T03:40:46.963 INFO:tasks.workunit.client.1.smithi136.stdout:2/931: getdents d3/d15/d19/d43/d54 0 2024-04-06T03:40:46.963 INFO:tasks.workunit.client.1.smithi136.stdout:2/932: fdatasync d3/d15/f2b 0 2024-04-06T03:40:46.963 INFO:tasks.workunit.client.1.smithi136.stdout:0/878: dwrite d1/d6/d2c/d1a/d36/f5d [0,4194304] 0 2024-04-06T03:40:46.966 INFO:tasks.workunit.client.1.smithi136.stdout:5/947: dwrite d1/d9b/fac [0,4194304] 0 2024-04-06T03:40:46.966 INFO:tasks.workunit.client.1.smithi136.stdout:5/948: chown d1/d16/d4b/l77 242946380 1 2024-04-06T03:40:46.967 INFO:tasks.workunit.client.1.smithi136.stdout:3/858: truncate d0/d4/d12/d2d/f39 199334 0 2024-04-06T03:40:46.971 INFO:tasks.workunit.client.0.smithi094.stdout:5/820: symlink db/l11f 0 2024-04-06T03:40:46.971 INFO:tasks.workunit.client.0.smithi094.stdout:5/821: write db/d19/dc1/ddf/f90 [244002,129078] 0 2024-04-06T03:40:46.972 INFO:tasks.workunit.client.1.smithi136.stdout:8/926: sync 2024-04-06T03:40:46.973 INFO:tasks.workunit.client.1.smithi136.stdout:7/985: creat d5/df/d12/d1b/d3a/d4a/d70/f146 x:0 0 0 2024-04-06T03:40:46.974 INFO:tasks.workunit.client.0.smithi094.stdout:8/857: rmdir d18/d3c/d49/d91/d94 39 2024-04-06T03:40:46.974 INFO:tasks.workunit.client.0.smithi094.stdout:8/858: write d18/d3c/dc4/fbf [1809651,20785] 0 2024-04-06T03:40:46.982 INFO:tasks.workunit.client.0.smithi094.stdout:5/822: write db/f11 [1085339,73703] 0 2024-04-06T03:40:46.983 INFO:tasks.workunit.client.1.smithi136.stdout:7/986: dread d5/df/d1d/d54/d5c/dc7/fd8 [0,4194304] 0 2024-04-06T03:40:46.983 INFO:tasks.workunit.client.1.smithi136.stdout:4/773: symlink d1d/dd0/l10e 0 2024-04-06T03:40:46.986 INFO:tasks.workunit.client.1.smithi136.stdout:2/933: dwrite d3/d4a/d9d/d18/f74 [0,4194304] 0 2024-04-06T03:40:46.986 INFO:tasks.workunit.client.1.smithi136.stdout:2/934: stat d3/d15/d19/d43/daa/dce 0 2024-04-06T03:40:46.986 INFO:tasks.workunit.client.1.smithi136.stdout:2/935: creat d3/d9/d3b/d87/db1/f138 x:0 0 0 2024-04-06T03:40:46.986 INFO:tasks.workunit.client.1.smithi136.stdout:0/879: dwrite d1/d6/d2c/d57/d43/d6d/d9b/d5c/f6a [0,4194304] 0 2024-04-06T03:40:46.987 INFO:tasks.workunit.client.0.smithi094.stdout:0/810: creat d1/d9/dd/d18/d1a/d44/dc2/f115 x:0 0 0 2024-04-06T03:40:46.989 INFO:tasks.workunit.client.1.smithi136.stdout:1/904: unlink d12/d3b/d60/d7e/d37/c3a 0 2024-04-06T03:40:46.989 INFO:tasks.workunit.client.1.smithi136.stdout:1/905: write d12/d1b/d28/d11c/d111/f12a [827275,33081] 0 2024-04-06T03:40:46.989 INFO:tasks.workunit.client.1.smithi136.stdout:1/906: dread - d12/d3b/d74/d7a/dea/d11a/dd1/fdf zero size 2024-04-06T03:40:46.989 INFO:tasks.workunit.client.1.smithi136.stdout:1/907: chown d12/c40 44 1 2024-04-06T03:40:46.991 INFO:tasks.workunit.client.0.smithi094.stdout:2/788: creat d0/dd/d1b/d72/da8/dd2/f10c x:0 0 0 2024-04-06T03:40:46.991 INFO:tasks.workunit.client.0.smithi094.stdout:2/789: creat d0/dd/d1b/d72/da8/dd2/f10d x:0 0 0 2024-04-06T03:40:46.991 INFO:tasks.workunit.client.0.smithi094.stdout:2/790: dread - d0/dd/d1b/d72/da8/dd2/f10d zero size 2024-04-06T03:40:46.994 INFO:tasks.workunit.client.1.smithi136.stdout:2/936: read d3/d4a/d9d/d18/d4b/f5d [389242,45405] 0 2024-04-06T03:40:46.994 INFO:tasks.workunit.client.1.smithi136.stdout:2/937: getdents d3/d4a/d9d/d105 0 2024-04-06T03:40:46.994 INFO:tasks.workunit.client.0.smithi094.stdout:4/918: rename d0/d4/d4b/fca to d0/d77/d123/d49/db2/dc7/f137 0 2024-04-06T03:40:46.996 INFO:tasks.workunit.client.1.smithi136.stdout:6/786: getdents d2/d7/d9e 0 2024-04-06T03:40:46.996 INFO:tasks.workunit.client.1.smithi136.stdout:6/787: write d2/d7/d13/d89/ffd [1367843,83636] 0 2024-04-06T03:40:46.998 INFO:tasks.workunit.client.0.smithi094.stdout:8/859: mkdir d18/d3c/d11a/d127 0 2024-04-06T03:40:46.998 INFO:tasks.workunit.client.0.smithi094.stdout:8/860: write d18/d3c/d49/d91/d94/daa/ddc/ff6 [504002,52456] 0 2024-04-06T03:40:46.999 INFO:tasks.workunit.client.1.smithi136.stdout:9/917: mknod d17/d19/da3/dc5/deb/d9e/dfb/c13b 0 2024-04-06T03:40:46.999 INFO:tasks.workunit.client.0.smithi094.stdout:6/775: truncate fa 935980 0 2024-04-06T03:40:47.009 INFO:tasks.workunit.client.1.smithi136.stdout:5/949: rmdir d1/d9b/dea 39 2024-04-06T03:40:47.011 INFO:tasks.workunit.client.1.smithi136.stdout:3/859: link d0/d4/d91/lcf d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/dd4/l11f 0 2024-04-06T03:40:47.013 INFO:tasks.workunit.client.1.smithi136.stdout:8/927: mkdir d3/de/d18/d7d/ddf/d126 0 2024-04-06T03:40:47.014 INFO:tasks.workunit.client.1.smithi136.stdout:3/860: dread d0/f11 [0,4194304] 0 2024-04-06T03:40:47.014 INFO:tasks.workunit.client.1.smithi136.stdout:8/928: write d3/da/d26/d35/d53/fac [4383853,50713] 0 2024-04-06T03:40:47.014 INFO:tasks.workunit.client.1.smithi136.stdout:3/861: creat d0/d4/d12/d19/d55/d3c/d5a/f120 x:0 0 0 2024-04-06T03:40:47.016 INFO:tasks.workunit.client.0.smithi094.stdout:6/776: dwrite f7 [0,4194304] 0 2024-04-06T03:40:47.017 INFO:tasks.workunit.client.1.smithi136.stdout:7/987: truncate d5/fd 4908713 0 2024-04-06T03:40:47.021 INFO:tasks.workunit.client.1.smithi136.stdout:4/774: creat d1d/d31/d3e/d55/f10f x:0 0 0 2024-04-06T03:40:47.026 INFO:tasks.workunit.client.0.smithi094.stdout:2/791: creat d0/d8/d4f/f10e x:0 0 0 2024-04-06T03:40:47.026 INFO:tasks.workunit.client.0.smithi094.stdout:2/792: dread - d0/d8/d1d/d77/fd4 zero size 2024-04-06T03:40:47.027 INFO:tasks.workunit.client.1.smithi136.stdout:0/880: link d1/d6/d2c/d57/d43/d6d/ff4 d1/d6/d2c/d1a/d82/f12f 0 2024-04-06T03:40:47.028 INFO:tasks.workunit.client.1.smithi136.stdout:3/862: dwrite d0/d4/d12/d19/d55/d6f/ddb/fe7 [0,4194304] 0 2024-04-06T03:40:47.029 INFO:tasks.workunit.client.0.smithi094.stdout:0/811: write d1/d9/dd/d18/d1e/def/dab/f2e [1067341,72952] 0 2024-04-06T03:40:47.030 INFO:tasks.workunit.client.1.smithi136.stdout:1/908: rename d12/d3b/d74/d7a/dea/d77/c98 to d12/d3b/d74/d7a/dea/d11a/d6f/d11e/c12f 0 2024-04-06T03:40:47.030 INFO:tasks.workunit.client.1.smithi136.stdout:1/909: readlink d12/d1b/d28/d11c/d111/lc3 0 2024-04-06T03:40:47.031 INFO:tasks.workunit.client.1.smithi136.stdout:1/910: chown l7 11689 1 2024-04-06T03:40:47.031 INFO:tasks.workunit.client.0.smithi094.stdout:5/823: rename db/d19/d21/d11e/d28/d85/dd5/dfa/ffc to db/d80/deb/f120 0 2024-04-06T03:40:47.033 INFO:tasks.workunit.client.1.smithi136.stdout:2/938: mknod d3/d9/d3b/d87/db7/c139 0 2024-04-06T03:40:47.037 INFO:tasks.workunit.client.0.smithi094.stdout:6/777: mkdir dc/d3a/df4/df6/d7b/dc6/d113 0 2024-04-06T03:40:47.039 INFO:tasks.workunit.client.1.smithi136.stdout:9/918: getdents d17/d19/da3/d101 0 2024-04-06T03:40:47.039 INFO:tasks.workunit.client.1.smithi136.stdout:9/919: chown d17/d21/d32/dc6/d113/d46/d9a/fd7 9 1 2024-04-06T03:40:47.039 INFO:tasks.workunit.client.1.smithi136.stdout:9/920: chown d17/d19/fc8 85970331 1 2024-04-06T03:40:47.039 INFO:tasks.workunit.client.1.smithi136.stdout:9/921: chown d17/d20/d28/d49/d75/d60/lb7 79452613 1 2024-04-06T03:40:47.040 INFO:tasks.workunit.client.0.smithi094.stdout:0/812: write d1/d9/dd/d18/d1e/d86/fc5 [2918837,69539] 0 2024-04-06T03:40:47.040 INFO:tasks.workunit.client.0.smithi094.stdout:0/813: read d1/d9/dd/d18/d5d/d60/dfd/dac/ff4 [335181,33402] 0 2024-04-06T03:40:47.040 INFO:tasks.workunit.client.0.smithi094.stdout:0/814: dread - d1/d9/dd/d18/d32/d42/fd3 zero size 2024-04-06T03:40:47.045 INFO:tasks.workunit.client.1.smithi136.stdout:3/863: dwrite d0/d4/f22 [0,4194304] 0 2024-04-06T03:40:47.045 INFO:tasks.workunit.client.1.smithi136.stdout:3/864: write d0/d4/d12/d3a/d51/ff3 [5036789,52442] 0 2024-04-06T03:40:47.045 INFO:tasks.workunit.client.0.smithi094.stdout:8/861: rename d18/d2c/l105 to d18/d2c/d37/d60/da3/l128 0 2024-04-06T03:40:47.045 INFO:tasks.workunit.client.0.smithi094.stdout:8/862: chown d18/d3c/d49/d91/d94/daa/f10f 38 1 2024-04-06T03:40:47.045 INFO:tasks.workunit.client.0.smithi094.stdout:8/863: readlink d18/d3c/d49/db5/l108 0 2024-04-06T03:40:47.048 INFO:tasks.workunit.client.1.smithi136.stdout:2/939: dwrite d3/d4a/d9d/d18/d1d/f35 [0,4194304] 0 2024-04-06T03:40:47.048 INFO:tasks.workunit.client.1.smithi136.stdout:2/940: fdatasync d3/d15/d19/d43/f94 0 2024-04-06T03:40:47.048 INFO:tasks.workunit.client.1.smithi136.stdout:2/941: chown d3/d4a/d77/f122 54363 1 2024-04-06T03:40:47.048 INFO:tasks.workunit.client.1.smithi136.stdout:2/942: chown d3/d4a/d9d 138 1 2024-04-06T03:40:47.048 INFO:tasks.workunit.client.1.smithi136.stdout:2/943: stat d3/d4a/d9d/d8a/fd9 0 2024-04-06T03:40:47.049 INFO:tasks.workunit.client.0.smithi094.stdout:5/824: mknod db/d19/d2c/d6d/de7/c121 0 2024-04-06T03:40:47.050 INFO:tasks.workunit.client.1.smithi136.stdout:2/944: dread d3/ddc/f113 [0,4194304] 0 2024-04-06T03:40:47.050 INFO:tasks.workunit.client.1.smithi136.stdout:2/945: fdatasync d3/d9/d3b/d87/f88 0 2024-04-06T03:40:47.054 INFO:tasks.workunit.client.1.smithi136.stdout:7/988: unlink d5/df/d12/d1b/d3a/d4a/ddf/f144 0 2024-04-06T03:40:47.057 INFO:tasks.workunit.client.1.smithi136.stdout:4/775: creat d1d/d31/d3e/d56/d102/f110 x:0 0 0 2024-04-06T03:40:47.058 INFO:tasks.workunit.client.0.smithi094.stdout:2/793: rename d0/dd/d1b/l6d to d0/dd/d1b/l10f 0 2024-04-06T03:40:47.058 INFO:tasks.workunit.client.0.smithi094.stdout:0/815: dwrite d1/d9/dd/d18/d1e/def/dab/d4e/f8f [0,4194304] 0 2024-04-06T03:40:47.068 INFO:tasks.workunit.client.1.smithi136.stdout:1/911: rename d12/d3b/d44/fa5 to d12/d1b/d120/da2/f130 0 2024-04-06T03:40:47.068 INFO:tasks.workunit.client.0.smithi094.stdout:6/778: write dc/d11/d97/fa1 [2526414,101472] 0 2024-04-06T03:40:47.073 INFO:tasks.workunit.client.1.smithi136.stdout:4/776: dread d1d/d2f/d6c/f98 [0,4194304] 0 2024-04-06T03:40:47.073 INFO:tasks.workunit.client.1.smithi136.stdout:4/777: fsync d1d/f20 0 2024-04-06T03:40:47.074 INFO:tasks.workunit.client.0.smithi094.stdout:4/919: write d0/d15/d3a/d61/d81/f89 [3500191,88355] 0 2024-04-06T03:40:47.075 INFO:tasks.workunit.client.0.smithi094.stdout:2/794: dwrite d0/d8/f11 [0,4194304] 0 2024-04-06T03:40:47.075 INFO:tasks.workunit.client.0.smithi094.stdout:2/795: fsync d0/d18/fd6 0 2024-04-06T03:40:47.075 INFO:tasks.workunit.client.1.smithi136.stdout:2/946: dwrite d3/d9/d3b/d87/fa5 [0,4194304] 0 2024-04-06T03:40:47.075 INFO:tasks.workunit.client.1.smithi136.stdout:3/865: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/dd4/f121 x:0 0 0 2024-04-06T03:40:47.075 INFO:tasks.workunit.client.1.smithi136.stdout:3/866: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/f122 x:0 0 0 2024-04-06T03:40:47.075 INFO:tasks.workunit.client.1.smithi136.stdout:7/989: dwrite d5/df/d1d/d6f/d2a/f10a [0,4194304] 0 2024-04-06T03:40:47.075 INFO:tasks.workunit.client.1.smithi136.stdout:7/990: fsync d5/df/d12/f123 0 2024-04-06T03:40:47.079 INFO:tasks.workunit.client.0.smithi094.stdout:6/779: creat dc/d3a/d4a/d54/dde/d50/d61/f114 x:0 0 0 2024-04-06T03:40:47.079 INFO:tasks.workunit.client.0.smithi094.stdout:6/780: chown dc/d3a/d4a/f6b 101871 1 2024-04-06T03:40:47.083 INFO:tasks.workunit.client.0.smithi094.stdout:4/920: unlink d0/d15/c8d 0 2024-04-06T03:40:47.085 INFO:tasks.workunit.client.0.smithi094.stdout:0/816: getdents d1/d9/dd/d18/d1e/d86/d96/daa 0 2024-04-06T03:40:47.085 INFO:tasks.workunit.client.1.smithi136.stdout:8/929: rmdir d3/da/d26/d90/dae 39 2024-04-06T03:40:47.089 INFO:tasks.workunit.client.1.smithi136.stdout:9/922: rename d17/d20/d25/da4/dbb/f127 to d17/d21/d32/dc6/d113/d66/de1/d51/d116/f13c 0 2024-04-06T03:40:47.090 INFO:tasks.workunit.client.0.smithi094.stdout:4/921: read d0/df/d13/f5a [150017,101069] 0 2024-04-06T03:40:47.090 INFO:tasks.workunit.client.0.smithi094.stdout:2/796: dwrite d0/dd/d19/d31/f6e [0,4194304] 0 2024-04-06T03:40:47.091 INFO:tasks.workunit.client.0.smithi094.stdout:2/797: creat d0/dd/d3d/d49/f110 x:0 0 0 2024-04-06T03:40:47.091 INFO:tasks.workunit.client.1.smithi136.stdout:4/778: creat d1d/d31/d78/dcb/f111 x:0 0 0 2024-04-06T03:40:47.094 INFO:tasks.workunit.client.0.smithi094.stdout:4/922: creat d0/d77/d123/d49/f138 x:0 0 0 2024-04-06T03:40:47.098 INFO:tasks.workunit.client.1.smithi136.stdout:2/947: link d3/d4a/d9d/d8a/d93/d5f/df0/l131 d3/d15/d19/d43/daa/dec/d12c/l13a 0 2024-04-06T03:40:47.098 INFO:tasks.workunit.client.1.smithi136.stdout:2/948: truncate d3/d4a/d9d/d18/d1d/fd8 526405 0 2024-04-06T03:40:47.098 INFO:tasks.workunit.client.1.smithi136.stdout:2/949: write d3/d15/fed [104581,42582] 0 2024-04-06T03:40:47.098 INFO:tasks.workunit.client.1.smithi136.stdout:2/950: fdatasync d3/d15/d19/d43/d54/fa8 0 2024-04-06T03:40:47.100 INFO:tasks.workunit.client.0.smithi094.stdout:4/923: mkdir d0/de8/d139 0 2024-04-06T03:40:47.104 INFO:tasks.workunit.client.0.smithi094.stdout:4/924: mknod d0/d4/d67/df9/c13a 0 2024-04-06T03:40:47.105 INFO:tasks.workunit.client.1.smithi136.stdout:3/867: creat d0/d4/d12/d19/d2e/f123 x:0 0 0 2024-04-06T03:40:47.109 INFO:tasks.workunit.client.1.smithi136.stdout:7/991: rename d5/df/d53/l101 to d5/df/d12/l147 0 2024-04-06T03:40:47.109 INFO:tasks.workunit.client.1.smithi136.stdout:7/992: stat d5/df/d3d/c78 0 2024-04-06T03:40:47.110 INFO:tasks.workunit.client.1.smithi136.stdout:9/923: link d17/d20/d28/d49/lf1 d17/d20/d25/l13d 0 2024-04-06T03:40:47.110 INFO:tasks.workunit.client.1.smithi136.stdout:9/924: chown d17/d20/d25/da4/l105 22370 1 2024-04-06T03:40:47.110 INFO:tasks.workunit.client.1.smithi136.stdout:9/925: write d17/d20/d25/da4/dbb/de4/f114 [135373,24750] 0 2024-04-06T03:40:47.110 INFO:tasks.workunit.client.1.smithi136.stdout:9/926: write d17/d21/d32/d3f/fb4 [1230497,9983] 0 2024-04-06T03:40:47.110 INFO:tasks.workunit.client.1.smithi136.stdout:9/927: read - d17/d21/d32/dc6/d113/d46/d9a/f106 zero size 2024-04-06T03:40:47.111 INFO:tasks.workunit.client.0.smithi094.stdout:6/781: dwrite dc/dd3/d72/d87/fbc [0,4194304] 0 2024-04-06T03:40:47.111 INFO:tasks.workunit.client.1.smithi136.stdout:2/951: mknod d3/d4a/d9d/d18/d1d/d3c/df7/c13b 0 2024-04-06T03:40:47.113 INFO:tasks.workunit.client.1.smithi136.stdout:8/930: rmdir d3/da/d26/d35/d3d/dc0 39 2024-04-06T03:40:47.114 INFO:tasks.workunit.client.1.smithi136.stdout:8/931: write d3/da/d26/f85 [976494,123304] 0 2024-04-06T03:40:47.114 INFO:tasks.workunit.client.1.smithi136.stdout:8/932: write d3/da/d26/d35/d53/d86/fdc [310497,79015] 0 2024-04-06T03:40:47.116 INFO:tasks.workunit.client.1.smithi136.stdout:7/993: rename d5/df/d12/d1b/d3a/d4a/f64 to d5/df/d12/d1b/d3a/d4a/ddf/f148 0 2024-04-06T03:40:47.119 INFO:tasks.workunit.client.0.smithi094.stdout:2/798: dwrite d0/d8/d1d/fc0 [0,4194304] 0 2024-04-06T03:40:47.119 INFO:tasks.workunit.client.0.smithi094.stdout:0/817: dwrite d1/d9/dd/d18/d1e/f71 [0,4194304] 0 2024-04-06T03:40:47.119 INFO:tasks.workunit.client.0.smithi094.stdout:0/818: readlink d1/d9/dd/d18/d78/l7f 0 2024-04-06T03:40:47.119 INFO:tasks.workunit.client.0.smithi094.stdout:0/819: write d1/d9/dd/d18/d1e/d86/d96/daa/fee [170624,103528] 0 2024-04-06T03:40:47.119 INFO:tasks.workunit.client.0.smithi094.stdout:0/820: dread - d1/d9/dd/d18/d1e/def/f106 zero size 2024-04-06T03:40:47.119 INFO:tasks.workunit.client.0.smithi094.stdout:0/821: truncate d1/d9/dd/d18/d1e/d86/d96/daa/fca 2071792 0 2024-04-06T03:40:47.119 INFO:tasks.workunit.client.0.smithi094.stdout:0/822: creat d1/d9/dd/d18/d5d/d60/dbc/dda/f116 x:0 0 0 2024-04-06T03:40:47.119 INFO:tasks.workunit.client.0.smithi094.stdout:2/799: write d0/dd/da4/f5f [395800,81762] 0 2024-04-06T03:40:47.119 INFO:tasks.workunit.client.0.smithi094.stdout:2/800: fsync d0/d8/d4f/f10e 0 2024-04-06T03:40:47.120 INFO:tasks.workunit.client.1.smithi136.stdout:4/779: dwrite d1d/d31/d8e/f9e [0,4194304] 0 2024-04-06T03:40:47.121 INFO:tasks.workunit.client.1.smithi136.stdout:9/928: mkdir d17/d19/d13e 0 2024-04-06T03:40:47.121 INFO:tasks.workunit.client.1.smithi136.stdout:9/929: write d17/d21/d32/fc7 [875578,53590] 0 2024-04-06T03:40:47.123 INFO:tasks.workunit.client.1.smithi136.stdout:2/952: creat d3/d9/dde/f13c x:0 0 0 2024-04-06T03:40:47.128 INFO:tasks.workunit.client.1.smithi136.stdout:7/994: symlink d5/df/d1d/l149 0 2024-04-06T03:40:47.128 INFO:tasks.workunit.client.1.smithi136.stdout:7/995: write d5/df/d12/d1b/ffa [141585,20890] 0 2024-04-06T03:40:47.131 INFO:tasks.workunit.client.1.smithi136.stdout:3/868: dwrite d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/fb5 [0,4194304] 0 2024-04-06T03:40:47.131 INFO:tasks.workunit.client.1.smithi136.stdout:3/869: write d0/d4/d12/d19/d115/dc9/f10a [398791,19283] 0 2024-04-06T03:40:47.132 INFO:tasks.workunit.client.1.smithi136.stdout:6/788: sync 2024-04-06T03:40:47.132 INFO:tasks.workunit.client.1.smithi136.stdout:6/789: stat d2/d7/d26/d68/f98 0 2024-04-06T03:40:47.132 INFO:tasks.workunit.client.0.smithi094.stdout:2/801: link d0/dd/d1b/f105 d0/d8/da0/f111 0 2024-04-06T03:40:47.135 INFO:tasks.workunit.client.0.smithi094.stdout:2/802: link d0/d8/f11 d0/f112 0 2024-04-06T03:40:47.138 INFO:tasks.workunit.client.0.smithi094.stdout:6/782: dwrite dc/d3a/d4a/d54/dde/d50/daf/f10e [0,4194304] 0 2024-04-06T03:40:47.148 INFO:tasks.workunit.client.1.smithi136.stdout:4/780: dwrite d1d/d31/d3e/ff4 [0,4194304] 0 2024-04-06T03:40:47.149 INFO:tasks.workunit.client.0.smithi094.stdout:0/823: dwrite d1/d9/dd/d18/d1a/d44/f80 [0,4194304] 0 2024-04-06T03:40:47.152 INFO:tasks.workunit.client.0.smithi094.stdout:0/824: truncate d1/d9/dd/d18/d1e/d86/fc5 1598779 0 2024-04-06T03:40:47.152 INFO:tasks.workunit.client.0.smithi094.stdout:0/825: readlink d1/d9/dd/d18/d1e/def/d67/df6/lfb 0 2024-04-06T03:40:47.154 INFO:tasks.workunit.client.1.smithi136.stdout:3/870: creat d0/d4/d12/d19/d2e/d3b/d49/f124 x:0 0 0 2024-04-06T03:40:47.155 INFO:tasks.workunit.client.1.smithi136.stdout:3/871: fsync d0/d4/d12/d19/d55/d3c/d5a/fb8 0 2024-04-06T03:40:47.156 INFO:tasks.workunit.client.1.smithi136.stdout:6/790: creat d2/d7/d26/d68/d9b/f10d x:0 0 0 2024-04-06T03:40:47.158 INFO:tasks.workunit.client.1.smithi136.stdout:9/930: rmdir d17/d20/d25/d71 39 2024-04-06T03:40:47.163 INFO:tasks.workunit.client.1.smithi136.stdout:7/996: dwrite d5/df/d12/dd6/f115 [0,4194304] 0 2024-04-06T03:40:47.163 INFO:tasks.workunit.client.1.smithi136.stdout:7/997: write d5/df/d12/d1b/d3a/d4a/d70/ddb/dfd/f5e [1930462,87873] 0 2024-04-06T03:40:47.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/953: dwrite d3/f61 [0,4194304] 0 2024-04-06T03:40:47.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/954: chown d3/d4a/c33 719 1 2024-04-06T03:40:47.163 INFO:tasks.workunit.client.1.smithi136.stdout:2/955: chown d3/d4a/c14 109328 1 2024-04-06T03:40:47.164 INFO:tasks.workunit.client.0.smithi094.stdout:2/803: dwrite d0/d8/d1d/f79 [0,4194304] 0 2024-04-06T03:40:47.164 INFO:tasks.workunit.client.0.smithi094.stdout:2/804: read - d0/dd/d1b/d72/da8/dd2/f10d zero size 2024-04-06T03:40:47.172 INFO:tasks.workunit.client.1.smithi136.stdout:3/872: write d0/d4/d12/d3a/d51/ff3 [2320757,43848] 0 2024-04-06T03:40:47.172 INFO:tasks.workunit.client.1.smithi136.stdout:3/873: write d0/d4/d12/d19/d55/d3c/d5a/f9a [378807,50179] 0 2024-04-06T03:40:47.173 INFO:tasks.workunit.client.0.smithi094.stdout:2/805: rmdir d0/dd/d3d/d49 39 2024-04-06T03:40:47.173 INFO:tasks.workunit.client.0.smithi094.stdout:2/806: getdents d0/dd/d1b/d72/df3 0 2024-04-06T03:40:47.174 INFO:tasks.workunit.client.1.smithi136.stdout:8/933: dwrite d3/da/d26/d35/d3d/dc7/dce/df2/fa2 [0,4194304] 0 2024-04-06T03:40:47.174 INFO:tasks.workunit.client.1.smithi136.stdout:8/934: chown d3/da/d26/d90 489972971 1 2024-04-06T03:40:47.175 INFO:tasks.workunit.client.0.smithi094.stdout:2/807: mkdir d0/dd/d1b/de2/d113 0 2024-04-06T03:40:47.175 INFO:tasks.workunit.client.0.smithi094.stdout:2/808: chown d0/dd/d19/d31/d67/lf0 287674368 1 2024-04-06T03:40:47.176 INFO:tasks.workunit.client.0.smithi094.stdout:6/783: dwrite dc/dd3/f74 [0,4194304] 0 2024-04-06T03:40:47.176 INFO:tasks.workunit.client.0.smithi094.stdout:6/784: stat dc/d3a/df4/df6/d7b/l9a 0 2024-04-06T03:40:47.176 INFO:tasks.workunit.client.0.smithi094.stdout:0/826: dwrite d1/d9/dd/d18/d1e/def/dab/d4e/de4/f57 [0,4194304] 0 2024-04-06T03:40:47.176 INFO:tasks.workunit.client.1.smithi136.stdout:4/781: dwrite d1d/d31/d3e/d55/fe3 [0,4194304] 0 2024-04-06T03:40:47.177 INFO:tasks.workunit.client.1.smithi136.stdout:0/881: sync 2024-04-06T03:40:47.177 INFO:tasks.workunit.client.1.smithi136.stdout:5/950: sync 2024-04-06T03:40:47.177 INFO:tasks.workunit.client.1.smithi136.stdout:2/956: creat d3/d9/f13d x:0 0 0 2024-04-06T03:40:47.177 INFO:tasks.workunit.client.1.smithi136.stdout:2/957: stat d3/d4a/d77/l9f 0 2024-04-06T03:40:47.178 INFO:tasks.workunit.client.1.smithi136.stdout:7/998: rename d5/df/d3d/fab to d5/df/d53/d68/da3/f14a 0 2024-04-06T03:40:47.178 INFO:tasks.workunit.client.1.smithi136.stdout:7/999: chown d5/c6 128761 1 2024-04-06T03:40:47.179 INFO:tasks.workunit.client.1.smithi136.stdout:3/874: stat d0/lf7 0 2024-04-06T03:40:47.179 INFO:tasks.workunit.client.1.smithi136.stdout:3/875: dread - d0/d4/d12/d19/d55/d3c/d5a/d79/d98/fcc zero size 2024-04-06T03:40:47.179 INFO:tasks.workunit.client.1.smithi136.stdout:3/876: write d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/ffd [309584,21287] 0 2024-04-06T03:40:47.183 INFO:tasks.workunit.client.0.smithi094.stdout:6/785: creat dc/d3a/d7f/dd4/f115 x:0 0 0 2024-04-06T03:40:47.187 INFO:tasks.workunit.client.1.smithi136.stdout:4/782: write d1d/d31/d8e/fc9 [367036,127796] 0 2024-04-06T03:40:47.188 INFO:tasks.workunit.client.1.smithi136.stdout:0/882: mknod d1/d6/ded/c130 0 2024-04-06T03:40:47.190 INFO:tasks.workunit.client.1.smithi136.stdout:3/877: dread d0/d4/d12/d19/d55/d3c/faa [0,4194304] 0 2024-04-06T03:40:47.190 INFO:tasks.workunit.client.1.smithi136.stdout:3/878: creat d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/f125 x:0 0 0 2024-04-06T03:40:47.190 INFO:tasks.workunit.client.1.smithi136.stdout:5/951: mknod d1/dc/c120 0 2024-04-06T03:40:47.190 INFO:tasks.workunit.client.1.smithi136.stdout:1/912: sync 2024-04-06T03:40:47.191 INFO:tasks.workunit.client.0.smithi094.stdout:6/786: symlink dc/db6/l116 0 2024-04-06T03:40:47.191 INFO:tasks.workunit.client.0.smithi094.stdout:6/787: fsync dc/d3a/d4a/d54/dde/d50/d61/f6c 0 2024-04-06T03:40:47.191 INFO:tasks.workunit.client.0.smithi094.stdout:6/788: write dc/d3a/d4a/d54/dde/d50/f69 [1389031,93019] 0 2024-04-06T03:40:47.191 INFO:tasks.workunit.client.0.smithi094.stdout:0/827: write d1/d9/dd/d18/d32/f5f [2725380,66922] 0 2024-04-06T03:40:47.192 INFO:tasks.workunit.client.1.smithi136.stdout:2/958: rmdir d3/d4a/d9d/d18/d4b/dc0/dd1 39 2024-04-06T03:40:47.193 INFO:tasks.workunit.client.0.smithi094.stdout:2/809: dwrite d0/f10a [0,4194304] 0 2024-04-06T03:40:47.194 INFO:tasks.workunit.client.1.smithi136.stdout:8/935: rename d3/de/f56 to d3/da/d26/dfb/dff/f127 0 2024-04-06T03:40:47.195 INFO:tasks.workunit.client.1.smithi136.stdout:8/936: read d3/de/d18/d7d/ddf/f72 [924530,122046] 0 2024-04-06T03:40:47.199 INFO:tasks.workunit.client.1.smithi136.stdout:4/783: symlink d1d/d31/d3e/d55/d89/dea/l112 0 2024-04-06T03:40:47.201 INFO:tasks.workunit.client.1.smithi136.stdout:5/952: dread d1/dc/d1c/d32/d8b/f7f [0,4194304] 0 2024-04-06T03:40:47.204 INFO:tasks.workunit.client.0.smithi094.stdout:6/789: dread dc/d3a/d4a/d54/dde/f57 [0,4194304] 0 2024-04-06T03:40:47.204 INFO:tasks.workunit.client.0.smithi094.stdout:6/790: readlink dc/d11/la9 0 2024-04-06T03:40:47.209 INFO:tasks.workunit.client.1.smithi136.stdout:6/791: dwrite d2/d7/d26/d68/d9b/f10d [0,4194304] 0 2024-04-06T03:40:47.209 INFO:tasks.workunit.client.1.smithi136.stdout:6/792: write d2/d93/fbc [726666,113499] 0 2024-04-06T03:40:47.210 INFO:tasks.workunit.client.1.smithi136.stdout:1/913: unlink d12/f38 0 2024-04-06T03:40:47.211 INFO:tasks.workunit.client.1.smithi136.stdout:2/959: rmdir d3 39 2024-04-06T03:40:47.211 INFO:tasks.workunit.client.1.smithi136.stdout:2/960: chown d3/d4a/d77/lda 0 1 2024-04-06T03:40:47.212 INFO:tasks.workunit.client.1.smithi136.stdout:2/961: read - d3/d4a/d9d/d18/df8/f135 zero size 2024-04-06T03:40:47.214 INFO:tasks.workunit.client.0.smithi094.stdout:6/791: dread dc/d3a/d4a/d54/dde/f57 [0,4194304] 0 2024-04-06T03:40:47.216 INFO:tasks.workunit.client.1.smithi136.stdout:8/937: creat d3/de/d60/f128 x:0 0 0 2024-04-06T03:40:47.223 INFO:tasks.workunit.client.1.smithi136.stdout:3/879: dwrite d0/d4/d12/d2d/f39 [0,4194304] 0 2024-04-06T03:40:47.224 INFO:tasks.workunit.client.1.smithi136.stdout:0/883: dwrite d1/d6/d2c/f88 [0,4194304] 0 2024-04-06T03:40:47.224 INFO:tasks.workunit.client.1.smithi136.stdout:5/953: dwrite d1/dc/d10/f112 [0,4194304] 0 2024-04-06T03:40:47.224 INFO:tasks.workunit.client.1.smithi136.stdout:0/884: read - d1/d6/d2c/d84/f119 zero size 2024-04-06T03:40:47.224 INFO:tasks.workunit.client.1.smithi136.stdout:5/954: truncate d1/d98/dbc/fc5 1289888 0 2024-04-06T03:40:47.224 INFO:tasks.workunit.client.1.smithi136.stdout:5/955: write d1/dc/d1c/d32/f38 [913256,121615] 0 2024-04-06T03:40:47.231 INFO:tasks.workunit.client.1.smithi136.stdout:8/938: dwrite d3/de/d18/d7d/ddf/d6c/dab/f122 [0,4194304] 0 2024-04-06T03:40:47.237 INFO:tasks.workunit.client.0.smithi094.stdout:0/828: dwrite d1/d9/dd/d18/d32/f5f [0,4194304] 0 2024-04-06T03:40:47.330 INFO:tasks.workunit.client.1.smithi136.stdout:4/784: symlink d1d/d31/d3e/d55/d7f/l113 0 2024-04-06T03:40:47.335 INFO:tasks.workunit.client.1.smithi136.stdout:6/793: rmdir d2/d7/d26/d30/d6e/d3d/d47 39 2024-04-06T03:40:47.336 INFO:tasks.workunit.client.0.smithi094.stdout:2/810: mknod d0/d8/d1d/c114 0 2024-04-06T03:40:47.336 INFO:tasks.workunit.client.1.smithi136.stdout:2/962: mknod d3/d9/dde/c13e 0 2024-04-06T03:40:47.337 INFO:tasks.workunit.client.0.smithi094.stdout:6/792: fsync dc/fed 0 2024-04-06T03:40:47.341 INFO:tasks.workunit.client.0.smithi094.stdout:0/829: rename d1/d9/dd/d18/d78/deb/ff3 to d1/d9/dd/d18/d78/deb/f117 0 2024-04-06T03:40:47.343 INFO:tasks.workunit.client.1.smithi136.stdout:6/794: dread d2/d104/fcb [0,4194304] 0 2024-04-06T03:40:47.344 INFO:tasks.workunit.client.0.smithi094.stdout:6/793: truncate dc/d3a/d4a/d54/d71/fcb 355465 0 2024-04-06T03:40:47.348 INFO:tasks.workunit.client.1.smithi136.stdout:3/880: mkdir d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/d126 0 2024-04-06T03:40:47.350 INFO:tasks.workunit.client.0.smithi094.stdout:6/794: creat dc/dd3/d72/d88/dfb/d112/f117 x:0 0 0 2024-04-06T03:40:47.352 INFO:tasks.workunit.client.0.smithi094.stdout:0/830: getdents d1/d9/dd/d18/d1a/d44 0 2024-04-06T03:40:47.352 INFO:tasks.workunit.client.0.smithi094.stdout:0/831: chown d1/f83 120701943 1 2024-04-06T03:40:47.353 INFO:tasks.workunit.client.0.smithi094.stdout:2/811: truncate d0/dd/d19/d31/f6e 718268 0 2024-04-06T03:40:47.353 INFO:tasks.workunit.client.0.smithi094.stdout:2/812: write d0/dd/f95 [571755,46309] 0 2024-04-06T03:40:47.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/767: sync 2024-04-06T03:40:47.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/768: write d2/dc/d21/d26/f29 [277199,129573] 0 2024-04-06T03:40:47.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/769: write d2/f7b [1604930,39184] 0 2024-04-06T03:40:47.355 INFO:tasks.workunit.client.0.smithi094.stdout:9/770: fsync d2/dc/d21/d26/d78/d69/da8/d96/fa0 0 2024-04-06T03:40:47.356 INFO:tasks.workunit.client.0.smithi094.stdout:0/832: mknod d1/d9/dd/d18/d5d/de9/c118 0 2024-04-06T03:40:47.358 INFO:tasks.workunit.client.0.smithi094.stdout:6/795: dread dc/d3a/d4a/d54/dde/f9e [0,4194304] 0 2024-04-06T03:40:47.359 INFO:tasks.workunit.client.0.smithi094.stdout:9/771: mknod d2/dc/c113 0 2024-04-06T03:40:47.359 INFO:tasks.workunit.client.0.smithi094.stdout:9/772: fsync d2/dc/d46/d111/d33/d64/dc6/fdb 0 2024-04-06T03:40:47.359 INFO:tasks.workunit.client.0.smithi094.stdout:9/773: read - d2/dc/d21/d26/d78/d69/f10f zero size 2024-04-06T03:40:47.361 INFO:tasks.workunit.client.1.smithi136.stdout:0/885: truncate d1/d6/d2c/d57/d43/d6d/d85/d86/fbc 3909391 0 2024-04-06T03:40:47.364 INFO:tasks.workunit.client.0.smithi094.stdout:0/833: truncate d1/d9/dd/d18/d1a/f46 1805636 0 2024-04-06T03:40:47.366 INFO:tasks.workunit.client.1.smithi136.stdout:5/956: creat d1/dc/d1c/d32/d8b/db0/f121 x:0 0 0 2024-04-06T03:40:47.366 INFO:tasks.workunit.client.1.smithi136.stdout:5/957: fdatasync d1/dc/d1c/d32/d8b/db8/fdf 0 2024-04-06T03:40:47.369 INFO:tasks.workunit.client.1.smithi136.stdout:8/939: mkdir d3/da/d26/d35/d3d/dc0/d11b/d129 0 2024-04-06T03:40:47.369 INFO:tasks.workunit.client.1.smithi136.stdout:8/940: truncate d3/f77 1870027 0 2024-04-06T03:40:47.373 INFO:tasks.workunit.client.0.smithi094.stdout:2/813: dwrite d0/dd/d19/d31/d67/dae/fec [0,4194304] 0 2024-04-06T03:40:47.375 INFO:tasks.workunit.client.0.smithi094.stdout:9/774: dwrite d2/dc/d21/d23/db0/fbf [0,4194304] 0 2024-04-06T03:40:47.377 INFO:tasks.workunit.client.1.smithi136.stdout:4/785: creat d1d/d2f/d6c/da5/dc1/d109/f114 x:0 0 0 2024-04-06T03:40:47.378 INFO:tasks.workunit.client.1.smithi136.stdout:2/963: symlink d3/d9/d3b/d87/l13f 0 2024-04-06T03:40:47.380 INFO:tasks.workunit.client.1.smithi136.stdout:6/795: creat d2/d7/d9e/ddf/f10e x:0 0 0 2024-04-06T03:40:47.380 INFO:tasks.workunit.client.1.smithi136.stdout:6/796: chown d2/d7/d13/f76 95491 1 2024-04-06T03:40:47.381 INFO:tasks.workunit.client.1.smithi136.stdout:9/931: sync 2024-04-06T03:40:47.382 INFO:tasks.workunit.client.1.smithi136.stdout:3/881: creat d0/f127 x:0 0 0 2024-04-06T03:40:47.382 INFO:tasks.workunit.client.1.smithi136.stdout:3/882: write d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/fd0 [730128,123221] 0 2024-04-06T03:40:47.384 INFO:tasks.workunit.client.1.smithi136.stdout:0/886: symlink d1/d6/d2c/d57/d62/l131 0 2024-04-06T03:40:47.384 INFO:tasks.workunit.client.0.smithi094.stdout:0/834: getdents d1/d9/dd/d18/d5d/d60 0 2024-04-06T03:40:47.387 INFO:tasks.workunit.client.1.smithi136.stdout:5/958: mkdir d1/d16/d89/d8a/df1/d106/d122 0 2024-04-06T03:40:47.389 INFO:tasks.workunit.client.0.smithi094.stdout:9/775: dread d2/dc/d21/d26/d78/d69/da8/d96/f9f [0,4194304] 0 2024-04-06T03:40:47.390 INFO:tasks.workunit.client.0.smithi094.stdout:0/835: creat d1/d9/f119 x:0 0 0 2024-04-06T03:40:47.390 INFO:tasks.workunit.client.0.smithi094.stdout:0/836: write d1/d9/dd/d18/d32/d42/f68 [1097197,52120] 0 2024-04-06T03:40:47.391 INFO:tasks.workunit.client.1.smithi136.stdout:6/797: write d2/d7/d26/d30/f4c [1757120,110654] 0 2024-04-06T03:40:47.392 INFO:tasks.workunit.client.1.smithi136.stdout:8/941: rename d3/l15 to d3/de/d18/d6d/l12a 0 2024-04-06T03:40:47.395 INFO:tasks.workunit.client.0.smithi094.stdout:2/814: dwrite d0/f64 [0,4194304] 0 2024-04-06T03:40:47.395 INFO:tasks.workunit.client.0.smithi094.stdout:2/815: fsync d0/dd/d1b/d72/d8f/fc4 0 2024-04-06T03:40:47.395 INFO:tasks.workunit.client.0.smithi094.stdout:2/816: write d0/dd/da4/ff5 [804306,120254] 0 2024-04-06T03:40:47.395 INFO:tasks.workunit.client.1.smithi136.stdout:4/786: symlink d1d/d2f/da6/de1/d10a/l115 0 2024-04-06T03:40:47.396 INFO:tasks.workunit.client.0.smithi094.stdout:9/776: symlink d2/dc/d80/ddd/def/l114 0 2024-04-06T03:40:47.396 INFO:tasks.workunit.client.0.smithi094.stdout:9/777: stat d2/dc/d21/d23/d47/d51/cfb 0 2024-04-06T03:40:47.398 INFO:tasks.workunit.client.1.smithi136.stdout:2/964: mkdir d3/d15/d19/d43/d140 0 2024-04-06T03:40:47.400 INFO:tasks.workunit.client.0.smithi094.stdout:0/837: link d1/l6 d1/d9/dd/d18/d1e/def/d67/df6/l11a 0 2024-04-06T03:40:47.400 INFO:tasks.workunit.client.1.smithi136.stdout:9/932: dwrite d17/d21/d29/de5/da5/fd6 [4194304,4194304] 0 2024-04-06T03:40:47.400 INFO:tasks.workunit.client.1.smithi136.stdout:9/933: chown fa 3389 1 2024-04-06T03:40:47.402 INFO:tasks.workunit.client.1.smithi136.stdout:1/914: sync 2024-04-06T03:40:47.403 INFO:tasks.workunit.client.1.smithi136.stdout:3/883: mkdir d0/d4/d12/d19/d2e/d128 0 2024-04-06T03:40:47.403 INFO:tasks.workunit.client.1.smithi136.stdout:3/884: fsync d0/d4/d12/d19/d55/d6f/ddb/f11a 0 2024-04-06T03:40:47.404 INFO:tasks.workunit.client.0.smithi094.stdout:2/817: chown d0/dd/d1b/l5c 3030 1 2024-04-06T03:40:47.408 INFO:tasks.workunit.client.0.smithi094.stdout:2/818: creat d0/d8/d27/d2b/d8e/f115 x:0 0 0 2024-04-06T03:40:47.410 INFO:tasks.workunit.client.1.smithi136.stdout:6/798: creat d2/d7/d26/d30/d6e/d58/f10f x:0 0 0 2024-04-06T03:40:47.411 INFO:tasks.workunit.client.1.smithi136.stdout:8/942: creat d3/d2d/d5f/f12b x:0 0 0 2024-04-06T03:40:47.411 INFO:tasks.workunit.client.1.smithi136.stdout:8/943: chown d3/de/d18/d7d/ddf/d126 3429 1 2024-04-06T03:40:47.413 INFO:tasks.workunit.client.0.smithi094.stdout:0/838: dread d1/d9/dd/d18/d5d/de9/f100 [0,4194304] 0 2024-04-06T03:40:47.414 INFO:tasks.workunit.client.1.smithi136.stdout:5/959: dwrite d1/d98/dbc/fdc [0,4194304] 0 2024-04-06T03:40:47.414 INFO:tasks.workunit.client.0.smithi094.stdout:2/819: creat d0/dd/d1b/de2/de7/f116 x:0 0 0 2024-04-06T03:40:47.414 INFO:tasks.workunit.client.0.smithi094.stdout:9/778: dwrite d2/dc/d21/d23/db0/fe5 [0,4194304] 0 2024-04-06T03:40:47.414 INFO:tasks.workunit.client.0.smithi094.stdout:9/779: creat d2/dc/d21/d23/d47/d51/f115 x:0 0 0 2024-04-06T03:40:47.415 INFO:tasks.workunit.client.1.smithi136.stdout:4/787: truncate d1d/f20 880975 0 2024-04-06T03:40:47.415 INFO:tasks.workunit.client.1.smithi136.stdout:4/788: readlink d1d/d2f/d6c/da5/la7 0 2024-04-06T03:40:47.416 INFO:tasks.workunit.client.1.smithi136.stdout:9/934: write d17/d20/d25/d3c/d55/f64 [772155,52852] 0 2024-04-06T03:40:47.416 INFO:tasks.workunit.client.1.smithi136.stdout:2/965: dwrite d3/d15/d19/d43/daa/dce/f11d [0,4194304] 0 2024-04-06T03:40:47.418 INFO:tasks.workunit.client.0.smithi094.stdout:0/839: mknod d1/d9/dd/d18/d5d/d8d/c11b 0 2024-04-06T03:40:47.419 INFO:tasks.workunit.client.1.smithi136.stdout:4/789: dread d1d/d31/f44 [0,4194304] 0 2024-04-06T03:40:47.419 INFO:tasks.workunit.client.1.smithi136.stdout:4/790: write d1d/d31/d78/fbd [1416428,39440] 0 2024-04-06T03:40:47.421 INFO:tasks.workunit.client.0.smithi094.stdout:2/820: rename d0/dd/d1b/l10f to d0/dd/d1b/d72/d8f/l117 0 2024-04-06T03:40:47.422 INFO:tasks.workunit.client.1.smithi136.stdout:1/915: mknod d12/d3b/d74/d7a/dea/d11a/d6f/d11e/c131 0 2024-04-06T03:40:47.424 INFO:tasks.workunit.client.1.smithi136.stdout:3/885: stat d0/d4/d12/d3a/d51/l63 0 2024-04-06T03:40:47.424 INFO:tasks.workunit.client.1.smithi136.stdout:3/886: fsync d0/d4/d12/d19/d55/d6f/fb1 0 2024-04-06T03:40:47.427 INFO:tasks.workunit.client.1.smithi136.stdout:9/935: dread d17/d21/d32/d3f/fb4 [0,4194304] 0 2024-04-06T03:40:47.427 INFO:tasks.workunit.client.1.smithi136.stdout:9/936: write d17/d20/f48 [1299899,69245] 0 2024-04-06T03:40:47.429 INFO:tasks.workunit.client.1.smithi136.stdout:5/960: dwrite d1/d9b/faa [0,4194304] 0 2024-04-06T03:40:47.429 INFO:tasks.workunit.client.1.smithi136.stdout:5/961: getdents d1/dc/d1c/d32/d70/dc1 0 2024-04-06T03:40:47.429 INFO:tasks.workunit.client.1.smithi136.stdout:4/791: dread d1d/d2f/f32 [0,4194304] 0 2024-04-06T03:40:47.430 INFO:tasks.workunit.client.1.smithi136.stdout:8/944: symlink d3/da/d26/d35/d3d/dc7/dce/l12c 0 2024-04-06T03:40:47.431 INFO:tasks.workunit.client.0.smithi094.stdout:9/780: symlink d2/dc/d21/d23/l116 0 2024-04-06T03:40:47.432 INFO:tasks.workunit.client.1.smithi136.stdout:5/962: write d1/dc/d1c/d32/d8b/f7f [651861,118211] 0 2024-04-06T03:40:47.432 INFO:tasks.workunit.client.1.smithi136.stdout:5/963: stat d1/dc/d10/d1e/f9d 0 2024-04-06T03:40:47.434 INFO:tasks.workunit.client.0.smithi094.stdout:0/840: creat d1/d9/dd/d18/d5d/d60/f11c x:0 0 0 2024-04-06T03:40:47.434 INFO:tasks.workunit.client.0.smithi094.stdout:0/841: chown d1/d9/dd/d18/f31 414520 1 2024-04-06T03:40:47.437 INFO:tasks.workunit.client.0.smithi094.stdout:2/821: unlink d0/dd/d3d/d49/f110 0 2024-04-06T03:40:47.437 INFO:tasks.workunit.client.0.smithi094.stdout:2/822: chown d0/dd/d1b/f105 2 1 2024-04-06T03:40:47.437 INFO:tasks.workunit.client.0.smithi094.stdout:1/851: sync 2024-04-06T03:40:47.437 INFO:tasks.workunit.client.0.smithi094.stdout:3/828: sync 2024-04-06T03:40:47.437 INFO:tasks.workunit.client.0.smithi094.stdout:3/829: truncate d9/d20/d2d/d128/d90/f119 565668 0 2024-04-06T03:40:47.439 INFO:tasks.workunit.client.0.smithi094.stdout:0/842: mknod d1/d9/dd/d18/d1e/d86/d96/dc0/c11d 0 2024-04-06T03:40:47.439 INFO:tasks.workunit.client.0.smithi094.stdout:0/843: dread - d1/d9/dd/d18/d5d/d60/f11c zero size 2024-04-06T03:40:47.442 INFO:tasks.workunit.client.0.smithi094.stdout:3/830: mkdir d9/d9d/d129 0 2024-04-06T03:40:47.442 INFO:tasks.workunit.client.0.smithi094.stdout:3/831: read - d9/d10/ffe zero size 2024-04-06T03:40:47.444 INFO:tasks.workunit.client.1.smithi136.stdout:8/945: dread d3/da/d26/d35/f3c [0,4194304] 0 2024-04-06T03:40:47.445 INFO:tasks.workunit.client.0.smithi094.stdout:2/823: dread d0/dd/d19/d31/d67/d98/fb7 [0,4194304] 0 2024-04-06T03:40:47.446 INFO:tasks.workunit.client.1.smithi136.stdout:2/966: getdents d3/d15/d19/d43 0 2024-04-06T03:40:47.447 INFO:tasks.workunit.client.0.smithi094.stdout:3/832: truncate d9/d20/d2d/d128/dba/d97/fd0 30847 0 2024-04-06T03:40:47.449 INFO:tasks.workunit.client.1.smithi136.stdout:1/916: mknod d12/d1b/da7/dcc/c132 0 2024-04-06T03:40:47.449 INFO:tasks.workunit.client.1.smithi136.stdout:5/964: dwrite d1/d16/d89/d8a/f104 [0,4194304] 0 2024-04-06T03:40:47.449 INFO:tasks.workunit.client.1.smithi136.stdout:5/965: chown d1/dc/d10/d99/fd2 1008668666 1 2024-04-06T03:40:47.449 INFO:tasks.workunit.client.1.smithi136.stdout:1/917: chown d12/d3b/d74/d7a/f118 3516111 1 2024-04-06T03:40:47.450 INFO:tasks.workunit.client.0.smithi094.stdout:9/781: dwrite d2/dc/d80/ddd/fa7 [0,4194304] 0 2024-04-06T03:40:47.450 INFO:tasks.workunit.client.0.smithi094.stdout:1/852: write d0/d7/f46 [1380336,61402] 0 2024-04-06T03:40:47.452 INFO:tasks.workunit.client.0.smithi094.stdout:0/844: unlink d1/d9/dd/d18/d1e/l29 0 2024-04-06T03:40:47.455 INFO:tasks.workunit.client.0.smithi094.stdout:2/824: symlink d0/dd/d1b/d72/l118 0 2024-04-06T03:40:47.456 INFO:tasks.workunit.client.1.smithi136.stdout:5/966: dread d1/d16/d4b/f76 [0,4194304] 0 2024-04-06T03:40:47.456 INFO:tasks.workunit.client.1.smithi136.stdout:5/967: creat d1/d16/d17/db2/f123 x:0 0 0 2024-04-06T03:40:47.457 INFO:tasks.workunit.client.1.smithi136.stdout:9/937: rename d17/d21/d29/de5/d4b/f80 to d17/d20/d25/da4/d112/f13f 0 2024-04-06T03:40:47.457 INFO:tasks.workunit.client.1.smithi136.stdout:9/938: chown d17/d20/d25/d3c/d55/d8a/f11e 65356781 1 2024-04-06T03:40:47.457 INFO:tasks.workunit.client.1.smithi136.stdout:9/939: stat d17/d21/d32/dc6/d113/d66/de1/d51/d116 0 2024-04-06T03:40:47.459 INFO:tasks.workunit.client.0.smithi094.stdout:3/833: link d9/d20/d76/l66 d9/d10/dbb/de1/l12a 0 2024-04-06T03:40:47.460 INFO:tasks.workunit.client.1.smithi136.stdout:9/940: fdatasync d17/d20/d25/d3c/d55/f64 0 2024-04-06T03:40:47.460 INFO:tasks.workunit.client.1.smithi136.stdout:6/799: rmdir d2/d92 39 2024-04-06T03:40:47.465 INFO:tasks.workunit.client.1.smithi136.stdout:4/792: mknod d1d/d2f/d6c/da5/dc1/c116 0 2024-04-06T03:40:47.465 INFO:tasks.workunit.client.1.smithi136.stdout:4/793: fsync d1d/d70/ff1 0 2024-04-06T03:40:47.468 INFO:tasks.workunit.client.0.smithi094.stdout:0/845: dwrite d1/d9/dd/d18/d1a/d44/f80 [0,4194304] 0 2024-04-06T03:40:47.470 INFO:tasks.workunit.client.0.smithi094.stdout:9/782: rename d2/dc/f68 to d2/dc/d46/d111/d33/d64/dc6/f117 0 2024-04-06T03:40:47.472 INFO:tasks.workunit.client.1.smithi136.stdout:8/946: creat d3/da/d26/d90/dae/db5/dec/f12d x:0 0 0 2024-04-06T03:40:47.473 INFO:tasks.workunit.client.1.smithi136.stdout:8/947: dread d3/da/d26/d35/d53/f101 [0,4194304] 0 2024-04-06T03:40:47.474 INFO:tasks.workunit.client.0.smithi094.stdout:1/853: mkdir d0/d1/d27/da5/d40/db1/dc3/d5f/d126 0 2024-04-06T03:40:47.474 INFO:tasks.workunit.client.0.smithi094.stdout:1/854: write d0/d1/d27/f2e [1708172,14745] 0 2024-04-06T03:40:47.475 INFO:tasks.workunit.client.1.smithi136.stdout:2/967: truncate d3/d15/f73 3273630 0 2024-04-06T03:40:47.475 INFO:tasks.workunit.client.1.smithi136.stdout:2/968: write d3/d4a/d9d/d18/d4b/f65 [2297566,33880] 0 2024-04-06T03:40:47.475 INFO:tasks.workunit.client.1.smithi136.stdout:2/969: stat d3/d9 0 2024-04-06T03:40:47.475 INFO:tasks.workunit.client.0.smithi094.stdout:0/846: write d1/d9/fc [488509,45145] 0 2024-04-06T03:40:47.484 INFO:tasks.workunit.client.1.smithi136.stdout:1/918: link d12/d3b/d44/d5f/d5a/de2/cff d12/d3b/d74/dd6/c133 0 2024-04-06T03:40:47.487 INFO:tasks.workunit.client.1.smithi136.stdout:3/887: getdents d0 0 2024-04-06T03:40:47.491 INFO:tasks.workunit.client.0.smithi094.stdout:2/825: mknod d0/dd/d3d/c119 0 2024-04-06T03:40:47.491 INFO:tasks.workunit.client.0.smithi094.stdout:9/783: dwrite d2/dc/d10/d2a/fa2 [0,4194304] 0 2024-04-06T03:40:47.493 INFO:tasks.workunit.client.1.smithi136.stdout:8/948: dwrite d3/d2d/f55 [4194304,4194304] 0 2024-04-06T03:40:47.495 INFO:tasks.workunit.client.1.smithi136.stdout:5/968: rename d1/dc/d1c/d32/d8b/db0/le5 to d1/dc/d1c/d32/d8b/d20/d66/d113/l124 0 2024-04-06T03:40:47.497 INFO:tasks.workunit.client.0.smithi094.stdout:0/847: symlink d1/d9/dd/d18/dd8/l11e 0 2024-04-06T03:40:47.497 INFO:tasks.workunit.client.0.smithi094.stdout:0/848: creat d1/d9/dd/d18/d1e/def/d67/df6/f11f x:0 0 0 2024-04-06T03:40:47.498 INFO:tasks.workunit.client.1.smithi136.stdout:3/888: dread d0/d4/d12/f15 [4194304,4194304] 0 2024-04-06T03:40:47.498 INFO:tasks.workunit.client.1.smithi136.stdout:9/941: mkdir d17/d21/d32/dc6/d113/d66/de1/d51/d57/d140 0 2024-04-06T03:40:47.499 INFO:tasks.workunit.client.0.smithi094.stdout:2/826: creat d0/dd/d1b/d72/f11a x:0 0 0 2024-04-06T03:40:47.505 INFO:tasks.workunit.client.1.smithi136.stdout:5/969: read d1/d16/d17/f44 [6030862,94465] 0 2024-04-06T03:40:47.506 INFO:tasks.workunit.client.0.smithi094.stdout:9/784: write d2/dc/d21/d23/d2f/f90 [3881472,97386] 0 2024-04-06T03:40:47.513 INFO:tasks.workunit.client.1.smithi136.stdout:2/970: creat d3/d15/d19/d4e/f141 x:0 0 0 2024-04-06T03:40:47.516 INFO:tasks.workunit.client.1.smithi136.stdout:5/970: dread d1/d16/d4b/f80 [4194304,4194304] 0 2024-04-06T03:40:47.516 INFO:tasks.workunit.client.1.smithi136.stdout:3/889: dwrite d0/f23 [0,4194304] 0 2024-04-06T03:40:47.516 INFO:tasks.workunit.client.0.smithi094.stdout:3/834: rename d9/d1e/d6e/l100 to d9/d20/d76/d6b/l12b 0 2024-04-06T03:40:47.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/919: symlink d12/d3b/d60/d7e/d37/l134 0 2024-04-06T03:40:47.517 INFO:tasks.workunit.client.1.smithi136.stdout:1/920: creat d12/d3b/d60/d7e/d37/dbb/f135 x:0 0 0 2024-04-06T03:40:47.517 INFO:tasks.workunit.client.1.smithi136.stdout:5/971: dread d1/d9b/dea/fff [0,4194304] 0 2024-04-06T03:40:47.520 INFO:tasks.workunit.client.1.smithi136.stdout:8/949: symlink d3/da/d26/d35/d3d/dc0/d11f/l12e 0 2024-04-06T03:40:47.520 INFO:tasks.workunit.client.1.smithi136.stdout:8/950: creat d3/da/d26/d35/dcb/d10d/f12f x:0 0 0 2024-04-06T03:40:47.522 INFO:tasks.workunit.client.0.smithi094.stdout:2/827: rmdir d0/d8/d27/d2b 39 2024-04-06T03:40:47.522 INFO:tasks.workunit.client.0.smithi094.stdout:2/828: chown d0/dd/d1b/d72/f11a 39019878 1 2024-04-06T03:40:47.523 INFO:tasks.workunit.client.1.smithi136.stdout:0/887: sync 2024-04-06T03:40:47.524 INFO:tasks.workunit.client.0.smithi094.stdout:9/785: mkdir d2/dc/d80/d118 0 2024-04-06T03:40:47.526 INFO:tasks.workunit.client.1.smithi136.stdout:6/800: rename d2/d104/f109 to d2/d7/d13/d89/f110 0 2024-04-06T03:40:47.526 INFO:tasks.workunit.client.0.smithi094.stdout:3/835: dread d9/d20/d30/f37 [0,4194304] 0 2024-04-06T03:40:47.527 INFO:tasks.workunit.client.1.smithi136.stdout:6/801: read d2/d7/d26/d30/d6e/d58/d5d/fd0 [899173,123318] 0 2024-04-06T03:40:47.528 INFO:tasks.workunit.client.1.smithi136.stdout:2/971: dwrite d3/d4a/d9d/d8a/d93/d5f/df0/f11e [0,4194304] 0 2024-04-06T03:40:47.529 INFO:tasks.workunit.client.1.smithi136.stdout:2/972: read - d3/d15/d19/d43/f94 zero size 2024-04-06T03:40:47.529 INFO:tasks.workunit.client.0.smithi094.stdout:7/744: sync 2024-04-06T03:40:47.529 INFO:tasks.workunit.client.1.smithi136.stdout:9/942: getdents d17/d21/d29/de5/da5/db8 0 2024-04-06T03:40:47.530 INFO:tasks.workunit.client.0.smithi094.stdout:9/786: dread d2/dc/d21/d23/d5b/ff0 [0,4194304] 0 2024-04-06T03:40:47.532 INFO:tasks.workunit.client.0.smithi094.stdout:3/836: unlink d9/de5/d10b/c11b 0 2024-04-06T03:40:47.533 INFO:tasks.workunit.client.1.smithi136.stdout:3/890: symlink d0/d4/d12/d19/d55/d6f/l129 0 2024-04-06T03:40:47.536 INFO:tasks.workunit.client.0.smithi094.stdout:7/745: truncate d2/d6/dc/d12/d29/fa7 2137151 0 2024-04-06T03:40:47.537 INFO:tasks.workunit.client.1.smithi136.stdout:2/973: write d3/d4a/d9d/d18/d4b/f85 [3101149,30947] 0 2024-04-06T03:40:47.538 INFO:tasks.workunit.client.0.smithi094.stdout:9/787: mknod d2/dc/d80/c119 0 2024-04-06T03:40:47.538 INFO:tasks.workunit.client.0.smithi094.stdout:9/788: dread - d2/dc/d21/d23/d47/d51/fed zero size 2024-04-06T03:40:47.538 INFO:tasks.workunit.client.0.smithi094.stdout:3/837: dread d9/d20/f3d [0,4194304] 0 2024-04-06T03:40:47.539 INFO:tasks.workunit.client.0.smithi094.stdout:3/838: write d9/d10/dbb/de1/f10a [2466479,58512] 0 2024-04-06T03:40:47.540 INFO:tasks.workunit.client.0.smithi094.stdout:9/789: dread d2/dc/d80/db7/fc7 [0,4194304] 0 2024-04-06T03:40:47.540 INFO:tasks.workunit.client.0.smithi094.stdout:9/790: fsync d2/dc/d21/d23/db0/fce 0 2024-04-06T03:40:47.540 INFO:tasks.workunit.client.0.smithi094.stdout:2/829: getdents d0/dd/d19/d31/dcc 0 2024-04-06T03:40:47.541 INFO:tasks.workunit.client.1.smithi136.stdout:8/951: symlink d3/da/d26/d35/d3d/dc0/d11b/l130 0 2024-04-06T03:40:47.541 INFO:tasks.workunit.client.1.smithi136.stdout:8/952: fdatasync d3/da/d26/d35/d53/d86/f8f 0 2024-04-06T03:40:47.541 INFO:tasks.workunit.client.1.smithi136.stdout:8/953: fsync d3/da/d26/d35/d3d/dc0/f117 0 2024-04-06T03:40:47.541 INFO:tasks.workunit.client.1.smithi136.stdout:8/954: truncate d3/da/d26/d90/fde 668491 0 2024-04-06T03:40:47.541 INFO:tasks.workunit.client.1.smithi136.stdout:8/955: write d3/da/d26/d35/d3d/dc0/f117 [391973,6627] 0 2024-04-06T03:40:47.541 INFO:tasks.workunit.client.1.smithi136.stdout:0/888: mkdir d1/d6/d2c/d57/d43/d6d/d85/d11a/d132 0 2024-04-06T03:40:47.542 INFO:tasks.workunit.client.0.smithi094.stdout:1/855: rename d0/d1/d27/da5/d40/db1/d2f/d76 to d0/d127 0 2024-04-06T03:40:47.544 INFO:tasks.workunit.client.1.smithi136.stdout:1/921: rename d12/d3b/d74/d7a/dea/d11a/d4d/f68 to d12/d3b/d60/f136 0 2024-04-06T03:40:47.544 INFO:tasks.workunit.client.1.smithi136.stdout:6/802: truncate d2/d104/fcb 1571167 0 2024-04-06T03:40:47.545 INFO:tasks.workunit.client.0.smithi094.stdout:7/746: creat d2/d6/dc/d12/d9a/ff1 x:0 0 0 2024-04-06T03:40:47.547 INFO:tasks.workunit.client.0.smithi094.stdout:7/747: dread d2/d6/dc/dd/f31 [0,4194304] 0 2024-04-06T03:40:47.547 INFO:tasks.workunit.client.0.smithi094.stdout:3/839: link d9/d20/d76/c7a d9/d20/d2d/d128/dba/d97/c12c 0 2024-04-06T03:40:47.548 INFO:tasks.workunit.client.0.smithi094.stdout:3/840: write d9/f3f [1353240,85760] 0 2024-04-06T03:40:47.548 INFO:tasks.workunit.client.0.smithi094.stdout:9/791: rmdir d2/dc/d21/d23/d47 39 2024-04-06T03:40:47.551 INFO:tasks.workunit.client.1.smithi136.stdout:9/943: creat d17/d20/d25/d71/dd5/f141 x:0 0 0 2024-04-06T03:40:47.552 INFO:tasks.workunit.client.0.smithi094.stdout:2/830: truncate d0/dd/d19/f4c 1084873 0 2024-04-06T03:40:47.556 INFO:tasks.workunit.client.0.smithi094.stdout:1/856: symlink d0/d127/dff/d115/l128 0 2024-04-06T03:40:47.560 INFO:tasks.workunit.client.0.smithi094.stdout:7/748: dread d2/d6/dc/dd/df/f2d [0,4194304] 0 2024-04-06T03:40:47.562 INFO:tasks.workunit.client.0.smithi094.stdout:3/841: rename d9/d1e/d109 to d9/d20/d2d/d128/dba/d3a/d12d 0 2024-04-06T03:40:47.562 INFO:tasks.workunit.client.0.smithi094.stdout:3/842: chown d9/d9d 89 1 2024-04-06T03:40:47.567 INFO:tasks.workunit.client.0.smithi094.stdout:9/792: unlink d2/dc/d21/d23/db0/fce 0 2024-04-06T03:40:47.570 INFO:tasks.workunit.client.1.smithi136.stdout:2/974: unlink d3/d15/d19/dac/l11c 0 2024-04-06T03:40:47.570 INFO:tasks.workunit.client.1.smithi136.stdout:2/975: stat d3/d15/d19 0 2024-04-06T03:40:47.573 INFO:tasks.workunit.client.1.smithi136.stdout:2/976: dread d3/fef [0,4194304] 0 2024-04-06T03:40:47.573 INFO:tasks.workunit.client.1.smithi136.stdout:2/977: chown d3/d4a/d9d/d18/f81 59224 1 2024-04-06T03:40:47.573 INFO:tasks.workunit.client.1.smithi136.stdout:2/978: chown d3/d4a/d9d/d18/d1d/f35 5986 1 2024-04-06T03:40:47.573 INFO:tasks.workunit.client.0.smithi094.stdout:1/857: dwrite d0/d1/d27/da5/d40/db1/d2f/d4b/fef [0,4194304] 0 2024-04-06T03:40:47.574 INFO:tasks.workunit.client.1.smithi136.stdout:9/944: dwrite d17/d21/d32/dc6/d113/d66/de1/d51/d57/f10b [0,4194304] 0 2024-04-06T03:40:47.574 INFO:tasks.workunit.client.1.smithi136.stdout:9/945: creat d17/d20/d25/da4/f142 x:0 0 0 2024-04-06T03:40:47.574 INFO:tasks.workunit.client.1.smithi136.stdout:9/946: truncate d17/d21/d29/de5/d4b/fab 661551 0 2024-04-06T03:40:47.575 INFO:tasks.workunit.client.1.smithi136.stdout:9/947: write d17/d19/da3/dc5/deb/f118 [270875,5978] 0 2024-04-06T03:40:47.575 INFO:tasks.workunit.client.1.smithi136.stdout:9/948: readlink d17/d19/da3/dc5/l10e 0 2024-04-06T03:40:47.575 INFO:tasks.workunit.client.1.smithi136.stdout:9/949: truncate d17/d20/d25/f100 995239 0 2024-04-06T03:40:47.575 INFO:tasks.workunit.client.0.smithi094.stdout:2/831: link d0/d8/d27/c82 d0/dd/d1b/d72/df3/c11b 0 2024-04-06T03:40:47.578 INFO:tasks.workunit.client.1.smithi136.stdout:5/972: rename d1/dc/d1c/d32/d8b/c3a to d1/d16/c125 0 2024-04-06T03:40:47.579 INFO:tasks.workunit.client.1.smithi136.stdout:1/922: symlink d12/d3b/d60/d7e/d37/dbb/l137 0 2024-04-06T03:40:47.581 INFO:tasks.workunit.client.0.smithi094.stdout:7/749: mknod d2/d6/dc/d12/d29/d3f/db9/cf2 0 2024-04-06T03:40:47.581 INFO:tasks.workunit.client.1.smithi136.stdout:4/794: sync 2024-04-06T03:40:47.582 INFO:tasks.workunit.client.1.smithi136.stdout:6/803: symlink d2/d7/d26/l111 0 2024-04-06T03:40:47.582 INFO:tasks.workunit.client.1.smithi136.stdout:5/973: dread d1/dc/d1c/d32/d8b/d20/f42 [0,4194304] 0 2024-04-06T03:40:47.582 INFO:tasks.workunit.client.1.smithi136.stdout:6/804: chown d2/d7/d26/d30/d6e/f69 622929 1 2024-04-06T03:40:47.582 INFO:tasks.workunit.client.1.smithi136.stdout:6/805: creat d2/d7/d9e/ddf/f112 x:0 0 0 2024-04-06T03:40:47.583 INFO:tasks.workunit.client.0.smithi094.stdout:3/843: creat d9/d20/d76/d6b/f12e x:0 0 0 2024-04-06T03:40:47.583 INFO:tasks.workunit.client.0.smithi094.stdout:3/844: fsync d9/f3f 0 2024-04-06T03:40:47.585 INFO:tasks.workunit.client.1.smithi136.stdout:8/956: link d3/da/d26/d35/dcb/d10d/f11a d3/da/d26/d35/d53/f131 0 2024-04-06T03:40:47.586 INFO:tasks.workunit.client.1.smithi136.stdout:8/957: fsync d3/de/d18/d7d/ddf/d6c/d7f/fbf 0 2024-04-06T03:40:47.587 INFO:tasks.workunit.client.0.smithi094.stdout:9/793: rename d2/dc/d80/db7/fe4 to d2/dc/d21/d23/d2f/f11a 0 2024-04-06T03:40:47.587 INFO:tasks.workunit.client.0.smithi094.stdout:9/794: write d2/dc/d21/d26/f3c [5432600,95338] 0 2024-04-06T03:40:47.588 INFO:tasks.workunit.client.1.smithi136.stdout:2/979: stat d3/d15/d19/d110/l12d 0 2024-04-06T03:40:47.588 INFO:tasks.workunit.client.1.smithi136.stdout:2/980: chown d3/d4a/d9d/l5b 468 1 2024-04-06T03:40:47.597 INFO:tasks.workunit.client.0.smithi094.stdout:7/750: dwrite d2/d6/dc/dd/df/f2d [0,4194304] 0 2024-04-06T03:40:47.600 INFO:tasks.workunit.client.0.smithi094.stdout:2/832: link d0/dd/d3d/d49/l90 d0/dd/d1b/de2/de7/l11c 0 2024-04-06T03:40:47.602 INFO:tasks.workunit.client.1.smithi136.stdout:3/891: rename d0/d4/d12/d19/d55/d6f/d82/d5d/dfc to d0/d4/d12/d2d/d37/d116/d12a 0 2024-04-06T03:40:47.603 INFO:tasks.workunit.client.0.smithi094.stdout:3/845: truncate d9/d20/d76/f83 452502 0 2024-04-06T03:40:47.608 INFO:tasks.workunit.client.0.smithi094.stdout:9/795: truncate d2/dc/d21/d23/d2f/f5f 6148337 0 2024-04-06T03:40:47.608 INFO:tasks.workunit.client.0.smithi094.stdout:9/796: truncate d2/dc/d46/f7e 1153620 0 2024-04-06T03:40:47.613 INFO:tasks.workunit.client.0.smithi094.stdout:2/833: unlink d0/d8/d27/lbc 0 2024-04-06T03:40:47.613 INFO:tasks.workunit.client.0.smithi094.stdout:2/834: fsync d0/dd/d19/d31/f3a 0 2024-04-06T03:40:47.613 INFO:tasks.workunit.client.0.smithi094.stdout:2/835: dread - d0/d8/d1d/d77/fd4 zero size 2024-04-06T03:40:47.615 INFO:tasks.workunit.client.0.smithi094.stdout:3/846: creat d9/d1e/d26/d6c/d8e/df8/f12f x:0 0 0 2024-04-06T03:40:47.618 INFO:tasks.workunit.client.1.smithi136.stdout:5/974: link d1/dc/d1c/cf9 d1/d16/d89/d8a/df1/d106/d122/c126 0 2024-04-06T03:40:47.618 INFO:tasks.workunit.client.1.smithi136.stdout:5/975: readlink d1/d16/d4b/l10a 0 2024-04-06T03:40:47.618 INFO:tasks.workunit.client.1.smithi136.stdout:4/795: dread d1d/d70/f5b [0,4194304] 0 2024-04-06T03:40:47.622 INFO:tasks.workunit.client.0.smithi094.stdout:9/797: rename d2/dc/d46/d111/d33/d64/dc6/c10b to d2/dc/d21/d26/d8a/c11b 0 2024-04-06T03:40:47.625 INFO:tasks.workunit.client.1.smithi136.stdout:8/958: creat d3/da/d26/d90/dae/db5/dec/f132 x:0 0 0 2024-04-06T03:40:47.626 INFO:tasks.workunit.client.0.smithi094.stdout:1/858: dwrite d0/d1/f10 [0,4194304] 0 2024-04-06T03:40:47.634 INFO:tasks.workunit.client.1.smithi136.stdout:0/889: rename d1/d4e/f10c to d1/d6/d2c/d1a/d36/d12a/f133 0 2024-04-06T03:40:47.639 INFO:tasks.workunit.client.1.smithi136.stdout:1/923: getdents d12/d1b/da7/db8 0 2024-04-06T03:40:47.639 INFO:tasks.workunit.client.0.smithi094.stdout:2/836: creat d0/dd/f11d x:0 0 0 2024-04-06T03:40:47.643 INFO:tasks.workunit.client.0.smithi094.stdout:3/847: rename d9/d10/f81 to d9/d20/d2d/d128/dba/dc5/f130 0 2024-04-06T03:40:47.643 INFO:tasks.workunit.client.0.smithi094.stdout:1/859: dwrite d0/d1/d27/da5/d40/d50/fbf [0,4194304] 0 2024-04-06T03:40:47.643 INFO:tasks.workunit.client.0.smithi094.stdout:1/860: write d0/d7/d1a/ff2 [884173,58154] 0 2024-04-06T03:40:47.645 INFO:tasks.workunit.client.1.smithi136.stdout:5/976: mknod d1/d98/c127 0 2024-04-06T03:40:47.646 INFO:tasks.workunit.client.0.smithi094.stdout:9/798: rmdir d2/dc/d80/ddd/d6b/d103 39 2024-04-06T03:40:47.647 INFO:tasks.workunit.client.1.smithi136.stdout:4/796: mknod d1d/c117 0 2024-04-06T03:40:47.648 INFO:tasks.workunit.client.1.smithi136.stdout:1/924: write d12/d53/f95 [1338524,42797] 0 2024-04-06T03:40:47.648 INFO:tasks.workunit.client.1.smithi136.stdout:6/806: getdents d2/d7/d13/d79/dc2 0 2024-04-06T03:40:47.651 INFO:tasks.workunit.client.1.smithi136.stdout:8/959: symlink d3/da/d26/l133 0 2024-04-06T03:40:47.653 INFO:tasks.workunit.client.0.smithi094.stdout:3/848: read d9/d10/fd8 [1532940,31354] 0 2024-04-06T03:40:47.653 INFO:tasks.workunit.client.0.smithi094.stdout:3/849: write d9/d20/d2d/d128/dba/d7c/f92 [880875,5225] 0 2024-04-06T03:40:47.653 INFO:tasks.workunit.client.0.smithi094.stdout:3/850: dread - d9/d9d/dc1/f10c zero size 2024-04-06T03:40:47.656 INFO:tasks.workunit.client.0.smithi094.stdout:2/837: creat d0/dd/d3d/d49/f11e x:0 0 0 2024-04-06T03:40:47.656 INFO:tasks.workunit.client.0.smithi094.stdout:2/838: fsync d0/dd/d3d/f88 0 2024-04-06T03:40:47.657 INFO:tasks.workunit.client.0.smithi094.stdout:2/839: fdatasync d0/dd/d19/d31/f3a 0 2024-04-06T03:40:47.657 INFO:tasks.workunit.client.1.smithi136.stdout:5/977: mkdir d1/d98/dc6/d128 0 2024-04-06T03:40:47.657 INFO:tasks.workunit.client.1.smithi136.stdout:5/978: chown d1/dc/d1c/d32/d8b/l2d 22094 1 2024-04-06T03:40:47.658 INFO:tasks.workunit.client.1.smithi136.stdout:4/797: dread d1d/d31/d3e/f71 [0,4194304] 0 2024-04-06T03:40:47.658 INFO:tasks.workunit.client.1.smithi136.stdout:4/798: fsync d1d/d31/f44 0 2024-04-06T03:40:47.659 INFO:tasks.workunit.client.1.smithi136.stdout:6/807: dread d2/d25/f56 [0,4194304] 0 2024-04-06T03:40:47.659 INFO:tasks.workunit.client.1.smithi136.stdout:0/890: dwrite d1/d6/d2c/d84/fd7 [4194304,4194304] 0 2024-04-06T03:40:47.659 INFO:tasks.workunit.client.1.smithi136.stdout:6/808: write d2/d7/d9e/ddf/f10e [686761,81160] 0 2024-04-06T03:40:47.659 INFO:tasks.workunit.client.1.smithi136.stdout:6/809: write d2/d104/fcb [1905666,113605] 0 2024-04-06T03:40:47.659 INFO:tasks.workunit.client.1.smithi136.stdout:6/810: write d2/d7/d9e/ddf/f112 [277614,108026] 0 2024-04-06T03:40:47.661 INFO:tasks.workunit.client.1.smithi136.stdout:1/925: symlink d12/d3b/d74/l138 0 2024-04-06T03:40:47.663 INFO:tasks.workunit.client.1.smithi136.stdout:8/960: mknod d3/da/d26/d35/d3d/dc7/dce/c134 0 2024-04-06T03:40:47.664 INFO:tasks.workunit.client.0.smithi094.stdout:2/840: write d0/d8/d27/d2b/f52 [2495232,89573] 0 2024-04-06T03:40:47.665 INFO:tasks.workunit.client.0.smithi094.stdout:2/841: truncate d0/dd/d1b/de2/de7/f116 366211 0 2024-04-06T03:40:47.665 INFO:tasks.workunit.client.0.smithi094.stdout:1/861: rename d0/d1/d27/da5/d40/db1/d2f/f77 to d0/d1/d27/da5/d40/db1/d2f/d3c/d48/dcb/d10c/f129 0 2024-04-06T03:40:47.668 INFO:tasks.workunit.client.1.smithi136.stdout:5/979: write d1/d16/d4b/f95 [1709088,43204] 0 2024-04-06T03:40:47.668 INFO:tasks.workunit.client.1.smithi136.stdout:5/980: truncate d1/dc/d10/d1e/f9d 297768 0 2024-04-06T03:40:47.669 INFO:tasks.workunit.client.1.smithi136.stdout:5/981: write d1/d16/d17/fd9 [799907,76926] 0 2024-04-06T03:40:47.669 INFO:tasks.workunit.client.1.smithi136.stdout:5/982: stat d1/dc 0 2024-04-06T03:40:47.671 INFO:tasks.workunit.client.0.smithi094.stdout:3/851: link d9/d1e/d6e/f107 d9/d10/dbb/de1/f131 0 2024-04-06T03:40:47.674 INFO:tasks.workunit.client.0.smithi094.stdout:1/862: fsync d0/d1/d27/da5/d40/db1/d2f/d3c/d48/dcb/d10c/f129 0 2024-04-06T03:40:47.676 INFO:tasks.workunit.client.0.smithi094.stdout:1/863: read d0/d1/d27/da5/d40/db1/d2f/d3c/da0/fdf [1508541,859] 0 2024-04-06T03:40:47.676 INFO:tasks.workunit.client.0.smithi094.stdout:1/864: chown d0/d7/fad 60622 1 2024-04-06T03:40:47.678 INFO:tasks.workunit.client.1.smithi136.stdout:2/981: rename d3/d9/d3b to d3/d15/d19/d43/daa/dec/d12c/d142 0 2024-04-06T03:40:47.678 INFO:tasks.workunit.client.1.smithi136.stdout:2/982: readlink d3/d4a/d9d/d18/df8/l120 0 2024-04-06T03:40:47.678 INFO:tasks.workunit.client.1.smithi136.stdout:2/983: creat d3/d15/d19/dac/f143 x:0 0 0 2024-04-06T03:40:47.678 INFO:tasks.workunit.client.1.smithi136.stdout:2/984: creat d3/d15/d19/dac/f144 x:0 0 0 2024-04-06T03:40:47.678 INFO:tasks.workunit.client.1.smithi136.stdout:0/891: dwrite d1/d6/d2c/d57/d43/d6d/d9b/d5c/fc7 [0,4194304] 0 2024-04-06T03:40:47.678 INFO:tasks.workunit.client.1.smithi136.stdout:0/892: write d1/d4e/fb0 [1342973,18103] 0 2024-04-06T03:40:47.678 INFO:tasks.workunit.client.1.smithi136.stdout:0/893: chown d1/d6/c110 51 1 2024-04-06T03:40:47.680 INFO:tasks.workunit.client.0.smithi094.stdout:9/799: rename d2/dc/d80/db7/fc7 to d2/dc/d46/d111/d33/d64/dc6/f11c 0 2024-04-06T03:40:47.681 INFO:tasks.workunit.client.0.smithi094.stdout:9/800: fsync d2/dc/d21/d26/f3c 0 2024-04-06T03:40:47.682 INFO:tasks.workunit.client.1.smithi136.stdout:4/799: mkdir d1d/d2f/d6c/da5/d118 0 2024-04-06T03:40:47.682 INFO:tasks.workunit.client.1.smithi136.stdout:8/961: dwrite d3/d2d/dad/ff3 [4194304,4194304] 0 2024-04-06T03:40:47.683 INFO:tasks.workunit.client.0.smithi094.stdout:3/852: write d9/d20/d30/f4d [5768878,29386] 0 2024-04-06T03:40:47.686 INFO:tasks.workunit.client.0.smithi094.stdout:1/865: creat d0/d1/d27/da5/d40/db1/d2f/d3c/d48/dcb/f12a x:0 0 0 2024-04-06T03:40:47.686 INFO:tasks.workunit.client.0.smithi094.stdout:1/866: readlink d0/d1/d27/da5/d40/d50/l51 0 2024-04-06T03:40:47.687 INFO:tasks.workunit.client.0.smithi094.stdout:2/842: dwrite d0/d8/d1d/d77/fd4 [0,4194304] 0 2024-04-06T03:40:47.687 INFO:tasks.workunit.client.1.smithi136.stdout:6/811: link d2/d7/d26/d30/d6e/c49 d2/d7/d26/d30/d6e/d3d/d47/c113 0 2024-04-06T03:40:47.690 INFO:tasks.workunit.client.1.smithi136.stdout:1/926: unlink d12/d3b/d74/d7a/dea/dd2/f12b 0 2024-04-06T03:40:47.690 INFO:tasks.workunit.client.1.smithi136.stdout:1/927: write d12/d1b/da7/daa/ff9 [634091,99030] 0 2024-04-06T03:40:47.693 INFO:tasks.workunit.client.1.smithi136.stdout:3/892: rename d0/d4/d12/d3a/d51/l63 to d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/l12b 0 2024-04-06T03:40:47.694 INFO:tasks.workunit.client.0.smithi094.stdout:1/867: symlink d0/d127/d30/d38/db9/l12b 0 2024-04-06T03:40:47.697 INFO:tasks.workunit.client.1.smithi136.stdout:2/985: dwrite d3/d4a/d9d/f6b [0,4194304] 0 2024-04-06T03:40:47.698 INFO:tasks.workunit.client.0.smithi094.stdout:2/843: rename d0/dd/d19/d31/c99 to d0/dd/d1b/d72/d8f/c11f 0 2024-04-06T03:40:47.698 INFO:tasks.workunit.client.1.smithi136.stdout:2/986: write d3/d15/d19/dac/f126 [239490,117002] 0 2024-04-06T03:40:47.698 INFO:tasks.workunit.client.1.smithi136.stdout:2/987: write d3/d4a/d9d/d8a/fb2 [5091305,16466] 0 2024-04-06T03:40:47.699 INFO:tasks.workunit.client.0.smithi094.stdout:9/801: dwrite d2/dc/d46/d111/d33/d64/dc6/f117 [0,4194304] 0 2024-04-06T03:40:47.700 INFO:tasks.workunit.client.0.smithi094.stdout:3/853: dwrite d9/d20/d2d/f67 [4194304,4194304] 0 2024-04-06T03:40:47.703 INFO:tasks.workunit.client.1.smithi136.stdout:0/894: getdents d1/d6/d2c/d57/d43/d6d/d9b/d5c/d96 0 2024-04-06T03:40:47.703 INFO:tasks.workunit.client.1.smithi136.stdout:0/895: readlink d1/d6/d2c/d1a/d36/d59/l24 0 2024-04-06T03:40:47.703 INFO:tasks.workunit.client.1.smithi136.stdout:0/896: write d1/d6/d2c/d1a/d36/fc2 [1124751,69809] 0 2024-04-06T03:40:47.703 INFO:tasks.workunit.client.1.smithi136.stdout:9/950: sync 2024-04-06T03:40:47.704 INFO:tasks.workunit.client.1.smithi136.stdout:9/951: read - d17/d20/d28/f120 zero size 2024-04-06T03:40:47.704 INFO:tasks.workunit.client.0.smithi094.stdout:9/802: chown d2/dc/d10/l11 0 1 2024-04-06T03:40:47.704 INFO:tasks.workunit.client.0.smithi094.stdout:8/864: sync 2024-04-06T03:40:47.704 INFO:tasks.workunit.client.0.smithi094.stdout:5/825: sync 2024-04-06T03:40:47.705 INFO:tasks.workunit.client.0.smithi094.stdout:4/925: sync 2024-04-06T03:40:47.706 INFO:tasks.workunit.client.1.smithi136.stdout:4/800: mknod d1d/d2f/d6c/c119 0 2024-04-06T03:40:47.709 INFO:tasks.workunit.client.0.smithi094.stdout:3/854: read d9/d20/f34 [813100,70441] 0 2024-04-06T03:40:47.709 INFO:tasks.workunit.client.0.smithi094.stdout:3/855: truncate d9/d20/f50 754960 0 2024-04-06T03:40:47.709 INFO:tasks.workunit.client.0.smithi094.stdout:3/856: readlink d9/d20/d2d/d128/dba/d97/l99 0 2024-04-06T03:40:47.710 INFO:tasks.workunit.client.1.smithi136.stdout:8/962: creat d3/de/d18/d7d/de0/f135 x:0 0 0 2024-04-06T03:40:47.710 INFO:tasks.workunit.client.1.smithi136.stdout:8/963: chown d3/de/d18/d7d/ddf/d6c/d7f/d105 5368880 1 2024-04-06T03:40:47.711 INFO:tasks.workunit.client.0.smithi094.stdout:9/803: stat d2/c25 0 2024-04-06T03:40:47.713 INFO:tasks.workunit.client.0.smithi094.stdout:2/844: dread d0/dd/d19/d31/f8c [4194304,4194304] 0 2024-04-06T03:40:47.713 INFO:tasks.workunit.client.0.smithi094.stdout:2/845: creat d0/d8/d4f/dab/f120 x:0 0 0 2024-04-06T03:40:47.713 INFO:tasks.workunit.client.0.smithi094.stdout:2/846: write d0/dd/d19/f81 [1198455,29476] 0 2024-04-06T03:40:47.715 INFO:tasks.workunit.client.1.smithi136.stdout:6/812: mkdir d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/d114 0 2024-04-06T03:40:47.715 INFO:tasks.workunit.client.1.smithi136.stdout:6/813: write d2/d7/d26/d30/d6e/db7/fbd [789526,125201] 0 2024-04-06T03:40:47.718 INFO:tasks.workunit.client.1.smithi136.stdout:1/928: mkdir d12/d3b/d44/d5f/d5a/d139 0 2024-04-06T03:40:47.718 INFO:tasks.workunit.client.1.smithi136.stdout:1/929: readlink d12/d3b/d74/d7a/dea/d11a/dd1/ld5 0 2024-04-06T03:40:47.719 INFO:tasks.workunit.client.0.smithi094.stdout:5/826: mkdir db/d7c/d8b/d122 0 2024-04-06T03:40:47.720 INFO:tasks.workunit.client.0.smithi094.stdout:2/847: dread d0/dd/d19/f4c [0,4194304] 0 2024-04-06T03:40:47.723 INFO:tasks.workunit.client.1.smithi136.stdout:5/983: rename d1/dc/d1c/d32/fc7 to d1/dc/d1c/d32/d8b/d20/d66/d113/f129 0 2024-04-06T03:40:47.723 INFO:tasks.workunit.client.0.smithi094.stdout:9/804: mkdir d2/dc/d46/d111/d33/d11d 0 2024-04-06T03:40:47.725 INFO:tasks.workunit.client.1.smithi136.stdout:1/930: read d12/d3b/d44/d5f/f7f [3963856,79627] 0 2024-04-06T03:40:47.726 INFO:tasks.workunit.client.0.smithi094.stdout:4/926: dread d0/df/d1e/d34/f39 [0,4194304] 0 2024-04-06T03:40:47.726 INFO:tasks.workunit.client.0.smithi094.stdout:3/857: dwrite d9/fd5 [0,4194304] 0 2024-04-06T03:40:47.732 INFO:tasks.workunit.client.0.smithi094.stdout:2/848: dread d0/dd/d19/d31/d67/d98/ddc/ff8 [4194304,4194304] 0 2024-04-06T03:40:47.732 INFO:tasks.workunit.client.0.smithi094.stdout:9/805: write d2/dc/d21/d23/db0/fe5 [2429826,126435] 0 2024-04-06T03:40:47.732 INFO:tasks.workunit.client.0.smithi094.stdout:9/806: truncate d2/dc/d21/d23/f7c 1251782 0 2024-04-06T03:40:47.733 INFO:tasks.workunit.client.0.smithi094.stdout:9/807: write d2/dc/d21/d26/f3c [4046668,54298] 0 2024-04-06T03:40:47.733 INFO:tasks.workunit.client.0.smithi094.stdout:5/827: creat db/d19/d21/d11e/d28/d85/f123 x:0 0 0 2024-04-06T03:40:47.734 INFO:tasks.workunit.client.0.smithi094.stdout:5/828: write db/d19/dc1/dd4/d77/fd8 [768337,31200] 0 2024-04-06T03:40:47.734 INFO:tasks.workunit.client.0.smithi094.stdout:5/829: chown db/d19/d30/d83/caa 94 1 2024-04-06T03:40:47.734 INFO:tasks.workunit.client.0.smithi094.stdout:5/830: chown db/d19/d21/f26 9 1 2024-04-06T03:40:47.734 INFO:tasks.workunit.client.0.smithi094.stdout:5/831: readlink db/d80/l107 0 2024-04-06T03:40:47.735 INFO:tasks.workunit.client.0.smithi094.stdout:4/927: symlink d0/df/l13b 0 2024-04-06T03:40:47.736 INFO:tasks.workunit.client.1.smithi136.stdout:4/801: dwrite d1d/d2f/da6/fcf [0,4194304] 0 2024-04-06T03:40:47.740 INFO:tasks.workunit.client.0.smithi094.stdout:8/865: dwrite d18/d30/f40 [0,4194304] 0 2024-04-06T03:40:47.741 INFO:tasks.workunit.client.1.smithi136.stdout:0/897: symlink d1/d6/d2c/d57/d43/dcf/de1/l134 0 2024-04-06T03:40:47.741 INFO:tasks.workunit.client.1.smithi136.stdout:0/898: creat d1/d6/d2c/d57/d43/d6d/d9b/d5c/f135 x:0 0 0 2024-04-06T03:40:47.741 INFO:tasks.workunit.client.1.smithi136.stdout:0/899: chown d1/d6/d2c/d57/d43/d6d/d85/d11a 118 1 2024-04-06T03:40:47.741 INFO:tasks.workunit.client.1.smithi136.stdout:0/900: write d1/d6/d2c/d57/d43/d6d/d85/da7/f126 [681639,23687] 0 2024-04-06T03:40:47.741 INFO:tasks.workunit.client.1.smithi136.stdout:0/901: truncate d1/d6/d2c/d1a/d36/ffd 932603 0 2024-04-06T03:40:47.745 INFO:tasks.workunit.client.0.smithi094.stdout:9/808: dwrite d2/dc/d21/d26/d78/d69/da8/f3a [0,4194304] 0 2024-04-06T03:40:47.748 INFO:tasks.workunit.client.0.smithi094.stdout:5/832: symlink db/d19/d21/l124 0 2024-04-06T03:40:47.748 INFO:tasks.workunit.client.0.smithi094.stdout:5/833: readlink db/d19/d30/l88 0 2024-04-06T03:40:47.749 INFO:tasks.workunit.client.1.smithi136.stdout:8/964: symlink d3/de/d18/d7d/ddf/dc9/l136 0 2024-04-06T03:40:47.749 INFO:tasks.workunit.client.1.smithi136.stdout:8/965: fdatasync d3/de/d18/d7d/ddf/d6c/d7f/fbf 0 2024-04-06T03:40:47.749 INFO:tasks.workunit.client.1.smithi136.stdout:8/966: chown d3/da/d26/d35/d53/d86/db0 9070 1 2024-04-06T03:40:47.750 INFO:tasks.workunit.client.0.smithi094.stdout:4/928: creat d0/d15/f13c x:0 0 0 2024-04-06T03:40:47.752 INFO:tasks.workunit.client.1.smithi136.stdout:6/814: mkdir d2/d7/d26/d97/d115 0 2024-04-06T03:40:47.754 INFO:tasks.workunit.client.0.smithi094.stdout:8/866: unlink d18/d3c/d8b/fe0 0 2024-04-06T03:40:47.756 INFO:tasks.workunit.client.1.smithi136.stdout:2/988: rename d3/d15/d19/d43/daa/dec/cfa to d3/d9/dde/c145 0 2024-04-06T03:40:47.756 INFO:tasks.workunit.client.1.smithi136.stdout:2/989: getdents d3/d15/d19/d43/d54 0 2024-04-06T03:40:47.756 INFO:tasks.workunit.client.1.smithi136.stdout:2/990: write d3/d15/d19/d43/daa/dec/d12c/d142/d87/db1/f138 [875063,3884] 0 2024-04-06T03:40:47.758 INFO:tasks.workunit.client.1.smithi136.stdout:5/984: truncate d1/d98/dbc/fc5 434669 0 2024-04-06T03:40:47.758 INFO:tasks.workunit.client.1.smithi136.stdout:5/985: fdatasync d1/d16/d4b/d7b/f83 0 2024-04-06T03:40:47.758 INFO:tasks.workunit.client.1.smithi136.stdout:5/986: fdatasync d1/d16/d89/d8a/f103 0 2024-04-06T03:40:47.760 INFO:tasks.workunit.client.0.smithi094.stdout:4/929: link d0/d4/fd d0/d4/d4b/f13d 0 2024-04-06T03:40:47.760 INFO:tasks.workunit.client.0.smithi094.stdout:4/930: fdatasync d0/d4/d4b/f13d 0 2024-04-06T03:40:47.760 INFO:tasks.workunit.client.0.smithi094.stdout:4/931: getdents d0/de8/d139 0 2024-04-06T03:40:47.760 INFO:tasks.workunit.client.0.smithi094.stdout:4/932: readlink d0/df/dbd/d107/l130 0 2024-04-06T03:40:47.760 INFO:tasks.workunit.client.0.smithi094.stdout:4/933: dread - d0/d15/d3a/d61/d81/fc0 zero size 2024-04-06T03:40:47.761 INFO:tasks.workunit.client.0.smithi094.stdout:4/934: truncate d0/d4/d67/d9f/fdb 439967 0 2024-04-06T03:40:47.761 INFO:tasks.workunit.client.1.smithi136.stdout:1/931: truncate d12/d3b/d74/d7a/dea/d11a/d4d/f51 3061498 0 2024-04-06T03:40:47.761 INFO:tasks.workunit.client.1.smithi136.stdout:1/932: creat d12/d3b/d74/d7a/f13a x:0 0 0 2024-04-06T03:40:47.762 INFO:tasks.workunit.client.0.smithi094.stdout:9/809: dwrite d2/dc/d21/d26/f72 [0,4194304] 0 2024-04-06T03:40:47.762 INFO:tasks.workunit.client.0.smithi094.stdout:8/867: dread d18/d3c/d49/d91/d94/daa/fe9 [0,4194304] 0 2024-04-06T03:40:47.762 INFO:tasks.workunit.client.0.smithi094.stdout:5/834: getdents db/d7c/dd7 0 2024-04-06T03:40:47.769 INFO:tasks.workunit.client.1.smithi136.stdout:4/802: mkdir d1d/d2f/d6c/da5/d11a 0 2024-04-06T03:40:47.770 INFO:tasks.workunit.client.1.smithi136.stdout:4/803: fdatasync d1d/f4f 0 2024-04-06T03:40:47.770 INFO:tasks.workunit.client.1.smithi136.stdout:4/804: write d1d/d31/d3e/d56/d7b/ff8 [857293,61327] 0 2024-04-06T03:40:47.770 INFO:tasks.workunit.client.1.smithi136.stdout:4/805: write d1d/d2f/fab [3692989,128777] 0 2024-04-06T03:40:47.770 INFO:tasks.workunit.client.0.smithi094.stdout:8/868: truncate d18/f82 991592 0 2024-04-06T03:40:47.770 INFO:tasks.workunit.client.0.smithi094.stdout:8/869: write d18/d3c/dc4/f104 [2179924,109432] 0 2024-04-06T03:40:47.772 INFO:tasks.workunit.client.0.smithi094.stdout:5/835: creat db/d19/d2c/d6d/de6/dfd/d10d/f125 x:0 0 0 2024-04-06T03:40:47.774 INFO:tasks.workunit.client.0.smithi094.stdout:9/810: dwrite d2/dc/d46/d111/d99/faf [0,4194304] 0 2024-04-06T03:40:47.774 INFO:tasks.workunit.client.0.smithi094.stdout:9/811: write d2/dc/d46/d111/d33/f4a [1685172,78532] 0 2024-04-06T03:40:47.775 INFO:tasks.workunit.client.1.smithi136.stdout:8/967: rmdir d3/de/d18/d7d/ddf/dc9 39 2024-04-06T03:40:47.777 INFO:tasks.workunit.client.1.smithi136.stdout:6/815: mkdir d2/d7/d26/dcd/dad/d116 0 2024-04-06T03:40:47.779 INFO:tasks.workunit.client.1.smithi136.stdout:9/952: rename d17/d19/da3/dc5/deb/d9e/dfb/f12e to d17/d21/d29/de5/da5/dce/dd1/f143 0 2024-04-06T03:40:47.780 INFO:tasks.workunit.client.0.smithi094.stdout:8/870: symlink d18/d3c/d49/d9f/d109/l129 0 2024-04-06T03:40:47.780 INFO:tasks.workunit.client.0.smithi094.stdout:8/871: write d18/ff9 [743562,54506] 0 2024-04-06T03:40:47.781 INFO:tasks.workunit.client.1.smithi136.stdout:2/991: mknod d3/d15/d19/c146 0 2024-04-06T03:40:47.781 INFO:tasks.workunit.client.0.smithi094.stdout:5/836: truncate db/f1e 1674709 0 2024-04-06T03:40:47.783 INFO:tasks.workunit.client.1.smithi136.stdout:5/987: mknod d1/dc/d1c/d32/d8b/d20/d7c/c12a 0 2024-04-06T03:40:47.783 INFO:tasks.workunit.client.1.smithi136.stdout:8/968: write d3/da/d26/f75 [2060377,105914] 0 2024-04-06T03:40:47.783 INFO:tasks.workunit.client.1.smithi136.stdout:8/969: fdatasync d3/de/d60/fb1 0 2024-04-06T03:40:47.784 INFO:tasks.workunit.client.1.smithi136.stdout:6/816: dread d2/d25/f38 [0,4194304] 0 2024-04-06T03:40:47.785 INFO:tasks.workunit.client.0.smithi094.stdout:9/812: rename d2/dc/d10/f66 to d2/dc/d21/dc8/f11e 0 2024-04-06T03:40:47.786 INFO:tasks.workunit.client.0.smithi094.stdout:9/813: read d2/dc/d46/d111/d33/f44 [3196500,113357] 0 2024-04-06T03:40:47.786 INFO:tasks.workunit.client.0.smithi094.stdout:9/814: fsync d2/dc/d21/d23/fd3 0 2024-04-06T03:40:47.786 INFO:tasks.workunit.client.0.smithi094.stdout:9/815: readlink d2/dc/d21/d26/d8a/laa 0 2024-04-06T03:40:47.787 INFO:tasks.workunit.client.1.smithi136.stdout:1/933: creat d12/d3b/d74/d7a/dea/d77/f13b x:0 0 0 2024-04-06T03:40:47.792 INFO:tasks.workunit.client.1.smithi136.stdout:8/970: write d3/da/fb [7445447,91961] 0 2024-04-06T03:40:47.792 INFO:tasks.workunit.client.1.smithi136.stdout:8/971: chown d3/de/d18/l91 47893 1 2024-04-06T03:40:47.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/816: dread d2/dc/d21/d23/d2f/f83 [0,4194304] 0 2024-04-06T03:40:47.793 INFO:tasks.workunit.client.0.smithi094.stdout:9/817: read - d2/dc/d21/d26/d78/d69/f10f zero size 2024-04-06T03:40:47.802 INFO:tasks.workunit.client.1.smithi136.stdout:1/934: dread d12/d3b/d74/d7a/dea/d11a/d4d/f51 [0,4194304] 0 2024-04-06T03:40:47.803 INFO:tasks.workunit.client.1.smithi136.stdout:1/935: write d12/d3b/d44/d5f/d5a/fce [939210,95429] 0 2024-04-06T03:40:47.803 INFO:tasks.workunit.client.1.smithi136.stdout:1/936: chown c5 47 1 2024-04-06T03:40:47.803 INFO:tasks.workunit.client.0.smithi094.stdout:9/818: dread d2/dc/d21/f43 [0,4194304] 0 2024-04-06T03:40:47.809 INFO:tasks.workunit.client.1.smithi136.stdout:1/937: write d12/d3b/d60/d7e/d37/f4e [4339819,76913] 0 2024-04-06T03:40:47.809 INFO:tasks.workunit.client.1.smithi136.stdout:1/938: stat d12/d1b/da7/ff6 0 2024-04-06T03:40:47.813 INFO:tasks.workunit.client.0.smithi094.stdout:9/819: dwrite d2/dc/d46/f7e [0,4194304] 0 2024-04-06T03:40:47.830 INFO:tasks.workunit.client.0.smithi094.stdout:9/820: dwrite d2/dc/d10/fb2 [0,4194304] 0 2024-04-06T03:40:47.831 INFO:tasks.workunit.client.0.smithi094.stdout:9/821: write d2/f7 [2011467,34827] 0 2024-04-06T03:40:47.843 INFO:tasks.workunit.client.0.smithi094.stdout:4/935: write d0/d4/dde/ff3 [216740,12574] 0 2024-04-06T03:40:47.846 INFO:tasks.workunit.client.0.smithi094.stdout:9/822: dwrite d2/dc/da6/f100 [0,4194304] 0 2024-04-06T03:40:47.851 INFO:tasks.workunit.client.1.smithi136.stdout:4/806: mkdir d1d/d2f/d6c/da5/d11b 0 2024-04-06T03:40:47.851 INFO:tasks.workunit.client.1.smithi136.stdout:4/807: chown d1d/d70/f94 3 1 2024-04-06T03:40:47.854 INFO:tasks.workunit.client.1.smithi136.stdout:0/902: getdents d1/d6/d2c/d57/d43/d6d/d9b/d5c 0 2024-04-06T03:40:47.854 INFO:tasks.workunit.client.1.smithi136.stdout:0/903: write d1/fc9 [8654449,78752] 0 2024-04-06T03:40:47.856 INFO:tasks.workunit.client.1.smithi136.stdout:9/953: mkdir d17/d21/d144 0 2024-04-06T03:40:47.857 INFO:tasks.workunit.client.0.smithi094.stdout:8/872: mkdir d18/d3c/d49/d9f/d12a 0 2024-04-06T03:40:47.857 INFO:tasks.workunit.client.0.smithi094.stdout:8/873: readlink d18/d34/d52/l57 0 2024-04-06T03:40:47.859 INFO:tasks.workunit.client.0.smithi094.stdout:4/936: dwrite d0/d4/d129/fa4 [0,4194304] 0 2024-04-06T03:40:47.861 INFO:tasks.workunit.client.1.smithi136.stdout:2/992: link d3/d15/d19/d43/daa/dec/d12c/d142/d87/lb0 d3/d15/d19/d43/daa/l147 0 2024-04-06T03:40:47.861 INFO:tasks.workunit.client.1.smithi136.stdout:2/993: dread - d3/d4a/f103 zero size 2024-04-06T03:40:47.863 INFO:tasks.workunit.client.1.smithi136.stdout:5/988: mkdir d1/d98/dc0/d11e/d12b 0 2024-04-06T03:40:47.863 INFO:tasks.workunit.client.1.smithi136.stdout:5/989: stat d1/d16/d4b/d7b 0 2024-04-06T03:40:47.863 INFO:tasks.workunit.client.1.smithi136.stdout:5/990: chown d1/dc/d10/d99 80 1 2024-04-06T03:40:47.863 INFO:tasks.workunit.client.1.smithi136.stdout:5/991: chown d1/dc/d1c/ce0 1 1 2024-04-06T03:40:47.863 INFO:tasks.workunit.client.1.smithi136.stdout:0/904: write d1/d6/d2c/d1a/d36/f5d [3247601,49756] 0 2024-04-06T03:40:47.863 INFO:tasks.workunit.client.1.smithi136.stdout:0/905: fdatasync d1/d6/d2c/d84/fd7 0 2024-04-06T03:40:47.865 INFO:tasks.workunit.client.0.smithi094.stdout:9/823: dwrite d2/dc/d46/d111/d99/f9e [0,4194304] 0 2024-04-06T03:40:47.866 INFO:tasks.workunit.client.0.smithi094.stdout:5/837: creat db/d7c/dd7/f126 x:0 0 0 2024-04-06T03:40:47.866 INFO:tasks.workunit.client.0.smithi094.stdout:5/838: write db/d19/d21/d11e/d28/d85/dd5/fe3 [137776,118281] 0 2024-04-06T03:40:47.866 INFO:tasks.workunit.client.0.smithi094.stdout:5/839: truncate db/d80/deb/f120 156230 0 2024-04-06T03:40:47.867 INFO:tasks.workunit.client.0.smithi094.stdout:5/840: read db/d80/f97 [2755536,35023] 0 2024-04-06T03:40:47.868 INFO:tasks.workunit.client.1.smithi136.stdout:6/817: symlink d2/d7/d26/d30/d6e/d58/d5d/l117 0 2024-04-06T03:40:47.869 INFO:tasks.workunit.client.0.smithi094.stdout:4/937: dread d0/df/d1e/f25 [0,4194304] 0 2024-04-06T03:40:47.869 INFO:tasks.workunit.client.0.smithi094.stdout:4/938: creat d0/d15/d3a/d61/d81/f13e x:0 0 0 2024-04-06T03:40:47.874 INFO:tasks.workunit.client.1.smithi136.stdout:5/992: dread d1/dc/d10/f11 [0,4194304] 0 2024-04-06T03:40:47.874 INFO:tasks.workunit.client.0.smithi094.stdout:8/874: dwrite d18/d2c/ffc [0,4194304] 0 2024-04-06T03:40:47.875 INFO:tasks.workunit.client.1.smithi136.stdout:8/972: rename d3/da/d26/d90/dae/db5 to d3/da/d26/d35/d53/dd2/d137 0 2024-04-06T03:40:47.876 INFO:tasks.workunit.client.1.smithi136.stdout:8/973: chown d3/da/d26/d35/d3d/dc7/dce/df2 29 1 2024-04-06T03:40:47.879 INFO:tasks.workunit.client.1.smithi136.stdout:1/939: mkdir d12/d3b/d44/d5f/d5a/d139/d13c 0 2024-04-06T03:40:47.881 INFO:tasks.workunit.client.1.smithi136.stdout:5/993: dread d1/dc/d1c/f56 [0,4194304] 0 2024-04-06T03:40:47.881 INFO:tasks.workunit.client.1.smithi136.stdout:3/893: sync 2024-04-06T03:40:47.883 INFO:tasks.workunit.client.1.smithi136.stdout:4/808: unlink d1d/d2f/d6c/da5/dc1/fcc 0 2024-04-06T03:40:47.886 INFO:tasks.workunit.client.1.smithi136.stdout:9/954: symlink d17/d21/d32/dc6/d113/d66/l145 0 2024-04-06T03:40:47.889 INFO:tasks.workunit.client.1.smithi136.stdout:6/818: dwrite d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/fa9 [0,4194304] 0 2024-04-06T03:40:47.892 INFO:tasks.workunit.client.1.smithi136.stdout:2/994: rename d3/d15/c7d to d3/d15/d19/d43/daa/dce/c148 0 2024-04-06T03:40:47.892 INFO:tasks.workunit.client.1.smithi136.stdout:2/995: chown d3/d15/d19/d43/daa/dec/d12c/d142/l55 336628 1 2024-04-06T03:40:47.898 INFO:tasks.workunit.client.1.smithi136.stdout:8/974: symlink d3/de/d18/d7d/ddf/d6c/d7f/l138 0 2024-04-06T03:40:47.898 INFO:tasks.workunit.client.1.smithi136.stdout:8/975: read - d3/da/d26/d90/dd5/f108 zero size 2024-04-06T03:40:47.898 INFO:tasks.workunit.client.1.smithi136.stdout:1/940: creat d12/d3b/d74/d7a/dea/d11a/d4d/f13d x:0 0 0 2024-04-06T03:40:47.899 INFO:tasks.workunit.client.1.smithi136.stdout:5/994: creat d1/dc/d10/da3/f12c x:0 0 0 2024-04-06T03:40:47.900 INFO:tasks.workunit.client.1.smithi136.stdout:5/995: read d1/d16/d17/fd9 [1385916,10705] 0 2024-04-06T03:40:47.901 INFO:tasks.workunit.client.1.smithi136.stdout:3/894: creat d0/d4/d12/d19/d55/d3c/d5a/d79/f12c x:0 0 0 2024-04-06T03:40:47.903 INFO:tasks.workunit.client.1.smithi136.stdout:3/895: dread d0/d4/d12/d2d/d37/f62 [0,4194304] 0 2024-04-06T03:40:47.904 INFO:tasks.workunit.client.1.smithi136.stdout:4/809: symlink d1d/d2f/d6c/l11c 0 2024-04-06T03:40:47.907 INFO:tasks.workunit.client.0.smithi094.stdout:9/824: unlink d2/dc/d80/fac 0 2024-04-06T03:40:47.907 INFO:tasks.workunit.client.0.smithi094.stdout:9/825: chown d2/dc/d21/d23/l116 474 1 2024-04-06T03:40:47.907 INFO:tasks.workunit.client.0.smithi094.stdout:9/826: truncate d2/f7b 2374686 0 2024-04-06T03:40:47.908 INFO:tasks.workunit.client.0.smithi094.stdout:9/827: read d2/dc/d21/d23/fe1 [67215,81081] 0 2024-04-06T03:40:47.909 INFO:tasks.workunit.client.1.smithi136.stdout:9/955: creat d17/d21/d32/dc6/d113/d66/de1/d51/d116/f146 x:0 0 0 2024-04-06T03:40:47.910 INFO:tasks.workunit.client.1.smithi136.stdout:9/956: stat d17/d20/d25/f26 0 2024-04-06T03:40:47.914 INFO:tasks.workunit.client.1.smithi136.stdout:4/810: dread d1d/d31/f3b [0,4194304] 0 2024-04-06T03:40:47.915 INFO:tasks.workunit.client.1.smithi136.stdout:6/819: symlink d2/d7/d26/d30/d6e/d58/d5d/d62/l118 0 2024-04-06T03:40:47.916 INFO:tasks.workunit.client.0.smithi094.stdout:4/939: symlink d0/df/db3/d11c/l13f 0 2024-04-06T03:40:47.916 INFO:tasks.workunit.client.0.smithi094.stdout:9/828: write d2/dc/d21/d23/db0/fbf [1417360,50790] 0 2024-04-06T03:40:47.918 INFO:tasks.workunit.client.1.smithi136.stdout:0/906: mkdir d1/d6/df5/d136 0 2024-04-06T03:40:47.920 INFO:tasks.workunit.client.1.smithi136.stdout:2/996: creat d3/d4a/d9d/d18/d4b/f149 x:0 0 0 2024-04-06T03:40:47.920 INFO:tasks.workunit.client.1.smithi136.stdout:2/997: creat d3/d15/d19/d43/daa/dec/f14a x:0 0 0 2024-04-06T03:40:47.922 INFO:tasks.workunit.client.1.smithi136.stdout:8/976: link d3/da/d26/d35/d53/l9f d3/da/d26/d35/d53/d86/l139 0 2024-04-06T03:40:47.923 INFO:tasks.workunit.client.1.smithi136.stdout:3/896: dwrite d0/d4/d12/d19/d55/d3c/d46/f78 [0,4194304] 0 2024-04-06T03:40:47.923 INFO:tasks.workunit.client.1.smithi136.stdout:3/897: write d0/d4/d12/d19/d55/d3c/d5a/f101 [328962,44140] 0 2024-04-06T03:40:47.923 INFO:tasks.workunit.client.1.smithi136.stdout:3/898: write d0/d4/d12/d19/d55/d3c/d5a/d79/d98/fcc [668232,72268] 0 2024-04-06T03:40:47.924 INFO:tasks.workunit.client.1.smithi136.stdout:1/941: mkdir d12/d101/d13e 0 2024-04-06T03:40:47.926 INFO:tasks.workunit.client.1.smithi136.stdout:5/996: link d1/dc/d10/d1e/c47 d1/d9b/dea/c12d 0 2024-04-06T03:40:47.926 INFO:tasks.workunit.client.0.smithi094.stdout:0/849: sync 2024-04-06T03:40:47.926 INFO:tasks.workunit.client.0.smithi094.stdout:6/796: sync 2024-04-06T03:40:47.927 INFO:tasks.workunit.client.0.smithi094.stdout:0/850: fdatasync d1/d9/dd/d18/d5d/d60/fc1 0 2024-04-06T03:40:47.927 INFO:tasks.workunit.client.0.smithi094.stdout:0/851: chown d1/d9/dd/d18/d1a/ld0 474300 1 2024-04-06T03:40:47.927 INFO:tasks.workunit.client.1.smithi136.stdout:9/957: readlink d17/d19/da3/dc5/deb/d9e/lb5 0 2024-04-06T03:40:47.928 INFO:tasks.workunit.client.1.smithi136.stdout:1/942: dread d12/d1b/d22/fb0 [0,4194304] 0 2024-04-06T03:40:47.929 INFO:tasks.workunit.client.1.smithi136.stdout:4/811: mknod d1d/d31/d3e/d55/d89/c11d 0 2024-04-06T03:40:47.930 INFO:tasks.workunit.client.1.smithi136.stdout:6/820: creat d2/d7/d26/f119 x:0 0 0 2024-04-06T03:40:47.932 INFO:tasks.workunit.client.1.smithi136.stdout:0/907: unlink d1/d6/d2c/d1a/l3c 0 2024-04-06T03:40:47.933 INFO:tasks.workunit.client.0.smithi094.stdout:9/829: rmdir d2/dc/d80/ddd/d6b/d103 39 2024-04-06T03:40:47.933 INFO:tasks.workunit.client.0.smithi094.stdout:9/830: readlink d2/dc/d21/d23/d47/d51/dd8/le0 0 2024-04-06T03:40:47.936 INFO:tasks.workunit.client.1.smithi136.stdout:2/998: creat d3/d4a/d9d/f14b x:0 0 0 2024-04-06T03:40:47.938 INFO:tasks.workunit.client.0.smithi094.stdout:8/875: mknod d18/d3c/d49/d80/c12b 0 2024-04-06T03:40:47.939 INFO:tasks.workunit.client.0.smithi094.stdout:8/876: write d18/d3c/d49/d80/fea [41796,27007] 0 2024-04-06T03:40:47.939 INFO:tasks.workunit.client.1.smithi136.stdout:8/977: creat d3/da/f13a x:0 0 0 2024-04-06T03:40:47.943 INFO:tasks.workunit.client.0.smithi094.stdout:6/797: dwrite dc/d3a/fc8 [0,4194304] 0 2024-04-06T03:40:47.944 INFO:tasks.workunit.client.0.smithi094.stdout:0/852: unlink d1/d9/dd/d18/d1a/d44/l8b 0 2024-04-06T03:40:47.947 INFO:tasks.workunit.client.0.smithi094.stdout:9/831: mknod d2/dc/c11f 0 2024-04-06T03:40:47.948 INFO:tasks.workunit.client.0.smithi094.stdout:9/832: dread - d2/dc/d80/db7/dd0/feb zero size 2024-04-06T03:40:47.948 INFO:tasks.workunit.client.0.smithi094.stdout:9/833: readlink d2/dc/d10/l13 0 2024-04-06T03:40:47.948 INFO:tasks.workunit.client.0.smithi094.stdout:9/834: chown d2/dc/d80 55 1 2024-04-06T03:40:47.949 INFO:tasks.workunit.client.1.smithi136.stdout:3/899: symlink d0/d4/d12/dee/l12d 0 2024-04-06T03:40:47.951 INFO:tasks.workunit.client.0.smithi094.stdout:8/877: mknod d18/d34/c12c 0 2024-04-06T03:40:47.954 INFO:tasks.workunit.client.1.smithi136.stdout:5/997: truncate d1/dc/d10/d1e/f2f 4891396 0 2024-04-06T03:40:47.957 INFO:tasks.workunit.client.1.smithi136.stdout:3/900: dread d0/f2 [0,4194304] 0 2024-04-06T03:40:47.959 INFO:tasks.workunit.client.1.smithi136.stdout:9/958: rename d17/d21/d29/de5/f11f to d17/d20/d25/da4/dbb/de4/f147 0 2024-04-06T03:40:47.961 INFO:tasks.workunit.client.0.smithi094.stdout:0/853: dwrite d1/f7a [4194304,4194304] 0 2024-04-06T03:40:47.968 INFO:tasks.workunit.client.1.smithi136.stdout:8/978: dwrite d3/de/d18/d7d/ddf/f66 [4194304,4194304] 0 2024-04-06T03:40:47.970 INFO:tasks.workunit.client.1.smithi136.stdout:1/943: mkdir d12/d1b/d28/d13f 0 2024-04-06T03:40:47.970 INFO:tasks.workunit.client.0.smithi094.stdout:7/751: sync 2024-04-06T03:40:47.970 INFO:tasks.workunit.client.0.smithi094.stdout:1/868: sync 2024-04-06T03:40:47.970 INFO:tasks.workunit.client.0.smithi094.stdout:7/752: chown d2/d6/dc/dd/df/f1a 8289950 1 2024-04-06T03:40:47.970 INFO:tasks.workunit.client.0.smithi094.stdout:1/869: read d0/d1/d27/da5/d40/db1/d2f/d4b/fa1 [460064,79462] 0 2024-04-06T03:40:47.973 INFO:tasks.workunit.client.0.smithi094.stdout:6/798: getdents dc/d3a/d4a/d94/dac 0 2024-04-06T03:40:47.973 INFO:tasks.workunit.client.0.smithi094.stdout:6/799: write dc/d3a/d4a/d94/f109 [1992602,86725] 0 2024-04-06T03:40:47.974 INFO:tasks.workunit.client.1.smithi136.stdout:0/908: truncate d1/d6/d2c/d57/d43/d6d/d9b/d5c/fc7 506114 0 2024-04-06T03:40:47.977 INFO:tasks.workunit.client.1.smithi136.stdout:2/999: truncate d3/d4a/d9d/d18/d1d/d3c/f51 1294806 0 2024-04-06T03:40:47.978 INFO:tasks.workunit.client.0.smithi094.stdout:0/854: dread d1/d9/dd/fa2 [0,4194304] 0 2024-04-06T03:40:47.980 INFO:tasks.workunit.client.1.smithi136.stdout:5/998: dwrite d1/f96 [0,4194304] 0 2024-04-06T03:40:47.980 INFO:tasks.workunit.client.1.smithi136.stdout:5/999: read - d1/dc/d1c/d32/d8b/d20/fef zero size 2024-04-06T03:40:47.980 INFO:tasks.workunit.client.1.smithi136.stdout:3/901: dwrite d0/f23 [0,4194304] 0 2024-04-06T03:40:47.984 INFO:tasks.workunit.client.0.smithi094.stdout:7/753: mknod d2/d6/d93/cf3 0 2024-04-06T03:40:47.987 INFO:tasks.workunit.client.1.smithi136.stdout:6/821: rename d2/d7/d13/d79/dc2/ddd/fe0 to d2/d7/d26/d30/dc4/f11a 0 2024-04-06T03:40:47.989 INFO:tasks.workunit.client.0.smithi094.stdout:1/870: rename d0/d1/d27/da5/d40/db1/dc3/d5f/la3 to d0/d1/d27/da5/db2/l12c 0 2024-04-06T03:40:47.992 INFO:tasks.workunit.client.1.smithi136.stdout:8/979: dwrite d3/de/fd7 [0,4194304] 0 2024-04-06T03:40:47.992 INFO:tasks.workunit.client.0.smithi094.stdout:9/835: stat d2/dc/d21/d26/d78/d69/da8/c39 0 2024-04-06T03:40:47.992 INFO:tasks.workunit.client.0.smithi094.stdout:9/836: creat d2/dc/d80/ddd/d6b/f120 x:0 0 0 2024-04-06T03:40:47.995 INFO:tasks.workunit.client.0.smithi094.stdout:6/800: dwrite dc/d3a/d4a/d94/f104 [0,4194304] 0 2024-04-06T03:40:48.003 INFO:tasks.workunit.client.0.smithi094.stdout:8/878: getdents d18/d2c 0 2024-04-06T03:40:48.003 INFO:tasks.workunit.client.0.smithi094.stdout:8/879: stat d18/d2c/d37/d60/da3/deb/ffa 0 2024-04-06T03:40:48.003 INFO:tasks.workunit.client.0.smithi094.stdout:8/880: write d18/d3c/d49/db5/fc2 [1445092,48477] 0 2024-04-06T03:40:48.004 INFO:tasks.workunit.client.0.smithi094.stdout:1/871: dwrite d0/d7/f86 [0,4194304] 0 2024-04-06T03:40:48.007 INFO:tasks.workunit.client.1.smithi136.stdout:8/980: dread d3/d2d/d5f/fb4 [0,4194304] 0 2024-04-06T03:40:48.007 INFO:tasks.workunit.client.1.smithi136.stdout:6/822: read d2/d7/d26/d30/d6e/d58/d5d/f65 [4099020,96001] 0 2024-04-06T03:40:48.008 INFO:tasks.workunit.client.0.smithi094.stdout:7/754: creat d2/d6/dc/d12/d3a/dcc/ff4 x:0 0 0 2024-04-06T03:40:48.009 INFO:tasks.workunit.client.1.smithi136.stdout:1/944: symlink d12/d3b/d60/d7e/d37/l140 0 2024-04-06T03:40:48.009 INFO:tasks.workunit.client.1.smithi136.stdout:1/945: chown d12/d1b/d22/fb0 944756 1 2024-04-06T03:40:48.009 INFO:tasks.workunit.client.1.smithi136.stdout:1/946: write d12/d3b/d44/d5f/d5a/f62 [3838099,90717] 0 2024-04-06T03:40:48.009 INFO:tasks.workunit.client.1.smithi136.stdout:1/947: dread - d12/d3b/d44/f110 zero size 2024-04-06T03:40:48.013 INFO:tasks.workunit.client.1.smithi136.stdout:4/812: getdents d1d/d2f/da6 0 2024-04-06T03:40:48.013 INFO:tasks.workunit.client.1.smithi136.stdout:4/813: creat d1d/d31/d3e/d55/f11e x:0 0 0 2024-04-06T03:40:48.015 INFO:tasks.workunit.client.0.smithi094.stdout:8/881: write d18/f74 [64252,116079] 0 2024-04-06T03:40:48.015 INFO:tasks.workunit.client.0.smithi094.stdout:8/882: write d18/d30/f3e [4519104,116223] 0 2024-04-06T03:40:48.017 INFO:tasks.workunit.client.1.smithi136.stdout:0/909: mkdir d1/d6/d2c/d57/d43/d6d/d9b/d5c/d96/d137 0 2024-04-06T03:40:48.019 INFO:tasks.workunit.client.0.smithi094.stdout:9/837: creat d2/dc/d21/d23/d47/f121 x:0 0 0 2024-04-06T03:40:48.020 INFO:tasks.workunit.client.0.smithi094.stdout:0/855: dwrite d1/d9/dd/d18/d1e/fbb [0,4194304] 0 2024-04-06T03:40:48.020 INFO:tasks.workunit.client.0.smithi094.stdout:0/856: write d1/d9/f90 [442344,76242] 0 2024-04-06T03:40:48.023 INFO:tasks.workunit.client.1.smithi136.stdout:3/902: mkdir d0/d4/d12/d19/d115/dc9/d12e 0 2024-04-06T03:40:48.028 INFO:tasks.workunit.client.0.smithi094.stdout:8/883: dwrite d18/d3c/d49/d91/d94/fce [0,4194304] 0 2024-04-06T03:40:48.029 INFO:tasks.workunit.client.0.smithi094.stdout:8/884: readlink d18/la7 0 2024-04-06T03:40:48.029 INFO:tasks.workunit.client.0.smithi094.stdout:8/885: write d18/d2c/d37/d60/da3/f11f [2908127,10342] 0 2024-04-06T03:40:48.029 INFO:tasks.workunit.client.0.smithi094.stdout:8/886: dread - d18/d2c/d37/d60/d70/fee zero size 2024-04-06T03:40:48.029 INFO:tasks.workunit.client.0.smithi094.stdout:8/887: fdatasync d18/d34/f3b 0 2024-04-06T03:40:48.031 INFO:tasks.workunit.client.0.smithi094.stdout:6/801: symlink dc/d3a/d7f/df0/l118 0 2024-04-06T03:40:48.031 INFO:tasks.workunit.client.0.smithi094.stdout:6/802: dread - dc/d3a/df4/df6/ff9 zero size 2024-04-06T03:40:48.031 INFO:tasks.workunit.client.0.smithi094.stdout:8/888: dread d18/d3c/d49/db5/fc2 [0,4194304] 0 2024-04-06T03:40:48.033 INFO:tasks.workunit.client.0.smithi094.stdout:6/803: dread dc/d3a/df4/df6/fe2 [0,4194304] 0 2024-04-06T03:40:48.038 INFO:tasks.workunit.client.1.smithi136.stdout:8/981: symlink d3/da/d26/d35/d53/dd2/d137/dec/l13b 0 2024-04-06T03:40:48.040 INFO:tasks.workunit.client.0.smithi094.stdout:8/889: read d18/f22 [4511056,44767] 0 2024-04-06T03:40:48.040 INFO:tasks.workunit.client.0.smithi094.stdout:8/890: chown d18/d2c/d37/d60/d70/cb6 62 1 2024-04-06T03:40:48.040 INFO:tasks.workunit.client.0.smithi094.stdout:8/891: fdatasync d18/d2c/d37/d60/da3/db8/f102 0 2024-04-06T03:40:48.041 INFO:tasks.workunit.client.1.smithi136.stdout:6/823: creat d2/d7/d26/d97/f11b x:0 0 0 2024-04-06T03:40:48.045 INFO:tasks.workunit.client.0.smithi094.stdout:6/804: dwrite dc/d3a/d4a/d54/fb2 [0,4194304] 0 2024-04-06T03:40:48.046 INFO:tasks.workunit.client.0.smithi094.stdout:1/872: rename d0/d7/f46 to d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/f12d 0 2024-04-06T03:40:48.046 INFO:tasks.workunit.client.0.smithi094.stdout:1/873: fsync d0/d1/f116 0 2024-04-06T03:40:48.047 INFO:tasks.workunit.client.1.smithi136.stdout:1/948: creat d12/d1b/d28/d13f/f141 x:0 0 0 2024-04-06T03:40:48.048 INFO:tasks.workunit.client.0.smithi094.stdout:7/755: symlink d2/d6/dc/dd/df/d95/dc9/lf5 0 2024-04-06T03:40:48.050 INFO:tasks.workunit.client.0.smithi094.stdout:7/756: dread d2/d6/f8 [0,4194304] 0 2024-04-06T03:40:48.051 INFO:tasks.workunit.client.0.smithi094.stdout:9/838: getdents d2/dc/d21/d23/d5b 0 2024-04-06T03:40:48.051 INFO:tasks.workunit.client.0.smithi094.stdout:9/839: write d2/dc/d21/d23/d5b/f89 [3025454,6941] 0 2024-04-06T03:40:48.053 INFO:tasks.workunit.client.1.smithi136.stdout:1/949: dread d12/d3b/f84 [0,4194304] 0 2024-04-06T03:40:48.053 INFO:tasks.workunit.client.1.smithi136.stdout:1/950: readlink d12/d3b/d44/d5f/l8c 0 2024-04-06T03:40:48.053 INFO:tasks.workunit.client.1.smithi136.stdout:1/951: creat d12/d3b/d74/d7a/dea/d11a/dd1/f142 x:0 0 0 2024-04-06T03:40:48.053 INFO:tasks.workunit.client.1.smithi136.stdout:1/952: stat d12/d3b/d44/d5f/f7f 0 2024-04-06T03:40:48.055 INFO:tasks.workunit.client.1.smithi136.stdout:0/910: creat d1/d6/df5/f138 x:0 0 0 2024-04-06T03:40:48.055 INFO:tasks.workunit.client.1.smithi136.stdout:0/911: write d1/d6/d2c/f87 [3831925,44997] 0 2024-04-06T03:40:48.057 INFO:tasks.workunit.client.1.smithi136.stdout:8/982: creat d3/de/d18/d6d/f13c x:0 0 0 2024-04-06T03:40:48.058 INFO:tasks.workunit.client.1.smithi136.stdout:8/983: chown d3/da/d26/d90/fe7 3 1 2024-04-06T03:40:48.058 INFO:tasks.workunit.client.0.smithi094.stdout:3/858: sync 2024-04-06T03:40:48.058 INFO:tasks.workunit.client.0.smithi094.stdout:0/857: mkdir d1/d9/dd/d120 0 2024-04-06T03:40:48.060 INFO:tasks.workunit.client.1.smithi136.stdout:6/824: mkdir d2/d7/d26/d30/d6e/d11c 0 2024-04-06T03:40:48.062 INFO:tasks.workunit.client.1.smithi136.stdout:4/814: getdents d1d/d31/d3e/d56 0 2024-04-06T03:40:48.062 INFO:tasks.workunit.client.1.smithi136.stdout:4/815: fdatasync d1d/d2f/da6/dbf/fe4 0 2024-04-06T03:40:48.069 INFO:tasks.workunit.client.1.smithi136.stdout:1/953: symlink d12/l143 0 2024-04-06T03:40:48.070 INFO:tasks.workunit.client.0.smithi094.stdout:8/892: mknod d18/d2c/dda/c12d 0 2024-04-06T03:40:48.071 INFO:tasks.workunit.client.1.smithi136.stdout:9/959: sync 2024-04-06T03:40:48.071 INFO:tasks.workunit.client.1.smithi136.stdout:9/960: write d17/d21/d32/dc6/d113/d66/de1/d51/f6d [705714,68347] 0 2024-04-06T03:40:48.071 INFO:tasks.workunit.client.1.smithi136.stdout:9/961: fdatasync d17/d21/d32/dc6/d113/d46/d9a/ff9 0 2024-04-06T03:40:48.071 INFO:tasks.workunit.client.1.smithi136.stdout:9/962: fdatasync fa 0 2024-04-06T03:40:48.072 INFO:tasks.workunit.client.1.smithi136.stdout:9/963: creat d17/d20/d28/d49/d75/d60/f148 x:0 0 0 2024-04-06T03:40:48.072 INFO:tasks.workunit.client.1.smithi136.stdout:6/825: creat d2/d7/d26/d30/d6e/d11c/f11d x:0 0 0 2024-04-06T03:40:48.076 INFO:tasks.workunit.client.0.smithi094.stdout:7/757: dwrite d2/d6/dc/d12/d9a/da0/fc3 [0,4194304] 0 2024-04-06T03:40:48.077 INFO:tasks.workunit.client.1.smithi136.stdout:1/954: mknod d12/d3b/d74/d7a/dea/d11a/d9f/c144 0 2024-04-06T03:40:48.077 INFO:tasks.workunit.client.1.smithi136.stdout:1/955: write d12/d3b/d60/d7e/f12d [650941,76325] 0 2024-04-06T03:40:48.079 INFO:tasks.workunit.client.0.smithi094.stdout:1/874: dwrite d0/d1/dd3/f10d [0,4194304] 0 2024-04-06T03:40:48.083 INFO:tasks.workunit.client.1.smithi136.stdout:8/984: dread d3/de/f3a [0,4194304] 0 2024-04-06T03:40:48.083 INFO:tasks.workunit.client.1.smithi136.stdout:0/912: dwrite d1/f5e [0,4194304] 0 2024-04-06T03:40:48.083 INFO:tasks.workunit.client.1.smithi136.stdout:8/985: chown d3/de/d18/d7d/l123 0 1 2024-04-06T03:40:48.083 INFO:tasks.workunit.client.1.smithi136.stdout:0/913: write d1/d4e/f75 [5561043,71039] 0 2024-04-06T03:40:48.083 INFO:tasks.workunit.client.1.smithi136.stdout:0/914: chown d1/d6/d2c/d1a/d36/d59/lce 32434 1 2024-04-06T03:40:48.085 INFO:tasks.workunit.client.0.smithi094.stdout:6/805: truncate dc/dd3/f74 1208942 0 2024-04-06T03:40:48.086 INFO:tasks.workunit.client.1.smithi136.stdout:8/986: mknod d3/da/d26/d35/dcb/d10d/c13d 0 2024-04-06T03:40:48.088 INFO:tasks.workunit.client.1.smithi136.stdout:9/964: dread d17/d21/d32/dc6/d113/d66/de1/d51/d57/f10b [0,4194304] 0 2024-04-06T03:40:48.088 INFO:tasks.workunit.client.1.smithi136.stdout:9/965: read - d17/d20/d25/da4/dbb/de4/f147 zero size 2024-04-06T03:40:48.090 INFO:tasks.workunit.client.1.smithi136.stdout:0/915: symlink d1/d6/d2c/d1a/d36/d59/l139 0 2024-04-06T03:40:48.092 INFO:tasks.workunit.client.0.smithi094.stdout:9/840: rmdir d2/d112 0 2024-04-06T03:40:48.094 INFO:tasks.workunit.client.1.smithi136.stdout:4/816: rename d1d/d2f/d6c/d91 to d1d/d2f/da6/d11f 0 2024-04-06T03:40:48.094 INFO:tasks.workunit.client.1.smithi136.stdout:4/817: stat d1d/d2f/f43 0 2024-04-06T03:40:48.094 INFO:tasks.workunit.client.1.smithi136.stdout:4/818: chown d1d/d31/d3e/l63 0 1 2024-04-06T03:40:48.095 INFO:tasks.workunit.client.0.smithi094.stdout:0/858: mknod d1/d9/dd/d18/d32/c121 0 2024-04-06T03:40:48.095 INFO:tasks.workunit.client.0.smithi094.stdout:0/859: truncate d1/f2c 5248654 0 2024-04-06T03:40:48.095 INFO:tasks.workunit.client.0.smithi094.stdout:0/860: chown d1/d9/dd/d18/d32/d42/cf1 2234129 1 2024-04-06T03:40:48.097 INFO:tasks.workunit.client.1.smithi136.stdout:9/966: mkdir d17/d21/d149 0 2024-04-06T03:40:48.098 INFO:tasks.workunit.client.1.smithi136.stdout:6/826: dwrite d2/d7/d26/d97/f11b [0,4194304] 0 2024-04-06T03:40:48.098 INFO:tasks.workunit.client.1.smithi136.stdout:6/827: truncate d2/d7/d26/dcd/dad/fbb 1110203 0 2024-04-06T03:40:48.098 INFO:tasks.workunit.client.1.smithi136.stdout:9/967: write d17/d20/d25/f26 [6637255,83721] 0 2024-04-06T03:40:48.103 INFO:tasks.workunit.client.1.smithi136.stdout:1/956: dwrite d12/d3b/d44/d5f/f7f [4194304,4194304] 0 2024-04-06T03:40:48.103 INFO:tasks.workunit.client.1.smithi136.stdout:0/916: creat d1/d6/d2c/d57/d43/d6b/f13a x:0 0 0 2024-04-06T03:40:48.103 INFO:tasks.workunit.client.0.smithi094.stdout:9/841: dread d2/dc/d21/d23/db0/fe5 [0,4194304] 0 2024-04-06T03:40:48.103 INFO:tasks.workunit.client.0.smithi094.stdout:9/842: readlink d2/l3 0 2024-04-06T03:40:48.104 INFO:tasks.workunit.client.0.smithi094.stdout:9/843: write d2/f7b [2986956,30318] 0 2024-04-06T03:40:48.104 INFO:tasks.workunit.client.0.smithi094.stdout:9/844: fdatasync d2/dc/d10/ff4 0 2024-04-06T03:40:48.104 INFO:tasks.workunit.client.0.smithi094.stdout:3/859: creat d9/d1d/d7d/dd7/f132 x:0 0 0 2024-04-06T03:40:48.104 INFO:tasks.workunit.client.0.smithi094.stdout:3/860: chown d9/d10/f22 323171 1 2024-04-06T03:40:48.104 INFO:tasks.workunit.client.0.smithi094.stdout:3/861: write d9/d20/d2d/f115 [1031882,102974] 0 2024-04-06T03:40:48.104 INFO:tasks.workunit.client.0.smithi094.stdout:8/893: mknod d18/d3c/d49/d91/d94/daa/c12e 0 2024-04-06T03:40:48.104 INFO:tasks.workunit.client.0.smithi094.stdout:0/861: dread d1/f7a [4194304,4194304] 0 2024-04-06T03:40:48.105 INFO:tasks.workunit.client.1.smithi136.stdout:4/819: truncate d1d/d31/d3e/d56/d7b/fd5 212364 0 2024-04-06T03:40:48.105 INFO:tasks.workunit.client.0.smithi094.stdout:7/758: symlink d2/d6/dc/dd/df/d95/lf6 0 2024-04-06T03:40:48.107 INFO:tasks.workunit.client.1.smithi136.stdout:8/987: dwrite d3/d2d/d5f/f12b [0,4194304] 0 2024-04-06T03:40:48.107 INFO:tasks.workunit.client.1.smithi136.stdout:8/988: write d3/de/d18/d6d/f7b [792384,39709] 0 2024-04-06T03:40:48.107 INFO:tasks.workunit.client.1.smithi136.stdout:8/989: chown d3/da/d26/d35/d53/f114 2737 1 2024-04-06T03:40:48.108 INFO:tasks.workunit.client.1.smithi136.stdout:6/828: stat d2/d7/lc 0 2024-04-06T03:40:48.108 INFO:tasks.workunit.client.1.smithi136.stdout:6/829: write d2/d7/d26/d30/d6e/d58/fbe [1400567,31814] 0 2024-04-06T03:40:48.113 INFO:tasks.workunit.client.1.smithi136.stdout:9/968: dwrite d17/d21/d32/dc6/d113/f11a [0,4194304] 0 2024-04-06T03:40:48.113 INFO:tasks.workunit.client.1.smithi136.stdout:9/969: write d17/d20/d28/f9c [823401,122078] 0 2024-04-06T03:40:48.113 INFO:tasks.workunit.client.1.smithi136.stdout:9/970: chown c9 1228 1 2024-04-06T03:40:48.114 INFO:tasks.workunit.client.1.smithi136.stdout:1/957: unlink d12/d3b/d60/d7e/d37/f4e 0 2024-04-06T03:40:48.116 INFO:tasks.workunit.client.0.smithi094.stdout:1/875: creat d0/d1/d27/da5/d40/db1/dc3/d5f/f12e x:0 0 0 2024-04-06T03:40:48.116 INFO:tasks.workunit.client.0.smithi094.stdout:1/876: chown d0/d1/d27/da5/d40/db1 4902 1 2024-04-06T03:40:48.119 INFO:tasks.workunit.client.1.smithi136.stdout:0/917: unlink d1/d6/d2c/d1a/d36/ff3 0 2024-04-06T03:40:48.119 INFO:tasks.workunit.client.0.smithi094.stdout:0/862: dwrite d1/d9/dd/d18/d1e/d86/d96/daa/fee [0,4194304] 0 2024-04-06T03:40:48.119 INFO:tasks.workunit.client.0.smithi094.stdout:0/863: stat d1/d9/dd/d18/d1e/def/f81 0 2024-04-06T03:40:48.120 INFO:tasks.workunit.client.0.smithi094.stdout:6/806: rename dc/dd3/d24/c5d to dc/dd3/d24/c119 0 2024-04-06T03:40:48.124 INFO:tasks.workunit.client.1.smithi136.stdout:4/820: rename d1d/d31/d3e/f5f to d1d/d2f/d6c/da5/df0/f120 0 2024-04-06T03:40:48.126 INFO:tasks.workunit.client.0.smithi094.stdout:9/845: link d2/dc/d46/d111/d33/f44 d2/dc/d21/d26/d8a/de9/f122 0 2024-04-06T03:40:48.126 INFO:tasks.workunit.client.0.smithi094.stdout:9/846: chown d2/dc/d80/ddd/fa7 590 1 2024-04-06T03:40:48.126 INFO:tasks.workunit.client.0.smithi094.stdout:2/849: sync 2024-04-06T03:40:48.126 INFO:tasks.workunit.client.0.smithi094.stdout:5/841: sync 2024-04-06T03:40:48.127 INFO:tasks.workunit.client.0.smithi094.stdout:5/842: read db/d19/dc1/dd4/f55 [2810242,44796] 0 2024-04-06T03:40:48.127 INFO:tasks.workunit.client.0.smithi094.stdout:5/843: chown db/d19/dc1/ddf/f90 3 1 2024-04-06T03:40:48.129 INFO:tasks.workunit.client.0.smithi094.stdout:3/862: mknod d9/d1e/d26/d6c/ddd/c133 0 2024-04-06T03:40:48.129 INFO:tasks.workunit.client.0.smithi094.stdout:3/863: write d9/d1e/d26/d6c/f125 [817143,59788] 0 2024-04-06T03:40:48.130 INFO:tasks.workunit.client.1.smithi136.stdout:8/990: dwrite d3/de/f3a [4194304,4194304] 0 2024-04-06T03:40:48.130 INFO:tasks.workunit.client.1.smithi136.stdout:8/991: chown d3/da/d26/d35/d3d/dc0/d11f 0 1 2024-04-06T03:40:48.130 INFO:tasks.workunit.client.1.smithi136.stdout:8/992: readlink d3/de/d18/d7d/ddf/d6c/dab/l106 0 2024-04-06T03:40:48.130 INFO:tasks.workunit.client.1.smithi136.stdout:8/993: stat d3/da/d26/d35/d3d/dc7/cd1 0 2024-04-06T03:40:48.130 INFO:tasks.workunit.client.1.smithi136.stdout:8/994: creat d3/da/d26/d90/f13e x:0 0 0 2024-04-06T03:40:48.131 INFO:tasks.workunit.client.0.smithi094.stdout:8/894: rmdir d18/d3c/dc4 39 2024-04-06T03:40:48.132 INFO:tasks.workunit.client.1.smithi136.stdout:6/830: creat d2/d7/d26/dcd/dad/f11e x:0 0 0 2024-04-06T03:40:48.132 INFO:tasks.workunit.client.1.smithi136.stdout:6/831: fsync d2/fa 0 2024-04-06T03:40:48.133 INFO:tasks.workunit.client.1.smithi136.stdout:6/832: read - d2/d104/d75/ffb zero size 2024-04-06T03:40:48.133 INFO:tasks.workunit.client.1.smithi136.stdout:0/918: dwrite d1/d6/d2c/d57/d43/d6b/f13a [0,4194304] 0 2024-04-06T03:40:48.133 INFO:tasks.workunit.client.1.smithi136.stdout:6/833: chown d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/cf1 15654670 1 2024-04-06T03:40:48.133 INFO:tasks.workunit.client.0.smithi094.stdout:1/877: dwrite d0/d1/d27/da5/d40/db1/d2f/d3c/d48/fee [0,4194304] 0 2024-04-06T03:40:48.133 INFO:tasks.workunit.client.0.smithi094.stdout:7/759: truncate d2/d6/dc/d12/d3a/fa2 3922749 0 2024-04-06T03:40:48.139 INFO:tasks.workunit.client.0.smithi094.stdout:4/940: sync 2024-04-06T03:40:48.139 INFO:tasks.workunit.client.0.smithi094.stdout:4/941: write d0/d77/d123/d49/db2/dc7/fd3 [389139,66054] 0 2024-04-06T03:40:48.139 INFO:tasks.workunit.client.0.smithi094.stdout:4/942: truncate d0/d15/d3a/d61/d81/da0/fdd 4763580 0 2024-04-06T03:40:48.140 INFO:tasks.workunit.client.0.smithi094.stdout:0/864: creat d1/d9/dd/d18/d1e/def/dab/d111/f122 x:0 0 0 2024-04-06T03:40:48.142 INFO:tasks.workunit.client.0.smithi094.stdout:6/807: symlink dc/dd3/d72/d88/dfb/l11a 0 2024-04-06T03:40:48.145 INFO:tasks.workunit.client.1.smithi136.stdout:6/834: dwrite d2/d7/d26/d30/f7e [4194304,4194304] 0 2024-04-06T03:40:48.145 INFO:tasks.workunit.client.0.smithi094.stdout:9/847: mkdir d2/dc/d80/ddd/d6b/d123 0 2024-04-06T03:40:48.148 INFO:tasks.workunit.client.0.smithi094.stdout:2/850: symlink d0/dd/d1b/d72/da8/dd2/l121 0 2024-04-06T03:40:48.151 INFO:tasks.workunit.client.0.smithi094.stdout:5/844: symlink db/d19/d21/d11e/d28/d85/db5/l127 0 2024-04-06T03:40:48.156 INFO:tasks.workunit.client.0.smithi094.stdout:3/864: creat d9/d20/d2d/d128/d90/f134 x:0 0 0 2024-04-06T03:40:48.158 INFO:tasks.workunit.client.0.smithi094.stdout:8/895: mkdir d18/d3c/d49/d91/d114/d12f 0 2024-04-06T03:40:48.162 INFO:tasks.workunit.client.1.smithi136.stdout:9/971: mknod d17/d21/d32/dc6/d113/d66/de1/d51/c14a 0 2024-04-06T03:40:48.164 INFO:tasks.workunit.client.0.smithi094.stdout:1/878: symlink d0/d127/d30/df6/l12f 0 2024-04-06T03:40:48.164 INFO:tasks.workunit.client.0.smithi094.stdout:1/879: creat d0/d1/d8f/f130 x:0 0 0 2024-04-06T03:40:48.164 INFO:tasks.workunit.client.0.smithi094.stdout:1/880: dread - d0/d1/d8f/f130 zero size 2024-04-06T03:40:48.166 INFO:tasks.workunit.client.0.smithi094.stdout:8/896: write fd [2045088,72814] 0 2024-04-06T03:40:48.168 INFO:tasks.workunit.client.1.smithi136.stdout:1/958: link d12/d3b/d74/d7a/dea/d11a/d23/c2b d12/d1b/da7/dcc/c145 0 2024-04-06T03:40:48.168 INFO:tasks.workunit.client.0.smithi094.stdout:7/760: symlink d2/d6/dc/d12/d29/daa/lf7 0 2024-04-06T03:40:48.170 INFO:tasks.workunit.client.0.smithi094.stdout:9/848: dwrite d2/dc/d10/d2a/fcc [0,4194304] 0 2024-04-06T03:40:48.171 INFO:tasks.workunit.client.0.smithi094.stdout:2/851: dwrite d0/f10a [0,4194304] 0 2024-04-06T03:40:48.172 INFO:tasks.workunit.client.1.smithi136.stdout:4/821: unlink d1d/d31/d8e/f9a 0 2024-04-06T03:40:48.173 INFO:tasks.workunit.client.0.smithi094.stdout:0/865: creat d1/d9/dd/d18/d5d/d8d/f123 x:0 0 0 2024-04-06T03:40:48.176 INFO:tasks.workunit.client.0.smithi094.stdout:6/808: unlink dc/d11/c9c 0 2024-04-06T03:40:48.176 INFO:tasks.workunit.client.1.smithi136.stdout:8/995: truncate d3/da/d26/d35/d53/d86/db0/f121 3429930 0 2024-04-06T03:40:48.178 INFO:tasks.workunit.client.0.smithi094.stdout:5/845: truncate db/d19/d30/d83/fe5 173421 0 2024-04-06T03:40:48.179 INFO:tasks.workunit.client.0.smithi094.stdout:5/846: write db/d19/d21/d11e/d2e/f48 [4782432,13440] 0 2024-04-06T03:40:48.181 INFO:tasks.workunit.client.0.smithi094.stdout:3/865: symlink d9/d9d/l135 0 2024-04-06T03:40:48.181 INFO:tasks.workunit.client.0.smithi094.stdout:3/866: truncate d9/d1d/fe6 991576 0 2024-04-06T03:40:48.184 INFO:tasks.workunit.client.1.smithi136.stdout:3/903: sync 2024-04-06T03:40:48.184 INFO:tasks.workunit.client.1.smithi136.stdout:3/904: write d0/d4/d12/d19/d55/d6f/d82/fd2 [497369,33889] 0 2024-04-06T03:40:48.184 INFO:tasks.workunit.client.1.smithi136.stdout:4/822: dwrite d1d/d31/d3e/d55/d7f/f83 [0,4194304] 0 2024-04-06T03:40:48.184 INFO:tasks.workunit.client.1.smithi136.stdout:0/919: mknod d1/d6/d2c/d1a/d82/c13b 0 2024-04-06T03:40:48.184 INFO:tasks.workunit.client.1.smithi136.stdout:0/920: dread - d1/d6/d2c/d57/d43/d6d/d9b/d5c/f135 zero size 2024-04-06T03:40:48.185 INFO:tasks.workunit.client.0.smithi094.stdout:1/881: mkdir d0/d127/d131 0 2024-04-06T03:40:48.186 INFO:tasks.workunit.client.1.smithi136.stdout:6/835: mknod d2/d104/d75/c11f 0 2024-04-06T03:40:48.190 INFO:tasks.workunit.client.0.smithi094.stdout:8/897: mknod d18/d3c/d49/d91/d94/daa/c130 0 2024-04-06T03:40:48.192 INFO:tasks.workunit.client.1.smithi136.stdout:9/972: creat d17/d20/d25/d3c/f14b x:0 0 0 2024-04-06T03:40:48.194 INFO:tasks.workunit.client.1.smithi136.stdout:3/905: dread d0/f105 [0,4194304] 0 2024-04-06T03:40:48.196 INFO:tasks.workunit.client.0.smithi094.stdout:7/761: symlink d2/d6/dc/d12/lf8 0 2024-04-06T03:40:48.200 INFO:tasks.workunit.client.0.smithi094.stdout:8/898: dread fd [0,4194304] 0 2024-04-06T03:40:48.200 INFO:tasks.workunit.client.0.smithi094.stdout:8/899: chown d18/d3c/d49/d91/d94/daa/c130 64 1 2024-04-06T03:40:48.201 INFO:tasks.workunit.client.1.smithi136.stdout:8/996: mknod d3/da/d26/dfb/c13f 0 2024-04-06T03:40:48.202 INFO:tasks.workunit.client.1.smithi136.stdout:6/836: dwrite d2/d7/d13/fd6 [0,4194304] 0 2024-04-06T03:40:48.208 INFO:tasks.workunit.client.0.smithi094.stdout:9/849: truncate d2/dc/d21/d26/f72 2185171 0 2024-04-06T03:40:48.209 INFO:tasks.workunit.client.0.smithi094.stdout:1/882: dwrite d0/d127/d30/f32 [0,4194304] 0 2024-04-06T03:40:48.210 INFO:tasks.workunit.client.1.smithi136.stdout:4/823: creat d1d/d31/d8e/f121 x:0 0 0 2024-04-06T03:40:48.213 INFO:tasks.workunit.client.0.smithi094.stdout:2/852: creat d0/dd/d1b/de2/f122 x:0 0 0 2024-04-06T03:40:48.213 INFO:tasks.workunit.client.0.smithi094.stdout:2/853: fsync d0/dd/d19/fe5 0 2024-04-06T03:40:48.214 INFO:tasks.workunit.client.0.smithi094.stdout:0/866: readlink d1/d9/dd/d18/d1e/l10b 0 2024-04-06T03:40:48.214 INFO:tasks.workunit.client.0.smithi094.stdout:0/867: chown d1/d9/dd/d18/d1e/def/dab/d4e/de4/fc3 21 1 2024-04-06T03:40:48.215 INFO:tasks.workunit.client.1.smithi136.stdout:9/973: creat d17/d21/d29/de5/da5/df5/f14c x:0 0 0 2024-04-06T03:40:48.215 INFO:tasks.workunit.client.1.smithi136.stdout:9/974: truncate d17/f12d 956869 0 2024-04-06T03:40:48.216 INFO:tasks.workunit.client.1.smithi136.stdout:3/906: rename d0/d4/d12/d19/d55/d3c/d46/f72 to d0/d4/d12/d19/d55/d6f/f12f 0 2024-04-06T03:40:48.218 INFO:tasks.workunit.client.0.smithi094.stdout:6/809: mknod dc/d3a/df4/df6/d7b/dc6/c11b 0 2024-04-06T03:40:48.220 INFO:tasks.workunit.client.1.smithi136.stdout:1/959: getdents d12/d1b/d28/d11c/d111 0 2024-04-06T03:40:48.223 INFO:tasks.workunit.client.0.smithi094.stdout:5/847: mknod db/c128 0 2024-04-06T03:40:48.223 INFO:tasks.workunit.client.0.smithi094.stdout:5/848: chown db/d19/d2c/d6d/de7 27432 1 2024-04-06T03:40:48.227 INFO:tasks.workunit.client.0.smithi094.stdout:3/867: getdents d9/d20/d2d/d128/dba/dc5 0 2024-04-06T03:40:48.227 INFO:tasks.workunit.client.0.smithi094.stdout:3/868: write d9/f19 [969384,49369] 0 2024-04-06T03:40:48.227 INFO:tasks.workunit.client.0.smithi094.stdout:3/869: readlink d9/l118 0 2024-04-06T03:40:48.227 INFO:tasks.workunit.client.0.smithi094.stdout:3/870: write d9/d1e/d26/f10e [1195523,85385] 0 2024-04-06T03:40:48.227 INFO:tasks.workunit.client.0.smithi094.stdout:3/871: fsync d9/d1e/f59 0 2024-04-06T03:40:48.229 INFO:tasks.workunit.client.0.smithi094.stdout:7/762: symlink d2/d6/dc/d12/d9a/lf9 0 2024-04-06T03:40:48.230 INFO:tasks.workunit.client.0.smithi094.stdout:7/763: truncate d2/d6/dc/d12/d3a/dcc/dde/db7/fba 110845 0 2024-04-06T03:40:48.230 INFO:tasks.workunit.client.0.smithi094.stdout:7/764: chown d2/d6/dc/dd/d90 12527 1 2024-04-06T03:40:48.231 INFO:tasks.workunit.client.1.smithi136.stdout:4/824: symlink d1d/d31/d3e/d56/d7b/l122 0 2024-04-06T03:40:48.232 INFO:tasks.workunit.client.0.smithi094.stdout:4/943: dwrite d0/d77/d123/d46/f95 [0,4194304] 0 2024-04-06T03:40:48.232 INFO:tasks.workunit.client.0.smithi094.stdout:4/944: truncate d0/d4/d67/dcb/ffb 421675 0 2024-04-06T03:40:48.232 INFO:tasks.workunit.client.0.smithi094.stdout:8/900: creat d18/d3c/d49/db5/f131 x:0 0 0 2024-04-06T03:40:48.232 INFO:tasks.workunit.client.1.smithi136.stdout:8/997: dwrite d3/de/d18/d7d/ddf/d6c/f10b [0,4194304] 0 2024-04-06T03:40:48.233 INFO:tasks.workunit.client.1.smithi136.stdout:8/998: readlink d3/da/d26/d35/d3d/dc0/d11f/l12e 0 2024-04-06T03:40:48.233 INFO:tasks.workunit.client.1.smithi136.stdout:8/999: dread - d3/da/d26/d35/dcb/ff4 zero size 2024-04-06T03:40:48.234 INFO:tasks.workunit.client.0.smithi094.stdout:9/850: symlink d2/dc/da6/l124 0 2024-04-06T03:40:48.236 INFO:tasks.workunit.client.1.smithi136.stdout:0/921: dwrite d1/d6/d2c/fc0 [0,4194304] 0 2024-04-06T03:40:48.236 INFO:tasks.workunit.client.1.smithi136.stdout:3/907: link d0/d4/d12/dc3/lca d0/d4/d12/d3a/d51/d8e/d10f/l130 0 2024-04-06T03:40:48.237 INFO:tasks.workunit.client.0.smithi094.stdout:1/883: truncate d0/d7/f26 5790731 0 2024-04-06T03:40:48.237 INFO:tasks.workunit.client.0.smithi094.stdout:1/884: chown d0/d1/d27/da5/d40/db1/d2f/d4b/lc2 30594 1 2024-04-06T03:40:48.237 INFO:tasks.workunit.client.0.smithi094.stdout:2/854: dwrite d0/dd/d3d/f100 [0,4194304] 0 2024-04-06T03:40:48.237 INFO:tasks.workunit.client.0.smithi094.stdout:2/855: stat d0/d18/lb0 0 2024-04-06T03:40:48.238 INFO:tasks.workunit.client.1.smithi136.stdout:9/975: dwrite d17/d21/d32/fa6 [0,4194304] 0 2024-04-06T03:40:48.239 INFO:tasks.workunit.client.1.smithi136.stdout:1/960: rename d12/d3b/d60/d7e/dbe/l10e to d12/d3b/d44/d5f/d5a/daf/l146 0 2024-04-06T03:40:48.239 INFO:tasks.workunit.client.1.smithi136.stdout:1/961: chown d12/d3b/d74/d7a/fad 2 1 2024-04-06T03:40:48.240 INFO:tasks.workunit.client.0.smithi094.stdout:0/868: symlink d1/d9/dd/d18/d1e/def/l124 0 2024-04-06T03:40:48.244 INFO:tasks.workunit.client.0.smithi094.stdout:6/810: chown dc/dd3/d72/d88/fad 14 1 2024-04-06T03:40:48.245 INFO:tasks.workunit.client.1.smithi136.stdout:4/825: mkdir d1d/d2f/d123 0 2024-04-06T03:40:48.245 INFO:tasks.workunit.client.1.smithi136.stdout:4/826: dread - d1d/d31/d3e/d55/f10f zero size 2024-04-06T03:40:48.245 INFO:tasks.workunit.client.1.smithi136.stdout:4/827: chown d1d/d2f/da6/dbf/ld3 309 1 2024-04-06T03:40:48.246 INFO:tasks.workunit.client.0.smithi094.stdout:5/849: mknod db/d19/d2c/d57/c129 0 2024-04-06T03:40:48.252 INFO:tasks.workunit.client.0.smithi094.stdout:3/872: symlink d9/d9d/d129/l136 0 2024-04-06T03:40:48.255 INFO:tasks.workunit.client.0.smithi094.stdout:7/765: unlink d2/d6/dac/ddd/fe0 0 2024-04-06T03:40:48.255 INFO:tasks.workunit.client.0.smithi094.stdout:7/766: dread - d2/d6/dc/d12/d29/d3f/fee zero size 2024-04-06T03:40:48.256 INFO:tasks.workunit.client.1.smithi136.stdout:9/976: dwrite d17/d20/d25/da4/dbb/de4/f147 [0,4194304] 0 2024-04-06T03:40:48.259 INFO:tasks.workunit.client.0.smithi094.stdout:4/945: creat d0/d77/d123/d49/db2/dc7/f140 x:0 0 0 2024-04-06T03:40:48.259 INFO:tasks.workunit.client.0.smithi094.stdout:4/946: write d0/df/d1e/d34/db6/fbe [3896,105865] 0 2024-04-06T03:40:48.259 INFO:tasks.workunit.client.0.smithi094.stdout:4/947: stat d0/d4/d67/dcb/d134 0 2024-04-06T03:40:48.260 INFO:tasks.workunit.client.0.smithi094.stdout:8/901: getdents d18/d3c/d49/d91/d94 0 2024-04-06T03:40:48.262 INFO:tasks.workunit.client.0.smithi094.stdout:9/851: stat d2/dc/d21/d23/d47/c67 0 2024-04-06T03:40:48.265 INFO:tasks.workunit.client.1.smithi136.stdout:4/828: dwrite d1d/d31/d78/fb0 [0,4194304] 0 2024-04-06T03:40:48.265 INFO:tasks.workunit.client.1.smithi136.stdout:4/829: creat d1d/f124 x:0 0 0 2024-04-06T03:40:48.266 INFO:tasks.workunit.client.0.smithi094.stdout:4/948: dread d0/df/d13/f37 [0,4194304] 0 2024-04-06T03:40:48.266 INFO:tasks.workunit.client.0.smithi094.stdout:4/949: chown d0/df/d57/f86 251786 1 2024-04-06T03:40:48.266 INFO:tasks.workunit.client.0.smithi094.stdout:4/950: fsync d0/df/db3/dd8/fed 0 2024-04-06T03:40:48.266 INFO:tasks.workunit.client.0.smithi094.stdout:1/885: unlink d0/d127/dff/d115/l128 0 2024-04-06T03:40:48.267 INFO:tasks.workunit.client.0.smithi094.stdout:1/886: truncate d0/d1/d27/da5/d40/ff3 1168097 0 2024-04-06T03:40:48.270 INFO:tasks.workunit.client.1.smithi136.stdout:0/922: creat d1/d6/d2c/d57/d43/d6d/f13c x:0 0 0 2024-04-06T03:40:48.271 INFO:tasks.workunit.client.0.smithi094.stdout:2/856: unlink d0/dd/d3d/c119 0 2024-04-06T03:40:48.272 INFO:tasks.workunit.client.1.smithi136.stdout:4/830: write d1d/d2f/f57 [3847950,4128] 0 2024-04-06T03:40:48.277 INFO:tasks.workunit.client.1.smithi136.stdout:3/908: rename d0/d4/d12/d3a/d51/da6/d100 to d0/d4/d12/d19/d55/d3c/d46/d131 0 2024-04-06T03:40:48.277 INFO:tasks.workunit.client.1.smithi136.stdout:3/909: truncate d0/d4/d12/d19/d115/f7f 1171406 0 2024-04-06T03:40:48.277 INFO:tasks.workunit.client.1.smithi136.stdout:3/910: write d0/f2 [3304306,83120] 0 2024-04-06T03:40:48.279 INFO:tasks.workunit.client.1.smithi136.stdout:4/831: dread d1d/d2f/f32 [0,4194304] 0 2024-04-06T03:40:48.280 INFO:tasks.workunit.client.1.smithi136.stdout:1/962: mknod d12/d3b/d74/dd6/c147 0 2024-04-06T03:40:48.280 INFO:tasks.workunit.client.1.smithi136.stdout:1/963: readlink d12/d3b/d74/d7a/dea/d11a/d23/lf8 0 2024-04-06T03:40:48.284 INFO:tasks.workunit.client.1.smithi136.stdout:9/977: getdents d17/d19/da3 0 2024-04-06T03:40:48.285 INFO:tasks.workunit.client.0.smithi094.stdout:3/873: creat d9/d1e/d26/d55/f137 x:0 0 0 2024-04-06T03:40:48.288 INFO:tasks.workunit.client.0.smithi094.stdout:7/767: rename d2/d6/dc/d12/f2a to d2/d6/dc/d12/d3a/ffa 0 2024-04-06T03:40:48.288 INFO:tasks.workunit.client.0.smithi094.stdout:7/768: fsync d2/d6/dc/d12/d9a/da0/faf 0 2024-04-06T03:40:48.289 INFO:tasks.workunit.client.0.smithi094.stdout:1/887: mknod d0/d127/d131/c132 0 2024-04-06T03:40:48.289 INFO:tasks.workunit.client.0.smithi094.stdout:1/888: write d0/d7/d1a/f9b [4385336,116885] 0 2024-04-06T03:40:48.289 INFO:tasks.workunit.client.0.smithi094.stdout:1/889: write d0/d1/f65 [2055828,91996] 0 2024-04-06T03:40:48.290 INFO:tasks.workunit.client.0.smithi094.stdout:2/857: dwrite d0/d8/da0/fd9 [0,4194304] 0 2024-04-06T03:40:48.290 INFO:tasks.workunit.client.0.smithi094.stdout:2/858: write d0/dd/d19/d31/dcc/ff4 [857379,119850] 0 2024-04-06T03:40:48.290 INFO:tasks.workunit.client.0.smithi094.stdout:5/850: getdents db/d19/d2c/d6d/de6 0 2024-04-06T03:40:48.292 INFO:tasks.workunit.client.0.smithi094.stdout:3/874: symlink d9/d20/d2d/d128/dba/d7c/da2/df2/l138 0 2024-04-06T03:40:48.292 INFO:tasks.workunit.client.0.smithi094.stdout:3/875: dread - d9/d20/d2d/d128/d90/f105 zero size 2024-04-06T03:40:48.294 INFO:tasks.workunit.client.1.smithi136.stdout:3/911: rename d0/c7b to d0/d4/d12/d19/d55/d3c/d5a/d79/d98/dc6/c132 0 2024-04-06T03:40:48.294 INFO:tasks.workunit.client.1.smithi136.stdout:3/912: readlink d0/d4/d12/d3a/d69/lc2 0 2024-04-06T03:40:48.294 INFO:tasks.workunit.client.1.smithi136.stdout:3/913: truncate d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/f83 645171 0 2024-04-06T03:40:48.295 INFO:tasks.workunit.client.1.smithi136.stdout:1/964: mkdir d12/d3b/d74/d7a/dea/d11a/d4d/d10d/d148 0 2024-04-06T03:40:48.297 INFO:tasks.workunit.client.0.smithi094.stdout:7/769: link d2/d6/dc/dd/df/d36/c46 d2/d6/dc/d12/d9a/da0/dd4/cfb 0 2024-04-06T03:40:48.297 INFO:tasks.workunit.client.1.smithi136.stdout:9/978: creat d17/d19/d13e/f14d x:0 0 0 2024-04-06T03:40:48.297 INFO:tasks.workunit.client.1.smithi136.stdout:0/923: dread d1/d6/d2c/f20 [0,4194304] 0 2024-04-06T03:40:48.298 INFO:tasks.workunit.client.1.smithi136.stdout:1/965: mknod d12/d3b/d44/d5f/db7/c149 0 2024-04-06T03:40:48.298 INFO:tasks.workunit.client.1.smithi136.stdout:1/966: dread - d12/d1b/da7/daa/f10c zero size 2024-04-06T03:40:48.299 INFO:tasks.workunit.client.0.smithi094.stdout:1/890: mkdir d0/d1/d27/da5/d133 0 2024-04-06T03:40:48.300 INFO:tasks.workunit.client.1.smithi136.stdout:9/979: unlink d17/d21/d29/de5/da5/db8/de0/l132 0 2024-04-06T03:40:48.301 INFO:tasks.workunit.client.1.smithi136.stdout:0/924: mknod d1/d6/d2c/d57/d43/d6d/d85/d11a/c13d 0 2024-04-06T03:40:48.302 INFO:tasks.workunit.client.0.smithi094.stdout:2/859: truncate d0/dd/d19/f4c 960263 0 2024-04-06T03:40:48.303 INFO:tasks.workunit.client.1.smithi136.stdout:3/914: write d0/d4/d12/d19/d55/d6f/fad [1486425,107889] 0 2024-04-06T03:40:48.303 INFO:tasks.workunit.client.0.smithi094.stdout:3/876: creat d9/d10/dbb/de1/f139 x:0 0 0 2024-04-06T03:40:48.304 INFO:tasks.workunit.client.0.smithi094.stdout:3/877: write d9/d10/d7e/da3/fdc [288576,112294] 0 2024-04-06T03:40:48.304 INFO:tasks.workunit.client.0.smithi094.stdout:3/878: creat d9/d9d/dc1/f13a x:0 0 0 2024-04-06T03:40:48.305 INFO:tasks.workunit.client.1.smithi136.stdout:1/967: write d12/d53/f66 [52428,65370] 0 2024-04-06T03:40:48.305 INFO:tasks.workunit.client.1.smithi136.stdout:1/968: write d12/d3b/d74/d7a/dea/d11a/dd1/f129 [360873,42828] 0 2024-04-06T03:40:48.310 INFO:tasks.workunit.client.1.smithi136.stdout:9/980: rename d17/d19/da3/dc5/deb to d17/d21/d29/de5/d4b/d14e 0 2024-04-06T03:40:48.311 INFO:tasks.workunit.client.0.smithi094.stdout:7/770: unlink d2/d6/dc/dd/df/d36/c62 0 2024-04-06T03:40:48.313 INFO:tasks.workunit.client.0.smithi094.stdout:1/891: mknod d0/d1/d27/da5/db2/c134 0 2024-04-06T03:40:48.315 INFO:tasks.workunit.client.1.smithi136.stdout:9/981: link d17/d19/l38 d17/d20/d28/d49/d75/d60/l14f 0 2024-04-06T03:40:48.317 INFO:tasks.workunit.client.0.smithi094.stdout:4/951: rename d0/df/d91 to d0/d4/d4b/d141 0 2024-04-06T03:40:48.319 INFO:tasks.workunit.client.0.smithi094.stdout:1/892: rename d0/d127/ced to d0/d127/dff/c135 0 2024-04-06T03:40:48.320 INFO:tasks.workunit.client.0.smithi094.stdout:1/893: creat d0/d1/d27/da5/d40/d50/f136 x:0 0 0 2024-04-06T03:40:48.320 INFO:tasks.workunit.client.0.smithi094.stdout:4/952: mkdir d0/df/d13/d4a/dfa/d142 0 2024-04-06T03:40:48.322 INFO:tasks.workunit.client.0.smithi094.stdout:1/894: mkdir d0/d127/dff/d137 0 2024-04-06T03:40:48.323 INFO:tasks.workunit.client.0.smithi094.stdout:1/895: write d0/d1/d8f/f11a [957796,42619] 0 2024-04-06T03:40:48.328 INFO:tasks.workunit.client.0.smithi094.stdout:9/852: dread d2/dc/d21/d23/d2f/f83 [0,4194304] 0 2024-04-06T03:40:48.330 INFO:tasks.workunit.client.0.smithi094.stdout:9/853: creat d2/dc/d80/ddd/d6b/d123/f125 x:0 0 0 2024-04-06T03:40:48.330 INFO:tasks.workunit.client.0.smithi094.stdout:9/854: chown d2/dc/d21/d23/d47/d51/c7d 3 1 2024-04-06T03:40:48.330 INFO:tasks.workunit.client.0.smithi094.stdout:9/855: chown d2/dc/d21/d23/d5b/cdf 12944 1 2024-04-06T03:40:48.330 INFO:tasks.workunit.client.0.smithi094.stdout:9/856: read - d2/ffc zero size 2024-04-06T03:40:48.331 INFO:tasks.workunit.client.0.smithi094.stdout:9/857: creat d2/dc/d21/d23/d47/d51/dee/f126 x:0 0 0 2024-04-06T03:40:48.333 INFO:tasks.workunit.client.0.smithi094.stdout:9/858: link d2/dc/d21/d23/d5b/lc0 d2/dc/d46/d111/d33/d11d/l127 0 2024-04-06T03:40:48.333 INFO:tasks.workunit.client.0.smithi094.stdout:6/811: dwrite dc/d3a/df4/df6/df2/f111 [0,4194304] 0 2024-04-06T03:40:48.333 INFO:tasks.workunit.client.0.smithi094.stdout:6/812: stat dc/dd3/d4e/c10c 0 2024-04-06T03:40:48.334 INFO:tasks.workunit.client.0.smithi094.stdout:8/902: dread f14 [0,4194304] 0 2024-04-06T03:40:48.339 INFO:tasks.workunit.client.1.smithi136.stdout:0/925: write d1/d6/d2c/d57/f46 [1449890,61049] 0 2024-04-06T03:40:48.339 INFO:tasks.workunit.client.1.smithi136.stdout:0/926: chown d1/d6/d2c/d57/d43/dcf/de1 52895158 1 2024-04-06T03:40:48.343 INFO:tasks.workunit.client.0.smithi094.stdout:6/813: rename dc/dd3/d72/d88/f102 to dc/dd3/d72/d87/f11c 0 2024-04-06T03:40:48.343 INFO:tasks.workunit.client.0.smithi094.stdout:6/814: readlink dc/d3a/d7f/dd4/le7 0 2024-04-06T03:40:48.347 INFO:tasks.workunit.client.0.smithi094.stdout:6/815: symlink dc/d3a/d7f/dd9/l11d 0 2024-04-06T03:40:48.348 INFO:tasks.workunit.client.0.smithi094.stdout:1/896: dread d0/d1/d27/da5/d40/db1/f55 [0,4194304] 0 2024-04-06T03:40:48.349 INFO:tasks.workunit.client.1.smithi136.stdout:1/969: dwrite d12/d1b/d28/f5e [0,4194304] 0 2024-04-06T03:40:48.349 INFO:tasks.workunit.client.1.smithi136.stdout:1/970: fsync d12/d3b/d8e/fbc 0 2024-04-06T03:40:48.350 INFO:tasks.workunit.client.0.smithi094.stdout:1/897: creat d0/d7/d1a/d108/f138 x:0 0 0 2024-04-06T03:40:48.353 INFO:tasks.workunit.client.0.smithi094.stdout:3/879: dwrite d9/d20/d2d/f67 [0,4194304] 0 2024-04-06T03:40:48.353 INFO:tasks.workunit.client.1.smithi136.stdout:1/971: mkdir d12/d3b/d74/d7a/dea/d77/d14a 0 2024-04-06T03:40:48.354 INFO:tasks.workunit.client.1.smithi136.stdout:1/972: dread - d12/d3b/d60/fda zero size 2024-04-06T03:40:48.355 INFO:tasks.workunit.client.0.smithi094.stdout:1/898: rename d0/d127/d30/l45 to d0/l139 0 2024-04-06T03:40:48.356 INFO:tasks.workunit.client.0.smithi094.stdout:6/816: write dc/dd3/f2f [2118125,126945] 0 2024-04-06T03:40:48.359 INFO:tasks.workunit.client.0.smithi094.stdout:3/880: dread d9/d10/f47 [0,4194304] 0 2024-04-06T03:40:48.364 INFO:tasks.workunit.client.0.smithi094.stdout:9/859: dwrite d2/dc/d80/ddd/d6b/f73 [4194304,4194304] 0 2024-04-06T03:40:48.364 INFO:tasks.workunit.client.0.smithi094.stdout:9/860: chown d2/dc/d46/d111/d33/f5c 2889176 1 2024-04-06T03:40:48.364 INFO:tasks.workunit.client.0.smithi094.stdout:3/881: link d9/d20/d76/c111 d9/d10/dd4/c13b 0 2024-04-06T03:40:48.364 INFO:tasks.workunit.client.0.smithi094.stdout:3/882: truncate d9/d20/d2d/d128/d90/f105 392312 0 2024-04-06T03:40:48.364 INFO:tasks.workunit.client.0.smithi094.stdout:3/883: chown d9/d20/d76/fac 5246195 1 2024-04-06T03:40:48.365 INFO:tasks.workunit.client.0.smithi094.stdout:4/953: dwrite d0/d4/d67/d9f/fa9 [0,4194304] 0 2024-04-06T03:40:48.365 INFO:tasks.workunit.client.0.smithi094.stdout:4/954: read d0/d15/fa2 [2584994,68932] 0 2024-04-06T03:40:48.365 INFO:tasks.workunit.client.0.smithi094.stdout:4/955: chown d0/df/d1e/f24 3 1 2024-04-06T03:40:48.366 INFO:tasks.workunit.client.1.smithi136.stdout:1/973: truncate d12/d3b/d74/d7a/fe9 1471654 0 2024-04-06T03:40:48.366 INFO:tasks.workunit.client.1.smithi136.stdout:1/974: chown d12/d3b/d74/d7a/dea/d11a/d9f/l119 15481579 1 2024-04-06T03:40:48.366 INFO:tasks.workunit.client.1.smithi136.stdout:1/975: read - d12/d3b/d74/d7a/f118 zero size 2024-04-06T03:40:48.367 INFO:tasks.workunit.client.0.smithi094.stdout:8/903: dwrite d18/f4e [0,4194304] 0 2024-04-06T03:40:48.368 INFO:tasks.workunit.client.0.smithi094.stdout:9/861: unlink d2/dc/d46/d111/ld7 0 2024-04-06T03:40:48.368 INFO:tasks.workunit.client.0.smithi094.stdout:9/862: write d2/dc/d21/f43 [3451845,43143] 0 2024-04-06T03:40:48.369 INFO:tasks.workunit.client.0.smithi094.stdout:9/863: readlink d2/dc/d21/dc8/l101 0 2024-04-06T03:40:48.369 INFO:tasks.workunit.client.1.smithi136.stdout:0/927: dwrite d1/d6/d2c/d1a/f1d [0,4194304] 0 2024-04-06T03:40:48.369 INFO:tasks.workunit.client.1.smithi136.stdout:0/928: dread - d1/d6/d2c/d1a/d82/ffa zero size 2024-04-06T03:40:48.369 INFO:tasks.workunit.client.1.smithi136.stdout:0/929: creat d1/d6/ded/f13e x:0 0 0 2024-04-06T03:40:48.370 INFO:tasks.workunit.client.1.smithi136.stdout:1/976: unlink d12/d1b/da7/dcc/c145 0 2024-04-06T03:40:48.370 INFO:tasks.workunit.client.1.smithi136.stdout:1/977: read d12/d3b/d44/d5f/f30 [2056246,18290] 0 2024-04-06T03:40:48.372 INFO:tasks.workunit.client.0.smithi094.stdout:3/884: link d9/d20/d2d/d128/c8b d9/d10/d9b/db7/dc0/de2/c13c 0 2024-04-06T03:40:48.373 INFO:tasks.workunit.client.0.smithi094.stdout:3/885: creat d9/d20/d30/d88/f13d x:0 0 0 2024-04-06T03:40:48.376 INFO:tasks.workunit.client.0.smithi094.stdout:9/864: dread d2/dc/d46/d111/fa5 [0,4194304] 0 2024-04-06T03:40:48.376 INFO:tasks.workunit.client.0.smithi094.stdout:9/865: readlink d2/dc/d21/d26/d78/d69/da8/l109 0 2024-04-06T03:40:48.380 INFO:tasks.workunit.client.0.smithi094.stdout:4/956: getdents d0/d77 0 2024-04-06T03:40:48.380 INFO:tasks.workunit.client.0.smithi094.stdout:6/817: dwrite dc/dd3/d72/d88/f110 [0,4194304] 0 2024-04-06T03:40:48.380 INFO:tasks.workunit.client.0.smithi094.stdout:6/818: chown dc/d3a/d4a/d54/dde/f39 105506329 1 2024-04-06T03:40:48.380 INFO:tasks.workunit.client.0.smithi094.stdout:4/957: creat d0/d4/f143 x:0 0 0 2024-04-06T03:40:48.381 INFO:tasks.workunit.client.0.smithi094.stdout:1/899: dwrite d0/d1/d27/da5/d40/fc7 [0,4194304] 0 2024-04-06T03:40:48.381 INFO:tasks.workunit.client.0.smithi094.stdout:1/900: stat d0/d1/d27/da5/d3f/de9/lf4 0 2024-04-06T03:40:48.383 INFO:tasks.workunit.client.1.smithi136.stdout:1/978: creat d12/d1b/da7/db8/f14b x:0 0 0 2024-04-06T03:40:48.383 INFO:tasks.workunit.client.1.smithi136.stdout:1/979: readlink d12/d1b/d120/ld7 0 2024-04-06T03:40:48.384 INFO:tasks.workunit.client.1.smithi136.stdout:0/930: truncate d1/d6/fd 426838 0 2024-04-06T03:40:48.387 INFO:tasks.workunit.client.0.smithi094.stdout:8/904: unlink d18/d3c/d49/d91/d94/daa/ffd 0 2024-04-06T03:40:48.388 INFO:tasks.workunit.client.1.smithi136.stdout:1/980: mknod d12/d3b/d74/d7a/dea/d77/d14a/c14c 0 2024-04-06T03:40:48.390 INFO:tasks.workunit.client.0.smithi094.stdout:3/886: dwrite d9/d10/ffe [0,4194304] 0 2024-04-06T03:40:48.390 INFO:tasks.workunit.client.0.smithi094.stdout:3/887: dread - d9/d9d/dc1/f10c zero size 2024-04-06T03:40:48.390 INFO:tasks.workunit.client.0.smithi094.stdout:3/888: creat d9/d1e/d26/d6c/d8e/df8/f13e x:0 0 0 2024-04-06T03:40:48.394 INFO:tasks.workunit.client.0.smithi094.stdout:9/866: unlink d2/dc/d21/d23/d47/d51/fd4 0 2024-04-06T03:40:48.394 INFO:tasks.workunit.client.0.smithi094.stdout:9/867: chown d2/dc/c113 1418112 1 2024-04-06T03:40:48.396 INFO:tasks.workunit.client.0.smithi094.stdout:8/905: dread d18/f1c [0,4194304] 0 2024-04-06T03:40:48.396 INFO:tasks.workunit.client.0.smithi094.stdout:8/906: creat d18/d3c/d49/db5/f132 x:0 0 0 2024-04-06T03:40:48.401 INFO:tasks.workunit.client.0.smithi094.stdout:6/819: creat dc/d3a/d4a/d54/d71/f11e x:0 0 0 2024-04-06T03:40:48.402 INFO:tasks.workunit.client.0.smithi094.stdout:6/820: chown dc/dd3/d72/d87/la8 44 1 2024-04-06T03:40:48.402 INFO:tasks.workunit.client.0.smithi094.stdout:6/821: fdatasync dc/dd3/d72/d88/da3/dc4/ffc 0 2024-04-06T03:40:48.409 INFO:tasks.workunit.client.0.smithi094.stdout:9/868: dwrite d2/dc/d46/d111/d33/f4a [0,4194304] 0 2024-04-06T03:40:48.410 INFO:tasks.workunit.client.0.smithi094.stdout:9/869: chown d2/f14 1 1 2024-04-06T03:40:48.410 INFO:tasks.workunit.client.1.smithi136.stdout:0/931: dwrite d1/d6/d2c/d1a/d36/fb1 [0,4194304] 0 2024-04-06T03:40:48.410 INFO:tasks.workunit.client.1.smithi136.stdout:0/932: chown d1/d6/d2c/d1a/d36/d59/f38 31414 1 2024-04-06T03:40:48.410 INFO:tasks.workunit.client.1.smithi136.stdout:0/933: chown d1/d6/f45 50 1 2024-04-06T03:40:48.410 INFO:tasks.workunit.client.1.smithi136.stdout:0/934: creat d1/d6/d2c/d57/d62/f13f x:0 0 0 2024-04-06T03:40:48.412 INFO:tasks.workunit.client.0.smithi094.stdout:7/771: write d2/d6/dc/d12/d29/fa7 [966974,83088] 0 2024-04-06T03:40:48.416 INFO:tasks.workunit.client.0.smithi094.stdout:3/889: mknod d9/d9d/d129/c13f 0 2024-04-06T03:40:48.419 INFO:tasks.workunit.client.1.smithi136.stdout:0/935: dread d1/d6/d2c/d57/fe6 [0,4194304] 0 2024-04-06T03:40:48.419 INFO:tasks.workunit.client.1.smithi136.stdout:0/936: creat d1/d6/df5/f140 x:0 0 0 2024-04-06T03:40:48.420 INFO:tasks.workunit.client.0.smithi094.stdout:8/907: mkdir d18/d3c/d49/d91/d94/d133 0 2024-04-06T03:40:48.420 INFO:tasks.workunit.client.1.smithi136.stdout:0/937: dread d1/d6/d2c/d57/d43/d6d/d85/da7/f11c [0,4194304] 0 2024-04-06T03:40:48.421 INFO:tasks.workunit.client.1.smithi136.stdout:0/938: write d1/d6/d2c/d57/d43/d6d/d85/d86/fba [500541,18961] 0 2024-04-06T03:40:48.421 INFO:tasks.workunit.client.1.smithi136.stdout:0/939: creat d1/d6/d2c/d57/d43/d6d/d85/da7/f141 x:0 0 0 2024-04-06T03:40:48.423 INFO:tasks.workunit.client.0.smithi094.stdout:3/890: read d9/d20/d2d/d128/dba/d7c/faa [1948632,31139] 0 2024-04-06T03:40:48.428 INFO:tasks.workunit.client.1.smithi136.stdout:0/940: mkdir d1/d6/d2c/d57/d43/d6d/d85/da7/de5/dec/d11e/d142 0 2024-04-06T03:40:48.430 INFO:tasks.workunit.client.0.smithi094.stdout:9/870: link d2/dc/d21/d26/f29 d2/dc/d80/ddd/def/f128 0 2024-04-06T03:40:48.430 INFO:tasks.workunit.client.0.smithi094.stdout:9/871: write d2/dc/d21/d23/d47/d51/f92 [17202,116642] 0 2024-04-06T03:40:48.432 INFO:tasks.workunit.client.0.smithi094.stdout:7/772: symlink d2/d6/dc/d12/lfc 0 2024-04-06T03:40:48.432 INFO:tasks.workunit.client.0.smithi094.stdout:7/773: write d2/d6/dc/d12/d3a/fe8 [809451,82437] 0 2024-04-06T03:40:48.432 INFO:tasks.workunit.client.0.smithi094.stdout:7/774: dread - d2/d6/dc/d12/d9a/ff1 zero size 2024-04-06T03:40:48.433 INFO:tasks.workunit.client.1.smithi136.stdout:0/941: creat d1/d6/d2c/d57/d43/d6d/d85/da7/de5/dec/d11e/f143 x:0 0 0 2024-04-06T03:40:48.436 INFO:tasks.workunit.client.0.smithi094.stdout:8/908: symlink d18/d2c/d37/d60/l134 0 2024-04-06T03:40:48.436 INFO:tasks.workunit.client.0.smithi094.stdout:6/822: rmdir dc/d3a/d7f/dd4 39 2024-04-06T03:40:48.438 INFO:tasks.workunit.client.1.smithi136.stdout:6/837: sync 2024-04-06T03:40:48.438 INFO:tasks.workunit.client.1.smithi136.stdout:6/838: creat d2/d93/f120 x:0 0 0 2024-04-06T03:40:48.439 INFO:tasks.workunit.client.1.smithi136.stdout:0/942: rename d1/d6/d2c/d1a/d36/d59/dff to d1/d6/d2c/d57/d43/d6d/d85/d86/d144 0 2024-04-06T03:40:48.439 INFO:tasks.workunit.client.1.smithi136.stdout:0/943: read - d1/d6/d2c/d57/d43/d6d/d9b/f10a zero size 2024-04-06T03:40:48.439 INFO:tasks.workunit.client.1.smithi136.stdout:0/944: readlink d1/d6/d2c/d1a/d36/d59/l8e 0 2024-04-06T03:40:48.440 INFO:tasks.workunit.client.0.smithi094.stdout:3/891: symlink d9/d20/d2d/d128/l140 0 2024-04-06T03:40:48.440 INFO:tasks.workunit.client.0.smithi094.stdout:3/892: chown d9/fc4 89 1 2024-04-06T03:40:48.449 INFO:tasks.workunit.client.0.smithi094.stdout:6/823: symlink dc/d11/l11f 0 2024-04-06T03:40:48.449 INFO:tasks.workunit.client.0.smithi094.stdout:6/824: dread - dc/dd3/d72/d88/dfb/d112/f117 zero size 2024-04-06T03:40:48.450 INFO:tasks.workunit.client.1.smithi136.stdout:0/945: fsync d1/d6/d2c/f40 0 2024-04-06T03:40:48.451 INFO:tasks.workunit.client.1.smithi136.stdout:6/839: rmdir d2/d7/d26/d30/d6e/d11c 39 2024-04-06T03:40:48.453 INFO:tasks.workunit.client.1.smithi136.stdout:0/946: creat d1/d6/d2c/d57/d43/d6d/d85/da7/de5/dec/f145 x:0 0 0 2024-04-06T03:40:48.457 INFO:tasks.workunit.client.0.smithi094.stdout:6/825: read dc/dd3/f52 [1659461,60107] 0 2024-04-06T03:40:48.458 INFO:tasks.workunit.client.0.smithi094.stdout:6/826: mkdir dc/d3a/d120 0 2024-04-06T03:40:48.458 INFO:tasks.workunit.client.0.smithi094.stdout:9/872: dwrite d2/dc/d21/d23/d2f/f5f [0,4194304] 0 2024-04-06T03:40:48.459 INFO:tasks.workunit.client.0.smithi094.stdout:6/827: symlink dc/dd3/d72/l121 0 2024-04-06T03:40:48.460 INFO:tasks.workunit.client.0.smithi094.stdout:6/828: rename dc/d3a/f41 to dc/d3a/df4/df6/d7b/dc6/f122 0 2024-04-06T03:40:48.461 INFO:tasks.workunit.client.0.smithi094.stdout:6/829: mknod dc/d3a/df4/c123 0 2024-04-06T03:40:48.462 INFO:tasks.workunit.client.0.smithi094.stdout:6/830: link dc/dd3/l36 dc/d3a/df4/df6/d7b/dc6/d113/l124 0 2024-04-06T03:40:48.464 INFO:tasks.workunit.client.0.smithi094.stdout:6/831: link dc/d3a/df4/df6/df2/f108 dc/d3a/df4/f125 0 2024-04-06T03:40:48.464 INFO:tasks.workunit.client.0.smithi094.stdout:6/832: fsync dc/d3a/d4a/d54/d71/f11e 0 2024-04-06T03:40:48.465 INFO:tasks.workunit.client.0.smithi094.stdout:6/833: creat dc/d3a/d4a/d54/dde/f126 x:0 0 0 2024-04-06T03:40:48.465 INFO:tasks.workunit.client.0.smithi094.stdout:6/834: chown dc/d11/l4c 10132968 1 2024-04-06T03:40:48.465 INFO:tasks.workunit.client.0.smithi094.stdout:6/835: creat dc/d3a/d4a/d54/d71/f127 x:0 0 0 2024-04-06T03:40:48.465 INFO:tasks.workunit.client.0.smithi094.stdout:6/836: stat dc/d3a/df4/df6 0 2024-04-06T03:40:48.489 INFO:tasks.workunit.client.0.smithi094.stdout:7/775: dwrite d2/d6/dc/d12/d9a/da0/dd4/f9c [0,4194304] 0 2024-04-06T03:40:48.489 INFO:tasks.workunit.client.0.smithi094.stdout:7/776: write d2/d6/dc/dd/f48 [531848,18804] 0 2024-04-06T03:40:48.490 INFO:tasks.workunit.client.0.smithi094.stdout:7/777: creat d2/d6/dc/dd/df/d36/ffd x:0 0 0 2024-04-06T03:40:48.490 INFO:tasks.workunit.client.0.smithi094.stdout:8/909: dwrite d18/d2c/d37/d60/da3/f11f [0,4194304] 0 2024-04-06T03:40:48.490 INFO:tasks.workunit.client.0.smithi094.stdout:6/837: dwrite f5 [0,4194304] 0 2024-04-06T03:40:48.490 INFO:tasks.workunit.client.0.smithi094.stdout:6/838: fsync dc/fed 0 2024-04-06T03:40:48.490 INFO:tasks.workunit.client.0.smithi094.stdout:9/873: read d2/dc/d10/fb2 [1867223,64780] 0 2024-04-06T03:40:48.490 INFO:tasks.workunit.client.0.smithi094.stdout:9/874: chown d2/dc/c59 3448971 1 2024-04-06T03:40:48.491 INFO:tasks.workunit.client.0.smithi094.stdout:3/893: dwrite d9/d1e/d26/f10e [0,4194304] 0 2024-04-06T03:40:48.491 INFO:tasks.workunit.client.0.smithi094.stdout:1/901: dwrite d0/d1/d27/da5/d40/db1/d2f/d4b/f9e [0,4194304] 0 2024-04-06T03:40:48.492 INFO:tasks.workunit.client.0.smithi094.stdout:1/902: dread - d0/d1/d27/da5/fc4 zero size 2024-04-06T03:40:48.494 INFO:tasks.workunit.client.0.smithi094.stdout:7/778: mkdir d2/d6/dc/d12/d29/d3f/dce/dfe 0 2024-04-06T03:40:48.495 INFO:tasks.workunit.client.0.smithi094.stdout:6/839: creat dc/d3a/d4a/f128 x:0 0 0 2024-04-06T03:40:48.496 INFO:tasks.workunit.client.0.smithi094.stdout:9/875: creat d2/dc/d80/d118/f129 x:0 0 0 2024-04-06T03:40:48.496 INFO:tasks.workunit.client.0.smithi094.stdout:9/876: truncate d2/dc/d46/fde 1383095 0 2024-04-06T03:40:48.497 INFO:tasks.workunit.client.0.smithi094.stdout:9/877: write d2/dc/d10/d2a/fcc [4699437,14591] 0 2024-04-06T03:40:48.501 INFO:tasks.workunit.client.0.smithi094.stdout:3/894: read d9/d1e/d26/d6c/ddd/ffc [4153760,106414] 0 2024-04-06T03:40:48.501 INFO:tasks.workunit.client.0.smithi094.stdout:7/779: truncate d2/d6/dac/f70 5238285 0 2024-04-06T03:40:48.502 INFO:tasks.workunit.client.0.smithi094.stdout:6/840: stat dc/c6e 0 2024-04-06T03:40:48.504 INFO:tasks.workunit.client.0.smithi094.stdout:9/878: symlink d2/dc/d21/d23/db0/l12a 0 2024-04-06T03:40:48.506 INFO:tasks.workunit.client.0.smithi094.stdout:3/895: creat d9/d20/d2d/d128/dba/d3a/daf/f141 x:0 0 0 2024-04-06T03:40:48.508 INFO:tasks.workunit.client.0.smithi094.stdout:7/780: mkdir d2/d6/dc/d12/d29/d3f/dff 0 2024-04-06T03:40:48.510 INFO:tasks.workunit.client.0.smithi094.stdout:8/910: dwrite d18/d3c/dc4/dbc/f93 [0,4194304] 0 2024-04-06T03:40:48.510 INFO:tasks.workunit.client.0.smithi094.stdout:8/911: fsync d18/f63 0 2024-04-06T03:40:48.515 INFO:tasks.workunit.client.0.smithi094.stdout:9/879: creat d2/dc/d46/d111/d33/f12b x:0 0 0 2024-04-06T03:40:48.515 INFO:tasks.workunit.client.0.smithi094.stdout:8/912: rmdir d18/d34 39 2024-04-06T03:40:48.515 INFO:tasks.workunit.client.0.smithi094.stdout:1/903: dwrite d0/d1/d113/fc0 [0,4194304] 0 2024-04-06T03:40:48.516 INFO:tasks.workunit.client.0.smithi094.stdout:9/880: dread d2/dc/d21/d23/fe1 [0,4194304] 0 2024-04-06T03:40:48.516 INFO:tasks.workunit.client.0.smithi094.stdout:8/913: mknod d18/d3c/d49/d91/d114/c135 0 2024-04-06T03:40:48.516 INFO:tasks.workunit.client.0.smithi094.stdout:9/881: chown d2/dc/d21/d23/d47/d51/dd8 10398714 1 2024-04-06T03:40:48.516 INFO:tasks.workunit.client.0.smithi094.stdout:8/914: getdents d18/d3c/d11a/d127 0 2024-04-06T03:40:48.521 INFO:tasks.workunit.client.1.smithi136.stdout:1/981: truncate d12/d3b/d74/d7a/fe9 1465969 0 2024-04-06T03:40:48.521 INFO:tasks.workunit.client.1.smithi136.stdout:1/982: dread - d12/d3b/d60/d7e/d37/ffb zero size 2024-04-06T03:40:48.521 INFO:tasks.workunit.client.1.smithi136.stdout:1/983: write d12/d3b/d8e/fbc [1264575,31744] 0 2024-04-06T03:40:48.521 INFO:tasks.workunit.client.0.smithi094.stdout:1/904: mkdir d0/d127/dff/d115/d13a 0 2024-04-06T03:40:48.523 INFO:tasks.workunit.client.0.smithi094.stdout:8/915: link d18/f74 d18/d2c/f136 0 2024-04-06T03:40:48.524 INFO:tasks.workunit.client.1.smithi136.stdout:1/984: mknod d12/d3b/d74/d7a/dea/d11a/d9f/c14d 0 2024-04-06T03:40:48.526 INFO:tasks.workunit.client.0.smithi094.stdout:9/882: symlink d2/dc/d21/d23/d47/l12c 0 2024-04-06T03:40:48.528 INFO:tasks.workunit.client.0.smithi094.stdout:8/916: creat d18/d2c/d37/d115/f137 x:0 0 0 2024-04-06T03:40:48.529 INFO:tasks.workunit.client.0.smithi094.stdout:9/883: dread d2/dc/d21/fb5 [0,4194304] 0 2024-04-06T03:40:48.529 INFO:tasks.workunit.client.0.smithi094.stdout:9/884: chown d2/dc/d21/d26/d78/d69/da8/l106 3202 1 2024-04-06T03:40:48.530 INFO:tasks.workunit.client.0.smithi094.stdout:6/841: dwrite dc/f17 [0,4194304] 0 2024-04-06T03:40:48.530 INFO:tasks.workunit.client.0.smithi094.stdout:1/905: dread d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/f12d [0,4194304] 0 2024-04-06T03:40:48.531 INFO:tasks.workunit.client.1.smithi136.stdout:1/985: creat d12/d1b/d120/f14e x:0 0 0 2024-04-06T03:40:48.533 INFO:tasks.workunit.client.0.smithi094.stdout:8/917: creat d18/d3c/d76/dd9/f138 x:0 0 0 2024-04-06T03:40:48.538 INFO:tasks.workunit.client.0.smithi094.stdout:3/896: dwrite d9/f19 [0,4194304] 0 2024-04-06T03:40:48.542 INFO:tasks.workunit.client.0.smithi094.stdout:7/781: dwrite d2/d6/f24 [0,4194304] 0 2024-04-06T03:40:48.543 INFO:tasks.workunit.client.0.smithi094.stdout:9/885: dread d2/dc/d21/d26/d78/d69/da8/f3a [0,4194304] 0 2024-04-06T03:40:48.546 INFO:tasks.workunit.client.0.smithi094.stdout:1/906: dread d0/f93 [0,4194304] 0 2024-04-06T03:40:48.547 INFO:tasks.workunit.client.0.smithi094.stdout:3/897: write d9/d20/d30/f69 [8381101,67830] 0 2024-04-06T03:40:48.551 INFO:tasks.workunit.client.1.smithi136.stdout:1/986: dwrite d12/d3b/d44/d5f/f7f [4194304,4194304] 0 2024-04-06T03:40:48.551 INFO:tasks.workunit.client.1.smithi136.stdout:1/987: dread - d12/d3b/d60/d7e/d37/dbb/f135 zero size 2024-04-06T03:40:48.560 INFO:tasks.workunit.client.0.smithi094.stdout:9/886: dwrite d2/dc/d80/ddd/def/f128 [4194304,4194304] 0 2024-04-06T03:40:48.576 INFO:tasks.workunit.client.0.smithi094.stdout:9/887: dwrite d2/dc/d21/d23/d5b/ff0 [0,4194304] 0 2024-04-06T03:40:48.576 INFO:tasks.workunit.client.0.smithi094.stdout:9/888: chown d2/dc/d21/d23/d47/d51/dee/f126 20303 1 2024-04-06T03:40:48.611 INFO:tasks.workunit.client.0.smithi094.stdout:6/842: mknod dc/dd3/d72/d88/da3/dc4/c129 0 2024-04-06T03:40:48.611 INFO:tasks.workunit.client.0.smithi094.stdout:6/843: chown dc/d11/c45 384035 1 2024-04-06T03:40:48.616 INFO:tasks.workunit.client.1.smithi136.stdout:9/982: sync 2024-04-06T03:40:48.616 INFO:tasks.workunit.client.1.smithi136.stdout:4/832: sync 2024-04-06T03:40:48.616 INFO:tasks.workunit.client.1.smithi136.stdout:3/915: sync 2024-04-06T03:40:48.616 INFO:tasks.workunit.client.1.smithi136.stdout:3/916: write d0/d4/d12/d19/d55/fec [616900,38427] 0 2024-04-06T03:40:48.622 INFO:tasks.workunit.client.0.smithi094.stdout:7/782: rmdir d2/d6/d93 39 2024-04-06T03:40:48.634 INFO:tasks.workunit.client.0.smithi094.stdout:1/907: creat d0/d127/dff/d115/d13a/f13b x:0 0 0 2024-04-06T03:40:48.634 INFO:tasks.workunit.client.1.smithi136.stdout:6/840: sync 2024-04-06T03:40:48.634 INFO:tasks.workunit.client.1.smithi136.stdout:0/947: sync 2024-04-06T03:40:48.635 INFO:tasks.workunit.client.0.smithi094.stdout:7/783: dread d2/d6/dc/d12/d29/f76 [0,4194304] 0 2024-04-06T03:40:48.636 INFO:tasks.workunit.client.1.smithi136.stdout:4/833: dwrite d1d/d2f/da6/ffa [0,4194304] 0 2024-04-06T03:40:48.637 INFO:tasks.workunit.client.1.smithi136.stdout:4/834: dread - d1d/d31/d3e/d55/f11e zero size 2024-04-06T03:40:48.637 INFO:tasks.workunit.client.1.smithi136.stdout:4/835: fdatasync d1d/d31/d3e/d56/f6f 0 2024-04-06T03:40:48.637 INFO:tasks.workunit.client.1.smithi136.stdout:9/983: dwrite d17/d21/d29/de5/d4b/d14e/fb1 [0,4194304] 0 2024-04-06T03:40:48.639 INFO:tasks.workunit.client.1.smithi136.stdout:9/984: dread d17/d21/d32/dc6/d113/d66/de1/d51/f6d [0,4194304] 0 2024-04-06T03:40:48.644 INFO:tasks.workunit.client.1.smithi136.stdout:4/836: write d1d/d31/d3e/d56/f61 [3827539,14507] 0 2024-04-06T03:40:48.652 INFO:tasks.workunit.client.1.smithi136.stdout:4/837: write d1d/d2f/f97 [4140914,126449] 0 2024-04-06T03:40:48.652 INFO:tasks.workunit.client.1.smithi136.stdout:4/838: fsync d1d/d31/f48 0 2024-04-06T03:40:48.712 INFO:tasks.workunit.client.0.smithi094.stdout:9/889: creat d2/f12d x:0 0 0 2024-04-06T03:40:48.712 INFO:tasks.workunit.client.0.smithi094.stdout:9/890: fdatasync d2/dc/d46/d111/f36 0 2024-04-06T03:40:48.712 INFO:tasks.workunit.client.0.smithi094.stdout:9/891: fsync d2/dc/d46/d111/fa5 0 2024-04-06T03:40:48.776 INFO:tasks.workunit.client.1.smithi136.stdout:3/917: mkdir d0/d4/d12/dc3/dd3/d133 0 2024-04-06T03:40:48.778 INFO:tasks.workunit.client.1.smithi136.stdout:0/948: mknod d1/d6/ded/c146 0 2024-04-06T03:40:48.783 INFO:tasks.workunit.client.1.smithi136.stdout:9/985: mknod d17/d21/d32/d3f/c150 0 2024-04-06T03:40:48.784 INFO:tasks.workunit.client.1.smithi136.stdout:9/986: fdatasync d17/d21/d32/dc6/d113/d66/de1/f9f 0 2024-04-06T03:40:48.791 INFO:tasks.workunit.client.1.smithi136.stdout:4/839: mkdir d1d/d2f/da6/de1/d10a/d125 0 2024-04-06T03:40:48.794 INFO:tasks.workunit.client.1.smithi136.stdout:3/918: creat d0/d4/d12/d19/d55/d3c/d5a/d79/d98/dc6/f134 x:0 0 0 2024-04-06T03:40:48.794 INFO:tasks.workunit.client.1.smithi136.stdout:3/919: chown d0/d4/d12/d2d/f39 3421953 1 2024-04-06T03:40:48.794 INFO:tasks.workunit.client.1.smithi136.stdout:3/920: write d0/d4/d12/d19/d55/d6f/d82/f103 [398416,124875] 0 2024-04-06T03:40:48.794 INFO:tasks.workunit.client.1.smithi136.stdout:3/921: write d0/d4/d12/d19/d55/d3c/d5a/d79/d98/fda [823769,128655] 0 2024-04-06T03:40:48.796 INFO:tasks.workunit.client.1.smithi136.stdout:9/987: dwrite d17/d20/d25/f100 [0,4194304] 0 2024-04-06T03:40:48.796 INFO:tasks.workunit.client.1.smithi136.stdout:9/988: write d17/d20/d28/fae [4921146,34278] 0 2024-04-06T03:40:48.796 INFO:tasks.workunit.client.1.smithi136.stdout:9/989: write d17/d21/d29/de5/d4b/d14e/f118 [723765,83896] 0 2024-04-06T03:40:48.797 INFO:tasks.workunit.client.1.smithi136.stdout:6/841: rename d2/d7/d9e to d2/d7/d26/d30/d6e/d3d/d47/d121 0 2024-04-06T03:40:48.801 INFO:tasks.workunit.client.1.smithi136.stdout:3/922: mknod d0/d4/d12/d19/d55/d6f/ddb/df9/c135 0 2024-04-06T03:40:48.811 INFO:tasks.workunit.client.1.smithi136.stdout:1/988: symlink d12/d3b/d44/d5f/l14f 0 2024-04-06T03:40:48.812 INFO:tasks.workunit.client.1.smithi136.stdout:9/990: unlink d17/d21/d29/de5/da5/df5/f14c 0 2024-04-06T03:40:48.812 INFO:tasks.workunit.client.1.smithi136.stdout:9/991: readlink d17/d20/d25/d3c/d55/l128 0 2024-04-06T03:40:48.812 INFO:tasks.workunit.client.1.smithi136.stdout:9/992: creat d17/d20/d25/da4/dbb/de4/f151 x:0 0 0 2024-04-06T03:40:48.812 INFO:tasks.workunit.client.1.smithi136.stdout:9/993: creat d17/d21/d29/de5/d4b/f152 x:0 0 0 2024-04-06T03:40:48.814 INFO:tasks.workunit.client.1.smithi136.stdout:9/994: dread d17/d21/d32/fc7 [0,4194304] 0 2024-04-06T03:40:48.825 INFO:tasks.workunit.client.1.smithi136.stdout:0/949: rename d1/d6/ded/f13e to d1/d6/df5/f147 0 2024-04-06T03:40:48.828 INFO:tasks.workunit.client.1.smithi136.stdout:6/842: symlink d2/l122 0 2024-04-06T03:40:48.829 INFO:tasks.workunit.client.0.smithi094.stdout:6/844: mkdir dc/d3a/d4a/d54/dde/d50/daf/d12a 0 2024-04-06T03:40:48.829 INFO:tasks.workunit.client.0.smithi094.stdout:6/845: truncate dc/dd3/d4e/fd7 857969 0 2024-04-06T03:40:48.831 INFO:tasks.workunit.client.0.smithi094.stdout:1/908: stat d0/d7/d1a/da8/l11b 0 2024-04-06T03:40:48.835 INFO:tasks.workunit.client.1.smithi136.stdout:1/989: read fb [216420,27179] 0 2024-04-06T03:40:48.836 INFO:tasks.workunit.client.0.smithi094.stdout:6/846: write dc/dd3/d24/f2e [3417355,42669] 0 2024-04-06T03:40:48.837 INFO:tasks.workunit.client.0.smithi094.stdout:9/892: mkdir d2/dc/d21/d26/d8a/d12e 0 2024-04-06T03:40:48.838 INFO:tasks.workunit.client.1.smithi136.stdout:9/995: rmdir d17/d21/d29/de5/d4b/d14e/d9e/dfb 39 2024-04-06T03:40:48.846 INFO:tasks.workunit.client.1.smithi136.stdout:1/990: symlink d12/d3b/d74/l150 0 2024-04-06T03:40:48.846 INFO:tasks.workunit.client.1.smithi136.stdout:1/991: stat d12/d3b/f84 0 2024-04-06T03:40:48.846 INFO:tasks.workunit.client.1.smithi136.stdout:1/992: dread - d12/d3b/d60/d7e/d37/f10f zero size 2024-04-06T03:40:48.846 INFO:tasks.workunit.client.1.smithi136.stdout:1/993: write d12/d3b/d44/d5f/d5a/f62 [2114148,59063] 0 2024-04-06T03:40:48.849 INFO:tasks.workunit.client.1.smithi136.stdout:0/950: rmdir d1/d6/d2c/d1a/d9c 39 2024-04-06T03:40:48.849 INFO:tasks.workunit.client.1.smithi136.stdout:0/951: chown d1/d6/d2c/d57/d43/fa2 65922 1 2024-04-06T03:40:48.849 INFO:tasks.workunit.client.1.smithi136.stdout:0/952: write d1/d6/df5/f140 [364218,74905] 0 2024-04-06T03:40:48.850 INFO:tasks.workunit.client.0.smithi094.stdout:1/909: mknod d0/d1/d27/da5/db2/dce/c13c 0 2024-04-06T03:40:48.851 INFO:tasks.workunit.client.0.smithi094.stdout:1/910: stat d0/d7/d2d/f6e 0 2024-04-06T03:40:48.851 INFO:tasks.workunit.client.1.smithi136.stdout:0/953: dread d1/d6/d2c/d1a/d36/d59/fbe [0,4194304] 0 2024-04-06T03:40:48.851 INFO:tasks.workunit.client.1.smithi136.stdout:0/954: truncate d1/d6/d2c/d1a/d36/d59/f38 684288 0 2024-04-06T03:40:48.851 INFO:tasks.workunit.client.1.smithi136.stdout:0/955: fdatasync d1/d6/d2c/d1a/d36/d59/f92 0 2024-04-06T03:40:48.853 INFO:tasks.workunit.client.0.smithi094.stdout:6/847: dwrite dc/d3a/d4a/d54/dde/f126 [0,4194304] 0 2024-04-06T03:40:48.853 INFO:tasks.workunit.client.0.smithi094.stdout:6/848: write dc/dd3/d72/d88/da3/faa [150095,123467] 0 2024-04-06T03:40:48.853 INFO:tasks.workunit.client.0.smithi094.stdout:6/849: getdents dc/d3a/d120 0 2024-04-06T03:40:48.857 INFO:tasks.workunit.client.0.smithi094.stdout:1/911: symlink d0/d1/d27/da5/d40/d50/l13d 0 2024-04-06T03:40:48.861 INFO:tasks.workunit.client.1.smithi136.stdout:0/956: link d1/d6/d2c/d57/d43/d6d/d85/d78/da4/cdd d1/d6/ded/c148 0 2024-04-06T03:40:48.863 INFO:tasks.workunit.client.1.smithi136.stdout:3/923: rename d0/d4/d12/d19/d55/d6f/df0 to d0/d4/d12/d19/d55/d3c/d136 0 2024-04-06T03:40:48.863 INFO:tasks.workunit.client.1.smithi136.stdout:3/924: fsync d0/d4/d12/d19/d55/d6f/d82/d5d/d8c/ffd 0 2024-04-06T03:40:48.866 INFO:tasks.workunit.client.0.smithi094.stdout:1/912: mknod d0/d7/d2d/d11f/c13e 0 2024-04-06T03:40:48.869 INFO:tasks.workunit.client.1.smithi136.stdout:6/843: rename d2/d7/d26/d30/dc4/fda to d2/d7/d26/d97/f123 0 2024-04-06T03:40:48.869 INFO:tasks.workunit.client.1.smithi136.stdout:6/844: dread - d2/d7/d26/d30/dc4/ff7 zero size 2024-04-06T03:40:48.871 INFO:tasks.workunit.client.0.smithi094.stdout:1/913: truncate d0/d1/d27/f2e 569376 0 2024-04-06T03:40:48.871 INFO:tasks.workunit.client.0.smithi094.stdout:1/914: readlink d0/l3e 0 2024-04-06T03:40:48.871 INFO:tasks.workunit.client.0.smithi094.stdout:1/915: chown d0/d7/f86 2041535 1 2024-04-06T03:40:48.871 INFO:tasks.workunit.client.0.smithi094.stdout:1/916: write d0/d127/d7d/fae [1616683,109196] 0 2024-04-06T03:40:48.872 INFO:tasks.workunit.client.1.smithi136.stdout:0/957: write d1/fc9 [6041202,23935] 0 2024-04-06T03:40:48.872 INFO:tasks.workunit.client.1.smithi136.stdout:0/958: creat d1/d53/f149 x:0 0 0 2024-04-06T03:40:48.872 INFO:tasks.workunit.client.0.smithi094.stdout:6/850: dread dc/dd3/d72/d88/f9f [0,4194304] 0 2024-04-06T03:40:48.887 INFO:tasks.workunit.client.1.smithi136.stdout:3/925: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/f137 x:0 0 0 2024-04-06T03:40:48.887 INFO:tasks.workunit.client.1.smithi136.stdout:3/926: write d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/f84 [1216101,124191] 0 2024-04-06T03:40:48.888 INFO:tasks.workunit.client.1.smithi136.stdout:3/927: mknod d0/d4/d12/d19/d55/d6f/ddb/df9/c138 0 2024-04-06T03:40:48.888 INFO:tasks.workunit.client.1.smithi136.stdout:3/928: getdents d0/d4/d12/d3a/d69 0 2024-04-06T03:40:48.889 INFO:tasks.workunit.client.1.smithi136.stdout:3/929: truncate d0/d4/d12/d19/d55/d3c/f58 2682062 0 2024-04-06T03:40:48.889 INFO:tasks.workunit.client.1.smithi136.stdout:3/930: chown d0/d4/d12/d19/d2e/lc1 87531 1 2024-04-06T03:40:48.892 INFO:tasks.workunit.client.1.smithi136.stdout:3/931: rmdir d0/d4/d12/d19/d55/d6f/d82/d5d/d8c 39 2024-04-06T03:40:48.895 INFO:tasks.workunit.client.0.smithi094.stdout:1/917: dwrite d0/d127/fd0 [0,4194304] 0 2024-04-06T03:40:48.895 INFO:tasks.workunit.client.1.smithi136.stdout:6/845: dwrite d2/d104/d75/f82 [0,4194304] 0 2024-04-06T03:40:48.895 INFO:tasks.workunit.client.1.smithi136.stdout:6/846: chown d2/d7/d26/d30/d6e/d58/d5d/d62/d8d 15 1 2024-04-06T03:40:48.900 INFO:tasks.workunit.client.1.smithi136.stdout:1/994: unlink d12/d3b/d74/d7a/f8d 0 2024-04-06T03:40:48.902 INFO:tasks.workunit.client.1.smithi136.stdout:6/847: dread d2/d7/d26/d30/d6e/f6b [0,4194304] 0 2024-04-06T03:40:48.902 INFO:tasks.workunit.client.0.smithi094.stdout:1/918: write d0/d1/d27/da5/d40/d50/fbf [2884020,14300] 0 2024-04-06T03:40:48.902 INFO:tasks.workunit.client.0.smithi094.stdout:1/919: write d0/d1/d27/da5/d40/db1/d2f/f10e [755000,2199] 0 2024-04-06T03:40:48.903 INFO:tasks.workunit.client.1.smithi136.stdout:0/959: rmdir d1/d6/d2c/d57/d43/d6d/d85/da7/de5 39 2024-04-06T03:40:48.905 INFO:tasks.workunit.client.1.smithi136.stdout:6/848: mkdir d2/d124 0 2024-04-06T03:40:48.906 INFO:tasks.workunit.client.0.smithi094.stdout:1/920: getdents d0/d127/d7d 0 2024-04-06T03:40:48.908 INFO:tasks.workunit.client.0.smithi094.stdout:1/921: mkdir d0/d1/d27/da5/d40/d50/d13f 0 2024-04-06T03:40:48.910 INFO:tasks.workunit.client.1.smithi136.stdout:6/849: creat d2/d7/d26/f125 x:0 0 0 2024-04-06T03:40:48.922 INFO:tasks.workunit.client.1.smithi136.stdout:3/932: creat d0/d4/d12/d19/f139 x:0 0 0 2024-04-06T03:40:48.922 INFO:tasks.workunit.client.1.smithi136.stdout:3/933: write d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/f125 [836343,99501] 0 2024-04-06T03:40:48.922 INFO:tasks.workunit.client.1.smithi136.stdout:3/934: symlink d0/d4/dd6/l13a 0 2024-04-06T03:40:48.922 INFO:tasks.workunit.client.1.smithi136.stdout:3/935: write d0/d4/d12/d19/d55/f9f [22485,100287] 0 2024-04-06T03:40:48.923 INFO:tasks.workunit.client.1.smithi136.stdout:6/850: dread d2/d25/f38 [4194304,4194304] 0 2024-04-06T03:40:48.923 INFO:tasks.workunit.client.1.smithi136.stdout:6/851: rename d2/d7/d13/d79/dc2/fc9 to d2/d104/d75/f126 0 2024-04-06T03:40:48.928 INFO:tasks.workunit.client.1.smithi136.stdout:0/960: dwrite d1/d6/d2c/d84/fd7 [4194304,4194304] 0 2024-04-06T03:40:48.930 INFO:tasks.workunit.client.1.smithi136.stdout:0/961: link d1/f54 d1/d6/d2c/d1a/d82/f14a 0 2024-04-06T03:40:48.930 INFO:tasks.workunit.client.1.smithi136.stdout:0/962: stat d1/d6/d2c/d57/d62 0 2024-04-06T03:40:48.930 INFO:tasks.workunit.client.1.smithi136.stdout:0/963: chown d1/d6/d2c/d57/f66 60 1 2024-04-06T03:40:48.931 INFO:tasks.workunit.client.1.smithi136.stdout:0/964: truncate d1/d6/d2c/d57/d43/f50 2733438 0 2024-04-06T03:40:48.932 INFO:tasks.workunit.client.1.smithi136.stdout:0/965: chown d1/d6/d2c/d57/d43/d6d/d85/d86 812478166 1 2024-04-06T03:40:48.933 INFO:tasks.workunit.client.1.smithi136.stdout:3/936: write d0/d4/d12/d3a/d51/da6/fb6 [1094895,104075] 0 2024-04-06T03:40:48.934 INFO:tasks.workunit.client.0.smithi094.stdout:7/784: mkdir d2/d6/dc/d12/d29/d100 0 2024-04-06T03:40:48.934 INFO:tasks.workunit.client.0.smithi094.stdout:7/785: creat d2/d6/dc/d12/d29/daa/f101 x:0 0 0 2024-04-06T03:40:48.935 INFO:tasks.workunit.client.1.smithi136.stdout:3/937: rmdir d0 39 2024-04-06T03:40:48.936 INFO:tasks.workunit.client.1.smithi136.stdout:9/996: mknod d17/d21/d32/c153 0 2024-04-06T03:40:48.937 INFO:tasks.workunit.client.0.smithi094.stdout:7/786: mknod d2/d6/dc/d12/d9a/da0/dd4/c102 0 2024-04-06T03:40:48.941 INFO:tasks.workunit.client.1.smithi136.stdout:9/997: rename d17/d19/da3/d101/f119 to d17/d21/d29/f154 0 2024-04-06T03:40:48.943 INFO:tasks.workunit.client.1.smithi136.stdout:3/938: rmdir d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/dd4 39 2024-04-06T03:40:48.943 INFO:tasks.workunit.client.1.smithi136.stdout:3/939: dread - d0/d4/d12/d19/d2e/f123 zero size 2024-04-06T03:40:48.944 INFO:tasks.workunit.client.0.smithi094.stdout:7/787: truncate d2/d6/dc/f2b 2685085 0 2024-04-06T03:40:48.944 INFO:tasks.workunit.client.0.smithi094.stdout:7/788: chown d2/d6/f24 28 1 2024-04-06T03:40:48.946 INFO:tasks.workunit.client.1.smithi136.stdout:9/998: unlink d17/d21/d29/de5/da5/lde 0 2024-04-06T03:40:48.946 INFO:tasks.workunit.client.1.smithi136.stdout:9/999: dread - d17/d20/d25/da4/dbb/fee zero size 2024-04-06T03:40:48.947 INFO:tasks.workunit.client.0.smithi094.stdout:9/893: rmdir d2/dc/d21/d26 39 2024-04-06T03:40:48.947 INFO:tasks.workunit.client.0.smithi094.stdout:9/894: write d2/dc/f104 [134810,40937] 0 2024-04-06T03:40:48.947 INFO:tasks.workunit.client.0.smithi094.stdout:9/895: stat d2/dc/d10/fb2 0 2024-04-06T03:40:48.948 INFO:tasks.workunit.client.1.smithi136.stdout:3/940: mkdir d0/d4/d12/d19/d55/d6f/d82/d5d/d61/d13b 0 2024-04-06T03:40:48.950 INFO:tasks.workunit.client.1.smithi136.stdout:3/941: creat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/df2/f13c x:0 0 0 2024-04-06T03:40:48.953 INFO:tasks.workunit.client.1.smithi136.stdout:0/966: dwrite d1/d6/d2c/f5a [0,4194304] 0 2024-04-06T03:40:48.953 INFO:tasks.workunit.client.0.smithi094.stdout:7/789: dread d2/d6/dc/d12/d3a/fe8 [0,4194304] 0 2024-04-06T03:40:48.954 INFO:tasks.workunit.client.0.smithi094.stdout:8/918: rmdir d18/d2c/d37/d60/d70 39 2024-04-06T03:40:48.955 INFO:tasks.workunit.client.1.smithi136.stdout:6/852: dwrite d2/d7/d26/d30/f4c [0,4194304] 0 2024-04-06T03:40:48.955 INFO:tasks.workunit.client.1.smithi136.stdout:6/853: write d2/d7/d26/d30/d6e/d58/f10f [86125,63015] 0 2024-04-06T03:40:48.955 INFO:tasks.workunit.client.1.smithi136.stdout:6/854: stat d2/d7/d26/l2d 0 2024-04-06T03:40:48.958 INFO:tasks.workunit.client.1.smithi136.stdout:1/995: creat d12/d3b/d44/f151 x:0 0 0 2024-04-06T03:40:48.958 INFO:tasks.workunit.client.1.smithi136.stdout:1/996: read d12/d3b/d74/d7a/dea/d11a/d23/f92 [217692,17086] 0 2024-04-06T03:40:48.960 INFO:tasks.workunit.client.1.smithi136.stdout:0/967: mknod d1/d6/d2c/d1a/d36/d59/c14b 0 2024-04-06T03:40:48.962 INFO:tasks.workunit.client.0.smithi094.stdout:9/896: write d2/dc/d21/d26/f29 [7076671,82160] 0 2024-04-06T03:40:48.964 INFO:tasks.workunit.client.0.smithi094.stdout:8/919: mknod d18/d3c/d49/db5/c139 0 2024-04-06T03:40:48.968 INFO:tasks.workunit.client.1.smithi136.stdout:0/968: mkdir d1/d6/d2c/d57/d43/d6d/d85/d112/d14c 0 2024-04-06T03:40:48.969 INFO:tasks.workunit.client.1.smithi136.stdout:6/855: truncate d2/d93/dab/ff0 389968 0 2024-04-06T03:40:48.969 INFO:tasks.workunit.client.1.smithi136.stdout:6/856: creat d2/d7/d26/d68/ddc/f127 x:0 0 0 2024-04-06T03:40:48.972 INFO:tasks.workunit.client.0.smithi094.stdout:7/790: dwrite d2/d6/dc/d12/d9a/ff1 [0,4194304] 0 2024-04-06T03:40:48.975 INFO:tasks.workunit.client.1.smithi136.stdout:3/942: dwrite d0/d4/d12/d19/d2e/d3b/d49/f124 [0,4194304] 0 2024-04-06T03:40:48.977 INFO:tasks.workunit.client.0.smithi094.stdout:9/897: dread d2/dc/d46/f7e [0,4194304] 0 2024-04-06T03:40:48.979 INFO:tasks.workunit.client.0.smithi094.stdout:8/920: write d18/d30/f40 [3257189,5064] 0 2024-04-06T03:40:48.979 INFO:tasks.workunit.client.0.smithi094.stdout:8/921: write d18/d3c/d49/d91/d94/daa/ddc/ff6 [1368708,19577] 0 2024-04-06T03:40:48.979 INFO:tasks.workunit.client.0.smithi094.stdout:8/922: chown d18/d3c/d49/fd5 40 1 2024-04-06T03:40:48.983 INFO:tasks.workunit.client.0.smithi094.stdout:7/791: symlink d2/d6/dc/d12/d29/d3f/dff/l103 0 2024-04-06T03:40:48.989 INFO:tasks.workunit.client.0.smithi094.stdout:6/851: unlink dc/dd3/d72/d88/c9d 0 2024-04-06T03:40:48.990 INFO:tasks.workunit.client.1.smithi136.stdout:3/943: dwrite d0/d4/d12/d19/d115/f90 [0,4194304] 0 2024-04-06T03:40:48.992 INFO:tasks.workunit.client.1.smithi136.stdout:0/969: dwrite d1/d6/d2c/d1a/f39 [8388608,4194304] 0 2024-04-06T03:40:48.992 INFO:tasks.workunit.client.1.smithi136.stdout:0/970: chown d1/f5e 22 1 2024-04-06T03:40:48.992 INFO:tasks.workunit.client.1.smithi136.stdout:0/971: dread - d1/d6/d2c/d57/d43/d6d/f99 zero size 2024-04-06T03:40:48.992 INFO:tasks.workunit.client.0.smithi094.stdout:8/923: link d18/d3c/d49/d91/d94/daa/fe9 d18/d2c/d37/d60/da3/db1/f13a 0 2024-04-06T03:40:48.995 INFO:tasks.workunit.client.0.smithi094.stdout:7/792: creat d2/d6/dc/dd/df/f104 x:0 0 0 2024-04-06T03:40:48.997 INFO:tasks.workunit.client.1.smithi136.stdout:6/857: dwrite d2/d7/d26/d68/d9b/fae [0,4194304] 0 2024-04-06T03:40:48.998 INFO:tasks.workunit.client.1.smithi136.stdout:6/858: readlink d2/d7/d26/l2d 0 2024-04-06T03:40:48.998 INFO:tasks.workunit.client.0.smithi094.stdout:6/852: rename dc/d3a/d4a/d54/dde/d50/d61/f68 to dc/d3a/d4a/d54/d71/f12b 0 2024-04-06T03:40:49.001 INFO:tasks.workunit.client.0.smithi094.stdout:8/924: creat d18/d2c/dd2/f13b x:0 0 0 2024-04-06T03:40:49.001 INFO:tasks.workunit.client.0.smithi094.stdout:8/925: write d18/d3c/d76/dd9/f89 [1631373,93418] 0 2024-04-06T03:40:49.002 INFO:tasks.workunit.client.0.smithi094.stdout:7/793: write d2/d6/dc/dd/f3b [2307999,111630] 0 2024-04-06T03:40:49.004 INFO:tasks.workunit.client.0.smithi094.stdout:8/926: mkdir d18/d2c/d37/d60/da3/deb/d13c 0 2024-04-06T03:40:49.005 INFO:tasks.workunit.client.0.smithi094.stdout:8/927: fsync d18/d30/f3e 0 2024-04-06T03:40:49.005 INFO:tasks.workunit.client.0.smithi094.stdout:8/928: fdatasync d18/d2c/f47 0 2024-04-06T03:40:49.006 INFO:tasks.workunit.client.0.smithi094.stdout:9/898: dwrite d2/dc/d46/d111/d33/d64/dc6/f117 [0,4194304] 0 2024-04-06T03:40:49.006 INFO:tasks.workunit.client.0.smithi094.stdout:9/899: chown d2/dc/d21/d23/d47/d51/f92 18359 1 2024-04-06T03:40:49.007 INFO:tasks.workunit.client.1.smithi136.stdout:0/972: dwrite d1/d6/d2c/d1a/d36/d59/f48 [0,4194304] 0 2024-04-06T03:40:49.008 INFO:tasks.workunit.client.0.smithi094.stdout:7/794: truncate d2/d6/dc/dd/df/d36/f60 3195708 0 2024-04-06T03:40:49.012 INFO:tasks.workunit.client.0.smithi094.stdout:6/853: dread dc/d3a/d4a/d94/dac/fb8 [0,4194304] 0 2024-04-06T03:40:49.012 INFO:tasks.workunit.client.0.smithi094.stdout:9/900: symlink d2/dc/d46/d111/d33/d11d/l12f 0 2024-04-06T03:40:49.016 INFO:tasks.workunit.client.0.smithi094.stdout:7/795: creat d2/d6/dc/d12/d29/f105 x:0 0 0 2024-04-06T03:40:49.019 INFO:tasks.workunit.client.1.smithi136.stdout:3/944: link d0/f105 d0/d4/d12/dee/f13d 0 2024-04-06T03:40:49.021 INFO:tasks.workunit.client.0.smithi094.stdout:9/901: dread d2/dc/d46/d111/d99/faf [0,4194304] 0 2024-04-06T03:40:49.024 INFO:tasks.workunit.client.1.smithi136.stdout:6/859: creat d2/d7/f128 x:0 0 0 2024-04-06T03:40:49.024 INFO:tasks.workunit.client.1.smithi136.stdout:6/860: creat d2/d7/d26/d30/dc4/f129 x:0 0 0 2024-04-06T03:40:49.024 INFO:tasks.workunit.client.1.smithi136.stdout:6/861: creat d2/d7/d13/d89/f12a x:0 0 0 2024-04-06T03:40:49.026 INFO:tasks.workunit.client.0.smithi094.stdout:9/902: rename d2/dc/d46/d111/d99/fb4 to d2/dc/d21/d26/d78/d69/f130 0 2024-04-06T03:40:49.028 INFO:tasks.workunit.client.1.smithi136.stdout:0/973: symlink d1/d6/d2c/d57/d43/d6d/d9b/l14d 0 2024-04-06T03:40:49.028 INFO:tasks.workunit.client.1.smithi136.stdout:0/974: fsync d1/d6/d2c/d57/d43/d6d/d85/d78/f9a 0 2024-04-06T03:40:49.028 INFO:tasks.workunit.client.0.smithi094.stdout:4/958: sync 2024-04-06T03:40:49.029 INFO:tasks.workunit.client.0.smithi094.stdout:3/898: sync 2024-04-06T03:40:49.029 INFO:tasks.workunit.client.0.smithi094.stdout:5/851: sync 2024-04-06T03:40:49.029 INFO:tasks.workunit.client.0.smithi094.stdout:0/869: sync 2024-04-06T03:40:49.029 INFO:tasks.workunit.client.0.smithi094.stdout:2/860: sync 2024-04-06T03:40:49.029 INFO:tasks.workunit.client.0.smithi094.stdout:5/852: readlink db/d19/d2c/ld9 0 2024-04-06T03:40:49.029 INFO:tasks.workunit.client.0.smithi094.stdout:5/853: chown db/d19/d2c/d6d/c86 721100783 1 2024-04-06T03:40:49.031 INFO:tasks.workunit.client.1.smithi136.stdout:4/840: sync 2024-04-06T03:40:49.031 INFO:tasks.workunit.client.0.smithi094.stdout:3/899: creat d9/de5/f142 x:0 0 0 2024-04-06T03:40:49.031 INFO:tasks.workunit.client.0.smithi094.stdout:3/900: chown d9/d9d/d129/c13f 462 1 2024-04-06T03:40:49.035 INFO:tasks.workunit.client.0.smithi094.stdout:5/854: creat db/d19/d21/d11e/d28/d85/f12a x:0 0 0 2024-04-06T03:40:49.037 INFO:tasks.workunit.client.1.smithi136.stdout:6/862: symlink d2/d7/d26/d97/d115/l12b 0 2024-04-06T03:40:49.037 INFO:tasks.workunit.client.1.smithi136.stdout:4/841: rmdir d1d/d31/d3e/d55/d89/dea 39 2024-04-06T03:40:49.039 INFO:tasks.workunit.client.1.smithi136.stdout:0/975: dread d1/d6/d2c/d57/d43/d6b/f13a [0,4194304] 0 2024-04-06T03:40:49.040 INFO:tasks.workunit.client.1.smithi136.stdout:6/863: symlink d2/d92/l12c 0 2024-04-06T03:40:49.042 INFO:tasks.workunit.client.1.smithi136.stdout:0/976: unlink d1/d6/d2c/d57/d43/d6d/d85/da7/de5/c107 0 2024-04-06T03:40:49.042 INFO:tasks.workunit.client.1.smithi136.stdout:6/864: truncate d2/d92/fc1 969439 0 2024-04-06T03:40:49.043 INFO:tasks.workunit.client.1.smithi136.stdout:6/865: dread - d2/d7/d26/f125 zero size 2024-04-06T03:40:49.043 INFO:tasks.workunit.client.0.smithi094.stdout:2/861: write d0/dd/d3d/f88 [1225583,31899] 0 2024-04-06T03:40:49.043 INFO:tasks.workunit.client.0.smithi094.stdout:2/862: chown d0/d8/d4f/dab/dc3 14 1 2024-04-06T03:40:49.044 INFO:tasks.workunit.client.1.smithi136.stdout:6/866: truncate d2/d7/d26/d68/ddc/fe4 402038 0 2024-04-06T03:40:49.044 INFO:tasks.workunit.client.1.smithi136.stdout:6/867: truncate d2/d7/d26/d30/d6e/d3d/f46 2350381 0 2024-04-06T03:40:49.044 INFO:tasks.workunit.client.1.smithi136.stdout:4/842: mkdir d1d/d31/d126 0 2024-04-06T03:40:49.046 INFO:tasks.workunit.client.0.smithi094.stdout:2/863: rename d0/dd/d19/d31/dcc/c104 to d0/dd/d19/c123 0 2024-04-06T03:40:49.046 INFO:tasks.workunit.client.0.smithi094.stdout:2/864: creat d0/dd/f124 x:0 0 0 2024-04-06T03:40:49.046 INFO:tasks.workunit.client.0.smithi094.stdout:2/865: chown d0/dd/d19/d31/dcc/ff4 77073 1 2024-04-06T03:40:49.046 INFO:tasks.workunit.client.0.smithi094.stdout:2/866: write d0/d8/d27/d2b/fdd [5094654,8323] 0 2024-04-06T03:40:49.046 INFO:tasks.workunit.client.1.smithi136.stdout:0/977: symlink d1/d6/d2c/d57/de3/l14e 0 2024-04-06T03:40:49.047 INFO:tasks.workunit.client.1.smithi136.stdout:0/978: creat d1/d6/d2c/d57/d43/d9e/f14f x:0 0 0 2024-04-06T03:40:49.047 INFO:tasks.workunit.client.1.smithi136.stdout:6/868: dread d2/d7/d26/d30/d6e/d3d/f105 [0,4194304] 0 2024-04-06T03:40:49.050 INFO:tasks.workunit.client.1.smithi136.stdout:0/979: creat d1/d6/d2c/d1a/d36/d12a/f150 x:0 0 0 2024-04-06T03:40:49.051 INFO:tasks.workunit.client.1.smithi136.stdout:4/843: dread d1d/d31/d3e/d56/f9b [0,4194304] 0 2024-04-06T03:40:49.053 INFO:tasks.workunit.client.1.smithi136.stdout:0/980: mkdir d1/d6/d2c/d57/d43/d6d/d85/d112/d14c/d151 0 2024-04-06T03:40:49.068 INFO:tasks.workunit.client.0.smithi094.stdout:0/870: dwrite d1/d9/dd/d18/d1e/d86/d96/dc0/f104 [0,4194304] 0 2024-04-06T03:40:49.069 INFO:tasks.workunit.client.1.smithi136.stdout:0/981: dwrite d1/f29 [0,4194304] 0 2024-04-06T03:40:49.069 INFO:tasks.workunit.client.1.smithi136.stdout:0/982: getdents d1/d6/d2c/d57/d43/d6d/d85/da7/de5/dec/d11e/d142 0 2024-04-06T03:40:49.069 INFO:tasks.workunit.client.0.smithi094.stdout:0/871: readlink d1/d9/dd/d18/d1e/l4c 0 2024-04-06T03:40:49.070 INFO:tasks.workunit.client.0.smithi094.stdout:0/872: fdatasync d1/d9/fc 0 2024-04-06T03:40:49.071 INFO:tasks.workunit.client.0.smithi094.stdout:0/873: dread d1/d9/dd/d18/d1e/def/dab/f4f [0,4194304] 0 2024-04-06T03:40:49.072 INFO:tasks.workunit.client.0.smithi094.stdout:0/874: readlink d1/d9/dd/d18/d1e/def/dab/d4e/ddf/lb9 0 2024-04-06T03:40:49.073 INFO:tasks.workunit.client.0.smithi094.stdout:0/875: symlink d1/d9/dd/d18/d1e/def/dab/d4e/ddf/l125 0 2024-04-06T03:40:49.103 INFO:tasks.workunit.client.0.smithi094.stdout:6/854: dread dc/d3a/df4/df6/d7b/dc6/f122 [0,4194304] 0 2024-04-06T03:40:49.104 INFO:tasks.workunit.client.0.smithi094.stdout:6/855: creat dc/dd3/d72/d88/f12c x:0 0 0 2024-04-06T03:40:49.105 INFO:tasks.workunit.client.0.smithi094.stdout:6/856: truncate dc/d3a/d4a/f56 4239600 0 2024-04-06T03:40:49.105 INFO:tasks.workunit.client.0.smithi094.stdout:6/857: truncate dc/d3a/fc8 2464906 0 2024-04-06T03:40:49.226 INFO:tasks.workunit.client.1.smithi136.stdout:4/844: dread d1d/d31/d3e/d55/fc7 [0,4194304] 0 2024-04-06T03:40:49.245 INFO:tasks.workunit.client.0.smithi094.stdout:9/903: dwrite d2/dc/d80/ddd/d6b/f108 [0,4194304] 0 2024-04-06T03:40:49.245 INFO:tasks.workunit.client.0.smithi094.stdout:8/929: dwrite d18/d30/f40 [0,4194304] 0 2024-04-06T03:40:49.245 INFO:tasks.workunit.client.0.smithi094.stdout:8/930: truncate d18/d3c/dc4/dbc/fc1 4699866 0 2024-04-06T03:40:49.246 INFO:tasks.workunit.client.1.smithi136.stdout:0/983: dwrite d1/d6/d2c/d1a/d82/f12f [0,4194304] 0 2024-04-06T03:40:49.246 INFO:tasks.workunit.client.0.smithi094.stdout:4/959: dwrite d0/df/f10b [0,4194304] 0 2024-04-06T03:40:49.247 INFO:tasks.workunit.client.0.smithi094.stdout:7/796: dwrite d2/d6/dc/dd/df/d95/dc9/f3d [0,4194304] 0 2024-04-06T03:40:49.247 INFO:tasks.workunit.client.0.smithi094.stdout:7/797: rename d2/d6/dc/dd/df to d2/d6/dc/dd/df/d95/dc9/d57/d106 22 2024-04-06T03:40:49.247 INFO:tasks.workunit.client.0.smithi094.stdout:8/931: write d18/d3c/dc4/fbf [533448,112073] 0 2024-04-06T03:40:49.247 INFO:tasks.workunit.client.0.smithi094.stdout:8/932: fdatasync d18/d3c/fb2 0 2024-04-06T03:40:49.248 INFO:tasks.workunit.client.0.smithi094.stdout:6/858: dwrite dc/d3a/df4/df6/fe2 [0,4194304] 0 2024-04-06T03:40:49.249 INFO:tasks.workunit.client.1.smithi136.stdout:0/984: dread d1/d6/d2c/d57/d62/f65 [0,4194304] 0 2024-04-06T03:40:49.249 INFO:tasks.workunit.client.1.smithi136.stdout:0/985: chown d1/d6/d2c/d57/d43/d6d/d85/la5 840930196 1 2024-04-06T03:40:49.249 INFO:tasks.workunit.client.1.smithi136.stdout:3/945: dwrite d0/f13 [0,4194304] 0 2024-04-06T03:40:49.249 INFO:tasks.workunit.client.1.smithi136.stdout:4/845: dwrite fc [0,4194304] 0 2024-04-06T03:40:49.249 INFO:tasks.workunit.client.1.smithi136.stdout:4/846: fsync d1d/f26 0 2024-04-06T03:40:49.249 INFO:tasks.workunit.client.1.smithi136.stdout:0/986: write d1/d4e/f55 [569067,129440] 0 2024-04-06T03:40:49.249 INFO:tasks.workunit.client.1.smithi136.stdout:0/987: fsync d1/d6/df5/f138 0 2024-04-06T03:40:49.250 INFO:tasks.workunit.client.0.smithi094.stdout:3/901: dwrite d9/d10/dbb/de1/f10a [4194304,4194304] 0 2024-04-06T03:40:49.254 INFO:tasks.workunit.client.0.smithi094.stdout:3/902: dread d9/d20/d2d/d128/dba/d7c/da2/fc9 [0,4194304] 0 2024-04-06T03:40:49.257 INFO:tasks.workunit.client.0.smithi094.stdout:3/903: dread d9/d20/d2d/d128/dba/d7c/da2/df2/f11a [0,4194304] 0 2024-04-06T03:40:49.258 INFO:tasks.workunit.client.0.smithi094.stdout:3/904: stat d9/d20/d2d/d128/dba/d3a 0 2024-04-06T03:40:49.262 INFO:tasks.workunit.client.0.smithi094.stdout:6/859: dwrite dc/dd3/d72/d88/da3/f10b [0,4194304] 0 2024-04-06T03:40:49.263 INFO:tasks.workunit.client.1.smithi136.stdout:0/988: dwrite d1/d6/d2c/d57/d43/d6d/d85/d78/da4/f124 [0,4194304] 0 2024-04-06T03:40:49.284 INFO:tasks.workunit.client.0.smithi094.stdout:9/904: symlink d2/dc/d21/d26/d8a/d12e/l131 0 2024-04-06T03:40:49.284 INFO:tasks.workunit.client.0.smithi094.stdout:9/905: chown d2/dc/d21/d23/f7c 12 1 2024-04-06T03:40:49.292 INFO:tasks.workunit.client.0.smithi094.stdout:4/960: mknod d0/d77/dc4/c144 0 2024-04-06T03:40:49.293 INFO:tasks.workunit.client.0.smithi094.stdout:4/961: dread d0/df/d1e/d34/db6/fbe [0,4194304] 0 2024-04-06T03:40:49.294 INFO:tasks.workunit.client.0.smithi094.stdout:4/962: write d0/df/f1b [2378789,130479] 0 2024-04-06T03:40:49.294 INFO:tasks.workunit.client.0.smithi094.stdout:7/798: creat d2/d6/dc/dd/df/d95/f107 x:0 0 0 2024-04-06T03:40:49.297 INFO:tasks.workunit.client.1.smithi136.stdout:6/869: dread d2/d92/fc1 [0,4194304] 0 2024-04-06T03:40:49.298 INFO:tasks.workunit.client.0.smithi094.stdout:8/933: creat d18/d34/d52/f13d x:0 0 0 2024-04-06T03:40:49.298 INFO:tasks.workunit.client.0.smithi094.stdout:8/934: dread d18/d2c/f2e [4194304,4194304] 0 2024-04-06T03:40:49.298 INFO:tasks.workunit.client.0.smithi094.stdout:8/935: chown fd 0 1 2024-04-06T03:40:49.302 INFO:tasks.workunit.client.1.smithi136.stdout:3/946: symlink d0/d4/d12/d2d/d37/l13e 0 2024-04-06T03:40:49.303 INFO:tasks.workunit.client.1.smithi136.stdout:3/947: creat d0/d4/d12/d19/d55/d3c/d5a/f13f x:0 0 0 2024-04-06T03:40:49.304 INFO:tasks.workunit.client.1.smithi136.stdout:4/847: creat d1d/f127 x:0 0 0 2024-04-06T03:40:49.309 INFO:tasks.workunit.client.1.smithi136.stdout:0/989: symlink d1/l152 0 2024-04-06T03:40:49.309 INFO:tasks.workunit.client.1.smithi136.stdout:0/990: dread - d1/d6/d2c/d84/f128 zero size 2024-04-06T03:40:49.311 INFO:tasks.workunit.client.1.smithi136.stdout:6/870: symlink d2/d93/dab/l12d 0 2024-04-06T03:40:49.311 INFO:tasks.workunit.client.1.smithi136.stdout:6/871: chown d2/d7/d13/fd6 25355252 1 2024-04-06T03:40:49.316 INFO:tasks.workunit.client.0.smithi094.stdout:3/905: symlink d9/d1e/l143 0 2024-04-06T03:40:49.316 INFO:tasks.workunit.client.0.smithi094.stdout:3/906: creat d9/d9d/dc1/f144 x:0 0 0 2024-04-06T03:40:49.316 INFO:tasks.workunit.client.0.smithi094.stdout:3/907: fdatasync d9/d1d/d7d/fb0 0 2024-04-06T03:40:49.316 INFO:tasks.workunit.client.0.smithi094.stdout:3/908: chown d9/d20/d76/c106 114 1 2024-04-06T03:40:49.316 INFO:tasks.workunit.client.0.smithi094.stdout:3/909: chown d9/f8d 4 1 2024-04-06T03:40:49.317 INFO:tasks.workunit.client.0.smithi094.stdout:3/910: dread d9/f8d [0,4194304] 0 2024-04-06T03:40:49.317 INFO:tasks.workunit.client.1.smithi136.stdout:4/848: rename d1d/d31/d78/dcb/fe8 to d1d/d2f/da6/f128 0 2024-04-06T03:40:49.317 INFO:tasks.workunit.client.1.smithi136.stdout:4/849: chown d1d/d31/d78/fbd 1368550 1 2024-04-06T03:40:49.317 INFO:tasks.workunit.client.1.smithi136.stdout:4/850: truncate d1d/d31/d3e/d55/f11e 501862 0 2024-04-06T03:40:49.317 INFO:tasks.workunit.client.1.smithi136.stdout:4/851: getdents d1d/d31/d3e/d56 0 2024-04-06T03:40:49.319 INFO:tasks.workunit.client.1.smithi136.stdout:4/852: dread d1d/d31/d3e/d56/d7b/ff8 [0,4194304] 0 2024-04-06T03:40:49.319 INFO:tasks.workunit.client.1.smithi136.stdout:4/853: creat d1d/d31/d8e/f129 x:0 0 0 2024-04-06T03:40:49.319 INFO:tasks.workunit.client.1.smithi136.stdout:4/854: truncate d1d/d31/d3e/d56/ffc 962243 0 2024-04-06T03:40:49.319 INFO:tasks.workunit.client.1.smithi136.stdout:4/855: truncate d1d/d31/d3e/d56/d7b/ff8 1111214 0 2024-04-06T03:40:49.321 INFO:tasks.workunit.client.1.smithi136.stdout:0/991: mknod d1/d6/d2c/d57/d43/d6d/d85/d112/c153 0 2024-04-06T03:40:49.321 INFO:tasks.workunit.client.1.smithi136.stdout:0/992: creat d1/d6/d2c/d57/d43/d6b/f154 x:0 0 0 2024-04-06T03:40:49.321 INFO:tasks.workunit.client.1.smithi136.stdout:0/993: creat d1/d6/d2c/d57/d43/d6d/d85/d78/da4/f155 x:0 0 0 2024-04-06T03:40:49.322 INFO:tasks.workunit.client.0.smithi094.stdout:6/860: mknod dc/d3a/d4a/d54/dde/d50/daf/c12d 0 2024-04-06T03:40:49.322 INFO:tasks.workunit.client.1.smithi136.stdout:6/872: creat d2/d7/d13/d79/dc2/ddd/f12e x:0 0 0 2024-04-06T03:40:49.324 INFO:tasks.workunit.client.0.smithi094.stdout:9/906: rename d2/dc/d21/d26/d78/d69/da8/f3a to d2/dc/d10/d6a/f132 0 2024-04-06T03:40:49.333 INFO:tasks.workunit.client.1.smithi136.stdout:4/856: dwrite d1d/f20 [0,4194304] 0 2024-04-06T03:40:49.345 INFO:tasks.workunit.client.1.smithi136.stdout:6/873: dwrite d2/d25/f38 [4194304,4194304] 0 2024-04-06T03:40:49.345 INFO:tasks.workunit.client.1.smithi136.stdout:4/857: dread d1d/d2f/d6c/da5/dc1/d109/f10b [0,4194304] 0 2024-04-06T03:40:49.345 INFO:tasks.workunit.client.1.smithi136.stdout:6/874: readlink d2/d7/d26/d30/l2b 0 2024-04-06T03:40:49.413 INFO:tasks.workunit.client.0.smithi094.stdout:7/799: link d2/d6/dc/d12/d9a/ccb d2/d6/dc/d12/d9a/c108 0 2024-04-06T03:40:49.416 INFO:tasks.workunit.client.0.smithi094.stdout:8/936: getdents d18/d3c 0 2024-04-06T03:40:49.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:49 smithi094 ceph-mon[29713]: pgmap v24: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 325 MiB/s rd, 409 MiB/s wr, 588 op/s 2024-04-06T03:40:49.419 INFO:tasks.workunit.client.1.smithi136.stdout:1/997: sync 2024-04-06T03:40:49.421 INFO:tasks.workunit.client.0.smithi094.stdout:3/911: mknod d9/d1e/d26/d6c/ddd/c145 0 2024-04-06T03:40:49.421 INFO:tasks.workunit.client.0.smithi094.stdout:3/912: readlink d9/d20/d2d/d128/dba/d7c/da2/lb4 0 2024-04-06T03:40:49.423 INFO:tasks.workunit.client.1.smithi136.stdout:4/858: unlink d1d/d2f/f97 0 2024-04-06T03:40:49.423 INFO:tasks.workunit.client.1.smithi136.stdout:4/859: write f7 [3430290,128700] 0 2024-04-06T03:40:49.424 INFO:tasks.workunit.client.1.smithi136.stdout:1/998: dread d12/d3b/d74/d7a/fe9 [0,4194304] 0 2024-04-06T03:40:49.424 INFO:tasks.workunit.client.1.smithi136.stdout:1/999: fsync d12/d3b/d60/d7e/ffc 0 2024-04-06T03:40:49.425 INFO:tasks.workunit.client.0.smithi094.stdout:9/907: creat d2/dc/d21/d23/d47/d51/dd8/f133 x:0 0 0 2024-04-06T03:40:49.426 INFO:tasks.workunit.client.0.smithi094.stdout:6/861: write dc/f28 [1370601,102044] 0 2024-04-06T03:40:49.432 INFO:tasks.workunit.client.0.smithi094.stdout:8/937: link d18/d3c/d49/d9f/d109/l129 d18/d3c/d49/d91/d94/df3/l13e 0 2024-04-06T03:40:49.432 INFO:tasks.workunit.client.0.smithi094.stdout:7/800: dwrite d2/d6/dc/dd/d90/fa6 [0,4194304] 0 2024-04-06T03:40:49.432 INFO:tasks.workunit.client.0.smithi094.stdout:7/801: chown d2/d6/dc/dd/df/c6e 50838 1 2024-04-06T03:40:49.432 INFO:tasks.workunit.client.1.smithi136.stdout:4/860: dread d1d/d2f/da6/d11f/fc4 [0,4194304] 0 2024-04-06T03:40:49.433 INFO:tasks.workunit.client.0.smithi094.stdout:3/913: dread d9/f5f [0,4194304] 0 2024-04-06T03:40:49.433 INFO:tasks.workunit.client.0.smithi094.stdout:3/914: stat d9/d20/d2d/d128/dba/d7c/da2/lb4 0 2024-04-06T03:40:49.433 INFO:tasks.workunit.client.1.smithi136.stdout:0/994: mknod d1/d6/d2c/c156 0 2024-04-06T03:40:49.435 INFO:tasks.workunit.client.0.smithi094.stdout:9/908: rename d2/dc/d46/d111/d33/d64/dc6/f11c to d2/dc/d21/d23/d47/f134 0 2024-04-06T03:40:49.436 INFO:tasks.workunit.client.1.smithi136.stdout:4/861: creat d1d/d2f/d6c/da5/f12a x:0 0 0 2024-04-06T03:40:49.437 INFO:tasks.workunit.client.0.smithi094.stdout:6/862: mknod dc/d3a/d4a/c12e 0 2024-04-06T03:40:49.438 INFO:tasks.workunit.client.0.smithi094.stdout:6/863: write dc/d3a/df4/f125 [1685334,129825] 0 2024-04-06T03:40:49.438 INFO:tasks.workunit.client.0.smithi094.stdout:6/864: fdatasync dc/dd3/d72/d88/da3/f10b 0 2024-04-06T03:40:49.438 INFO:tasks.workunit.client.0.smithi094.stdout:8/938: mknod d18/d3c/d49/d91/d94/d133/c13f 0 2024-04-06T03:40:49.439 INFO:tasks.workunit.client.0.smithi094.stdout:8/939: readlink d18/d34/d52/l57 0 2024-04-06T03:40:49.440 INFO:tasks.workunit.client.0.smithi094.stdout:7/802: dread d2/d6/dc/d12/f5f [0,4194304] 0 2024-04-06T03:40:49.440 INFO:tasks.workunit.client.1.smithi136.stdout:4/862: unlink cb 0 2024-04-06T03:40:49.443 INFO:tasks.workunit.client.1.smithi136.stdout:0/995: dread d1/d6/d2c/f20 [0,4194304] 0 2024-04-06T03:40:49.443 INFO:tasks.workunit.client.1.smithi136.stdout:0/996: truncate d1/d4e/f12d 254336 0 2024-04-06T03:40:49.444 INFO:tasks.workunit.client.1.smithi136.stdout:4/863: mkdir d1d/d2f/d6c/da5/dc1/d109/d12b 0 2024-04-06T03:40:49.446 INFO:tasks.workunit.client.0.smithi094.stdout:3/915: unlink d9/d20/d2d/d128/dba/lec 0 2024-04-06T03:40:49.448 INFO:tasks.workunit.client.0.smithi094.stdout:9/909: creat d2/dc/d21/d26/d78/d69/da8/ddc/f135 x:0 0 0 2024-04-06T03:40:49.451 INFO:tasks.workunit.client.0.smithi094.stdout:8/940: creat d18/d30/f140 x:0 0 0 2024-04-06T03:40:49.451 INFO:tasks.workunit.client.0.smithi094.stdout:8/941: write d18/d3c/d49/db5/f131 [899670,86240] 0 2024-04-06T03:40:49.451 INFO:tasks.workunit.client.0.smithi094.stdout:6/865: write dc/dd3/d72/f81 [3751755,12450] 0 2024-04-06T03:40:49.451 INFO:tasks.workunit.client.1.smithi136.stdout:6/875: fsync d2/d7/d13/d79/dc2/ddd/f12e 0 2024-04-06T03:40:49.452 INFO:tasks.workunit.client.0.smithi094.stdout:9/910: write d2/dc/d21/f9c [186640,8974] 0 2024-04-06T03:40:49.453 INFO:tasks.workunit.client.1.smithi136.stdout:6/876: write d2/d7/d26/d30/d6e/d3d/d47/d121/ddf/f10e [358990,46599] 0 2024-04-06T03:40:49.453 INFO:tasks.workunit.client.1.smithi136.stdout:6/877: creat d2/d93/dab/f12f x:0 0 0 2024-04-06T03:40:49.456 INFO:tasks.workunit.client.0.smithi094.stdout:3/916: link d9/d9d/l135 d9/l146 0 2024-04-06T03:40:49.459 INFO:tasks.workunit.client.0.smithi094.stdout:8/942: rename d18/d3c/d76/dd9/f89 to d18/d2c/d37/d60/da3/d111/f141 0 2024-04-06T03:40:49.460 INFO:tasks.workunit.client.0.smithi094.stdout:8/943: truncate d18/d3c/d49/db5/f132 144674 0 2024-04-06T03:40:49.462 INFO:tasks.workunit.client.0.smithi094.stdout:6/866: getdents dc/d3a/d7f/dd9 0 2024-04-06T03:40:49.463 INFO:tasks.workunit.client.1.smithi136.stdout:0/997: dwrite d1/d6/d2c/d1a/d36/f49 [0,4194304] 0 2024-04-06T03:40:49.463 INFO:tasks.workunit.client.1.smithi136.stdout:0/998: readlink d1/d4e/lc8 0 2024-04-06T03:40:49.464 INFO:tasks.workunit.client.1.smithi136.stdout:0/999: truncate d1/d6/d2c/d57/d43/d6d/d85/da7/de5/f11d 276433 0 2024-04-06T03:40:49.465 INFO:tasks.workunit.client.0.smithi094.stdout:9/911: mknod d2/dc/d21/dc8/c136 0 2024-04-06T03:40:49.468 INFO:tasks.workunit.client.0.smithi094.stdout:3/917: unlink d9/d20/d2d/d128/c8b 0 2024-04-06T03:40:49.471 INFO:tasks.workunit.client.0.smithi094.stdout:8/944: mkdir d18/d2c/d37/d60/da3/d142 0 2024-04-06T03:40:49.472 INFO:tasks.workunit.client.0.smithi094.stdout:6/867: symlink dc/d3a/d4a/d54/dde/l12f 0 2024-04-06T03:40:49.473 INFO:tasks.workunit.client.0.smithi094.stdout:1/922: sync 2024-04-06T03:40:49.474 INFO:tasks.workunit.client.1.smithi136.stdout:3/948: sync 2024-04-06T03:40:49.475 INFO:tasks.workunit.client.1.smithi136.stdout:3/949: write d0/d4/d12/d19/d115/f7f [230522,32446] 0 2024-04-06T03:40:49.475 INFO:tasks.workunit.client.0.smithi094.stdout:9/912: dwrite d2/dc/d21/d23/d5b/ff0 [4194304,4194304] 0 2024-04-06T03:40:49.478 INFO:tasks.workunit.client.0.smithi094.stdout:8/945: creat d18/d2c/d37/d60/da3/deb/d13c/f143 x:0 0 0 2024-04-06T03:40:49.478 INFO:tasks.workunit.client.0.smithi094.stdout:8/946: chown d18/d3c/d8b/la9 14 1 2024-04-06T03:40:49.480 INFO:tasks.workunit.client.0.smithi094.stdout:6/868: creat dc/dd3/d72/d88/da3/db9/f130 x:0 0 0 2024-04-06T03:40:49.482 INFO:tasks.workunit.client.0.smithi094.stdout:1/923: symlink d0/d1/dd3/de7/l140 0 2024-04-06T03:40:49.485 INFO:tasks.workunit.client.0.smithi094.stdout:9/913: symlink d2/dc/d21/d23/db0/l137 0 2024-04-06T03:40:49.485 INFO:tasks.workunit.client.0.smithi094.stdout:9/914: creat d2/dc/d21/d23/d47/d51/dee/f138 x:0 0 0 2024-04-06T03:40:49.487 INFO:tasks.workunit.client.0.smithi094.stdout:3/918: rmdir d9/d1e/d6e 39 2024-04-06T03:40:49.488 INFO:tasks.workunit.client.0.smithi094.stdout:6/869: mknod dc/d11/c131 0 2024-04-06T03:40:49.488 INFO:tasks.workunit.client.0.smithi094.stdout:6/870: fdatasync dc/dd3/d4e/ffd 0 2024-04-06T03:40:49.488 INFO:tasks.workunit.client.0.smithi094.stdout:6/871: fsync dc/dd3/d72/d88/f9f 0 2024-04-06T03:40:49.490 INFO:tasks.workunit.client.0.smithi094.stdout:1/924: mknod d0/d7/d2d/dab/c141 0 2024-04-06T03:40:49.495 INFO:tasks.workunit.client.0.smithi094.stdout:3/919: symlink d9/d10/dd4/l147 0 2024-04-06T03:40:49.495 INFO:tasks.workunit.client.0.smithi094.stdout:8/947: dwrite d18/d3c/d49/d80/fea [0,4194304] 0 2024-04-06T03:40:49.498 INFO:tasks.workunit.client.1.smithi136.stdout:3/950: dwrite d0/d4/d12/d19/d55/d3c/d5a/d79/d98/fda [0,4194304] 0 2024-04-06T03:40:49.498 INFO:tasks.workunit.client.0.smithi094.stdout:1/925: rename d0/d1/d27/da5/d40/d50/l6a to d0/d127/d30/df6/l142 0 2024-04-06T03:40:49.500 INFO:tasks.workunit.client.0.smithi094.stdout:3/920: creat d9/d20/da6/de9/f148 x:0 0 0 2024-04-06T03:40:49.500 INFO:tasks.workunit.client.0.smithi094.stdout:3/921: fsync d9/f19 0 2024-04-06T03:40:49.502 INFO:tasks.workunit.client.0.smithi094.stdout:3/922: dread d9/d20/d2d/d128/dba/d7c/da2/df2/f11a [0,4194304] 0 2024-04-06T03:40:49.502 INFO:tasks.workunit.client.0.smithi094.stdout:3/923: fdatasync d9/d20/da6/de9/f148 0 2024-04-06T03:40:49.503 INFO:tasks.workunit.client.0.smithi094.stdout:6/872: dwrite dc/d3a/d4a/d54/d71/f11e [0,4194304] 0 2024-04-06T03:40:49.503 INFO:tasks.workunit.client.0.smithi094.stdout:6/873: write f7 [4692481,93904] 0 2024-04-06T03:40:49.507 INFO:tasks.workunit.client.0.smithi094.stdout:1/926: mknod d0/d1/dd3/c143 0 2024-04-06T03:40:49.510 INFO:tasks.workunit.client.1.smithi136.stdout:3/951: rmdir d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d11d 0 2024-04-06T03:40:49.511 INFO:tasks.workunit.client.0.smithi094.stdout:3/924: mkdir d9/d10/d9b/db7/dc0/de2/d149 0 2024-04-06T03:40:49.515 INFO:tasks.workunit.client.1.smithi136.stdout:3/952: rename d0/d4/d12/d2d/d37/f6d to d0/f140 0 2024-04-06T03:40:49.515 INFO:tasks.workunit.client.1.smithi136.stdout:3/953: truncate d0/d4/d12/d3a/d51/da6/fb6 5362020 0 2024-04-06T03:40:49.518 INFO:tasks.workunit.client.1.smithi136.stdout:3/954: link d0/d4/d12/d19/d2e/lc1 d0/d4/d12/d19/d55/d6f/ddb/df9/l141 0 2024-04-06T03:40:49.518 INFO:tasks.workunit.client.1.smithi136.stdout:3/955: stat d0/d4/d12/d19/d2e/d3b/f5f 0 2024-04-06T03:40:49.518 INFO:tasks.workunit.client.1.smithi136.stdout:3/956: write d0/d4/d12/d3a/d51/da6/fb6 [1996498,73835] 0 2024-04-06T03:40:49.518 INFO:tasks.workunit.client.1.smithi136.stdout:3/957: truncate d0/d4/d12/d19/d55/d3c/d5a/fb8 193923 0 2024-04-06T03:40:49.520 INFO:tasks.workunit.client.0.smithi094.stdout:6/874: dwrite dc/d3a/d4a/d54/dde/f39 [0,4194304] 0 2024-04-06T03:40:49.524 INFO:tasks.workunit.client.1.smithi136.stdout:3/958: write d0/f32 [1810923,36101] 0 2024-04-06T03:40:49.575 INFO:tasks.workunit.client.0.smithi094.stdout:1/927: mknod d0/d1/d27/da5/d3f/c144 0 2024-04-06T03:40:49.578 INFO:tasks.workunit.client.0.smithi094.stdout:3/925: rename d9/d10/l17 to d9/d20/d2d/d128/dba/dc5/l14a 0 2024-04-06T03:40:49.583 INFO:tasks.workunit.client.0.smithi094.stdout:1/928: dread d0/d7/f52 [4194304,4194304] 0 2024-04-06T03:40:49.585 INFO:tasks.workunit.client.0.smithi094.stdout:6/875: mknod dc/d3a/d4a/d54/d71/c132 0 2024-04-06T03:40:49.587 INFO:tasks.workunit.client.1.smithi136.stdout:3/959: mknod d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/c142 0 2024-04-06T03:40:49.588 INFO:tasks.workunit.client.0.smithi094.stdout:3/926: symlink d9/d20/d2d/d128/dba/d3a/l14b 0 2024-04-06T03:40:49.589 INFO:tasks.workunit.client.0.smithi094.stdout:1/929: mknod d0/d7/d1a/da8/c145 0 2024-04-06T03:40:49.591 INFO:tasks.workunit.client.1.smithi136.stdout:3/960: link d0/d4/d12/d19/d55/d3c/d5a/d79/d98/dc6/lf6 d0/d4/d12/d19/d55/d3c/d136/l143 0 2024-04-06T03:40:49.591 INFO:tasks.workunit.client.1.smithi136.stdout:3/961: truncate d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/f102 957147 0 2024-04-06T03:40:49.591 INFO:tasks.workunit.client.1.smithi136.stdout:3/962: fdatasync d0/d4/d12/d19/d55/d3c/d46/f78 0 2024-04-06T03:40:49.593 INFO:tasks.workunit.client.0.smithi094.stdout:3/927: rename d9/d1d/d7d/l84 to d9/d10/d9b/dee/l14c 0 2024-04-06T03:40:49.598 INFO:tasks.workunit.client.1.smithi136.stdout:3/963: creat d0/d4/d12/d19/d115/f144 x:0 0 0 2024-04-06T03:40:49.598 INFO:tasks.workunit.client.0.smithi094.stdout:5/855: sync 2024-04-06T03:40:49.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:49 smithi136 ceph-mon[36436]: pgmap v24: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 325 MiB/s rd, 409 MiB/s wr, 588 op/s 2024-04-06T03:40:49.600 INFO:tasks.workunit.client.0.smithi094.stdout:1/930: mknod d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/c146 0 2024-04-06T03:40:49.600 INFO:tasks.workunit.client.0.smithi094.stdout:1/931: chown d0/d1/d27/da5/d40/db1/d2f/d3c/d48/dcb/d10c 254 1 2024-04-06T03:40:49.602 INFO:tasks.workunit.client.0.smithi094.stdout:6/876: symlink dc/dd3/d72/d88/l133 0 2024-04-06T03:40:49.604 INFO:tasks.workunit.client.1.smithi136.stdout:3/964: creat d0/d4/d12/d19/d55/d3c/d53/f145 x:0 0 0 2024-04-06T03:40:49.604 INFO:tasks.workunit.client.1.smithi136.stdout:3/965: read - d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/dd4/f121 zero size 2024-04-06T03:40:49.604 INFO:tasks.workunit.client.1.smithi136.stdout:3/966: chown d0/d4/d12/d19/d55/d6f/ddb/f11a 393042 1 2024-04-06T03:40:49.606 INFO:tasks.workunit.client.0.smithi094.stdout:3/928: dwrite d9/d20/d2d/da9/fe0 [0,4194304] 0 2024-04-06T03:40:49.607 INFO:tasks.workunit.client.1.smithi136.stdout:3/967: rename d0/d4/d12/d3a to d0/d4/d12/d19/d55/d3c/d5a/d79/d146 0 2024-04-06T03:40:49.608 INFO:tasks.workunit.client.1.smithi136.stdout:3/968: stat d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d97/dd4/deb 0 2024-04-06T03:40:49.608 INFO:tasks.workunit.client.1.smithi136.stdout:3/969: write d0/d4/d12/d19/d55/d6f/d82/f103 [1562730,105291] 0 2024-04-06T03:40:49.611 INFO:tasks.workunit.client.0.smithi094.stdout:1/932: creat d0/d127/d30/f147 x:0 0 0 2024-04-06T03:40:49.611 INFO:tasks.workunit.client.0.smithi094.stdout:1/933: fdatasync d0/d127/d30/fe4 0 2024-04-06T03:40:49.612 INFO:tasks.workunit.client.0.smithi094.stdout:6/877: mknod dc/d3a/c134 0 2024-04-06T03:40:49.612 INFO:tasks.workunit.client.0.smithi094.stdout:6/878: readlink dc/d3a/l75 0 2024-04-06T03:40:49.612 INFO:tasks.workunit.client.0.smithi094.stdout:6/879: creat dc/dd3/d72/d88/dfb/d112/f135 x:0 0 0 2024-04-06T03:40:49.613 INFO:tasks.workunit.client.1.smithi136.stdout:4/864: sync 2024-04-06T03:40:49.613 INFO:tasks.workunit.client.1.smithi136.stdout:3/970: creat d0/d4/d12/d2d/d37/d116/d12a/f147 x:0 0 0 2024-04-06T03:40:49.615 INFO:tasks.workunit.client.1.smithi136.stdout:6/878: sync 2024-04-06T03:40:49.616 INFO:tasks.workunit.client.0.smithi094.stdout:1/934: creat d0/d1/d27/da5/d40/db1/d2f/d3c/d48/dcb/d10c/f148 x:0 0 0 2024-04-06T03:40:49.617 INFO:tasks.workunit.client.1.smithi136.stdout:3/971: dread d0/d4/d12/d19/f50 [0,4194304] 0 2024-04-06T03:40:49.617 INFO:tasks.workunit.client.1.smithi136.stdout:3/972: stat d0/d4/d12/d19/d55/d6f/d82/d5d/d61/cfa 0 2024-04-06T03:40:49.620 INFO:tasks.workunit.client.0.smithi094.stdout:6/880: mknod dc/dd3/d72/c136 0 2024-04-06T03:40:49.620 INFO:tasks.workunit.client.0.smithi094.stdout:3/929: dwrite d9/fd5 [4194304,4194304] 0 2024-04-06T03:40:49.622 INFO:tasks.workunit.client.1.smithi136.stdout:3/973: getdents d0/d4/d12/d19/d55/d3c/d5a/d79/d146/d51/d8e/d10f 0 2024-04-06T03:40:49.622 INFO:tasks.workunit.client.1.smithi136.stdout:3/974: dread - d0/d4/d12/d2d/d37/d116/d12a/f147 zero size 2024-04-06T03:40:49.627 INFO:tasks.workunit.client.0.smithi094.stdout:6/881: write dc/d3a/d4a/d54/dde/f9e [1567743,75277] 0 2024-04-06T03:40:49.627 INFO:tasks.workunit.client.1.smithi136.stdout:4/865: rename d1d/d31/d3e/d56/ff5 to d1d/f12c 0 2024-04-06T03:40:49.628 INFO:tasks.workunit.client.1.smithi136.stdout:6/879: dwrite d2/d7/d26/d68/ddc/f127 [0,4194304] 0 2024-04-06T03:40:49.629 INFO:tasks.workunit.client.0.smithi094.stdout:1/935: mknod d0/d7/d1a/d108/c149 0 2024-04-06T03:40:49.630 INFO:tasks.workunit.client.1.smithi136.stdout:3/975: symlink d0/d4/d12/d19/d115/dc9/l148 0 2024-04-06T03:40:49.631 INFO:tasks.workunit.client.0.smithi094.stdout:5/856: write db/f1e [2462475,1160] 0 2024-04-06T03:40:49.631 INFO:tasks.workunit.client.0.smithi094.stdout:5/857: dread - db/d7c/dd7/f126 zero size 2024-04-06T03:40:49.631 INFO:tasks.workunit.client.0.smithi094.stdout:5/858: write db/d19/d21/d11e/d2e/f10e [457564,56666] 0 2024-04-06T03:40:49.633 INFO:tasks.workunit.client.1.smithi136.stdout:4/866: truncate d1d/d31/f4c 1180784 0 2024-04-06T03:40:49.633 INFO:tasks.workunit.client.1.smithi136.stdout:4/867: chown d1d/d31/d3e/c3d 10 1 2024-04-06T03:40:49.635 INFO:tasks.workunit.client.0.smithi094.stdout:6/882: read dc/d3a/d4a/d54/f76 [1219946,63828] 0 2024-04-06T03:40:49.638 INFO:tasks.workunit.client.1.smithi136.stdout:6/880: dread d2/d7/f31 [0,4194304] 0 2024-04-06T03:40:49.638 INFO:tasks.workunit.client.1.smithi136.stdout:6/881: fdatasync d2/d7/d26/d68/ddc/fe4 0 2024-04-06T03:40:49.638 INFO:tasks.workunit.client.1.smithi136.stdout:6/882: chown d2/d7/d26/d30/d6e/db7/fbf 508865933 1 2024-04-06T03:40:49.638 INFO:tasks.workunit.client.1.smithi136.stdout:6/883: creat d2/d104/f130 x:0 0 0 2024-04-06T03:40:49.640 INFO:tasks.workunit.client.0.smithi094.stdout:1/936: write d0/d7/f86 [1987888,81044] 0 2024-04-06T03:40:49.642 INFO:tasks.workunit.client.0.smithi094.stdout:3/930: symlink d9/de5/l14d 0 2024-04-06T03:40:49.642 INFO:tasks.workunit.client.0.smithi094.stdout:3/931: chown d9/d10/f47 14226 1 2024-04-06T03:40:49.643 INFO:tasks.workunit.client.1.smithi136.stdout:3/976: dread d0/d4/d12/d19/d55/d3c/faa [0,4194304] 0 2024-04-06T03:40:49.643 INFO:tasks.workunit.client.1.smithi136.stdout:3/977: write d0/d4/d12/d19/d55/d3c/d5a/f120 [614350,104888] 0 2024-04-06T03:40:49.644 INFO:tasks.workunit.client.0.smithi094.stdout:5/859: mknod db/d7c/c12b 0 2024-04-06T03:40:49.649 INFO:tasks.workunit.client.1.smithi136.stdout:3/978: mknod d0/d4/d12/d19/d115/c149 0 2024-04-06T03:40:49.649 INFO:tasks.workunit.client.1.smithi136.stdout:3/979: fsync d0/f140 0 2024-04-06T03:40:49.649 INFO:tasks.workunit.client.0.smithi094.stdout:6/883: write dc/d3a/d4a/d94/f104 [1896577,34426] 0 2024-04-06T03:40:49.649 INFO:tasks.workunit.client.0.smithi094.stdout:6/884: read dc/dd3/f2f [1370887,89671] 0 2024-04-06T03:40:49.650 INFO:tasks.workunit.client.0.smithi094.stdout:1/937: symlink d0/d1/d27/da5/db2/dce/l14a 0 2024-04-06T03:40:49.654 INFO:tasks.workunit.client.0.smithi094.stdout:3/932: creat d9/d10/d9b/dee/f14e x:0 0 0 2024-04-06T03:40:49.655 INFO:tasks.workunit.client.1.smithi136.stdout:4/868: getdents d1d/d2f/d6c/da5 0 2024-04-06T03:40:49.655 INFO:tasks.workunit.client.1.smithi136.stdout:4/869: stat d1d/d31/d3e/d56/d7b 0 2024-04-06T03:40:49.655 INFO:tasks.workunit.client.1.smithi136.stdout:6/884: dwrite d2/d7/d26/d30/dc4/f129 [0,4194304] 0 2024-04-06T03:40:49.656 INFO:tasks.workunit.client.0.smithi094.stdout:5/860: mknod db/d19/dc1/dd4/dcf/c12c 0 2024-04-06T03:40:49.656 INFO:tasks.workunit.client.0.smithi094.stdout:5/861: fsync db/d19/d2c/fc2 0 2024-04-06T03:40:49.656 INFO:tasks.workunit.client.0.smithi094.stdout:5/862: write db/d19/d30/d83/fee [756084,86989] 0 2024-04-06T03:40:49.657 INFO:tasks.workunit.client.0.smithi094.stdout:4/963: sync 2024-04-06T03:40:49.657 INFO:tasks.workunit.client.0.smithi094.stdout:2/867: sync 2024-04-06T03:40:49.657 INFO:tasks.workunit.client.0.smithi094.stdout:0/876: sync 2024-04-06T03:40:49.658 INFO:tasks.workunit.client.1.smithi136.stdout:3/980: getdents d0/d4/d12/d19/d55/d3c/d5a/d79/d146 0 2024-04-06T03:40:49.658 INFO:tasks.workunit.client.0.smithi094.stdout:6/885: symlink dc/d3a/df4/df6/d7b/dc6/d113/l137 0 2024-04-06T03:40:49.659 INFO:tasks.workunit.client.0.smithi094.stdout:6/886: chown dc/d3a/d4a/d54/dde/f34 25517905 1 2024-04-06T03:40:49.659 INFO:tasks.workunit.client.0.smithi094.stdout:6/887: chown dc/d3a/d4a/d54/fb5 100739 1 2024-04-06T03:40:49.661 INFO:tasks.workunit.client.0.smithi094.stdout:5/863: creat db/d19/dc1/dd4/d11d/f12d x:0 0 0 2024-04-06T03:40:49.661 INFO:tasks.workunit.client.0.smithi094.stdout:5/864: readlink db/d19/d21/l124 0 2024-04-06T03:40:49.664 INFO:tasks.workunit.client.0.smithi094.stdout:4/964: creat d0/d4/d67/dec/d12a/f145 x:0 0 0 2024-04-06T03:40:49.664 INFO:tasks.workunit.client.0.smithi094.stdout:4/965: fsync d0/df/f10b 0 2024-04-06T03:40:49.664 INFO:tasks.workunit.client.1.smithi136.stdout:3/981: mknod d0/d4/d12/d19/d115/dc9/d12e/c14a 0 2024-04-06T03:40:49.668 INFO:tasks.workunit.client.0.smithi094.stdout:2/868: mkdir d0/dd/d1b/d72/da8/d125 0 2024-04-06T03:40:49.668 INFO:tasks.workunit.client.0.smithi094.stdout:2/869: chown d0/dd/d1b/lad 27 1 2024-04-06T03:40:49.668 INFO:tasks.workunit.client.0.smithi094.stdout:2/870: chown d0/dd/d19/cc6 422435 1 2024-04-06T03:40:49.668 INFO:tasks.workunit.client.0.smithi094.stdout:2/871: creat d0/dd/d1b/de2/f126 x:0 0 0 2024-04-06T03:40:49.668 INFO:tasks.workunit.client.0.smithi094.stdout:2/872: readlink d0/dd/d1b/lce 0 2024-04-06T03:40:49.671 INFO:tasks.workunit.client.0.smithi094.stdout:1/938: dwrite d0/d127/d30/f32 [0,4194304] 0 2024-04-06T03:40:49.672 INFO:tasks.workunit.client.0.smithi094.stdout:3/933: link d9/d9d/d129/l136 d9/d20/d76/d6b/l14f 0 2024-04-06T03:40:49.672 INFO:tasks.workunit.client.0.smithi094.stdout:3/934: truncate d9/d10/dbb/de1/f131 553324 0 2024-04-06T03:40:49.675 INFO:tasks.workunit.client.0.smithi094.stdout:0/877: dread d1/d9/dd/d18/d1e/d22/f37 [0,4194304] 0 2024-04-06T03:40:49.677 INFO:tasks.workunit.client.0.smithi094.stdout:2/873: rename d0/d8/d1d/d77/fa1 to d0/d8/da0/d10b/f127 0 2024-04-06T03:40:49.679 INFO:tasks.workunit.client.0.smithi094.stdout:6/888: mkdir dc/dab/d138 0 2024-04-06T03:40:49.681 INFO:tasks.workunit.client.0.smithi094.stdout:1/939: rmdir d0/d1/d27/da5/db2 39 2024-04-06T03:40:49.682 INFO:tasks.workunit.client.0.smithi094.stdout:3/935: unlink d9/d20/d2d/d128/dba/ca4 0 2024-04-06T03:40:49.683 INFO:tasks.workunit.client.0.smithi094.stdout:3/936: chown d9/d20/d2d/d128/dba/d7c/ld1 6193492 1 2024-04-06T03:40:49.683 INFO:tasks.workunit.client.0.smithi094.stdout:3/937: creat d9/d20/d2d/da9/f150 x:0 0 0 2024-04-06T03:40:49.683 INFO:tasks.workunit.client.0.smithi094.stdout:3/938: creat d9/d10/d9b/dee/f151 x:0 0 0 2024-04-06T03:40:49.683 INFO:tasks.workunit.client.0.smithi094.stdout:0/878: mknod d1/d9/dd/d18/d1e/d86/d96/dc0/c126 0 2024-04-06T03:40:49.684 INFO:tasks.workunit.client.1.smithi136.stdout:6/885: dwrite d2/d7/d26/f125 [0,4194304] 0 2024-04-06T03:40:49.685 INFO:tasks.workunit.client.0.smithi094.stdout:6/889: truncate dc/d3a/d4a/d94/f109 3289717 0 2024-04-06T03:40:49.685 INFO:tasks.workunit.client.0.smithi094.stdout:1/940: mknod d0/d7/d2d/d11f/c14b 0 2024-04-06T03:40:49.685 INFO:tasks.workunit.client.0.smithi094.stdout:4/966: dwrite d0/d4/dde/ff3 [0,4194304] 0 2024-04-06T03:40:49.685 INFO:tasks.workunit.client.0.smithi094.stdout:4/967: chown d0/df/d1e 130 1 2024-04-06T03:40:49.686 INFO:tasks.workunit.client.1.smithi136.stdout:4/870: dwrite d1d/d31/d3e/d56/d102/f110 [0,4194304] 0 2024-04-06T03:40:49.686 INFO:tasks.workunit.client.1.smithi136.stdout:3/982: dwrite d0/d4/d12/d19/d55/d6f/d82/d5d/d6a/d88/df2/f13c [0,4194304] 0 2024-04-06T03:40:49.687 INFO:tasks.workunit.client.0.smithi094.stdout:3/939: mkdir d9/d1e/d26/d54/d113/d152 0 2024-04-06T03:40:49.690 INFO:tasks.workunit.client.0.smithi094.stdout:0/879: creat d1/d9/dd/d18/f127 x:0 0 0 2024-04-06T03:40:49.690 INFO:tasks.workunit.client.0.smithi094.stdout:0/880: chown d1/d9/dd/d18/d1a/d44/dc2/f10c 119408166 1 2024-04-06T03:40:49.690 INFO:tasks.workunit.client.0.smithi094.stdout:0/881: readlink d1/d9/dd/d18/d1a/lcf 0 2024-04-06T03:40:49.690 INFO:tasks.workunit.client.0.smithi094.stdout:3/940: dread d9/d20/d2d/d128/dba/d7c/f92 [0,4194304] 0 2024-04-06T03:40:49.690 INFO:tasks.workunit.client.0.smithi094.stdout:3/941: chown d9/d20/cb5 177450191 1 2024-04-06T03:40:49.690 INFO:tasks.workunit.client.0.smithi094.stdout:5/865: rmdir db/d19/dc1/dd4 39 2024-04-06T03:40:49.691 INFO:tasks.workunit.client.1.smithi136.stdout:6/886: dread d2/d104/fcb [0,4194304] 0 2024-04-06T03:40:49.692 INFO:tasks.workunit.client.0.smithi094.stdout:6/890: mknod dc/d3a/d4a/d54/d71/d103/c139 0 2024-04-06T03:40:49.694 INFO:tasks.workunit.client.0.smithi094.stdout:1/941: unlink d0/d7/d2d/dab/c141 0 2024-04-06T03:40:49.698 INFO:tasks.workunit.client.1.smithi136.stdout:6/887: write d2/d7/d26/d30/dc4/f129 [1377767,48721] 0 2024-04-06T03:40:49.701 INFO:tasks.workunit.client.1.smithi136.stdout:3/983: mkdir d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/d126/d14b 0 2024-04-06T03:40:49.701 INFO:tasks.workunit.client.1.smithi136.stdout:3/984: dread - d0/d4/d12/d19/d55/d3c/d53/de6/fea zero size 2024-04-06T03:40:49.708 INFO:tasks.workunit.client.0.smithi094.stdout:4/968: creat d0/d4/d129/f146 x:0 0 0 2024-04-06T03:40:49.708 INFO:tasks.workunit.client.0.smithi094.stdout:4/969: chown d0/df/db3/dd8/fed 14840908 1 2024-04-06T03:40:49.708 INFO:tasks.workunit.client.0.smithi094.stdout:4/970: truncate d0/d15/d3a/d61/d81/da0/fdd 4927456 0 2024-04-06T03:40:49.710 INFO:tasks.workunit.client.1.smithi136.stdout:6/888: dwrite d2/d7/d26/f119 [0,4194304] 0 2024-04-06T03:40:49.720 INFO:tasks.workunit.client.1.smithi136.stdout:6/889: mkdir d2/d7/d26/d30/d6e/d58/d131 0 2024-04-06T03:40:49.720 INFO:tasks.workunit.client.1.smithi136.stdout:6/890: truncate d2/d7/d13/f9a 4345022 0 2024-04-06T03:40:49.721 INFO:tasks.workunit.client.0.smithi094.stdout:4/971: dwrite d0/d4/f143 [0,4194304] 0 2024-04-06T03:40:49.721 INFO:tasks.workunit.client.0.smithi094.stdout:0/882: symlink d1/l128 0 2024-04-06T03:40:49.725 INFO:tasks.workunit.client.0.smithi094.stdout:3/942: unlink d9/d10/d7e/da3/f11f 0 2024-04-06T03:40:49.728 INFO:tasks.workunit.client.0.smithi094.stdout:5/866: symlink db/d19/d2c/d6d/de7/l12e 0 2024-04-06T03:40:49.728 INFO:tasks.workunit.client.0.smithi094.stdout:5/867: fsync db/d19/d21/d11e/d28/d85/f9e 0 2024-04-06T03:40:49.729 INFO:tasks.workunit.client.0.smithi094.stdout:6/891: rmdir dc/dd3/d72/d87 39 2024-04-06T03:40:49.729 INFO:tasks.workunit.client.0.smithi094.stdout:6/892: fsync dc/dd3/d72/d88/da3/faa 0 2024-04-06T03:40:49.731 INFO:tasks.workunit.client.0.smithi094.stdout:1/942: symlink d0/d1/d27/da5/d40/db1/dc3/df0/l14c 0 2024-04-06T03:40:49.732 INFO:tasks.workunit.client.1.smithi136.stdout:3/985: truncate d0/d4/d12/d19/d55/d6f/d82/f103 1537940 0 2024-04-06T03:40:49.733 INFO:tasks.workunit.client.1.smithi136.stdout:3/986: write d0/d4/d12/d19/d55/d3c/d5a/f71 [338390,47522] 0 2024-04-06T03:40:49.737 INFO:tasks.workunit.client.0.smithi094.stdout:0/883: dwrite d1/f83 [0,4194304] 0 2024-04-06T03:40:49.738 INFO:tasks.workunit.client.1.smithi136.stdout:6/891: dwrite d2/d7/d13/d79/dc2/ddd/f12e [0,4194304] 0 2024-04-06T03:40:49.741 INFO:tasks.workunit.client.0.smithi094.stdout:5/868: dwrite db/d7c/dd7/f126 [0,4194304] 0 2024-04-06T03:40:49.744 INFO:tasks.workunit.client.0.smithi094.stdout:4/972: creat d0/d77/dc4/d11a/f147 x:0 0 0 2024-04-06T03:40:49.746 INFO:tasks.workunit.client.0.smithi094.stdout:3/943: mknod d9/d20/d2d/d128/dba/d7c/da2/c153 0 2024-04-06T03:40:49.751 INFO:tasks.workunit.client.0.smithi094.stdout:6/893: rename dc/dd3/d72/d87/la8 to dc/d3a/df4/l13a 0 2024-04-06T03:40:49.752 INFO:tasks.workunit.client.1.smithi136.stdout:6/892: mkdir d2/d7/d26/d30/d6e/d3d/d132 0 2024-04-06T03:40:49.752 INFO:tasks.workunit.client.1.smithi136.stdout:6/893: readlink d2/d7/d26/d30/d6e/lc6 0 2024-04-06T03:40:49.753 INFO:tasks.workunit.client.0.smithi094.stdout:6/894: dread dc/dd3/f74 [0,4194304] 0 2024-04-06T03:40:49.754 INFO:tasks.workunit.client.0.smithi094.stdout:6/895: chown dc/d3a/df4/df6/f4d 12180329 1 2024-04-06T03:40:49.756 INFO:tasks.workunit.client.1.smithi136.stdout:3/987: getdents d0/d4/d12/d19/d55/d3c/d5a 0 2024-04-06T03:40:49.759 INFO:tasks.workunit.client.1.smithi136.stdout:6/894: mkdir d2/d7/d26/dcd/dad/d116/d133 0 2024-04-06T03:40:49.759 INFO:tasks.workunit.client.1.smithi136.stdout:6/895: fsync d2/d7/d13/d89/f12a 0 2024-04-06T03:40:49.759 INFO:tasks.workunit.client.1.smithi136.stdout:4/871: sync 2024-04-06T03:40:49.760 INFO:tasks.workunit.client.0.smithi094.stdout:5/869: mkdir db/d19/d21/d12f 0 2024-04-06T03:40:49.762 INFO:tasks.workunit.client.0.smithi094.stdout:4/973: mknod d0/df/dbd/d107/c148 0 2024-04-06T03:40:49.765 INFO:tasks.workunit.client.0.smithi094.stdout:6/896: read dc/dd3/d72/f8c [808449,22992] 0 2024-04-06T03:40:49.765 INFO:tasks.workunit.client.0.smithi094.stdout:6/897: truncate dc/d3a/d4a/d54/dde/d50/f80 4426879 0 2024-04-06T03:40:49.765 INFO:tasks.workunit.client.1.smithi136.stdout:6/896: unlink d2/d7/d26/d30/d6e/d58/d5d/l70 0 2024-04-06T03:40:49.768 INFO:tasks.workunit.client.1.smithi136.stdout:4/872: symlink d1d/d31/d3e/d55/d89/db3/l12d 0 2024-04-06T03:40:49.770 INFO:tasks.workunit.client.0.smithi094.stdout:5/870: dread db/d19/d30/fb8 [0,4194304] 0 2024-04-06T03:40:49.770 INFO:tasks.workunit.client.0.smithi094.stdout:1/943: rename d0/d7/d1a/da8/l11b to d0/l14d 0 2024-04-06T03:40:49.770 INFO:tasks.workunit.client.0.smithi094.stdout:1/944: chown d0/d1/d27/da5/d3f/d8c/le2 426939561 1 2024-04-06T03:40:49.772 INFO:tasks.workunit.client.1.smithi136.stdout:6/897: mkdir d2/d7/d26/d30/d6e/d58/d131/d134 0 2024-04-06T03:40:49.774 INFO:tasks.workunit.client.0.smithi094.stdout:6/898: getdents dc/d3a/d4a/d54/dde 0 2024-04-06T03:40:49.774 INFO:tasks.workunit.client.0.smithi094.stdout:6/899: dread - dc/d3a/d7f/dd4/f115 zero size 2024-04-06T03:40:49.774 INFO:tasks.workunit.client.0.smithi094.stdout:7/803: sync 2024-04-06T03:40:49.774 INFO:tasks.workunit.client.0.smithi094.stdout:7/804: read d2/d6/dc/d12/d29/f89 [3109320,92293] 0 2024-04-06T03:40:49.774 INFO:tasks.workunit.client.0.smithi094.stdout:7/805: truncate d2/d6/dc/d12/d9a/da0/dd4/db1/fb2 365235 0 2024-04-06T03:40:49.775 INFO:tasks.workunit.client.0.smithi094.stdout:3/944: symlink d9/d20/da6/l154 0 2024-04-06T03:40:49.775 INFO:tasks.workunit.client.0.smithi094.stdout:3/945: chown d9/fc4 1337 1 2024-04-06T03:40:49.777 INFO:tasks.workunit.client.1.smithi136.stdout:6/898: link d2/d7/d26/d30/d6e/d58/d5d/d62/fb0 d2/d7/d26/dcd/dad/d116/f135 0 2024-04-06T03:40:49.778 INFO:tasks.workunit.client.0.smithi094.stdout:0/884: rename d1/d9/dd/d18/d32/d42/cf0 to d1/d9/dd/d18/d78/c129 0 2024-04-06T03:40:49.779 INFO:tasks.workunit.client.0.smithi094.stdout:5/871: mkdir db/d19/d2c/d6d/de6/dfd/d10d/d130 0 2024-04-06T03:40:49.780 INFO:tasks.workunit.client.1.smithi136.stdout:6/899: mkdir d2/d7/d13/d79/d136 0 2024-04-06T03:40:49.781 INFO:tasks.workunit.client.0.smithi094.stdout:1/945: rmdir d0/d127/d131 39 2024-04-06T03:40:49.781 INFO:tasks.workunit.client.0.smithi094.stdout:1/946: write d0/d1/d27/da5/d40/db1/fd9 [33202,90457] 0 2024-04-06T03:40:49.783 INFO:tasks.workunit.client.1.smithi136.stdout:6/900: rename d2/d7/d26/d30/d6e/d3d/d47/l9d to d2/d7/d26/d68/ddc/l137 0 2024-04-06T03:40:49.784 INFO:tasks.workunit.client.0.smithi094.stdout:7/806: mknod d2/d6/dac/ddd/c109 0 2024-04-06T03:40:49.785 INFO:tasks.workunit.client.0.smithi094.stdout:7/807: write d2/d6/dc/d12/d9a/da0/faf [1173386,128026] 0 2024-04-06T03:40:49.787 INFO:tasks.workunit.client.1.smithi136.stdout:6/901: unlink d2/d7/d26/d30/d6e/d58/d5d/d62/d53/fa1 0 2024-04-06T03:40:49.787 INFO:tasks.workunit.client.1.smithi136.stdout:6/902: creat d2/d93/dab/f138 x:0 0 0 2024-04-06T03:40:49.787 INFO:tasks.workunit.client.1.smithi136.stdout:6/903: readlink d2/d7/d26/l111 0 2024-04-06T03:40:49.788 INFO:tasks.workunit.client.0.smithi094.stdout:5/872: link db/d19/dc1/dd4/fe9 db/d19/dc1/f131 0 2024-04-06T03:40:49.790 INFO:tasks.workunit.client.1.smithi136.stdout:3/988: dwrite d0/d4/d12/d19/d55/d3c/d5a/d79/d146/f94 [4194304,4194304] 0 2024-04-06T03:40:49.791 INFO:tasks.workunit.client.0.smithi094.stdout:6/900: write dc/f10f [1368361,79305] 0 2024-04-06T03:40:49.791 INFO:tasks.workunit.client.0.smithi094.stdout:6/901: creat dc/d3a/d4a/d54/dde/d50/d61/f13b x:0 0 0 2024-04-06T03:40:49.791 INFO:tasks.workunit.client.0.smithi094.stdout:4/974: dwrite d0/df/d1e/fef [0,4194304] 0 2024-04-06T03:40:49.791 INFO:tasks.workunit.client.0.smithi094.stdout:4/975: readlink d0/l28 0 2024-04-06T03:40:49.791 INFO:tasks.workunit.client.1.smithi136.stdout:6/904: creat d2/d7/d26/d30/d6e/d3d/d47/f139 x:0 0 0 2024-04-06T03:40:49.793 INFO:tasks.workunit.client.0.smithi094.stdout:7/808: creat d2/d6/dc/d12/d29/d3f/dce/dfe/f10a x:0 0 0 2024-04-06T03:40:49.794 INFO:tasks.workunit.client.0.smithi094.stdout:7/809: write d2/d6/dc/d12/d9a/dbd/fe3 [607299,40203] 0 2024-04-06T03:40:49.794 INFO:tasks.workunit.client.0.smithi094.stdout:3/946: dread d9/d10/fd8 [0,4194304] 0 2024-04-06T03:40:49.794 INFO:tasks.workunit.client.1.smithi136.stdout:4/873: dwrite d1d/d31/d3e/d55/fe7 [0,4194304] 0 2024-04-06T03:40:49.795 INFO:tasks.workunit.client.1.smithi136.stdout:3/989: mkdir d0/d4/d12/d19/d55/d6f/ddb/d14c 0 2024-04-06T03:40:49.796 INFO:tasks.workunit.client.1.smithi136.stdout:4/874: dread d1d/d31/d78/feb [0,4194304] 0 2024-04-06T03:40:49.797 INFO:tasks.workunit.client.0.smithi094.stdout:0/885: link d1/d9/dd/d18/d1e/def/dab/d4e/de4/fc3 d1/d9/dd/d18/d1e/f12a 0 2024-04-06T03:40:49.797 INFO:tasks.workunit.client.0.smithi094.stdout:0/886: write d1/d9/dd/d18/d78/ffe [521857,73105] 0 2024-04-06T03:40:49.799 INFO:tasks.workunit.client.1.smithi136.stdout:6/905: mkdir d2/d7/d26/d30/d9c/d13a 0 2024-04-06T03:40:49.802 INFO:tasks.workunit.client.1.smithi136.stdout:3/990: rename d0/d4/d12/d2d/d37/d116/l119 to d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/d126/l14d 0 2024-04-06T03:40:49.802 INFO:tasks.workunit.client.1.smithi136.stdout:3/991: chown d0/d4/d12/d19/d55/d6f/d82/d5d/lb3 93297461 1 2024-04-06T03:40:49.805 INFO:tasks.workunit.client.0.smithi094.stdout:4/976: rename d0/d4/d4b/lc6 to d0/d77/d123/d49/db2/l149 0 2024-04-06T03:40:49.805 INFO:tasks.workunit.client.0.smithi094.stdout:1/947: dwrite d0/d1/d27/fa9 [0,4194304] 0 2024-04-06T03:40:49.806 INFO:tasks.workunit.client.1.smithi136.stdout:6/906: mkdir d2/d7/d26/d30/d6e/d85/d13b 0 2024-04-06T03:40:49.806 INFO:tasks.workunit.client.0.smithi094.stdout:7/810: creat d2/d6/dc/d12/d29/d100/f10b x:0 0 0 2024-04-06T03:40:49.809 INFO:tasks.workunit.client.1.smithi136.stdout:3/992: creat d0/d4/d12/d19/d55/d3c/d5a/d79/d146/d51/d8e/d10f/f14e x:0 0 0 2024-04-06T03:40:49.809 INFO:tasks.workunit.client.1.smithi136.stdout:3/993: fdatasync d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/f125 0 2024-04-06T03:40:49.809 INFO:tasks.workunit.client.0.smithi094.stdout:5/873: dwrite db/d19/d21/f93 [0,4194304] 0 2024-04-06T03:40:49.810 INFO:tasks.workunit.client.0.smithi094.stdout:3/947: creat d9/d20/d2d/d128/dba/f155 x:0 0 0 2024-04-06T03:40:49.813 INFO:tasks.workunit.client.1.smithi136.stdout:3/994: rename d0/d4/d12/d19/d55/d3c/d5a/d79/d7e/f102 to d0/d4/d12/d19/d55/d3c/d46/f14f 0 2024-04-06T03:40:49.814 INFO:tasks.workunit.client.0.smithi094.stdout:6/902: dwrite dc/d3a/d7f/fc0 [0,4194304] 0 2024-04-06T03:40:49.814 INFO:tasks.workunit.client.0.smithi094.stdout:0/887: mknod d1/d9/dd/d18/d1e/def/dab/d111/c12b 0 2024-04-06T03:40:49.823 INFO:tasks.workunit.client.1.smithi136.stdout:6/907: dwrite d2/d25/f59 [0,4194304] 0 2024-04-06T03:40:49.823 INFO:tasks.workunit.client.0.smithi094.stdout:4/977: rename d0/d15/d3a/d61/cea to d0/d77/d123/d49/db2/dc7/c14a 0 2024-04-06T03:40:49.824 INFO:tasks.workunit.client.0.smithi094.stdout:4/978: truncate d0/d15/d3a/d61/d81/f13e 336962 0 2024-04-06T03:40:49.824 INFO:tasks.workunit.client.0.smithi094.stdout:4/979: readlink d0/d4/d129/ld2 0 2024-04-06T03:40:49.825 INFO:tasks.workunit.client.1.smithi136.stdout:3/995: creat d0/d4/d12/d19/d115/dc9/d111/f150 x:0 0 0 2024-04-06T03:40:49.825 INFO:tasks.workunit.client.1.smithi136.stdout:3/996: stat d0/d4/d91 0 2024-04-06T03:40:49.825 INFO:tasks.workunit.client.1.smithi136.stdout:3/997: chown d0/d4/d12/d19/d2e/d3b/f5f 0 1 2024-04-06T03:40:49.826 INFO:tasks.workunit.client.0.smithi094.stdout:1/948: creat d0/d1/d27/da5/d40/db1/d2f/d4b/f14e x:0 0 0 2024-04-06T03:40:49.827 INFO:tasks.workunit.client.1.smithi136.stdout:4/875: dwrite d1d/f52 [0,4194304] 0 2024-04-06T03:40:49.828 INFO:tasks.workunit.client.0.smithi094.stdout:7/811: unlink d2/fa8 0 2024-04-06T03:40:49.830 INFO:tasks.workunit.client.1.smithi136.stdout:3/998: rename d0/l28 to d0/d4/d12/d2d/d37/d116/l151 0 2024-04-06T03:40:49.835 INFO:tasks.workunit.client.0.smithi094.stdout:4/980: write d0/d4/d67/f79 [400123,736] 0 2024-04-06T03:40:49.836 INFO:tasks.workunit.client.1.smithi136.stdout:4/876: link d1d/d31/d3e/d55/d89/c8f d1d/d31/d3e/d55/d89/dea/c12e 0 2024-04-06T03:40:49.839 INFO:tasks.workunit.client.1.smithi136.stdout:3/999: link d0/d4/d12/d19/d55/lcd d0/d4/d12/dc3/dd3/de1/l152 0 2024-04-06T03:40:49.839 INFO:tasks.workunit.client.1.smithi136.stdout:6/908: dwrite d2/d7/d26/f125 [0,4194304] 0 2024-04-06T03:40:49.840 INFO:tasks.workunit.client.1.smithi136.stdout:6/909: chown d2/d7/d26/d30/d6e/d58/cd2 3 1 2024-04-06T03:40:49.840 INFO:tasks.workunit.client.1.smithi136.stdout:6/910: truncate d2/d7/d26/dcd/dad/d116/f135 222365 0 2024-04-06T03:40:49.840 INFO:tasks.workunit.client.1.smithi136.stdout:6/911: readlink d2/d93/dab/l12d 0 2024-04-06T03:40:49.841 INFO:tasks.workunit.client.0.smithi094.stdout:7/812: dwrite d2/d6/dc/dd/f48 [4194304,4194304] 0 2024-04-06T03:40:49.845 INFO:tasks.workunit.client.0.smithi094.stdout:4/981: dread d0/d15/fe1 [0,4194304] 0 2024-04-06T03:40:49.846 INFO:tasks.workunit.client.1.smithi136.stdout:6/912: dread d2/d7/d26/d30/f50 [4194304,4194304] 0 2024-04-06T03:40:49.846 INFO:tasks.workunit.client.1.smithi136.stdout:6/913: chown d2/d104/c83 5 1 2024-04-06T03:40:49.847 INFO:tasks.workunit.client.1.smithi136.stdout:4/877: link d1d/d31/d3e/f5d d1d/dd0/d10d/f12f 0 2024-04-06T03:40:49.853 INFO:tasks.workunit.client.1.smithi136.stdout:4/878: symlink d1d/d31/d3e/dd4/l130 0 2024-04-06T03:40:49.855 INFO:tasks.workunit.client.1.smithi136.stdout:4/879: creat d1d/d31/d78/f131 x:0 0 0 2024-04-06T03:40:49.855 INFO:tasks.workunit.client.1.smithi136.stdout:6/914: dread d2/d7/d26/d30/d6e/d3d/f81 [0,4194304] 0 2024-04-06T03:40:49.856 INFO:tasks.workunit.client.1.smithi136.stdout:4/880: dread - d1d/f124 zero size 2024-04-06T03:40:49.858 INFO:tasks.workunit.client.1.smithi136.stdout:6/915: truncate d2/d7/d26/d30/f7e 7510767 0 2024-04-06T03:40:49.860 INFO:tasks.workunit.client.1.smithi136.stdout:4/881: mkdir d1d/d2f/d6c/da5/d132 0 2024-04-06T03:40:49.862 INFO:tasks.workunit.client.1.smithi136.stdout:6/916: rmdir d2/d7/d26/d30/d6e/d85/d13b 0 2024-04-06T03:40:49.862 INFO:tasks.workunit.client.1.smithi136.stdout:6/917: fdatasync d2/d7/d26/d30/d6e/d58/d5d/d62/fb0 0 2024-04-06T03:40:49.862 INFO:tasks.workunit.client.1.smithi136.stdout:6/918: dread - d2/d93/dab/f138 zero size 2024-04-06T03:40:49.865 INFO:tasks.workunit.client.1.smithi136.stdout:4/882: mknod d1d/d31/d3e/d56/c133 0 2024-04-06T03:40:49.865 INFO:tasks.workunit.client.1.smithi136.stdout:4/883: write d1d/d31/d3e/d55/d7f/f107 [845726,20780] 0 2024-04-06T03:40:49.865 INFO:tasks.workunit.client.1.smithi136.stdout:4/884: write d1d/d2f/d6c/da5/df0/f120 [4907489,117624] 0 2024-04-06T03:40:49.865 INFO:tasks.workunit.client.0.smithi094.stdout:0/888: truncate d1/d9/dd/d18/d32/f9c 5120257 0 2024-04-06T03:40:49.867 INFO:tasks.workunit.client.0.smithi094.stdout:1/949: creat d0/d127/daf/f14f x:0 0 0 2024-04-06T03:40:49.868 INFO:tasks.workunit.client.1.smithi136.stdout:6/919: symlink d2/d7/d26/d97/d115/l13c 0 2024-04-06T03:40:49.869 INFO:tasks.workunit.client.0.smithi094.stdout:0/889: write d1/d9/dd/d18/d32/d42/f4a [1170880,44204] 0 2024-04-06T03:40:49.870 INFO:tasks.workunit.client.0.smithi094.stdout:0/890: write d1/d9/dd/d18/d1e/d22/f97 [4861473,45755] 0 2024-04-06T03:40:49.870 INFO:tasks.workunit.client.1.smithi136.stdout:4/885: mknod d1d/d2f/da6/dbf/c134 0 2024-04-06T03:40:49.873 INFO:tasks.workunit.client.1.smithi136.stdout:6/920: link d2/d7/d26/d97/d115/l12b d2/l13d 0 2024-04-06T03:40:49.877 INFO:tasks.workunit.client.1.smithi136.stdout:6/921: symlink d2/d7/d26/dcd/dad/d116/d133/l13e 0 2024-04-06T03:40:49.883 INFO:tasks.workunit.client.0.smithi094.stdout:0/891: mknod d1/d9/dd/d18/d1e/def/dab/c12c 0 2024-04-06T03:40:49.883 INFO:tasks.workunit.client.1.smithi136.stdout:6/922: read d2/d7/d26/d30/d6e/f6b [1408589,52573] 0 2024-04-06T03:40:49.884 INFO:tasks.workunit.client.0.smithi094.stdout:7/813: getdents d2/d6/dc/d12/d29/d3f 0 2024-04-06T03:40:49.886 INFO:tasks.workunit.client.0.smithi094.stdout:1/950: dwrite d0/d7/d1a/da8/fbb [0,4194304] 0 2024-04-06T03:40:49.888 INFO:tasks.workunit.client.0.smithi094.stdout:4/982: rmdir d0/d4/d67/dcb 39 2024-04-06T03:40:49.889 INFO:tasks.workunit.client.0.smithi094.stdout:4/983: fsync d0/f83 0 2024-04-06T03:40:49.892 INFO:tasks.workunit.client.0.smithi094.stdout:0/892: rmdir d1/d9/dd/d18/d5d/d60/dfd 39 2024-04-06T03:40:49.892 INFO:tasks.workunit.client.0.smithi094.stdout:0/893: read d1/f23 [1482307,19520] 0 2024-04-06T03:40:49.892 INFO:tasks.workunit.client.0.smithi094.stdout:0/894: truncate d1/d9/dd/d18/d1e/def/d67/df6/f11f 842877 0 2024-04-06T03:40:49.892 INFO:tasks.workunit.client.0.smithi094.stdout:0/895: write d1/d9/dd/d18/d1e/def/d67/df6/ff9 [94895,32942] 0 2024-04-06T03:40:49.894 INFO:tasks.workunit.client.0.smithi094.stdout:7/814: creat d2/d6/dc/d12/d3a/dcc/dde/db7/f10c x:0 0 0 2024-04-06T03:40:49.894 INFO:tasks.workunit.client.0.smithi094.stdout:7/815: creat d2/d6/dc/dd/df/d95/dc9/f10d x:0 0 0 2024-04-06T03:40:49.895 INFO:tasks.workunit.client.1.smithi136.stdout:4/886: dwrite d1d/d31/d8e/f9e [0,4194304] 0 2024-04-06T03:40:49.898 INFO:tasks.workunit.client.0.smithi094.stdout:1/951: creat d0/d1/d27/da5/d40/db1/d2f/d3c/f150 x:0 0 0 2024-04-06T03:40:49.898 INFO:tasks.workunit.client.0.smithi094.stdout:1/952: readlink d0/d1/d27/da5/d3f/de9/lf4 0 2024-04-06T03:40:49.900 INFO:tasks.workunit.client.0.smithi094.stdout:4/984: getdents d0/df/dbd 0 2024-04-06T03:40:49.900 INFO:tasks.workunit.client.1.smithi136.stdout:6/923: dwrite d2/d7/d26/d97/f11b [0,4194304] 0 2024-04-06T03:40:49.900 INFO:tasks.workunit.client.1.smithi136.stdout:6/924: rename d2/d7/d26/d30 to d2/d7/d26/d30/d6e/d85/d13f 22 2024-04-06T03:40:49.904 INFO:tasks.workunit.client.1.smithi136.stdout:6/925: creat d2/d7/d26/d30/d6e/d3d/d132/f140 x:0 0 0 2024-04-06T03:40:49.906 INFO:tasks.workunit.client.0.smithi094.stdout:1/953: mkdir d0/d1/d27/da5/d40/db1/d2f/d3c/d151 0 2024-04-06T03:40:49.907 INFO:tasks.workunit.client.0.smithi094.stdout:1/954: read d0/d7/ff9 [100291,71791] 0 2024-04-06T03:40:49.908 INFO:tasks.workunit.client.1.smithi136.stdout:6/926: symlink d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/l141 0 2024-04-06T03:40:49.911 INFO:tasks.workunit.client.1.smithi136.stdout:4/887: dwrite d1d/d31/d3e/d55/d7f/f107 [0,4194304] 0 2024-04-06T03:40:49.911 INFO:tasks.workunit.client.1.smithi136.stdout:4/888: readlink d1d/d2f/d6c/da5/la7 0 2024-04-06T03:40:49.911 INFO:tasks.workunit.client.1.smithi136.stdout:4/889: creat d1d/d31/d3e/d55/f135 x:0 0 0 2024-04-06T03:40:49.911 INFO:tasks.workunit.client.1.smithi136.stdout:4/890: chown d1d/d31/d3e/d55/d7f/f107 3 1 2024-04-06T03:40:49.913 INFO:tasks.workunit.client.0.smithi094.stdout:0/896: rename d1/d9/dd/d18/d5d/d60/dfd/dac/cd2 to d1/d9/dd/d18/d1e/def/dab/d4e/ddf/c12d 0 2024-04-06T03:40:49.915 INFO:tasks.workunit.client.1.smithi136.stdout:6/927: creat d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/f142 x:0 0 0 2024-04-06T03:40:49.915 INFO:tasks.workunit.client.1.smithi136.stdout:6/928: write d2/fa [2616869,129111] 0 2024-04-06T03:40:49.915 INFO:tasks.workunit.client.1.smithi136.stdout:6/929: dread - d2/d7/d26/d30/d6e/d3d/d47/f139 zero size 2024-04-06T03:40:49.915 INFO:tasks.workunit.client.1.smithi136.stdout:6/930: getdents d2/d7/d26/d30/d6e/d3d/d47/d121 0 2024-04-06T03:40:49.919 INFO:tasks.workunit.client.1.smithi136.stdout:4/891: dread d1d/d2f/da6/fcf [0,4194304] 0 2024-04-06T03:40:49.923 INFO:tasks.workunit.client.1.smithi136.stdout:6/931: creat d2/d7/d26/d30/d6e/d11c/f143 x:0 0 0 2024-04-06T03:40:49.928 INFO:tasks.workunit.client.1.smithi136.stdout:4/892: dread d1d/fe2 [0,4194304] 0 2024-04-06T03:40:49.929 INFO:tasks.workunit.client.0.smithi094.stdout:0/897: dwrite d1/d9/dd/d18/d1e/d86/d96/daa/fee [0,4194304] 0 2024-04-06T03:40:49.929 INFO:tasks.workunit.client.0.smithi094.stdout:0/898: truncate d1/d9/dd/d18/d1a/d44/dc2/ffa 2714 0 2024-04-06T03:40:49.931 INFO:tasks.workunit.client.1.smithi136.stdout:4/893: mkdir d1d/d31/d3e/d56/d136 0 2024-04-06T03:40:49.932 INFO:tasks.workunit.client.0.smithi094.stdout:6/903: dwrite dc/dd3/d72/d87/f11c [0,4194304] 0 2024-04-06T03:40:49.933 INFO:tasks.workunit.client.1.smithi136.stdout:4/894: creat d1d/d2f/da6/d11f/f137 x:0 0 0 2024-04-06T03:40:49.934 INFO:tasks.workunit.client.1.smithi136.stdout:4/895: readlink d1d/d2f/d6c/l11c 0 2024-04-06T03:40:49.934 INFO:tasks.workunit.client.1.smithi136.stdout:4/896: write d1d/d31/d3e/d56/d7b/ff8 [1851100,61922] 0 2024-04-06T03:40:49.938 INFO:tasks.workunit.client.1.smithi136.stdout:4/897: write d1d/fed [17787,54232] 0 2024-04-06T03:40:49.939 INFO:tasks.workunit.client.1.smithi136.stdout:4/898: readlink d1d/dd0/l10e 0 2024-04-06T03:40:49.939 INFO:tasks.workunit.client.1.smithi136.stdout:4/899: fdatasync d1d/d2f/da6/d11f/fc4 0 2024-04-06T03:40:49.940 INFO:tasks.workunit.client.0.smithi094.stdout:7/816: rmdir d2 39 2024-04-06T03:40:49.940 INFO:tasks.workunit.client.0.smithi094.stdout:0/899: read d1/d9/dd/d18/d32/f5f [220194,76101] 0 2024-04-06T03:40:49.941 INFO:tasks.workunit.client.0.smithi094.stdout:1/955: symlink d0/d7/d1a/d64/l152 0 2024-04-06T03:40:49.942 INFO:tasks.workunit.client.0.smithi094.stdout:0/900: write d1/d9/dd/d18/d1a/f54 [1181860,80317] 0 2024-04-06T03:40:49.942 INFO:tasks.workunit.client.0.smithi094.stdout:0/901: write d1/d9/dd/d18/d1e/f12a [186423,37177] 0 2024-04-06T03:40:49.942 INFO:tasks.workunit.client.0.smithi094.stdout:0/902: fsync d1/d9/dd/d18/d5d/d60/dfd/f3c 0 2024-04-06T03:40:49.943 INFO:tasks.workunit.client.1.smithi136.stdout:4/900: link d1d/d2f/f7d d1d/d70/f138 0 2024-04-06T03:40:49.944 INFO:tasks.workunit.client.1.smithi136.stdout:4/901: dread - d1d/d2f/da6/d11f/f137 zero size 2024-04-06T03:40:49.944 INFO:tasks.workunit.client.0.smithi094.stdout:8/948: sync 2024-04-06T03:40:49.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/915: sync 2024-04-06T03:40:49.944 INFO:tasks.workunit.client.0.smithi094.stdout:2/874: sync 2024-04-06T03:40:49.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/916: chown d2/dc/d46/d111/d33 0 1 2024-04-06T03:40:49.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/917: fdatasync d2/dc/da6/f100 0 2024-04-06T03:40:49.944 INFO:tasks.workunit.client.0.smithi094.stdout:9/918: fsync d2/dc/d10/d2a/fcc 0 2024-04-06T03:40:49.944 INFO:tasks.workunit.client.0.smithi094.stdout:8/949: read d18/d2c/d37/d60/da3/db8/dc3/fd4 [254765,58861] 0 2024-04-06T03:40:49.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/903: dread d1/d9/dd/d18/d5d/d60/fc1 [4194304,4194304] 0 2024-04-06T03:40:49.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/904: fdatasync d1/d9/fd4 0 2024-04-06T03:40:49.948 INFO:tasks.workunit.client.0.smithi094.stdout:0/905: truncate d1/d9/dd/d18/d1e/def/fb6 1995952 0 2024-04-06T03:40:49.949 INFO:tasks.workunit.client.1.smithi136.stdout:4/902: chown d1d/l21 3530814 1 2024-04-06T03:40:49.949 INFO:tasks.workunit.client.1.smithi136.stdout:4/903: readlink d1d/l21 0 2024-04-06T03:40:49.949 INFO:tasks.workunit.client.1.smithi136.stdout:4/904: write d1d/d31/d3e/d55/d89/db3/fbe [497358,26384] 0 2024-04-06T03:40:49.952 INFO:tasks.workunit.client.0.smithi094.stdout:4/985: rmdir d0/d15 39 2024-04-06T03:40:49.952 INFO:tasks.workunit.client.0.smithi094.stdout:4/986: chown d0/d4/ce7 23 1 2024-04-06T03:40:49.952 INFO:tasks.workunit.client.0.smithi094.stdout:4/987: stat d0/d4/dde/lf4 0 2024-04-06T03:40:49.953 INFO:tasks.workunit.client.0.smithi094.stdout:6/904: dwrite dc/d3a/d7f/fc0 [4194304,4194304] 0 2024-04-06T03:40:49.955 INFO:tasks.workunit.client.1.smithi136.stdout:6/932: dwrite d2/d93/dab/f138 [0,4194304] 0 2024-04-06T03:40:49.955 INFO:tasks.workunit.client.1.smithi136.stdout:6/933: chown d2/d7/d26/d30/d6e/f35 873653971 1 2024-04-06T03:40:49.964 INFO:tasks.workunit.client.0.smithi094.stdout:9/919: dread d2/dc/d10/d2a/fa2 [0,4194304] 0 2024-04-06T03:40:49.966 INFO:tasks.workunit.client.0.smithi094.stdout:1/956: dwrite d0/d1/d27/da5/d40/db1/d2f/d3c/d48/dcb/d10c/f129 [0,4194304] 0 2024-04-06T03:40:49.966 INFO:tasks.workunit.client.0.smithi094.stdout:1/957: chown d0/d1/d27/da5/d40/db1/d2f/d4b/c11c 3056998 1 2024-04-06T03:40:49.967 INFO:tasks.workunit.client.0.smithi094.stdout:1/958: readlink d0/d7/l101 0 2024-04-06T03:40:49.969 INFO:tasks.workunit.client.0.smithi094.stdout:0/906: dread d1/d9/dd/d18/d5d/d60/dfd/dac/ff4 [0,4194304] 0 2024-04-06T03:40:49.969 INFO:tasks.workunit.client.0.smithi094.stdout:4/988: dwrite d0/df/d13/d4a/f9d [0,4194304] 0 2024-04-06T03:40:49.969 INFO:tasks.workunit.client.0.smithi094.stdout:8/950: dwrite d18/d2c/d37/d60/d70/fee [0,4194304] 0 2024-04-06T03:40:49.969 INFO:tasks.workunit.client.0.smithi094.stdout:2/875: truncate d0/d8/d1d/d77/fa6 2799172 0 2024-04-06T03:40:49.974 INFO:tasks.workunit.client.0.smithi094.stdout:9/920: dread d2/dc/d21/d26/d78/d69/da8/d96/fab [0,4194304] 0 2024-04-06T03:40:49.978 INFO:tasks.workunit.client.1.smithi136.stdout:6/934: dwrite d2/d7/d26/d30/d6e/d3d/d47/f139 [0,4194304] 0 2024-04-06T03:40:49.983 INFO:tasks.workunit.client.0.smithi094.stdout:7/817: truncate d2/d6/f9 1394328 0 2024-04-06T03:40:49.984 INFO:tasks.workunit.client.0.smithi094.stdout:4/989: dwrite d0/df/f19 [8388608,4194304] 0 2024-04-06T03:40:49.987 INFO:tasks.workunit.client.0.smithi094.stdout:1/959: link d0/d1/d27/da5/d40/db1/l84 d0/d1/d27/da5/d40/db1/d2f/d3c/da0/l153 0 2024-04-06T03:40:49.990 INFO:tasks.workunit.client.1.smithi136.stdout:6/935: mknod d2/d7/d26/d30/d6e/d58/d131/d134/c144 0 2024-04-06T03:40:49.992 INFO:tasks.workunit.client.0.smithi094.stdout:0/907: symlink d1/d9/dd/d18/d1e/d86/d96/l12e 0 2024-04-06T03:40:49.993 INFO:tasks.workunit.client.1.smithi136.stdout:6/936: dread d2/d104/f67 [0,4194304] 0 2024-04-06T03:40:49.993 INFO:tasks.workunit.client.1.smithi136.stdout:6/937: dread - d2/d93/dab/fc5 zero size 2024-04-06T03:40:49.995 INFO:tasks.workunit.client.0.smithi094.stdout:2/876: creat d0/d8/d27/f128 x:0 0 0 2024-04-06T03:40:49.996 INFO:tasks.workunit.client.1.smithi136.stdout:6/938: mknod d2/d7/d26/d30/d6e/d58/d131/c145 0 2024-04-06T03:40:49.996 INFO:tasks.workunit.client.1.smithi136.stdout:6/939: chown d2/d7/d26/d30/d6e/d3d/d47/c52 7197024 1 2024-04-06T03:40:49.997 INFO:tasks.workunit.client.0.smithi094.stdout:9/921: mknod d2/dc/d21/d23/d47/c139 0 2024-04-06T03:40:49.999 INFO:tasks.workunit.client.0.smithi094.stdout:9/922: dread d2/dc/d21/d23/d47/f134 [0,4194304] 0 2024-04-06T03:40:50.001 INFO:tasks.workunit.client.0.smithi094.stdout:4/990: mknod d0/df/d13/d4a/c14b 0 2024-04-06T03:40:50.003 INFO:tasks.workunit.client.0.smithi094.stdout:1/960: truncate d0/d7/d1a/da8/fbb 2808170 0 2024-04-06T03:40:50.006 INFO:tasks.workunit.client.0.smithi094.stdout:9/923: mkdir d2/dc/d21/d23/d47/dfa/d13a 0 2024-04-06T03:40:50.006 INFO:tasks.workunit.client.0.smithi094.stdout:9/924: stat d2/dc/d21/d23/d5b/de2/cea 0 2024-04-06T03:40:50.007 INFO:tasks.workunit.client.0.smithi094.stdout:1/961: link d0/d1/dd3/f10d d0/d7/d1a/d64/f154 0 2024-04-06T03:40:50.007 INFO:tasks.workunit.client.0.smithi094.stdout:1/962: write d0/f19 [3620093,127238] 0 2024-04-06T03:40:50.008 INFO:tasks.workunit.client.0.smithi094.stdout:0/908: symlink d1/d9/dd/d18/d1e/def/dab/d4e/ddf/l12f 0 2024-04-06T03:40:50.009 INFO:tasks.workunit.client.0.smithi094.stdout:0/909: dread - d1/d9/dd/d18/f127 zero size 2024-04-06T03:40:50.011 INFO:tasks.workunit.client.0.smithi094.stdout:0/910: creat d1/d9/dd/d18/d5d/d60/dfd/dac/f130 x:0 0 0 2024-04-06T03:40:50.012 INFO:tasks.workunit.client.0.smithi094.stdout:9/925: dread d2/dc/f104 [0,4194304] 0 2024-04-06T03:40:50.014 INFO:tasks.workunit.client.1.smithi136.stdout:6/940: dwrite d2/d7/f128 [0,4194304] 0 2024-04-06T03:40:50.015 INFO:tasks.workunit.client.1.smithi136.stdout:6/941: fsync d2/d7/d26/d30/d6e/db7/fbd 0 2024-04-06T03:40:50.015 INFO:tasks.workunit.client.1.smithi136.stdout:6/942: fdatasync d2/d7/d13/f100 0 2024-04-06T03:40:50.015 INFO:tasks.workunit.client.0.smithi094.stdout:4/991: write d0/d4/d67/d9f/fa9 [10890117,114320] 0 2024-04-06T03:40:50.015 INFO:tasks.workunit.client.0.smithi094.stdout:2/877: dwrite d0/dd/d19/dbe/fe9 [0,4194304] 0 2024-04-06T03:40:50.026 INFO:tasks.workunit.client.1.smithi136.stdout:6/943: rename d2/d7/d26/d30/d6e/d58/d5d/cb3 to d2/d7/d13/d79/dc2/ddd/c146 0 2024-04-06T03:40:50.027 INFO:tasks.workunit.client.0.smithi094.stdout:9/926: write d2/dc/d46/d111/f36 [1095139,49993] 0 2024-04-06T03:40:50.029 INFO:tasks.workunit.client.0.smithi094.stdout:0/911: dwrite d1/d9/dd/d18/d1e/def/dab/d4e/fb1 [0,4194304] 0 2024-04-06T03:40:50.029 INFO:tasks.workunit.client.0.smithi094.stdout:0/912: dread - d1/d9/dd/d18/d5d/d60/dfd/dac/f130 zero size 2024-04-06T03:40:50.029 INFO:tasks.workunit.client.0.smithi094.stdout:4/992: link d0/d77/d123/d46/l63 d0/df/db3/d11c/l14c 0 2024-04-06T03:40:50.032 INFO:tasks.workunit.client.1.smithi136.stdout:6/944: mknod d2/d7/d13/d79/c147 0 2024-04-06T03:40:50.032 INFO:tasks.workunit.client.1.smithi136.stdout:6/945: read - d2/d7/d13/d89/fb9 zero size 2024-04-06T03:40:50.034 INFO:tasks.workunit.client.0.smithi094.stdout:2/878: link d0/dd/d19/d31/f7d d0/d8/d1d/d77/f129 0 2024-04-06T03:40:50.034 INFO:tasks.workunit.client.0.smithi094.stdout:2/879: dread - d0/d8/d4f/dab/dc3/fe8 zero size 2024-04-06T03:40:50.037 INFO:tasks.workunit.client.0.smithi094.stdout:9/927: chown d2/dc/d21/d26/d78/c98 3093 1 2024-04-06T03:40:50.038 INFO:tasks.workunit.client.0.smithi094.stdout:9/928: creat d2/dc/f13b x:0 0 0 2024-04-06T03:40:50.041 INFO:tasks.workunit.client.1.smithi136.stdout:6/946: dread d2/d7/d26/d30/d6e/d3d/d47/f139 [0,4194304] 0 2024-04-06T03:40:50.042 INFO:tasks.workunit.client.0.smithi094.stdout:0/913: creat d1/f131 x:0 0 0 2024-04-06T03:40:50.042 INFO:tasks.workunit.client.0.smithi094.stdout:0/914: stat d1/d9/dd/d18/d1e/fbb 0 2024-04-06T03:40:50.043 INFO:tasks.workunit.client.0.smithi094.stdout:0/915: dread - d1/d9/dd/d18/d5d/d60/dbc/dda/f116 zero size 2024-04-06T03:40:50.043 INFO:tasks.workunit.client.0.smithi094.stdout:4/993: mknod d0/df/d57/c14d 0 2024-04-06T03:40:50.043 INFO:tasks.workunit.client.0.smithi094.stdout:4/994: write d0/d4/d4b/d141/fd4 [989324,54589] 0 2024-04-06T03:40:50.044 INFO:tasks.workunit.client.1.smithi136.stdout:6/947: mknod d2/d7/d26/d30/c148 0 2024-04-06T03:40:50.047 INFO:tasks.workunit.client.0.smithi094.stdout:9/929: getdents d2/dc/d21/d26/d8a/d12e 0 2024-04-06T03:40:50.047 INFO:tasks.workunit.client.0.smithi094.stdout:9/930: readlink d2/dc/l17 0 2024-04-06T03:40:50.047 INFO:tasks.workunit.client.0.smithi094.stdout:9/931: fsync d2/f7 0 2024-04-06T03:40:50.050 INFO:tasks.workunit.client.1.smithi136.stdout:6/948: read d2/d93/dab/ff0 [138086,80650] 0 2024-04-06T03:40:50.050 INFO:tasks.workunit.client.1.smithi136.stdout:6/949: dread - d2/d7/d26/d30/d6e/d3d/d132/f140 zero size 2024-04-06T03:40:50.050 INFO:tasks.workunit.client.1.smithi136.stdout:6/950: creat d2/d7/d26/d30/d6e/d58/d5d/f149 x:0 0 0 2024-04-06T03:40:50.050 INFO:tasks.workunit.client.1.smithi136.stdout:6/951: stat d2/d104/d75/c7f 0 2024-04-06T03:40:50.050 INFO:tasks.workunit.client.1.smithi136.stdout:4/905: sync 2024-04-06T03:40:50.051 INFO:tasks.workunit.client.0.smithi094.stdout:0/916: rmdir d1/d9/dd/d18/dad 39 2024-04-06T03:40:50.052 INFO:tasks.workunit.client.0.smithi094.stdout:4/995: creat d0/d77/d120/f14e x:0 0 0 2024-04-06T03:40:50.052 INFO:tasks.workunit.client.0.smithi094.stdout:4/996: chown d0/f83 1039243 1 2024-04-06T03:40:50.052 INFO:tasks.workunit.client.0.smithi094.stdout:4/997: readlink d0/df/d13/d4a/l10a 0 2024-04-06T03:40:50.052 INFO:tasks.workunit.client.0.smithi094.stdout:4/998: write d0/d15/f13c [509388,7949] 0 2024-04-06T03:40:50.053 INFO:tasks.workunit.client.0.smithi094.stdout:4/999: stat d0/df/db3/dd8/f133 0 2024-04-06T03:40:50.056 INFO:tasks.workunit.client.0.smithi094.stdout:9/932: creat d2/dc/d21/d26/d78/d69/da8/f13c x:0 0 0 2024-04-06T03:40:50.056 INFO:tasks.workunit.client.0.smithi094.stdout:9/933: dread - d2/dc/d46/d111/d99/fbd zero size 2024-04-06T03:40:50.057 INFO:tasks.workunit.client.1.smithi136.stdout:6/952: creat d2/d25/f14a x:0 0 0 2024-04-06T03:40:50.057 INFO:tasks.workunit.client.1.smithi136.stdout:6/953: chown d2/d93/dab/f138 6813 1 2024-04-06T03:40:50.057 INFO:tasks.workunit.client.1.smithi136.stdout:6/954: creat d2/d7/d13/f14b x:0 0 0 2024-04-06T03:40:50.058 INFO:tasks.workunit.client.1.smithi136.stdout:4/906: creat d1d/d2f/da6/de1/d10a/d125/f139 x:0 0 0 2024-04-06T03:40:50.059 INFO:tasks.workunit.client.0.smithi094.stdout:9/934: truncate d2/dc/d21/d26/f29 3082949 0 2024-04-06T03:40:50.061 INFO:tasks.workunit.client.1.smithi136.stdout:6/955: mkdir d2/d7/d26/d30/d6e/d3d/d47/d121/d14c 0 2024-04-06T03:40:50.062 INFO:tasks.workunit.client.0.smithi094.stdout:9/935: link d2/dc/d21/d23/d47/d51/le6 d2/dc/d21/d26/d8a/de9/dfd/l13d 0 2024-04-06T03:40:50.062 INFO:tasks.workunit.client.0.smithi094.stdout:9/936: dread - d2/dc/d21/d23/d47/d51/f115 zero size 2024-04-06T03:40:50.063 INFO:tasks.workunit.client.0.smithi094.stdout:0/917: dwrite d1/d9/dd/d18/d1e/def/dab/d4e/fb1 [0,4194304] 0 2024-04-06T03:40:50.063 INFO:tasks.workunit.client.1.smithi136.stdout:4/907: truncate d1d/d31/f58 573229 0 2024-04-06T03:40:50.064 INFO:tasks.workunit.client.0.smithi094.stdout:1/963: rmdir d0/d1/d27/da5/d40/db1/d2f/d3c 39 2024-04-06T03:40:50.066 INFO:tasks.workunit.client.0.smithi094.stdout:9/937: creat d2/dc/d21/d26/d8a/de9/dfd/f13e x:0 0 0 2024-04-06T03:40:50.069 INFO:tasks.workunit.client.1.smithi136.stdout:4/908: rename d1d/d31/d3e/d55/f135 to d1d/d2f/d6c/f13a 0 2024-04-06T03:40:50.069 INFO:tasks.workunit.client.1.smithi136.stdout:4/909: chown d1d/d31/d3e/d55/d7f/f107 5524 1 2024-04-06T03:40:50.069 INFO:tasks.workunit.client.1.smithi136.stdout:4/910: readlink d1d/d31/d3e/d56/lc2 0 2024-04-06T03:40:50.069 INFO:tasks.workunit.client.1.smithi136.stdout:4/911: chown d1d/d2f/da6/d11f/cb9 2 1 2024-04-06T03:40:50.070 INFO:tasks.workunit.client.0.smithi094.stdout:1/964: dread d0/d1/f65 [0,4194304] 0 2024-04-06T03:40:50.072 INFO:tasks.workunit.client.0.smithi094.stdout:0/918: symlink d1/d9/l132 0 2024-04-06T03:40:50.072 INFO:tasks.workunit.client.0.smithi094.stdout:0/919: write d1/d9/dd/d18/d5d/d60/dfd/dac/ff4 [5470209,107222] 0 2024-04-06T03:40:50.073 INFO:tasks.workunit.client.1.smithi136.stdout:4/912: symlink d1d/d2f/d6c/da5/d132/l13b 0 2024-04-06T03:40:50.074 INFO:tasks.workunit.client.0.smithi094.stdout:1/965: mknod d0/d7/d2d/dab/c155 0 2024-04-06T03:40:50.074 INFO:tasks.workunit.client.1.smithi136.stdout:4/913: fdatasync d1d/f20 0 2024-04-06T03:40:50.076 INFO:tasks.workunit.client.1.smithi136.stdout:6/956: dwrite d2/d7/d26/d30/d6e/d3d/d47/f8c [0,4194304] 0 2024-04-06T03:40:50.076 INFO:tasks.workunit.client.0.smithi094.stdout:1/966: rename d0/d1/dd3/f117 to d0/d1/d27/da5/db2/dce/f156 0 2024-04-06T03:40:50.077 INFO:tasks.workunit.client.1.smithi136.stdout:4/914: symlink d1d/d2f/d6c/da5/dc1/d109/l13c 0 2024-04-06T03:40:50.078 INFO:tasks.workunit.client.1.smithi136.stdout:6/957: rmdir d2/d92 39 2024-04-06T03:40:50.079 INFO:tasks.workunit.client.1.smithi136.stdout:4/915: mkdir d1d/d2f/d6c/da5/df0/d13d 0 2024-04-06T03:40:50.080 INFO:tasks.workunit.client.1.smithi136.stdout:6/958: rename d2/d7/d26/d30/f4c to d2/d7/d13/d79/dc2/ddd/f14d 0 2024-04-06T03:40:50.081 INFO:tasks.workunit.client.1.smithi136.stdout:6/959: dread - d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/f10a zero size 2024-04-06T03:40:50.083 INFO:tasks.workunit.client.1.smithi136.stdout:6/960: creat d2/d7/d26/d30/f14e x:0 0 0 2024-04-06T03:40:50.084 INFO:tasks.workunit.client.1.smithi136.stdout:4/916: symlink d1d/d31/l13e 0 2024-04-06T03:40:50.086 INFO:tasks.workunit.client.1.smithi136.stdout:4/917: readlink d1d/d31/l79 0 2024-04-06T03:40:50.086 INFO:tasks.workunit.client.1.smithi136.stdout:4/918: fsync d1d/d31/f4c 0 2024-04-06T03:40:50.088 INFO:tasks.workunit.client.1.smithi136.stdout:4/919: mknod d1d/d31/d78/dcb/c13f 0 2024-04-06T03:40:50.088 INFO:tasks.workunit.client.1.smithi136.stdout:6/961: dread d2/d7/d26/dcd/dad/fbb [0,4194304] 0 2024-04-06T03:40:50.094 INFO:tasks.workunit.client.0.smithi094.stdout:1/967: dwrite d0/d7/d1a/da8/fbb [0,4194304] 0 2024-04-06T03:40:50.094 INFO:tasks.workunit.client.0.smithi094.stdout:1/968: getdents d0/d1/d27/da5/d40/d50/d13f 0 2024-04-06T03:40:50.094 INFO:tasks.workunit.client.0.smithi094.stdout:0/920: dwrite d1/d9/dd/d18/d1a/f7e [0,4194304] 0 2024-04-06T03:40:50.098 INFO:tasks.workunit.client.0.smithi094.stdout:1/969: getdents d0/d127/d30/d38 0 2024-04-06T03:40:50.101 INFO:tasks.workunit.client.0.smithi094.stdout:1/970: truncate d0/d1/f116 2254056 0 2024-04-06T03:40:50.101 INFO:tasks.workunit.client.0.smithi094.stdout:1/971: readlink d0/d127/l28 0 2024-04-06T03:40:50.101 INFO:tasks.workunit.client.0.smithi094.stdout:1/972: chown d0/d127/d30/df6/l142 2299262 1 2024-04-06T03:40:50.101 INFO:tasks.workunit.client.0.smithi094.stdout:1/973: fsync d0/d127/fd0 0 2024-04-06T03:40:50.101 INFO:tasks.workunit.client.0.smithi094.stdout:1/974: stat d0/d1/d27/da5/d40/db1/d2f/d4b/f9e 0 2024-04-06T03:40:50.101 INFO:tasks.workunit.client.0.smithi094.stdout:1/975: chown d0/d1/d27/da5/d133 119349 1 2024-04-06T03:40:50.101 INFO:tasks.workunit.client.0.smithi094.stdout:1/976: fsync d0/d1/d27/da5/fca 0 2024-04-06T03:40:50.103 INFO:tasks.workunit.client.0.smithi094.stdout:1/977: creat d0/d1/d27/da5/d40/db1/dc3/df0/f157 x:0 0 0 2024-04-06T03:40:50.104 INFO:tasks.workunit.client.1.smithi136.stdout:4/920: write d1d/d31/f40 [602759,85207] 0 2024-04-06T03:40:50.108 INFO:tasks.workunit.client.1.smithi136.stdout:6/962: dwrite d2/d25/f17 [0,4194304] 0 2024-04-06T03:40:50.118 INFO:tasks.workunit.client.1.smithi136.stdout:6/963: dread d2/fa [0,4194304] 0 2024-04-06T03:40:50.120 INFO:tasks.workunit.client.1.smithi136.stdout:6/964: symlink d2/d7/d26/d30/d6e/db7/l14f 0 2024-04-06T03:40:50.120 INFO:tasks.workunit.client.0.smithi094.stdout:0/921: dwrite d1/d9/dd/d18/d1e/def/f59 [0,4194304] 0 2024-04-06T03:40:50.122 INFO:tasks.workunit.client.1.smithi136.stdout:6/965: mknod d2/d7/d26/d30/d6e/d3d/d47/d121/d14c/c150 0 2024-04-06T03:40:50.123 INFO:tasks.workunit.client.0.smithi094.stdout:0/922: mknod d1/d9/dd/d18/d1e/def/d67/d76/c133 0 2024-04-06T03:40:50.125 INFO:tasks.workunit.client.1.smithi136.stdout:6/966: link d2/d7/d13/d79/dc2/ddd/c146 d2/d7/d13/d89/c151 0 2024-04-06T03:40:50.125 INFO:tasks.workunit.client.1.smithi136.stdout:6/967: chown d2/d7/d26/dcd/lba 128570168 1 2024-04-06T03:40:50.126 INFO:tasks.workunit.client.1.smithi136.stdout:6/968: mknod d2/d7/d26/d30/d6e/d3d/d47/d121/d14c/c152 0 2024-04-06T03:40:50.126 INFO:tasks.workunit.client.1.smithi136.stdout:6/969: creat d2/d7/d26/d68/f153 x:0 0 0 2024-04-06T03:40:50.127 INFO:tasks.workunit.client.1.smithi136.stdout:4/921: dwrite d1d/d31/d3e/d56/f6f [0,4194304] 0 2024-04-06T03:40:50.132 INFO:tasks.workunit.client.1.smithi136.stdout:4/922: creat d1d/d2f/d6c/da5/f140 x:0 0 0 2024-04-06T03:40:50.137 INFO:tasks.workunit.client.0.smithi094.stdout:1/978: dwrite d0/d1/d27/fa9 [0,4194304] 0 2024-04-06T03:40:50.138 INFO:tasks.workunit.client.0.smithi094.stdout:9/938: dwrite d2/dc/d80/ddd/def/f128 [0,4194304] 0 2024-04-06T03:40:50.138 INFO:tasks.workunit.client.0.smithi094.stdout:9/939: write d2/dc/d21/d23/d5b/f89 [3939342,83289] 0 2024-04-06T03:40:50.138 INFO:tasks.workunit.client.0.smithi094.stdout:9/940: fdatasync d2/dc/d10/ff4 0 2024-04-06T03:40:50.138 INFO:tasks.workunit.client.0.smithi094.stdout:1/979: creat d0/d1/d27/da5/d40/db1/d2f/d3c/da0/f158 x:0 0 0 2024-04-06T03:40:50.140 INFO:tasks.workunit.client.0.smithi094.stdout:9/941: rename d2/dc/d21/d23/d47/d51/le6 to d2/dc/d46/d111/d33/l13f 0 2024-04-06T03:40:50.141 INFO:tasks.workunit.client.1.smithi136.stdout:6/970: dwrite d2/d7/d26/d30/d6e/d58/d5d/d62/d8d/f142 [0,4194304] 0 2024-04-06T03:40:50.141 INFO:tasks.workunit.client.0.smithi094.stdout:0/923: dwrite d1/d9/f20 [0,4194304] 0 2024-04-06T03:40:50.143 INFO:tasks.workunit.client.0.smithi094.stdout:9/942: unlink d2/dc/d21/d23/fe1 0 2024-04-06T03:40:50.143 INFO:tasks.workunit.client.0.smithi094.stdout:9/943: chown d2/dc/d21/d23/d47/dfa/d13a 2185269 1 2024-04-06T03:40:50.143 INFO:tasks.workunit.client.0.smithi094.stdout:0/924: dread d1/f2c [4194304,4194304] 0 2024-04-06T03:40:50.151 INFO:tasks.workunit.client.0.smithi094.stdout:0/925: creat d1/d9/dd/d18/d5d/d60/dbc/d105/f134 x:0 0 0 2024-04-06T03:40:50.151 INFO:tasks.workunit.client.0.smithi094.stdout:9/944: dread d2/dc/d46/d111/d99/fd9 [0,4194304] 0 2024-04-06T03:40:50.151 INFO:tasks.workunit.client.0.smithi094.stdout:9/945: chown d2/dc/c3e 93 1 2024-04-06T03:40:50.152 INFO:tasks.workunit.client.1.smithi136.stdout:6/971: getdents d2/d7/d26/d30/d6e/d3d 0 2024-04-06T03:40:50.154 INFO:tasks.workunit.client.0.smithi094.stdout:0/926: unlink d1/d9/dd/d18/d1e/l4c 0 2024-04-06T03:40:50.154 INFO:tasks.workunit.client.0.smithi094.stdout:0/927: write d1/d9/dd/d18/d1e/def/dab/d4e/f9e [1043509,16145] 0 2024-04-06T03:40:50.155 INFO:tasks.workunit.client.0.smithi094.stdout:9/946: mknod d2/dc/d21/d23/d2f/c140 0 2024-04-06T03:40:50.155 INFO:tasks.workunit.client.0.smithi094.stdout:9/947: creat d2/dc/d21/d26/d78/d69/da8/d96/f141 x:0 0 0 2024-04-06T03:40:50.157 INFO:tasks.workunit.client.0.smithi094.stdout:0/928: creat d1/d9/dd/d18/d78/deb/d112/f135 x:0 0 0 2024-04-06T03:40:50.160 INFO:tasks.workunit.client.0.smithi094.stdout:0/929: symlink d1/d9/dd/d120/l136 0 2024-04-06T03:40:50.162 INFO:tasks.workunit.client.0.smithi094.stdout:0/930: creat d1/d9/dd/f137 x:0 0 0 2024-04-06T03:40:50.169 INFO:tasks.workunit.client.0.smithi094.stdout:9/948: dwrite d2/dc/d46/d111/d33/f12b [0,4194304] 0 2024-04-06T03:40:50.169 INFO:tasks.workunit.client.0.smithi094.stdout:9/949: dread - d2/dc/d21/d26/d78/d69/da8/d96/f141 zero size 2024-04-06T03:40:50.176 INFO:tasks.workunit.client.0.smithi094.stdout:9/950: write d2/dc/da6/f100 [7623537,79267] 0 2024-04-06T03:40:50.177 INFO:tasks.workunit.client.0.smithi094.stdout:9/951: creat d2/dc/d21/d26/d8a/de9/f142 x:0 0 0 2024-04-06T03:40:50.177 INFO:tasks.workunit.client.0.smithi094.stdout:9/952: rmdir d2/dc/d46/d111 39 2024-04-06T03:40:50.178 INFO:tasks.workunit.client.0.smithi094.stdout:9/953: mknod d2/dc/d80/db7/c143 0 2024-04-06T03:40:50.179 INFO:tasks.workunit.client.0.smithi094.stdout:9/954: creat d2/dc/d21/d23/d47/f144 x:0 0 0 2024-04-06T03:40:50.196 INFO:tasks.workunit.client.0.smithi094.stdout:9/955: dwrite d2/dc/d21/d23/f10d [0,4194304] 0 2024-04-06T03:40:50.198 INFO:tasks.workunit.client.0.smithi094.stdout:9/956: symlink d2/dc/d80/l145 0 2024-04-06T03:40:50.200 INFO:tasks.workunit.client.0.smithi094.stdout:9/957: dread d2/dc/d21/d26/d78/fae [0,4194304] 0 2024-04-06T03:40:50.201 INFO:tasks.workunit.client.0.smithi094.stdout:9/958: symlink d2/dc/d21/d26/d8a/de9/l146 0 2024-04-06T03:40:50.201 INFO:tasks.workunit.client.0.smithi094.stdout:9/959: chown d2/dc/d21/d26/d78/d69/da8/l6e 132668 1 2024-04-06T03:40:50.202 INFO:tasks.workunit.client.0.smithi094.stdout:9/960: write d2/dc/d21/d23/d47/d51/dee/f126 [218030,61305] 0 2024-04-06T03:40:50.202 INFO:tasks.workunit.client.0.smithi094.stdout:9/961: getdents d2/dc/d46/d111/d33/d64/dc6 0 2024-04-06T03:40:50.208 INFO:tasks.workunit.client.0.smithi094.stdout:9/962: dread d2/dc/d10/d6a/f132 [0,4194304] 0 2024-04-06T03:40:50.208 INFO:tasks.workunit.client.0.smithi094.stdout:9/963: mknod d2/dc/d46/d111/d33/d64/c147 0 2024-04-06T03:40:50.209 INFO:tasks.workunit.client.0.smithi094.stdout:9/964: write d2/dc/d21/d26/d78/d69/fcf [120490,29940] 0 2024-04-06T03:40:50.209 INFO:tasks.workunit.client.0.smithi094.stdout:9/965: dread d2/dc/d46/f95 [0,4194304] 0 2024-04-06T03:40:50.210 INFO:tasks.workunit.client.0.smithi094.stdout:9/966: symlink d2/dc/d21/dc8/l148 0 2024-04-06T03:40:50.211 INFO:tasks.workunit.client.0.smithi094.stdout:9/967: mkdir d2/dc/d21/d23/d2f/d149 0 2024-04-06T03:40:50.211 INFO:tasks.workunit.client.0.smithi094.stdout:9/968: write d2/dc/d21/d23/d47/d51/f92 [1141484,51132] 0 2024-04-06T03:40:50.213 INFO:tasks.workunit.client.0.smithi094.stdout:9/969: dread d2/dc/d46/d111/d33/d64/dc6/fdb [0,4194304] 0 2024-04-06T03:40:50.213 INFO:tasks.workunit.client.0.smithi094.stdout:9/970: chown d2/dc/d10/l13 0 1 2024-04-06T03:40:50.213 INFO:tasks.workunit.client.0.smithi094.stdout:9/971: write d2/dc/d21/d23/d5b/f89 [1944879,108927] 0 2024-04-06T03:40:50.213 INFO:tasks.workunit.client.0.smithi094.stdout:9/972: chown d2/dc/l8d 400652554 1 2024-04-06T03:40:50.214 INFO:tasks.workunit.client.0.smithi094.stdout:9/973: creat d2/dc/d21/d26/d78/f14a x:0 0 0 2024-04-06T03:40:50.228 INFO:tasks.workunit.client.1.smithi136.stdout:4/923: sync 2024-04-06T03:40:50.228 INFO:tasks.workunit.client.1.smithi136.stdout:4/924: chown d1d/d31/d3e/d55/d89/db3/fc5 101758 1 2024-04-06T03:40:50.228 INFO:tasks.workunit.client.1.smithi136.stdout:4/925: readlink d1d/d31/d3e/d55/lb1 0 2024-04-06T03:40:50.228 INFO:tasks.workunit.client.1.smithi136.stdout:4/926: chown d1d/d31/d3e/l46 40 1 2024-04-06T03:40:50.229 INFO:tasks.workunit.client.1.smithi136.stdout:4/927: creat d1d/d2f/da6/de1/d10a/f141 x:0 0 0 2024-04-06T03:40:50.250 INFO:tasks.workunit.client.0.smithi094.stdout:9/974: dwrite d2/dc/d21/d23/d2f/f5f [0,4194304] 0 2024-04-06T03:40:50.256 INFO:tasks.workunit.client.0.smithi094.stdout:9/975: creat d2/dc/d21/f14b x:0 0 0 2024-04-06T03:40:50.256 INFO:tasks.workunit.client.0.smithi094.stdout:9/976: chown d2/dc/d21/d26/d78/d69/da8 221778952 1 2024-04-06T03:40:50.256 INFO:tasks.workunit.client.0.smithi094.stdout:9/977: dread - d2/dc/d21/d26/d78/d69/da8/ddc/f135 zero size 2024-04-06T03:40:50.261 INFO:tasks.workunit.client.0.smithi094.stdout:5/874: sync 2024-04-06T03:40:50.261 INFO:tasks.workunit.client.0.smithi094.stdout:3/948: sync 2024-04-06T03:40:50.262 INFO:tasks.workunit.client.0.smithi094.stdout:5/875: getdents db/d19/dc1/dd4/d77 0 2024-04-06T03:40:50.262 INFO:tasks.workunit.client.0.smithi094.stdout:5/876: write db/d19/dc1/ddf/fc4 [792807,48978] 0 2024-04-06T03:40:50.263 INFO:tasks.workunit.client.0.smithi094.stdout:3/949: mkdir d9/d20/d2d/d128/dba/d7c/d156 0 2024-04-06T03:40:50.263 INFO:tasks.workunit.client.0.smithi094.stdout:3/950: stat d9/d20/d2d/c49 0 2024-04-06T03:40:50.263 INFO:tasks.workunit.client.0.smithi094.stdout:3/951: read d9/d20/f4e [714541,130123] 0 2024-04-06T03:40:50.264 INFO:tasks.workunit.client.0.smithi094.stdout:3/952: symlink d9/d20/l157 0 2024-04-06T03:40:50.264 INFO:tasks.workunit.client.0.smithi094.stdout:3/953: write d9/d1e/d26/d6c/d8e/df8/f12f [539924,84959] 0 2024-04-06T03:40:50.264 INFO:tasks.workunit.client.0.smithi094.stdout:3/954: chown d9/d1e/d26/c127 49485120 1 2024-04-06T03:40:50.265 INFO:tasks.workunit.client.0.smithi094.stdout:3/955: creat d9/d20/d76/d6b/f158 x:0 0 0 2024-04-06T03:40:50.266 INFO:tasks.workunit.client.0.smithi094.stdout:3/956: write d9/d1e/d26/d55/d62/db8/fca [194563,87654] 0 2024-04-06T03:40:50.266 INFO:tasks.workunit.client.0.smithi094.stdout:3/957: write d9/d1d/d7d/dd7/f132 [221489,57749] 0 2024-04-06T03:40:50.272 INFO:tasks.workunit.client.0.smithi094.stdout:9/978: dwrite d2/dc/d80/ddd/fa7 [0,4194304] 0 2024-04-06T03:40:50.274 INFO:tasks.workunit.client.0.smithi094.stdout:9/979: rmdir d2/df9 0 2024-04-06T03:40:50.283 INFO:tasks.workunit.client.0.smithi094.stdout:3/958: dwrite d9/d20/d76/ff7 [0,4194304] 0 2024-04-06T03:40:50.283 INFO:tasks.workunit.client.0.smithi094.stdout:3/959: write d9/f3f [4665275,119607] 0 2024-04-06T03:40:50.285 INFO:tasks.workunit.client.1.smithi136.stdout:6/972: sync 2024-04-06T03:40:50.285 INFO:tasks.workunit.client.1.smithi136.stdout:6/973: creat d2/d7/d26/d30/f154 x:0 0 0 2024-04-06T03:40:50.285 INFO:tasks.workunit.client.1.smithi136.stdout:6/974: fsync d2/d7/fc7 0 2024-04-06T03:40:50.286 INFO:tasks.workunit.client.0.smithi094.stdout:3/960: rename d9/d20/d76/dcd to d9/d1e/d26/d54/d159 0 2024-04-06T03:40:50.286 INFO:tasks.workunit.client.0.smithi094.stdout:3/961: truncate d9/d10/d7e/fea 820972 0 2024-04-06T03:40:50.288 INFO:tasks.workunit.client.1.smithi136.stdout:6/975: stat d2/d7/d13/d89/le8 0 2024-04-06T03:40:50.288 INFO:tasks.workunit.client.0.smithi094.stdout:9/980: dwrite d2/dc/f13b [0,4194304] 0 2024-04-06T03:40:50.289 INFO:tasks.workunit.client.1.smithi136.stdout:6/976: creat d2/d7/d13/d79/dc2/f155 x:0 0 0 2024-04-06T03:40:50.289 INFO:tasks.workunit.client.1.smithi136.stdout:6/977: chown d2/d7/f5a 186009258 1 2024-04-06T03:40:50.292 INFO:tasks.workunit.client.0.smithi094.stdout:9/981: getdents d2/dc/d21/d26/d8a 0 2024-04-06T03:40:50.294 INFO:tasks.workunit.client.0.smithi094.stdout:9/982: link d2/dc/d21/c8f d2/dc/d21/d26/d8a/d12e/c14c 0 2024-04-06T03:40:50.295 INFO:tasks.workunit.client.0.smithi094.stdout:9/983: fdatasync d2/dc/d46/d111/f36 0 2024-04-06T03:40:50.295 INFO:tasks.workunit.client.0.smithi094.stdout:9/984: write d2/dc/d21/f9c [915711,83191] 0 2024-04-06T03:40:50.295 INFO:tasks.workunit.client.0.smithi094.stdout:9/985: fsync d2/dc/d10/d6a/f105 0 2024-04-06T03:40:50.296 INFO:tasks.workunit.client.0.smithi094.stdout:3/962: write d9/d20/d76/fac [2530451,7511] 0 2024-04-06T03:40:50.297 INFO:tasks.workunit.client.0.smithi094.stdout:3/963: creat d9/d1e/d26/d6c/d8e/f15a x:0 0 0 2024-04-06T03:40:50.297 INFO:tasks.workunit.client.0.smithi094.stdout:3/964: write d9/d20/d2d/d128/dba/dc5/f130 [66455,41622] 0 2024-04-06T03:40:50.299 INFO:tasks.workunit.client.0.smithi094.stdout:3/965: unlink d9/f8d 0 2024-04-06T03:40:50.312 INFO:tasks.workunit.client.0.smithi094.stdout:3/966: read d9/d10/fd8 [873315,10721] 0 2024-04-06T03:40:50.312 INFO:tasks.workunit.client.0.smithi094.stdout:3/967: stat d9/d10/d9b/db7 0 2024-04-06T03:40:50.314 INFO:tasks.workunit.client.0.smithi094.stdout:9/986: dwrite d2/dc/d21/d26/d78/d69/da8/d96/fa0 [0,4194304] 0 2024-04-06T03:40:50.315 INFO:tasks.workunit.client.0.smithi094.stdout:3/968: truncate d9/d1e/d26/f2c 3848376 0 2024-04-06T03:40:50.317 INFO:tasks.workunit.client.1.smithi136.stdout:6/978: dwrite d2/d93/dab/ff0 [0,4194304] 0 2024-04-06T03:40:50.318 INFO:tasks.workunit.client.0.smithi094.stdout:9/987: creat d2/dc/d80/ddd/def/f14d x:0 0 0 2024-04-06T03:40:50.318 INFO:tasks.workunit.client.0.smithi094.stdout:9/988: creat d2/dc/d80/ddd/d6b/d123/f14e x:0 0 0 2024-04-06T03:40:50.319 INFO:tasks.workunit.client.1.smithi136.stdout:4/928: sync 2024-04-06T03:40:50.319 INFO:tasks.workunit.client.1.smithi136.stdout:4/929: stat d1d/d31/d3e/d56/c133 0 2024-04-06T03:40:50.323 INFO:tasks.workunit.client.1.smithi136.stdout:4/930: symlink d1d/d2f/da6/l142 0 2024-04-06T03:40:50.323 INFO:tasks.workunit.client.1.smithi136.stdout:4/931: chown d1d/d70/ff1 329902 1 2024-04-06T03:40:50.323 INFO:tasks.workunit.client.1.smithi136.stdout:4/932: chown d1d/d31/d3e/l96 256 1 2024-04-06T03:40:50.324 INFO:tasks.workunit.client.0.smithi094.stdout:3/969: mknod d9/d1e/d6e/c15b 0 2024-04-06T03:40:50.325 INFO:tasks.workunit.client.0.smithi094.stdout:9/989: dread d2/dc/d10/d6a/f105 [0,4194304] 0 2024-04-06T03:40:50.326 INFO:tasks.workunit.client.0.smithi094.stdout:9/990: truncate d2/dc/d21/d26/f29 3902748 0 2024-04-06T03:40:50.328 INFO:tasks.workunit.client.0.smithi094.stdout:9/991: unlink d2/dc/d21/d26/d78/d69/da8/l106 0 2024-04-06T03:40:50.331 INFO:tasks.workunit.client.0.smithi094.stdout:9/992: getdents d2/dc/d21/d26/d78/d69 0 2024-04-06T03:40:50.331 INFO:tasks.workunit.client.0.smithi094.stdout:9/993: stat d2/dc/d80/db7/dd0/feb 0 2024-04-06T03:40:50.333 INFO:tasks.workunit.client.1.smithi136.stdout:4/933: write d1d/d31/d3e/d55/d7f/f83 [2595874,71737] 0 2024-04-06T03:40:50.333 INFO:tasks.workunit.client.1.smithi136.stdout:4/934: chown d1d/d31/d3e/d56/d7b 233 1 2024-04-06T03:40:50.333 INFO:tasks.workunit.client.1.smithi136.stdout:4/935: chown d1d/d2f/d6c/da5/dc1/cd9 1015 1 2024-04-06T03:40:50.334 INFO:tasks.workunit.client.0.smithi094.stdout:9/994: rename d2/dc/d21/d23/d47/c67 to d2/dc/d21/d26/d8a/d12e/c14f 0 2024-04-06T03:40:50.335 INFO:tasks.workunit.client.1.smithi136.stdout:6/979: dwrite d2/d7/d13/d89/ffd [0,4194304] 0 2024-04-06T03:40:50.337 INFO:tasks.workunit.client.1.smithi136.stdout:4/936: rename d1d/d31/d3e/d55/d89/dea/l112 to d1d/d2f/da6/l143 0 2024-04-06T03:40:50.338 INFO:tasks.workunit.client.0.smithi094.stdout:9/995: mkdir d2/dc/d21/d26/d8a/d150 0 2024-04-06T03:40:50.338 INFO:tasks.workunit.client.0.smithi094.stdout:9/996: fsync d2/dc/d10/d2a/fcc 0 2024-04-06T03:40:50.339 INFO:tasks.workunit.client.0.smithi094.stdout:3/970: dwrite d9/d20/d2d/da9/fe0 [4194304,4194304] 0 2024-04-06T03:40:50.341 INFO:tasks.workunit.client.0.smithi094.stdout:9/997: rename d2/dc/d21/d26/d8a/d150 to d2/dc/d21/d23/db0/d151 0 2024-04-06T03:40:50.342 INFO:tasks.workunit.client.1.smithi136.stdout:6/980: creat d2/d7/d26/d30/d6e/d58/d131/d134/f156 x:0 0 0 2024-04-06T03:40:50.342 INFO:tasks.workunit.client.0.smithi094.stdout:9/998: write d2/dc/d80/fa1 [430954,97779] 0 2024-04-06T03:40:50.344 INFO:tasks.workunit.client.1.smithi136.stdout:4/937: creat d1d/d2f/da6/de1/f144 x:0 0 0 2024-04-06T03:40:50.346 INFO:tasks.workunit.client.1.smithi136.stdout:6/981: creat d2/d7/d13/d79/f157 x:0 0 0 2024-04-06T03:40:50.348 INFO:tasks.workunit.client.1.smithi136.stdout:4/938: link d1d/dd0/l10e d1d/d2f/d6c/l145 0 2024-04-06T03:40:50.349 INFO:tasks.workunit.client.0.smithi094.stdout:9/999: rename d2/dc/d46/d111/d33/d64/dc6/f117 to d2/dc/d21/d23/db0/d151/f152 0 2024-04-06T03:40:50.357 INFO:tasks.workunit.client.1.smithi136.stdout:4/939: read d1d/f20 [227181,15152] 0 2024-04-06T03:40:50.357 INFO:tasks.workunit.client.1.smithi136.stdout:4/940: write d1d/d31/d3e/d55/f11e [854839,42306] 0 2024-04-06T03:40:50.359 INFO:tasks.workunit.client.1.smithi136.stdout:4/941: truncate d1d/d31/d3e/d56/f65 377250 0 2024-04-06T03:40:50.359 INFO:tasks.workunit.client.1.smithi136.stdout:4/942: stat d1d/d31/d3e/d55/dba 0 2024-04-06T03:40:50.385 INFO:tasks.workunit.client.1.smithi136.stdout:4/943: rmdir d1d/d70 39 2024-04-06T03:40:50.386 INFO:tasks.workunit.client.1.smithi136.stdout:4/944: creat d1d/d2f/d6c/da5/d11b/f146 x:0 0 0 2024-04-06T03:40:50.386 INFO:tasks.workunit.client.1.smithi136.stdout:4/945: chown d1d/d2f/f80 0 1 2024-04-06T03:40:50.389 INFO:tasks.workunit.client.1.smithi136.stdout:4/946: dread d1d/fbb [0,4194304] 0 2024-04-06T03:40:50.429 INFO:tasks.workunit.client.1.smithi136.stdout:6/982: sync 2024-04-06T03:40:50.430 INFO:tasks.workunit.client.1.smithi136.stdout:6/983: rmdir d2/d7/d26/dcd 39 2024-04-06T03:40:50.444 INFO:tasks.workunit.client.1.smithi136.stdout:6/984: dwrite d2/d7/d26/d30/d6e/d3d/f46 [0,4194304] 0 2024-04-06T03:40:50.444 INFO:tasks.workunit.client.1.smithi136.stdout:6/985: chown d2/d7/d26/d30/d6e/d58/d131/c145 854885384 1 2024-04-06T03:40:50.444 INFO:tasks.workunit.client.1.smithi136.stdout:6/986: fsync d2/d93/dab/ff0 0 2024-04-06T03:40:50.445 INFO:tasks.workunit.client.1.smithi136.stdout:6/987: link d2/d104/lc0 d2/d7/d26/d30/l158 0 2024-04-06T03:40:50.446 INFO:tasks.workunit.client.1.smithi136.stdout:6/988: read d2/d7/d26/dcd/dad/fbb [39219,24730] 0 2024-04-06T03:40:50.448 INFO:tasks.workunit.client.1.smithi136.stdout:6/989: rename d2/d7/d26/d30/c6f to d2/d7/d26/d30/d6e/d3d/c159 0 2024-04-06T03:40:50.448 INFO:tasks.workunit.client.1.smithi136.stdout:6/990: stat d2/d7/d26/d30/dc4/f11a 0 2024-04-06T03:40:50.448 INFO:tasks.workunit.client.1.smithi136.stdout:6/991: dread - d2/d7/d26/d68/fde zero size 2024-04-06T03:40:50.449 INFO:tasks.workunit.client.1.smithi136.stdout:4/947: sync 2024-04-06T03:40:50.456 INFO:tasks.workunit.client.1.smithi136.stdout:6/992: write d2/d93/dab/f138 [3201064,100525] 0 2024-04-06T03:40:50.456 INFO:tasks.workunit.client.1.smithi136.stdout:6/993: write d2/d7/d26/d30/d6e/d58/f10f [528590,101328] 0 2024-04-06T03:40:50.457 INFO:tasks.workunit.client.1.smithi136.stdout:6/994: rename d2/d7/d26/f119 to d2/d25/f15a 0 2024-04-06T03:40:50.461 INFO:tasks.workunit.client.1.smithi136.stdout:4/948: dwrite d1d/d70/f64 [0,4194304] 0 2024-04-06T03:40:50.470 INFO:tasks.workunit.client.1.smithi136.stdout:6/995: dwrite d2/d7/d26/faa [0,4194304] 0 2024-04-06T03:40:50.470 INFO:tasks.workunit.client.1.smithi136.stdout:6/996: truncate d2/d7/d26/d68/fed 79019 0 2024-04-06T03:40:50.470 INFO:tasks.workunit.client.1.smithi136.stdout:6/997: readlink d2/d7/d26/d30/d6e/d3d/l3e 0 2024-04-06T03:40:50.470 INFO:tasks.workunit.client.1.smithi136.stdout:6/998: chown d2/d7/d26/d68/c96 22 1 2024-04-06T03:40:50.470 INFO:tasks.workunit.client.1.smithi136.stdout:6/999: fsync d2/d7/d13/f9a 0 2024-04-06T03:40:50.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:50 smithi136 ceph-mon[36436]: pgmap v25: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 220 MiB/s rd, 272 MiB/s wr, 398 op/s 2024-04-06T03:40:50.529 INFO:tasks.workunit.client.0.smithi094.stdout:8/951: sync 2024-04-06T03:40:50.529 INFO:tasks.workunit.client.0.smithi094.stdout:7/818: sync 2024-04-06T03:40:50.529 INFO:tasks.workunit.client.0.smithi094.stdout:8/952: readlink d18/d3c/d49/db5/l108 0 2024-04-06T03:40:50.529 INFO:tasks.workunit.client.0.smithi094.stdout:7/819: readlink d2/d6/dc/d12/d9a/da0/dd4/l9d 0 2024-04-06T03:40:50.529 INFO:tasks.workunit.client.0.smithi094.stdout:6/905: sync 2024-04-06T03:40:50.530 INFO:tasks.workunit.client.0.smithi094.stdout:8/953: creat d18/d3c/d49/d91/f144 x:0 0 0 2024-04-06T03:40:50.530 INFO:tasks.workunit.client.0.smithi094.stdout:8/954: write d18/d2c/d37/d115/f137 [365505,18172] 0 2024-04-06T03:40:50.530 INFO:tasks.workunit.client.0.smithi094.stdout:6/906: mkdir dc/d3a/df4/df6/d7b/dc6/d13c 0 2024-04-06T03:40:50.532 INFO:tasks.workunit.client.0.smithi094.stdout:8/955: creat d18/d3c/dc4/f145 x:0 0 0 2024-04-06T03:40:50.532 INFO:tasks.workunit.client.0.smithi094.stdout:8/956: write d18/d3c/d49/db5/f132 [862588,128286] 0 2024-04-06T03:40:50.538 INFO:tasks.workunit.client.0.smithi094.stdout:7/820: read d2/d6/dc/dd/df/f1a [504032,23845] 0 2024-04-06T03:40:50.539 INFO:tasks.workunit.client.0.smithi094.stdout:8/957: write d18/f116 [2545579,91159] 0 2024-04-06T03:40:50.539 INFO:tasks.workunit.client.0.smithi094.stdout:8/958: truncate d18/d3c/d76/f9e 276286 0 2024-04-06T03:40:50.539 INFO:tasks.workunit.client.0.smithi094.stdout:8/959: chown d18/d3c 190 1 2024-04-06T03:40:50.539 INFO:tasks.workunit.client.0.smithi094.stdout:7/821: truncate d2/d6/dc/d12/d29/d3f/f5e 40039 0 2024-04-06T03:40:50.550 INFO:tasks.workunit.client.0.smithi094.stdout:6/907: dwrite dc/dd3/d72/d88/fad [0,4194304] 0 2024-04-06T03:40:50.550 INFO:tasks.workunit.client.0.smithi094.stdout:7/822: dread d2/d6/dc/dd/d90/fa6 [0,4194304] 0 2024-04-06T03:40:50.551 INFO:tasks.workunit.client.1.smithi136.stdout:4/949: sync 2024-04-06T03:40:50.552 INFO:tasks.workunit.client.1.smithi136.stdout:4/950: stat d1d/d2f/da6/de1/d10a/f141 0 2024-04-06T03:40:50.552 INFO:tasks.workunit.client.1.smithi136.stdout:4/951: fdatasync d1d/d2f/d6c/da5/dc1/d109/f10b 0 2024-04-06T03:40:50.552 INFO:tasks.workunit.client.1.smithi136.stdout:4/952: read d1d/d2f/d6c/f7e [382803,22177] 0 2024-04-06T03:40:50.552 INFO:tasks.workunit.client.1.smithi136.stdout:4/953: read - d1d/d31/d78/f131 zero size 2024-04-06T03:40:50.552 INFO:tasks.workunit.client.1.smithi136.stdout:4/954: chown d1d/d2f 19698797 1 2024-04-06T03:40:50.552 INFO:tasks.workunit.client.1.smithi136.stdout:4/955: read d1d/f20 [875515,34944] 0 2024-04-06T03:40:50.552 INFO:tasks.workunit.client.0.smithi094.stdout:6/908: creat dc/d11/f13d x:0 0 0 2024-04-06T03:40:50.553 INFO:tasks.workunit.client.0.smithi094.stdout:6/909: write dc/d3a/d4a/d54/d71/fcb [1089425,98287] 0 2024-04-06T03:40:50.554 INFO:tasks.workunit.client.1.smithi136.stdout:4/956: truncate d1d/d2f/fab 421293 0 2024-04-06T03:40:50.559 INFO:tasks.workunit.client.0.smithi094.stdout:6/910: rename dc/dd3/d72 to dc/dd3/d4e/d13e 0 2024-04-06T03:40:50.573 INFO:tasks.workunit.client.0.smithi094.stdout:7/823: dwrite d2/d6/dc/d12/d9a/da0/dd4/f71 [0,4194304] 0 2024-04-06T03:40:50.574 INFO:tasks.workunit.client.1.smithi136.stdout:4/957: dwrite d1d/fbb [0,4194304] 0 2024-04-06T03:40:50.575 INFO:tasks.workunit.client.0.smithi094.stdout:7/824: rmdir d2/d6/dc/dd/df/d36/d79 39 2024-04-06T03:40:50.575 INFO:tasks.workunit.client.0.smithi094.stdout:7/825: stat d2/d6/dc/d12/f78 0 2024-04-06T03:40:50.575 INFO:tasks.workunit.client.0.smithi094.stdout:7/826: stat d2/d6/dc/d12/d9a/da0/dd4/db1/ld9 0 2024-04-06T03:40:50.578 INFO:tasks.workunit.client.0.smithi094.stdout:7/827: getdents d2/d6/dc/d12/d29/d3f/dce 0 2024-04-06T03:40:50.580 INFO:tasks.workunit.client.1.smithi136.stdout:4/958: write d1d/d31/d3e/d55/d7f/fb6 [2032185,37646] 0 2024-04-06T03:40:50.581 INFO:tasks.workunit.client.0.smithi094.stdout:7/828: creat d2/d6/dc/d12/d29/d3f/dce/f10e x:0 0 0 2024-04-06T03:40:50.581 INFO:tasks.workunit.client.0.smithi094.stdout:7/829: write d2/d6/dc/dd/f31 [716568,101903] 0 2024-04-06T03:40:50.581 INFO:tasks.workunit.client.0.smithi094.stdout:7/830: chown d2/d6/dac/ddd/c109 1142050177 1 2024-04-06T03:40:50.581 INFO:tasks.workunit.client.0.smithi094.stdout:7/831: chown d2/d6/dc/dd/df/d95/dc9/c98 92319448 1 2024-04-06T03:40:50.582 INFO:tasks.workunit.client.1.smithi136.stdout:4/959: getdents d1d/d31 0 2024-04-06T03:40:50.584 INFO:tasks.workunit.client.0.smithi094.stdout:7/832: symlink d2/d6/l10f 0 2024-04-06T03:40:50.584 INFO:tasks.workunit.client.0.smithi094.stdout:7/833: truncate d2/d6/f8f 291500 0 2024-04-06T03:40:50.584 INFO:tasks.workunit.client.0.smithi094.stdout:7/834: dread - d2/d6/dc/dd/df/d95/dc9/fef zero size 2024-04-06T03:40:50.584 INFO:tasks.workunit.client.0.smithi094.stdout:7/835: write d2/d6/dc/d12/d29/d3f/fee [105203,43288] 0 2024-04-06T03:40:50.585 INFO:tasks.workunit.client.0.smithi094.stdout:7/836: symlink d2/d6/dc/dd/df/d36/d79/l110 0 2024-04-06T03:40:50.586 INFO:tasks.workunit.client.0.smithi094.stdout:7/837: rename d2/d6/dc/d12/d29/d100/f10b to d2/d6/dc/dd/f111 0 2024-04-06T03:40:50.595 INFO:tasks.workunit.client.1.smithi136.stdout:4/960: dwrite d1d/d31/d78/fbd [0,4194304] 0 2024-04-06T03:40:50.598 INFO:tasks.workunit.client.0.smithi094.stdout:7/838: write d2/d6/dc/d12/d9a/da0/dd4/f9c [3270914,27039] 0 2024-04-06T03:40:50.598 INFO:tasks.workunit.client.1.smithi136.stdout:4/961: rmdir d1d/d2f/da6/dbf 39 2024-04-06T03:40:50.600 INFO:tasks.workunit.client.1.smithi136.stdout:4/962: rename d1d/dd0 to d1d/d2f/d6c/da5/d11a/d147 0 2024-04-06T03:40:50.608 INFO:tasks.workunit.client.1.smithi136.stdout:4/963: dread f1c [0,4194304] 0 2024-04-06T03:40:50.609 INFO:tasks.workunit.client.1.smithi136.stdout:4/964: symlink d1d/d31/d8e/l148 0 2024-04-06T03:40:50.610 INFO:tasks.workunit.client.1.smithi136.stdout:4/965: symlink d1d/d31/d3e/d56/d7b/l149 0 2024-04-06T03:40:50.612 INFO:tasks.workunit.client.1.smithi136.stdout:4/966: rename d1d/d70/l6b to d1d/d2f/d6c/da5/d11a/d147/l14a 0 2024-04-06T03:40:50.616 INFO:tasks.workunit.client.0.smithi094.stdout:7/839: dwrite d2/d6/dc/d12/d9a/da0/dd4/f9c [0,4194304] 0 2024-04-06T03:40:50.617 INFO:tasks.workunit.client.0.smithi094.stdout:7/840: mknod d2/d6/dc/d12/d29/d3f/c112 0 2024-04-06T03:40:50.618 INFO:tasks.workunit.client.0.smithi094.stdout:7/841: mkdir d2/d6/dc/d12/d3a/dcc/dde/db7/d113 0 2024-04-06T03:40:50.619 INFO:tasks.workunit.client.0.smithi094.stdout:7/842: rename d2/d6/dc/d12/d9a/da0/dd4/db1/ld9 to d2/d6/dc/dd/df/l114 0 2024-04-06T03:40:50.627 INFO:tasks.workunit.client.0.smithi094.stdout:7/843: getdents d2/d6/dc/d12/d3a/dcc/dde/db7 0 2024-04-06T03:40:50.630 INFO:tasks.workunit.client.0.smithi094.stdout:7/844: creat d2/d6/dc/dd/df/d95/dc9/f115 x:0 0 0 2024-04-06T03:40:50.631 INFO:tasks.workunit.client.0.smithi094.stdout:7/845: creat d2/d6/dc/d12/d3a/f116 x:0 0 0 2024-04-06T03:40:50.631 INFO:tasks.workunit.client.0.smithi094.stdout:7/846: symlink d2/d6/dc/dd/df/d36/d79/l117 0 2024-04-06T03:40:50.632 INFO:tasks.workunit.client.0.smithi094.stdout:7/847: dread - d2/d6/dc/dd/df/d36/ffd zero size 2024-04-06T03:40:50.632 INFO:tasks.workunit.client.0.smithi094.stdout:7/848: mkdir d2/d6/dc/dd/df/d95/d118 0 2024-04-06T03:40:50.633 INFO:tasks.workunit.client.0.smithi094.stdout:7/849: mknod d2/d6/dc/dd/d90/c119 0 2024-04-06T03:40:50.633 INFO:tasks.workunit.client.0.smithi094.stdout:7/850: readlink d2/d6/dc/d12/lfc 0 2024-04-06T03:40:50.633 INFO:tasks.workunit.client.0.smithi094.stdout:7/851: truncate d2/d6/dc/dd/df/d95/f107 668930 0 2024-04-06T03:40:50.634 INFO:tasks.workunit.client.1.smithi136.stdout:4/967: dwrite d1d/d31/f3f [0,4194304] 0 2024-04-06T03:40:50.634 INFO:tasks.workunit.client.0.smithi094.stdout:7/852: symlink d2/d6/dc/d12/d3a/l11a 0 2024-04-06T03:40:50.635 INFO:tasks.workunit.client.1.smithi136.stdout:4/968: rmdir d1d/d2f/d6c/da5/df0 39 2024-04-06T03:40:50.636 INFO:tasks.workunit.client.1.smithi136.stdout:4/969: creat d1d/d2f/d6c/da5/d11a/d147/f14b x:0 0 0 2024-04-06T03:40:50.638 INFO:tasks.workunit.client.1.smithi136.stdout:4/970: link d1d/f9d d1d/d2f/da6/de1/d10a/d125/f14c 0 2024-04-06T03:40:50.640 INFO:tasks.workunit.client.1.smithi136.stdout:4/971: unlink d1d/d2f/da6/fcf 0 2024-04-06T03:40:50.641 INFO:tasks.workunit.client.1.smithi136.stdout:4/972: mknod d1d/d31/d3e/dd4/c14d 0 2024-04-06T03:40:50.646 INFO:tasks.workunit.client.0.smithi094.stdout:7/853: dwrite d2/d6/dc/d12/d29/f105 [0,4194304] 0 2024-04-06T03:40:50.646 INFO:tasks.workunit.client.0.smithi094.stdout:7/854: write d2/d6/dc/d12/d29/d3f/fee [273369,113724] 0 2024-04-06T03:40:50.655 INFO:tasks.workunit.client.0.smithi094.stdout:7/855: read d2/d6/dc/dd/df/d95/dc9/f54 [553579,78561] 0 2024-04-06T03:40:50.656 INFO:tasks.workunit.client.0.smithi094.stdout:7/856: getdents d2/d6/dc/dd/df/d36 0 2024-04-06T03:40:50.657 INFO:tasks.workunit.client.0.smithi094.stdout:7/857: creat d2/d6/dc/dd/df/d95/dc9/d57/f11b x:0 0 0 2024-04-06T03:40:50.657 INFO:tasks.workunit.client.1.smithi136.stdout:4/973: dwrite d1d/d31/d78/dcb/f111 [0,4194304] 0 2024-04-06T03:40:50.657 INFO:tasks.workunit.client.1.smithi136.stdout:4/974: chown d1d/d2f 456837 1 2024-04-06T03:40:50.658 INFO:tasks.workunit.client.0.smithi094.stdout:7/858: symlink d2/d6/d93/l11c 0 2024-04-06T03:40:50.658 INFO:tasks.workunit.client.0.smithi094.stdout:7/859: fsync d2/d6/dc/dd/df/d95/dc9/fbc 0 2024-04-06T03:40:50.659 INFO:tasks.workunit.client.0.smithi094.stdout:7/860: rmdir d2/d6/dc/d12/d9a/da0/dd4/db1 39 2024-04-06T03:40:50.660 INFO:tasks.workunit.client.1.smithi136.stdout:4/975: mkdir d1d/d2f/da6/de1/d14e 0 2024-04-06T03:40:50.660 INFO:tasks.workunit.client.1.smithi136.stdout:4/976: write d1d/d2f/da6/de1/f144 [1040385,38910] 0 2024-04-06T03:40:50.661 INFO:tasks.workunit.client.0.smithi094.stdout:7/861: mkdir d2/d6/dc/d12/d9a/da0/d11d 0 2024-04-06T03:40:50.661 INFO:tasks.workunit.client.0.smithi094.stdout:7/862: dread - d2/d6/dc/dd/df/d95/dc9/fb0 zero size 2024-04-06T03:40:50.661 INFO:tasks.workunit.client.0.smithi094.stdout:7/863: getdents d2/d6/dc/dd/df/d36 0 2024-04-06T03:40:50.662 INFO:tasks.workunit.client.1.smithi136.stdout:4/977: rmdir d1d/d31/d3e 39 2024-04-06T03:40:50.664 INFO:tasks.workunit.client.0.smithi094.stdout:7/864: rename d2/d6/dc/dd/df/d95/dc9/f3d to d2/d6/dc/d12/d3a/dcc/dde/f11e 0 2024-04-06T03:40:50.664 INFO:tasks.workunit.client.0.smithi094.stdout:7/865: stat d2/d6/dc/d12/d3a/ffa 0 2024-04-06T03:40:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:50 smithi094 ceph-mon[29713]: pgmap v25: 65 pgs: 65 active+clean; 3.8 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 220 MiB/s rd, 272 MiB/s wr, 398 op/s 2024-04-06T03:40:50.672 INFO:tasks.workunit.client.0.smithi094.stdout:7/866: dread d2/d6/dc/d12/d9a/da0/dd4/f71 [0,4194304] 0 2024-04-06T03:40:50.673 INFO:tasks.workunit.client.0.smithi094.stdout:7/867: truncate d2/d6/dc/d12/d29/d3f/f5e 155257 0 2024-04-06T03:40:50.677 INFO:tasks.workunit.client.0.smithi094.stdout:5/877: sync 2024-04-06T03:40:50.677 INFO:tasks.workunit.client.0.smithi094.stdout:1/980: sync 2024-04-06T03:40:50.677 INFO:tasks.workunit.client.0.smithi094.stdout:2/880: sync 2024-04-06T03:40:50.677 INFO:tasks.workunit.client.0.smithi094.stdout:0/931: sync 2024-04-06T03:40:50.677 INFO:tasks.workunit.client.0.smithi094.stdout:0/932: chown d1/d9/dd/d18/d5d/fe0 21197 1 2024-04-06T03:40:50.677 INFO:tasks.workunit.client.0.smithi094.stdout:0/933: read - d1/d9/dd/d18/d5d/d60/dbc/d105/f134 zero size 2024-04-06T03:40:50.677 INFO:tasks.workunit.client.0.smithi094.stdout:0/934: stat d1/d9/dd/d18/d1e/d86/d96/dc0/c11d 0 2024-04-06T03:40:50.677 INFO:tasks.workunit.client.0.smithi094.stdout:1/981: creat d0/d1/d27/da5/d40/d50/f159 x:0 0 0 2024-04-06T03:40:50.677 INFO:tasks.workunit.client.0.smithi094.stdout:0/935: read d1/d9/dd/d18/d32/d42/f4a [302484,62790] 0 2024-04-06T03:40:50.678 INFO:tasks.workunit.client.0.smithi094.stdout:5/878: write db/d19/d21/d11e/d28/d85/dd5/fe3 [105953,79398] 0 2024-04-06T03:40:50.678 INFO:tasks.workunit.client.0.smithi094.stdout:5/879: chown db/d19/d21/d11e/d2e/fb3 87 1 2024-04-06T03:40:50.679 INFO:tasks.workunit.client.0.smithi094.stdout:3/971: sync 2024-04-06T03:40:50.680 INFO:tasks.workunit.client.0.smithi094.stdout:0/936: mknod d1/d9/c138 0 2024-04-06T03:40:50.680 INFO:tasks.workunit.client.0.smithi094.stdout:0/937: readlink d1/d9/dd/d18/d32/d42/lf7 0 2024-04-06T03:40:50.680 INFO:tasks.workunit.client.0.smithi094.stdout:7/868: write d2/d6/dc/d12/d9a/da0/dd4/db1/fb5 [408246,64472] 0 2024-04-06T03:40:50.680 INFO:tasks.workunit.client.0.smithi094.stdout:6/911: sync 2024-04-06T03:40:50.680 INFO:tasks.workunit.client.0.smithi094.stdout:8/960: sync 2024-04-06T03:40:50.681 INFO:tasks.workunit.client.0.smithi094.stdout:7/869: write d2/d6/dc/dd/f2f [1778628,114237] 0 2024-04-06T03:40:50.681 INFO:tasks.workunit.client.0.smithi094.stdout:7/870: stat d2/d6/dc/dd/l1b 0 2024-04-06T03:40:50.681 INFO:tasks.workunit.client.0.smithi094.stdout:7/871: chown d2/d6/dc/d12/d61/l6a 0 1 2024-04-06T03:40:50.681 INFO:tasks.workunit.client.0.smithi094.stdout:7/872: write d2/d6/fa [287651,44788] 0 2024-04-06T03:40:50.681 INFO:tasks.workunit.client.0.smithi094.stdout:5/880: creat db/d19/d2c/d6d/de6/dfd/d10d/d130/f132 x:0 0 0 2024-04-06T03:40:50.683 INFO:tasks.workunit.client.0.smithi094.stdout:3/972: truncate d9/d1e/d26/d6c/f125 602313 0 2024-04-06T03:40:50.683 INFO:tasks.workunit.client.0.smithi094.stdout:3/973: fsync d9/d10/dbb/de1/f139 0 2024-04-06T03:40:50.684 INFO:tasks.workunit.client.0.smithi094.stdout:5/881: mknod db/d19/dc1/dd4/d77/c133 0 2024-04-06T03:40:50.686 INFO:tasks.workunit.client.0.smithi094.stdout:3/974: rename d9/d20/d2d/d128/dba/f38 to d9/d1d/f15c 0 2024-04-06T03:40:50.688 INFO:tasks.workunit.client.0.smithi094.stdout:0/938: write d1/d9/f20 [3802204,77516] 0 2024-04-06T03:40:50.688 INFO:tasks.workunit.client.0.smithi094.stdout:0/939: truncate d1/d9/f90 912321 0 2024-04-06T03:40:50.688 INFO:tasks.workunit.client.0.smithi094.stdout:8/961: getdents d18/d3c/d49/d91/d94/df3 0 2024-04-06T03:40:50.690 INFO:tasks.workunit.client.0.smithi094.stdout:5/882: creat db/d19/f134 x:0 0 0 2024-04-06T03:40:50.690 INFO:tasks.workunit.client.0.smithi094.stdout:5/883: readlink db/d19/d30/l39 0 2024-04-06T03:40:50.690 INFO:tasks.workunit.client.0.smithi094.stdout:8/962: write d18/d3c/dc4/db9/fc0 [309158,75331] 0 2024-04-06T03:40:50.693 INFO:tasks.workunit.client.0.smithi094.stdout:3/975: mknod d9/d20/d2d/d128/dba/c15d 0 2024-04-06T03:40:50.696 INFO:tasks.workunit.client.0.smithi094.stdout:7/873: dread d2/d6/dc/d12/d3a/dcc/dde/f11e [0,4194304] 0 2024-04-06T03:40:50.696 INFO:tasks.workunit.client.0.smithi094.stdout:3/976: mknod d9/de5/c15e 0 2024-04-06T03:40:50.701 INFO:tasks.workunit.client.0.smithi094.stdout:5/884: dread db/d19/d2c/f73 [4194304,4194304] 0 2024-04-06T03:40:50.701 INFO:tasks.workunit.client.0.smithi094.stdout:2/881: dwrite d0/d8/d1d/f38 [0,4194304] 0 2024-04-06T03:40:50.702 INFO:tasks.workunit.client.0.smithi094.stdout:6/912: dwrite dc/d3a/d7f/dd4/f115 [0,4194304] 0 2024-04-06T03:40:50.702 INFO:tasks.workunit.client.0.smithi094.stdout:6/913: write dc/dd3/d4e/d13e/d88/da3/faa [984100,126634] 0 2024-04-06T03:40:50.703 INFO:tasks.workunit.client.0.smithi094.stdout:5/885: creat db/d19/d21/f135 x:0 0 0 2024-04-06T03:40:50.704 INFO:tasks.workunit.client.0.smithi094.stdout:2/882: symlink d0/d8/d27/d2b/d8e/l12a 0 2024-04-06T03:40:50.704 INFO:tasks.workunit.client.0.smithi094.stdout:2/883: fsync d0/dd/d19/d31/d67/d98/ddc/ff8 0 2024-04-06T03:40:50.704 INFO:tasks.workunit.client.0.smithi094.stdout:2/884: write d0/dd/d19/d31/f8c [9166021,36740] 0 2024-04-06T03:40:50.704 INFO:tasks.workunit.client.0.smithi094.stdout:2/885: fsync d0/dd/d19/d31/f7d 0 2024-04-06T03:40:50.704 INFO:tasks.workunit.client.0.smithi094.stdout:2/886: readlink d0/d8/da0/lc2 0 2024-04-06T03:40:50.704 INFO:tasks.workunit.client.0.smithi094.stdout:2/887: write d0/d8/d4f/dab/dc3/fe8 [98473,43019] 0 2024-04-06T03:40:50.705 INFO:tasks.workunit.client.1.smithi136.stdout:4/978: sync 2024-04-06T03:40:50.708 INFO:tasks.workunit.client.0.smithi094.stdout:7/874: write d2/d6/f24 [3888767,97352] 0 2024-04-06T03:40:50.709 INFO:tasks.workunit.client.0.smithi094.stdout:0/940: dwrite d1/d9/dd/d18/d5d/d8d/f123 [0,4194304] 0 2024-04-06T03:40:50.711 INFO:tasks.workunit.client.0.smithi094.stdout:5/886: rmdir db/d19/dc1 39 2024-04-06T03:40:50.712 INFO:tasks.workunit.client.0.smithi094.stdout:8/963: dwrite d18/d3c/d49/d91/d94/daa/fe9 [0,4194304] 0 2024-04-06T03:40:50.712 INFO:tasks.workunit.client.0.smithi094.stdout:8/964: read - d18/d2c/d37/d60/da3/db8/dd8/ff4 zero size 2024-04-06T03:40:50.712 INFO:tasks.workunit.client.0.smithi094.stdout:8/965: readlink d18/d34/d52/l57 0 2024-04-06T03:40:50.712 INFO:tasks.workunit.client.0.smithi094.stdout:0/941: dread d1/d9/dd/d18/d1e/d86/fdd [0,4194304] 0 2024-04-06T03:40:50.713 INFO:tasks.workunit.client.0.smithi094.stdout:2/888: link d0/d8/d27/d2b/c32 d0/dd/d1b/d72/c12b 0 2024-04-06T03:40:50.715 INFO:tasks.workunit.client.1.smithi136.stdout:4/979: write d1d/d70/f138 [3004845,36585] 0 2024-04-06T03:40:50.716 INFO:tasks.workunit.client.1.smithi136.stdout:4/980: chown d1d/d2f/d6c/f98 236371307 1 2024-04-06T03:40:50.716 INFO:tasks.workunit.client.0.smithi094.stdout:7/875: mkdir d2/d6/dc/dd/df/d36/d79/d11f 0 2024-04-06T03:40:50.716 INFO:tasks.workunit.client.0.smithi094.stdout:7/876: fsync d2/d6/dc/dd/df/d95/dc9/d57/f82 0 2024-04-06T03:40:50.720 INFO:tasks.workunit.client.0.smithi094.stdout:5/887: dread db/d19/dc1/dd4/d77/f6a [0,4194304] 0 2024-04-06T03:40:50.723 INFO:tasks.workunit.client.0.smithi094.stdout:0/942: dread d1/d9/dd/d18/d1a/d44/f80 [0,4194304] 0 2024-04-06T03:40:50.723 INFO:tasks.workunit.client.0.smithi094.stdout:0/943: creat d1/d9/dd/d18/d1e/d86/d96/dc0/f139 x:0 0 0 2024-04-06T03:40:50.723 INFO:tasks.workunit.client.0.smithi094.stdout:5/888: dread db/d19/d30/f44 [0,4194304] 0 2024-04-06T03:40:50.723 INFO:tasks.workunit.client.0.smithi094.stdout:6/914: dwrite dc/f95 [4194304,4194304] 0 2024-04-06T03:40:50.724 INFO:tasks.workunit.client.0.smithi094.stdout:8/966: rename d18/d34/d52/l7f to d18/d3c/d49/d91/d94/l146 0 2024-04-06T03:40:50.725 INFO:tasks.workunit.client.0.smithi094.stdout:8/967: readlink d18/d3c/dc4/dbc/lc5 0 2024-04-06T03:40:50.725 INFO:tasks.workunit.client.0.smithi094.stdout:2/889: getdents d0/dd/d1b/d72/df3 0 2024-04-06T03:40:50.726 INFO:tasks.workunit.client.1.smithi136.stdout:4/981: write d1d/d31/f41 [3873340,75091] 0 2024-04-06T03:40:50.726 INFO:tasks.workunit.client.1.smithi136.stdout:4/982: chown d1d/d70/l54 863427747 1 2024-04-06T03:40:50.726 INFO:tasks.workunit.client.1.smithi136.stdout:4/983: write d1d/d2f/d6c/f98 [4559831,24481] 0 2024-04-06T03:40:50.728 INFO:tasks.workunit.client.0.smithi094.stdout:7/877: link d2/d6/dc/dd/df/d95/dc9/f59 d2/d6/dc/dd/df/d36/f120 0 2024-04-06T03:40:50.728 INFO:tasks.workunit.client.0.smithi094.stdout:7/878: stat d2/d6/dc/d12/d9a/dbd/ldf 0 2024-04-06T03:40:50.730 INFO:tasks.workunit.client.0.smithi094.stdout:0/944: getdents d1/d9/dd/d18/dad 0 2024-04-06T03:40:50.731 INFO:tasks.workunit.client.0.smithi094.stdout:0/945: fsync d1/d9/dd/d18/d5d/d60/fe6 0 2024-04-06T03:40:50.733 INFO:tasks.workunit.client.0.smithi094.stdout:5/889: unlink db/d80/f97 0 2024-04-06T03:40:50.734 INFO:tasks.workunit.client.1.smithi136.stdout:4/984: creat d1d/d2f/d6c/da5/dc1/d109/d12b/f14f x:0 0 0 2024-04-06T03:40:50.734 INFO:tasks.workunit.client.0.smithi094.stdout:8/968: mknod d18/d5c/c147 0 2024-04-06T03:40:50.734 INFO:tasks.workunit.client.0.smithi094.stdout:8/969: truncate d18/d34/d52/fec 572863 0 2024-04-06T03:40:50.735 INFO:tasks.workunit.client.0.smithi094.stdout:2/890: creat d0/d18/f12c x:0 0 0 2024-04-06T03:40:50.735 INFO:tasks.workunit.client.0.smithi094.stdout:2/891: write d0/d18/ff2 [167805,37716] 0 2024-04-06T03:40:50.736 INFO:tasks.workunit.client.1.smithi136.stdout:4/985: symlink d1d/d2f/da6/de1/d10a/l150 0 2024-04-06T03:40:50.736 INFO:tasks.workunit.client.1.smithi136.stdout:4/986: chown d1d/d2f/d6c/da5/d11b/f146 0 1 2024-04-06T03:40:50.737 INFO:tasks.workunit.client.1.smithi136.stdout:4/987: getdents d1d/d2f/d6c/da5/d11a 0 2024-04-06T03:40:50.737 INFO:tasks.workunit.client.1.smithi136.stdout:4/988: creat d1d/d31/d78/dcb/f151 x:0 0 0 2024-04-06T03:40:50.739 INFO:tasks.workunit.client.0.smithi094.stdout:7/879: symlink d2/d6/dc/d12/d29/l121 0 2024-04-06T03:40:50.740 INFO:tasks.workunit.client.0.smithi094.stdout:0/946: dread d1/d9/dd/d18/d32/f5f [0,4194304] 0 2024-04-06T03:40:50.741 INFO:tasks.workunit.client.0.smithi094.stdout:5/890: mknod db/d110/c136 0 2024-04-06T03:40:50.742 INFO:tasks.workunit.client.1.smithi136.stdout:4/989: rmdir d1d/d31/d3e 39 2024-04-06T03:40:50.742 INFO:tasks.workunit.client.1.smithi136.stdout:4/990: chown d1d/d31/d3e/d55/d89/db3/fbe 3 1 2024-04-06T03:40:50.743 INFO:tasks.workunit.client.0.smithi094.stdout:2/892: mknod d0/dd/d3d/c12d 0 2024-04-06T03:40:50.743 INFO:tasks.workunit.client.0.smithi094.stdout:2/893: truncate d0/dd/d1b/de2/f122 824458 0 2024-04-06T03:40:50.744 INFO:tasks.workunit.client.0.smithi094.stdout:0/947: truncate d1/d9/dd/d18/d1e/d22/f6a 858145 0 2024-04-06T03:40:50.748 INFO:tasks.workunit.client.0.smithi094.stdout:5/891: dread db/d19/d30/fb8 [0,4194304] 0 2024-04-06T03:40:50.748 INFO:tasks.workunit.client.0.smithi094.stdout:5/892: creat db/d19/d2c/d6d/de6/dfd/d10d/f137 x:0 0 0 2024-04-06T03:40:50.751 INFO:tasks.workunit.client.0.smithi094.stdout:8/970: dwrite d18/f74 [4194304,4194304] 0 2024-04-06T03:40:50.751 INFO:tasks.workunit.client.0.smithi094.stdout:8/971: creat d18/d2c/d37/d60/da3/db8/dd8/f148 x:0 0 0 2024-04-06T03:40:50.751 INFO:tasks.workunit.client.0.smithi094.stdout:8/972: read - d18/d3c/d76/f121 zero size 2024-04-06T03:40:50.753 INFO:tasks.workunit.client.0.smithi094.stdout:8/973: dread d18/d3c/d76/f9e [0,4194304] 0 2024-04-06T03:40:50.756 INFO:tasks.workunit.client.1.smithi136.stdout:4/991: dwrite d1d/d31/d3e/d55/d89/db3/fc6 [0,4194304] 0 2024-04-06T03:40:50.756 INFO:tasks.workunit.client.1.smithi136.stdout:4/992: chown d1d/d31/d3e/d55/d89/db3/fc5 1577 1 2024-04-06T03:40:50.756 INFO:tasks.workunit.client.0.smithi094.stdout:0/948: dread d1/f83 [0,4194304] 0 2024-04-06T03:40:50.757 INFO:tasks.workunit.client.0.smithi094.stdout:0/949: chown d1/d9/dd/d18/d1a/lcf 1 1 2024-04-06T03:40:50.757 INFO:tasks.workunit.client.0.smithi094.stdout:8/974: creat d18/d3c/d49/d91/d94/daa/ddc/f149 x:0 0 0 2024-04-06T03:40:50.757 INFO:tasks.workunit.client.0.smithi094.stdout:8/975: write d18/d3c/dc4/db9/fdf [432739,16316] 0 2024-04-06T03:40:50.757 INFO:tasks.workunit.client.0.smithi094.stdout:8/976: truncate d18/d2c/d37/d60/da3/deb/ffa 617208 0 2024-04-06T03:40:50.762 INFO:tasks.workunit.client.0.smithi094.stdout:2/894: read d0/d8/d27/f45 [3259864,35484] 0 2024-04-06T03:40:50.762 INFO:tasks.workunit.client.0.smithi094.stdout:2/895: rename d0/d8/d102 to d0/d8/d1d/d12e 0 2024-04-06T03:40:50.773 INFO:tasks.workunit.client.0.smithi094.stdout:5/893: dwrite db/d19/d21/d11e/d28/dec/f10c [0,4194304] 0 2024-04-06T03:40:50.777 INFO:tasks.workunit.client.0.smithi094.stdout:5/894: getdents db/d19/d2c/d6d 0 2024-04-06T03:40:50.778 INFO:tasks.workunit.client.0.smithi094.stdout:5/895: rmdir db/d19/d21/d11e/d28 39 2024-04-06T03:40:50.780 INFO:tasks.workunit.client.0.smithi094.stdout:5/896: mkdir db/d19/d2c/d6d/de6/dfd/d10d/d130/d138 0 2024-04-06T03:40:50.780 INFO:tasks.workunit.client.0.smithi094.stdout:5/897: readlink db/d19/l22 0 2024-04-06T03:40:50.781 INFO:tasks.workunit.client.0.smithi094.stdout:5/898: symlink db/d19/d2c/d6d/de6/dfd/d10d/d130/l139 0 2024-04-06T03:40:50.782 INFO:tasks.workunit.client.0.smithi094.stdout:5/899: symlink db/d19/dc1/l13a 0 2024-04-06T03:40:50.784 INFO:tasks.workunit.client.0.smithi094.stdout:5/900: mknod db/d19/d21/d11e/d28/d85/dd5/c13b 0 2024-04-06T03:40:50.785 INFO:tasks.workunit.client.0.smithi094.stdout:5/901: rename db/d19/l22 to db/d19/l13c 0 2024-04-06T03:40:50.787 INFO:tasks.workunit.client.0.smithi094.stdout:5/902: mknod db/d110/c13d 0 2024-04-06T03:40:50.787 INFO:tasks.workunit.client.0.smithi094.stdout:0/950: dwrite d1/d9/dd/d18/d32/d42/fd3 [0,4194304] 0 2024-04-06T03:40:50.788 INFO:tasks.workunit.client.1.smithi136.stdout:4/993: dwrite d1d/d31/d78/feb [0,4194304] 0 2024-04-06T03:40:50.788 INFO:tasks.workunit.client.0.smithi094.stdout:2/896: dwrite d0/d18/fd6 [0,4194304] 0 2024-04-06T03:40:50.791 INFO:tasks.workunit.client.0.smithi094.stdout:0/951: link d1/d9/dd/d18/d5d/d60/fc1 d1/d9/dd/d18/d1e/d86/d96/daa/f13a 0 2024-04-06T03:40:50.791 INFO:tasks.workunit.client.0.smithi094.stdout:0/952: write d1/d9/dd/d18/d1e/d86/fa3 [784110,122885] 0 2024-04-06T03:40:50.791 INFO:tasks.workunit.client.0.smithi094.stdout:0/953: chown d1/d9/dd/d18/d78/deb/f117 1 1 2024-04-06T03:40:50.791 INFO:tasks.workunit.client.0.smithi094.stdout:0/954: write d1/d9/dd/d18/d5d/d60/dfd/f5c [6063202,98619] 0 2024-04-06T03:40:50.791 INFO:tasks.workunit.client.0.smithi094.stdout:0/955: fdatasync d1/d9/dd/fa2 0 2024-04-06T03:40:50.794 INFO:tasks.workunit.client.0.smithi094.stdout:0/956: mkdir d1/d9/dd/d18/d1e/def/d67/d13b 0 2024-04-06T03:40:50.794 INFO:tasks.workunit.client.0.smithi094.stdout:0/957: truncate d1/d9/dd/d18/d1e/def/dab/f2e 1653304 0 2024-04-06T03:40:50.795 INFO:tasks.workunit.client.0.smithi094.stdout:5/903: dread db/d19/d30/d83/fa1 [0,4194304] 0 2024-04-06T03:40:50.796 INFO:tasks.workunit.client.0.smithi094.stdout:2/897: link d0/d8/d27/d2b/f5d d0/dd/d1b/f12f 0 2024-04-06T03:40:50.796 INFO:tasks.workunit.client.0.smithi094.stdout:2/898: write d0/dd/d19/d31/d67/f6c [544476,127684] 0 2024-04-06T03:40:50.797 INFO:tasks.workunit.client.0.smithi094.stdout:0/958: creat d1/d9/dd/d18/d1e/d22/d3e/f13c x:0 0 0 2024-04-06T03:40:50.797 INFO:tasks.workunit.client.0.smithi094.stdout:2/899: write d0/dd/da4/ff5 [592254,34823] 0 2024-04-06T03:40:50.798 INFO:tasks.workunit.client.1.smithi136.stdout:4/994: dread d1d/d31/d8e/fc9 [0,4194304] 0 2024-04-06T03:40:50.799 INFO:tasks.workunit.client.0.smithi094.stdout:0/959: symlink d1/d9/dd/d18/d32/l13d 0 2024-04-06T03:40:50.799 INFO:tasks.workunit.client.0.smithi094.stdout:0/960: chown d1/d9/dd/d18/d1a/d44/cd9 93033096 1 2024-04-06T03:40:50.799 INFO:tasks.workunit.client.0.smithi094.stdout:2/900: dread d0/dd/d1b/fd3 [0,4194304] 0 2024-04-06T03:40:50.800 INFO:tasks.workunit.client.1.smithi136.stdout:4/995: creat d1d/d31/d3e/d56/f152 x:0 0 0 2024-04-06T03:40:50.800 INFO:tasks.workunit.client.1.smithi136.stdout:4/996: truncate d1d/d2f/da6/d11f/ffd 551375 0 2024-04-06T03:40:50.801 INFO:tasks.workunit.client.0.smithi094.stdout:5/904: dread db/f11 [4194304,4194304] 0 2024-04-06T03:40:50.801 INFO:tasks.workunit.client.0.smithi094.stdout:5/905: dread - db/d7c/ffe zero size 2024-04-06T03:40:50.802 INFO:tasks.workunit.client.0.smithi094.stdout:0/961: creat d1/d9/dd/d18/d78/d109/f13e x:0 0 0 2024-04-06T03:40:50.803 INFO:tasks.workunit.client.0.smithi094.stdout:2/901: link d0/dd/d19/ca2 d0/dd/d1b/de2/d113/c130 0 2024-04-06T03:40:50.804 INFO:tasks.workunit.client.1.smithi136.stdout:4/997: mkdir d1d/d31/d78/d153 0 2024-04-06T03:40:50.804 INFO:tasks.workunit.client.0.smithi094.stdout:2/902: dread d0/dd/d1b/de2/f122 [0,4194304] 0 2024-04-06T03:40:50.804 INFO:tasks.workunit.client.0.smithi094.stdout:5/906: mknod db/d19/d21/d11e/d28/d85/dd5/dfa/c13e 0 2024-04-06T03:40:50.806 INFO:tasks.workunit.client.0.smithi094.stdout:0/962: unlink d1/d9/dd/d18/d1e/d22/ldc 0 2024-04-06T03:40:50.807 INFO:tasks.workunit.client.1.smithi136.stdout:4/998: mknod d1d/d2f/d6c/c154 0 2024-04-06T03:40:50.808 INFO:tasks.workunit.client.1.smithi136.stdout:4/999: truncate d1d/d2f/da6/de1/f144 243219 0 2024-04-06T03:40:50.809 INFO:tasks.workunit.client.0.smithi094.stdout:0/963: mknod d1/d9/dd/d18/d32/d42/c13f 0 2024-04-06T03:40:50.810 INFO:tasks.workunit.client.0.smithi094.stdout:2/903: truncate d0/dd/d1b/f12f 1140110 0 2024-04-06T03:40:50.811 INFO:tasks.workunit.client.0.smithi094.stdout:5/907: truncate db/d19/dc1/dd4/f55 2772569 0 2024-04-06T03:40:50.813 INFO:tasks.workunit.client.1.smithi136.stderr:+ rm -rf -- ./tmp.FkHTAOF2Rv 2024-04-06T03:40:50.817 INFO:tasks.workunit.client.0.smithi094.stdout:2/904: mkdir d0/dd/d1b/de2/de7/d131 0 2024-04-06T03:40:50.819 INFO:tasks.workunit.client.0.smithi094.stdout:2/905: creat d0/d8/d1d/d77/f132 x:0 0 0 2024-04-06T03:40:50.821 INFO:tasks.workunit.client.0.smithi094.stdout:2/906: mknod d0/dd/d19/d31/dcc/c133 0 2024-04-06T03:40:50.821 INFO:tasks.workunit.client.0.smithi094.stdout:2/907: write d0/dd/d1b/fd3 [878849,45706] 0 2024-04-06T03:40:50.821 INFO:tasks.workunit.client.0.smithi094.stdout:2/908: creat d0/dd/f134 x:0 0 0 2024-04-06T03:40:50.821 INFO:tasks.workunit.client.0.smithi094.stdout:2/909: chown d0/dd/d1b 37702 1 2024-04-06T03:40:50.824 INFO:tasks.workunit.client.0.smithi094.stdout:2/910: creat d0/dd/d19/d31/d67/d98/f135 x:0 0 0 2024-04-06T03:40:50.826 INFO:tasks.workunit.client.0.smithi094.stdout:5/908: dwrite db/d19/d21/d11e/d28/d85/f9e [0,4194304] 0 2024-04-06T03:40:50.827 INFO:tasks.workunit.client.0.smithi094.stdout:2/911: truncate d0/d8/d1d/d77/f129 5218963 0 2024-04-06T03:40:50.842 INFO:tasks.workunit.client.0.smithi094.stdout:2/912: dread d0/f10a [0,4194304] 0 2024-04-06T03:40:50.847 INFO:tasks.workunit.client.0.smithi094.stdout:5/909: dwrite db/d19/d30/f8e [0,4194304] 0 2024-04-06T03:40:50.849 INFO:tasks.workunit.client.0.smithi094.stdout:0/964: dwrite d1/d9/dd/d18/d5d/d60/dfd/f52 [0,4194304] 0 2024-04-06T03:40:50.863 INFO:tasks.workunit.client.0.smithi094.stdout:2/913: rmdir d0/dd 39 2024-04-06T03:40:50.866 INFO:tasks.workunit.client.0.smithi094.stdout:5/910: dwrite db/d19/d21/d11e/f72 [0,4194304] 0 2024-04-06T03:40:50.871 INFO:tasks.workunit.client.0.smithi094.stdout:0/965: link d1/d9/dd/d18/d1e/def/d67/df6/c10d d1/d9/dd/d18/d5d/d60/dfd/c140 0 2024-04-06T03:40:50.881 INFO:tasks.workunit.client.0.smithi094.stdout:2/914: truncate d0/d8/d1d/f79 3937606 0 2024-04-06T03:40:50.891 INFO:tasks.workunit.client.0.smithi094.stdout:5/911: creat db/d19/d2c/f13f x:0 0 0 2024-04-06T03:40:50.892 INFO:tasks.workunit.client.0.smithi094.stdout:5/912: fsync db/f12 0 2024-04-06T03:40:50.892 INFO:tasks.workunit.client.0.smithi094.stdout:5/913: readlink db/d19/d2c/d6d/de7/l12e 0 2024-04-06T03:40:50.892 INFO:tasks.workunit.client.0.smithi094.stdout:0/966: truncate d1/d9/dd/d18/d5d/d60/dfd/dac/fe7 2463422 0 2024-04-06T03:40:50.895 INFO:tasks.workunit.client.0.smithi094.stdout:5/914: creat db/d19/d21/d12f/f140 x:0 0 0 2024-04-06T03:40:50.895 INFO:tasks.workunit.client.0.smithi094.stdout:5/915: fdatasync db/d19/d2c/f106 0 2024-04-06T03:40:50.895 INFO:tasks.workunit.client.0.smithi094.stdout:5/916: chown db/d19/d30/d83/fd3 123483 1 2024-04-06T03:40:50.895 INFO:tasks.workunit.client.0.smithi094.stdout:5/917: truncate db/d19/d2c/d6d/de6/dfd/d10d/f125 550964 0 2024-04-06T03:40:50.895 INFO:tasks.workunit.client.0.smithi094.stdout:0/967: symlink d1/d9/dd/d18/d1a/d44/dc2/l141 0 2024-04-06T03:40:50.902 INFO:tasks.workunit.client.0.smithi094.stdout:2/915: rename d0/d8/da0 to d0/d8/d4f/d136 0 2024-04-06T03:40:50.906 INFO:tasks.workunit.client.0.smithi094.stdout:5/918: write db/fa8 [2362131,87308] 0 2024-04-06T03:40:50.908 INFO:tasks.workunit.client.0.smithi094.stdout:5/919: getdents db/d19/d21/d11e/d28/dec 0 2024-04-06T03:40:50.909 INFO:tasks.workunit.client.0.smithi094.stdout:2/916: read d0/dd/d1b/f1c [3101908,7527] 0 2024-04-06T03:40:50.910 INFO:tasks.workunit.client.0.smithi094.stdout:5/920: mkdir db/d19/d21/d11e/d28/d85/dd5/dfa/d141 0 2024-04-06T03:40:50.914 INFO:tasks.workunit.client.0.smithi094.stdout:0/968: dwrite d1/d9/dd/d18/d5d/d60/dbc/dda/f101 [0,4194304] 0 2024-04-06T03:40:50.919 INFO:tasks.workunit.client.0.smithi094.stdout:5/921: dread db/d7c/d98/f9f [0,4194304] 0 2024-04-06T03:40:50.979 INFO:tasks.workunit.client.0.smithi094.stdout:1/982: sync 2024-04-06T03:40:50.981 INFO:tasks.workunit.client.0.smithi094.stdout:0/969: mknod d1/d9/dd/d18/d32/d42/c142 0 2024-04-06T03:40:50.981 INFO:tasks.workunit.client.0.smithi094.stdout:1/983: dread d0/d1/f73 [0,4194304] 0 2024-04-06T03:40:50.982 INFO:tasks.workunit.client.0.smithi094.stdout:1/984: truncate d0/d7/d1a/ff2 1577608 0 2024-04-06T03:40:50.982 INFO:tasks.workunit.client.0.smithi094.stdout:5/922: mkdir db/d19/d2c/d6d/de6/dfd/d10d/d130/d142 0 2024-04-06T03:40:50.983 INFO:tasks.workunit.client.0.smithi094.stdout:5/923: chown db/d19/d21/d11e/d2e/f10e 884310 1 2024-04-06T03:40:50.983 INFO:tasks.workunit.client.0.smithi094.stdout:5/924: read - db/d19/fca zero size 2024-04-06T03:40:50.983 INFO:tasks.workunit.client.0.smithi094.stdout:5/925: stat db/d19/f134 0 2024-04-06T03:40:50.983 INFO:tasks.workunit.client.0.smithi094.stdout:0/970: dread d1/d9/dd/d18/d1e/def/f81 [0,4194304] 0 2024-04-06T03:40:50.983 INFO:tasks.workunit.client.0.smithi094.stdout:0/971: readlink d1/d9/l132 0 2024-04-06T03:40:50.986 INFO:tasks.workunit.client.0.smithi094.stdout:1/985: rmdir d0/d1/d27/da5/d40/d50 39 2024-04-06T03:40:50.988 INFO:tasks.workunit.client.0.smithi094.stdout:1/986: truncate d0/d1/d27/da5/d40/db1/d2f/d3c/d48/d67/f12d 1752477 0 2024-04-06T03:40:50.992 INFO:tasks.workunit.client.0.smithi094.stdout:1/987: creat d0/d127/d131/f15a x:0 0 0 2024-04-06T03:40:50.994 INFO:tasks.workunit.client.0.smithi094.stdout:1/988: truncate d0/d1/d27/da5/d40/db1/d2f/d4b/f91 3656583 0 2024-04-06T03:40:50.994 INFO:tasks.workunit.client.0.smithi094.stdout:1/989: write d0/d1/d27/da5/d40/db1/d2f/d3c/f150 [276164,18330] 0 2024-04-06T03:40:50.996 INFO:tasks.workunit.client.0.smithi094.stdout:5/926: dread db/d19/d21/f62 [0,4194304] 0 2024-04-06T03:40:50.999 INFO:tasks.workunit.client.0.smithi094.stdout:1/990: truncate d0/f19 4656065 0 2024-04-06T03:40:51.000 INFO:tasks.workunit.client.0.smithi094.stdout:1/991: fdatasync d0/d7/d1a/ff2 0 2024-04-06T03:40:51.001 INFO:tasks.workunit.client.0.smithi094.stdout:0/972: dwrite d1/d9/dd/d18/d1e/def/d67/df6/f11f [0,4194304] 0 2024-04-06T03:40:51.001 INFO:tasks.workunit.client.0.smithi094.stdout:5/927: creat db/d19/dc1/dd4/d77/f143 x:0 0 0 2024-04-06T03:40:51.001 INFO:tasks.workunit.client.0.smithi094.stdout:5/928: write db/d19/dc1/dd4/d77/f143 [771350,103915] 0 2024-04-06T03:40:51.001 INFO:tasks.workunit.client.0.smithi094.stdout:5/929: readlink db/d19/d21/d11e/d28/dec/l103 0 2024-04-06T03:40:51.001 INFO:tasks.workunit.client.0.smithi094.stdout:5/930: chown db/d19/d2c/f31 16560 1 2024-04-06T03:40:51.002 INFO:tasks.workunit.client.0.smithi094.stdout:1/992: creat d0/d1/d27/da5/d40/db1/d2f/f15b x:0 0 0 2024-04-06T03:40:51.007 INFO:tasks.workunit.client.0.smithi094.stdout:5/931: write db/d19/d30/fb8 [2612649,525] 0 2024-04-06T03:40:51.007 INFO:tasks.workunit.client.0.smithi094.stdout:1/993: creat d0/f15c x:0 0 0 2024-04-06T03:40:51.008 INFO:tasks.workunit.client.0.smithi094.stdout:5/932: dread db/d19/d21/d11e/d28/f3f [0,4194304] 0 2024-04-06T03:40:51.008 INFO:tasks.workunit.client.0.smithi094.stdout:5/933: chown db/d80 486408 1 2024-04-06T03:40:51.008 INFO:tasks.workunit.client.0.smithi094.stdout:5/934: fdatasync db/d19/d21/d11e/d2e/fb3 0 2024-04-06T03:40:51.008 INFO:tasks.workunit.client.0.smithi094.stdout:5/935: chown db/d19/d30/d83/c105 23 1 2024-04-06T03:40:51.008 INFO:tasks.workunit.client.0.smithi094.stdout:5/936: chown db/f1e 1404 1 2024-04-06T03:40:51.009 INFO:tasks.workunit.client.0.smithi094.stdout:0/973: unlink d1/d9/dd/d18/d1e/def/dab/d4e/ddf/f99 0 2024-04-06T03:40:51.010 INFO:tasks.workunit.client.0.smithi094.stdout:1/994: mknod d0/d1/d27/da5/d40/db1/d2f/d3c/c15d 0 2024-04-06T03:40:51.011 INFO:tasks.workunit.client.0.smithi094.stdout:5/937: symlink db/d7c/l144 0 2024-04-06T03:40:51.012 INFO:tasks.workunit.client.0.smithi094.stdout:0/974: dread d1/d9/dd/d18/d1e/f71 [0,4194304] 0 2024-04-06T03:40:51.012 INFO:tasks.workunit.client.0.smithi094.stdout:1/995: mkdir d0/d1/d27/da5/d40/db1/d15e 0 2024-04-06T03:40:51.013 INFO:tasks.workunit.client.0.smithi094.stdout:5/938: symlink db/d19/d30/l145 0 2024-04-06T03:40:51.014 INFO:tasks.workunit.client.0.smithi094.stdout:0/975: mkdir d1/d9/dd/d18/d1e/def/dab/d4e/de4/d143 0 2024-04-06T03:40:51.015 INFO:tasks.workunit.client.0.smithi094.stdout:1/996: truncate d0/d1/d27/da5/d40/db1/d2f/d4b/fa1 3853040 0 2024-04-06T03:40:51.018 INFO:tasks.workunit.client.0.smithi094.stdout:5/939: rename db/fae to db/d19/d21/d12f/f146 0 2024-04-06T03:40:51.018 INFO:tasks.workunit.client.0.smithi094.stdout:5/940: truncate db/d19/d30/fb8 4711644 0 2024-04-06T03:40:51.018 INFO:tasks.workunit.client.0.smithi094.stdout:5/941: write db/d19/d21/d11e/f72 [5118561,14909] 0 2024-04-06T03:40:51.019 INFO:tasks.workunit.client.0.smithi094.stdout:0/976: mknod d1/d9/dd/d18/d1a/d44/dc2/c144 0 2024-04-06T03:40:51.021 INFO:tasks.workunit.client.0.smithi094.stdout:1/997: truncate d0/d1/d27/da5/f53 3028 0 2024-04-06T03:40:51.027 INFO:tasks.workunit.client.0.smithi094.stdout:5/942: creat db/d19/d21/d11e/d28/f147 x:0 0 0 2024-04-06T03:40:51.032 INFO:tasks.workunit.client.0.smithi094.stdout:0/977: unlink d1/d9/c62 0 2024-04-06T03:40:51.037 INFO:tasks.workunit.client.0.smithi094.stdout:1/998: mkdir d0/d7/d1a/d64/d15f 0 2024-04-06T03:40:51.048 INFO:tasks.workunit.client.0.smithi094.stdout:5/943: mkdir db/d19/d21/d11e/d148 0 2024-04-06T03:40:51.051 INFO:tasks.workunit.client.0.smithi094.stdout:0/978: creat d1/d9/dd/d18/d1e/def/dab/f145 x:0 0 0 2024-04-06T03:40:51.051 INFO:tasks.workunit.client.0.smithi094.stdout:0/979: creat d1/d9/dd/d18/d1e/def/f146 x:0 0 0 2024-04-06T03:40:51.051 INFO:tasks.workunit.client.0.smithi094.stdout:1/999: truncate d0/d7/d1a/da8/fbb 2773817 0 2024-04-06T03:40:51.053 INFO:tasks.workunit.client.0.smithi094.stdout:5/944: stat db/d19/d21/d11e/d28/d85/dd5/cf8 0 2024-04-06T03:40:51.057 INFO:tasks.workunit.client.0.smithi094.stdout:0/980: creat d1/d9/dd/d18/d1e/def/dab/d4e/de4/f147 x:0 0 0 2024-04-06T03:40:51.060 INFO:tasks.workunit.client.0.smithi094.stdout:0/981: mkdir d1/d9/dd/d18/d32/d42/d148 0 2024-04-06T03:40:51.068 INFO:tasks.workunit.client.0.smithi094.stdout:0/982: symlink d1/d9/dd/d18/d78/deb/l149 0 2024-04-06T03:40:51.081 INFO:tasks.workunit.client.0.smithi094.stdout:0/983: dwrite d1/d9/dd/d18/d1e/d22/f97 [4194304,4194304] 0 2024-04-06T03:40:51.085 INFO:tasks.workunit.client.0.smithi094.stdout:0/984: mkdir d1/d9/dd/d18/d1a/d44/d14a 0 2024-04-06T03:40:51.085 INFO:tasks.workunit.client.0.smithi094.stdout:0/985: fsync d1/d9/dd/d18/d1a/d44/dc2/f10c 0 2024-04-06T03:40:51.087 INFO:tasks.workunit.client.0.smithi094.stdout:0/986: chown d1/d9/dd/d18/d1e/fe2 151642577 1 2024-04-06T03:40:51.087 INFO:tasks.workunit.client.0.smithi094.stdout:0/987: stat d1/d9/dd/d18/d1a/d44/dc2/ffa 0 2024-04-06T03:40:51.095 INFO:tasks.workunit.client.0.smithi094.stdout:0/988: dread d1/d9/dd/d18/d1e/d86/d96/daa/fee [0,4194304] 0 2024-04-06T03:40:51.096 INFO:tasks.workunit.client.0.smithi094.stdout:0/989: getdents d1/d9/dd/d18/d1a 0 2024-04-06T03:40:51.096 INFO:tasks.workunit.client.0.smithi094.stdout:0/990: stat d1/d9/dd/d18/d1e/f71 0 2024-04-06T03:40:51.096 INFO:tasks.workunit.client.0.smithi094.stdout:3/977: sync 2024-04-06T03:40:51.096 INFO:tasks.workunit.client.0.smithi094.stdout:3/978: chown d9/d20/d2d/d128/dba/ca5 359003 1 2024-04-06T03:40:51.096 INFO:tasks.workunit.client.0.smithi094.stdout:3/979: stat d9/d10/d9b/db7/dc0/de2 0 2024-04-06T03:40:51.096 INFO:tasks.workunit.client.0.smithi094.stdout:3/980: creat d9/d20/d2d/d128/dba/d3a/daf/f15f x:0 0 0 2024-04-06T03:40:51.097 INFO:tasks.workunit.client.0.smithi094.stdout:0/991: mknod d1/d9/dd/d18/d78/deb/d112/c14b 0 2024-04-06T03:40:51.097 INFO:tasks.workunit.client.0.smithi094.stdout:0/992: dread - d1/d9/dd/d18/d1e/d22/d3e/f13c zero size 2024-04-06T03:40:51.097 INFO:tasks.workunit.client.0.smithi094.stdout:0/993: write d1/d9/fc [7912105,26887] 0 2024-04-06T03:40:51.099 INFO:tasks.workunit.client.0.smithi094.stdout:3/981: unlink d9/l79 0 2024-04-06T03:40:51.099 INFO:tasks.workunit.client.0.smithi094.stdout:3/982: fdatasync d9/d20/d2d/d128/dba/d3a/daf/f141 0 2024-04-06T03:40:51.099 INFO:tasks.workunit.client.0.smithi094.stdout:3/983: readlink d9/d20/d2d/d128/l140 0 2024-04-06T03:40:51.099 INFO:tasks.workunit.client.0.smithi094.stdout:8/977: sync 2024-04-06T03:40:51.099 INFO:tasks.workunit.client.0.smithi094.stdout:7/880: sync 2024-04-06T03:40:51.099 INFO:tasks.workunit.client.0.smithi094.stdout:6/915: sync 2024-04-06T03:40:51.100 INFO:tasks.workunit.client.0.smithi094.stdout:8/978: write d18/f2d [412726,96905] 0 2024-04-06T03:40:51.100 INFO:tasks.workunit.client.0.smithi094.stdout:0/994: creat d1/d9/dd/d18/d1e/f14c x:0 0 0 2024-04-06T03:40:51.101 INFO:tasks.workunit.client.0.smithi094.stdout:7/881: dread d2/d6/f8 [0,4194304] 0 2024-04-06T03:40:51.101 INFO:tasks.workunit.client.0.smithi094.stdout:7/882: write d2/d6/dc/d12/d29/f76 [1782632,71081] 0 2024-04-06T03:40:51.103 INFO:tasks.workunit.client.0.smithi094.stdout:3/984: creat d9/d20/d2d/d128/dba/d7c/da2/df2/f160 x:0 0 0 2024-04-06T03:40:51.105 INFO:tasks.workunit.client.0.smithi094.stdout:6/916: creat dc/d11/d97/f13f x:0 0 0 2024-04-06T03:40:51.105 INFO:tasks.workunit.client.0.smithi094.stdout:6/917: write dc/d3a/d4a/d54/fb5 [988975,99261] 0 2024-04-06T03:40:51.107 INFO:tasks.workunit.client.0.smithi094.stdout:0/995: unlink d1/d9/dd/d18/d78/l84 0 2024-04-06T03:40:51.109 INFO:tasks.workunit.client.0.smithi094.stdout:7/883: creat d2/d6/dc/d12/d3a/dcc/f122 x:0 0 0 2024-04-06T03:40:51.110 INFO:tasks.workunit.client.0.smithi094.stdout:8/979: write d18/d3c/d49/d91/d94/df3/d100/f10e [3009963,10291] 0 2024-04-06T03:40:51.110 INFO:tasks.workunit.client.0.smithi094.stdout:2/917: sync 2024-04-06T03:40:51.112 INFO:tasks.workunit.client.0.smithi094.stdout:5/945: dwrite db/d19/d21/d11e/d2e/f36 [0,4194304] 0 2024-04-06T03:40:51.113 INFO:tasks.workunit.client.0.smithi094.stdout:6/918: mknod dc/dd3/d4e/d13e/d88/dfb/c140 0 2024-04-06T03:40:51.115 INFO:tasks.workunit.client.0.smithi094.stdout:8/980: mkdir d18/d2c/d37/d60/d70/d14a 0 2024-04-06T03:40:51.115 INFO:tasks.workunit.client.0.smithi094.stdout:8/981: dread - d18/d3c/d49/d91/f144 zero size 2024-04-06T03:40:51.117 INFO:tasks.workunit.client.0.smithi094.stdout:2/918: mkdir d0/dd/d1b/de2/de7/d137 0 2024-04-06T03:40:51.122 INFO:tasks.workunit.client.0.smithi094.stdout:5/946: dread db/d19/d30/d83/f87 [0,4194304] 0 2024-04-06T03:40:51.122 INFO:tasks.workunit.client.0.smithi094.stdout:3/985: symlink d9/d10/d9b/db7/dc0/de2/l161 0 2024-04-06T03:40:51.122 INFO:tasks.workunit.client.0.smithi094.stdout:3/986: chown d9/d10/dbb/de1/f131 32164456 1 2024-04-06T03:40:51.123 INFO:tasks.workunit.client.0.smithi094.stdout:8/982: rmdir d18/d3c/d49/d80 39 2024-04-06T03:40:51.123 INFO:tasks.workunit.client.0.smithi094.stdout:8/983: dread - d18/d3c/dc4/f145 zero size 2024-04-06T03:40:51.123 INFO:tasks.workunit.client.0.smithi094.stdout:6/919: write dc/dd3/f1f [558391,23762] 0 2024-04-06T03:40:51.127 INFO:tasks.workunit.client.0.smithi094.stdout:5/947: mknod db/d19/c149 0 2024-04-06T03:40:51.129 INFO:tasks.workunit.client.0.smithi094.stdout:8/984: creat d18/d3c/d49/d91/d114/f14b x:0 0 0 2024-04-06T03:40:51.129 INFO:tasks.workunit.client.0.smithi094.stdout:8/985: chown d18/d34/lca 0 1 2024-04-06T03:40:51.130 INFO:tasks.workunit.client.0.smithi094.stdout:5/948: creat db/d19/dc1/dd4/dcf/f14a x:0 0 0 2024-04-06T03:40:51.132 INFO:tasks.workunit.client.0.smithi094.stdout:8/986: truncate d18/d3c/d76/f9e 125906 0 2024-04-06T03:40:51.133 INFO:tasks.workunit.client.0.smithi094.stdout:0/996: dwrite d1/d9/dd/d18/d5d/de9/f100 [0,4194304] 0 2024-04-06T03:40:51.135 INFO:tasks.workunit.client.0.smithi094.stdout:7/884: dwrite d2/d6/dc/d12/d29/d3f/fee [0,4194304] 0 2024-04-06T03:40:51.136 INFO:tasks.workunit.client.0.smithi094.stdout:7/885: truncate d2/d6/dc/d12/d3a/ffa 4605259 0 2024-04-06T03:40:51.136 INFO:tasks.workunit.client.0.smithi094.stdout:2/919: dread d0/d8/d1d/f79 [0,4194304] 0 2024-04-06T03:40:51.138 INFO:tasks.workunit.client.0.smithi094.stdout:0/997: mkdir d1/d9/dd/d18/d5d/d60/dbc/d105/d14d 0 2024-04-06T03:40:51.139 INFO:tasks.workunit.client.0.smithi094.stdout:0/998: write d1/d9/dd/d18/d5d/fe0 [955910,47471] 0 2024-04-06T03:40:51.144 INFO:tasks.workunit.client.0.smithi094.stdout:6/920: dread dc/d3a/d4a/d54/dde/f39 [0,4194304] 0 2024-04-06T03:40:51.145 INFO:tasks.workunit.client.0.smithi094.stdout:3/987: dwrite d9/d20/d2d/d128/dba/d3a/ffa [0,4194304] 0 2024-04-06T03:40:51.151 INFO:tasks.workunit.client.0.smithi094.stdout:3/988: unlink d9/d10/dbb/fff 0 2024-04-06T03:40:51.152 INFO:tasks.workunit.client.0.smithi094.stdout:5/949: dwrite db/d19/d30/d83/fe1 [0,4194304] 0 2024-04-06T03:40:51.152 INFO:tasks.workunit.client.0.smithi094.stdout:7/886: dread d2/d6/f30 [0,4194304] 0 2024-04-06T03:40:51.152 INFO:tasks.workunit.client.0.smithi094.stdout:7/887: read d2/d6/dc/dd/f2f [871290,104818] 0 2024-04-06T03:40:51.152 INFO:tasks.workunit.client.0.smithi094.stdout:7/888: stat d2/d6/dc/d12/ce4 0 2024-04-06T03:40:51.154 INFO:tasks.workunit.client.0.smithi094.stdout:6/921: mkdir dc/dd3/d4e/d13e/d88/d141 0 2024-04-06T03:40:51.154 INFO:tasks.workunit.client.0.smithi094.stdout:6/922: chown dc/d3a/l67 7430 1 2024-04-06T03:40:51.155 INFO:tasks.workunit.client.0.smithi094.stdout:6/923: read - dc/dd3/d4e/d13e/d88/f12c zero size 2024-04-06T03:40:51.155 INFO:tasks.workunit.client.0.smithi094.stdout:2/920: write d0/dd/d3d/f100 [552455,19127] 0 2024-04-06T03:40:51.159 INFO:tasks.workunit.client.0.smithi094.stdout:8/987: dwrite d18/f4e [0,4194304] 0 2024-04-06T03:40:51.159 INFO:tasks.workunit.client.0.smithi094.stdout:8/988: write f17 [4206819,79354] 0 2024-04-06T03:40:51.159 INFO:tasks.workunit.client.0.smithi094.stdout:8/989: fdatasync d18/d3c/fb2 0 2024-04-06T03:40:51.159 INFO:tasks.workunit.client.0.smithi094.stdout:8/990: read d18/d2c/dda/f118 [1524372,89251] 0 2024-04-06T03:40:51.160 INFO:tasks.workunit.client.0.smithi094.stdout:3/989: symlink d9/d20/d2d/d128/dba/d7c/da2/df2/l162 0 2024-04-06T03:40:51.167 INFO:tasks.workunit.client.0.smithi094.stdout:0/999: dwrite d1/d9/dd/d18/d1e/def/d67/df6/f11f [0,4194304] 0 2024-04-06T03:40:51.170 INFO:tasks.workunit.client.0.smithi094.stdout:5/950: mknod db/d7c/d8b/c14b 0 2024-04-06T03:40:51.172 INFO:tasks.workunit.client.0.smithi094.stdout:6/924: truncate dc/f95 8387400 0 2024-04-06T03:40:51.172 INFO:tasks.workunit.client.0.smithi094.stdout:6/925: dread - dc/d3a/d4a/d54/dde/d50/d61/f101 zero size 2024-04-06T03:40:51.172 INFO:tasks.workunit.client.0.smithi094.stdout:6/926: write dc/d3a/f89 [1279795,35588] 0 2024-04-06T03:40:51.174 INFO:tasks.workunit.client.0.smithi094.stdout:2/921: dwrite d0/dd/d19/f81 [4194304,4194304] 0 2024-04-06T03:40:51.174 INFO:tasks.workunit.client.0.smithi094.stdout:2/922: write d0/dd/f11d [502348,60608] 0 2024-04-06T03:40:51.174 INFO:tasks.workunit.client.0.smithi094.stdout:3/990: dwrite d9/d20/f3d [0,4194304] 0 2024-04-06T03:40:51.178 INFO:tasks.workunit.client.0.smithi094.stdout:8/991: rmdir d18/d5c 39 2024-04-06T03:40:51.185 INFO:tasks.workunit.client.0.smithi094.stdout:8/992: write d18/d30/f3e [4102831,17033] 0 2024-04-06T03:40:51.185 INFO:tasks.workunit.client.0.smithi094.stdout:8/993: write d18/d3c/d49/d91/fcf [452299,80354] 0 2024-04-06T03:40:51.190 INFO:tasks.workunit.client.0.smithi094.stdout:5/951: link db/d19/d2c/la6 db/d19/d21/l14c 0 2024-04-06T03:40:51.190 INFO:tasks.workunit.client.0.smithi094.stdout:5/952: getdents db/d19/d2c/d6d/de6/dfd/d10d/d130/d142 0 2024-04-06T03:40:51.190 INFO:tasks.workunit.client.0.smithi094.stdout:5/953: fdatasync db/d19/d21/d11e/f37 0 2024-04-06T03:40:51.193 INFO:tasks.workunit.client.0.smithi094.stdout:6/927: creat dc/d3a/df4/df6/d7b/f142 x:0 0 0 2024-04-06T03:40:51.193 INFO:tasks.workunit.client.0.smithi094.stdout:6/928: chown dc/d3a/df4/df6/cf3 10656893 1 2024-04-06T03:40:51.193 INFO:tasks.workunit.client.0.smithi094.stdout:6/929: write dc/d3a/df4/df6/df2/f108 [2024069,6066] 0 2024-04-06T03:40:51.198 INFO:tasks.workunit.client.0.smithi094.stdout:8/994: link d18/d2c/d37/d60/d70/c98 d18/d5c/de3/c14c 0 2024-04-06T03:40:51.200 INFO:tasks.workunit.client.0.smithi094.stdout:5/954: mkdir db/d19/d21/d11e/d28/d14d 0 2024-04-06T03:40:51.201 INFO:tasks.workunit.client.0.smithi094.stdout:6/930: rmdir dc/dd3/d4e/d13e/d88/dfb/d112 39 2024-04-06T03:40:51.201 INFO:tasks.workunit.client.0.smithi094.stdout:6/931: fsync dc/d3a/d4a/d54/d71/f11e 0 2024-04-06T03:40:51.203 INFO:tasks.workunit.client.0.smithi094.stdout:2/923: rename d0/d8/d4f/ld1 to d0/d8/d4f/dab/l138 0 2024-04-06T03:40:51.204 INFO:tasks.workunit.client.0.smithi094.stdout:3/991: mkdir d9/d20/da6/d163 0 2024-04-06T03:40:51.204 INFO:tasks.workunit.client.0.smithi094.stdout:3/992: read - d9/d20/d2d/d128/dba/d3a/daf/f141 zero size 2024-04-06T03:40:51.206 INFO:tasks.workunit.client.0.smithi094.stdout:5/955: dread db/d19/dc1/dd4/f55 [0,4194304] 0 2024-04-06T03:40:51.207 INFO:tasks.workunit.client.0.smithi094.stdout:8/995: mknod d18/d3c/d49/d9f/c14d 0 2024-04-06T03:40:51.208 INFO:tasks.workunit.client.0.smithi094.stdout:2/924: mknod d0/d8/d27/c139 0 2024-04-06T03:40:51.211 INFO:tasks.workunit.client.0.smithi094.stdout:6/932: dread dc/dd3/f31 [0,4194304] 0 2024-04-06T03:40:51.211 INFO:tasks.workunit.client.0.smithi094.stdout:5/956: symlink db/d19/d2c/d6d/de6/dfd/d10d/d130/l14e 0 2024-04-06T03:40:51.214 INFO:tasks.workunit.client.0.smithi094.stdout:8/996: dread d18/d3c/d49/d91/d94/daa/fe9 [0,4194304] 0 2024-04-06T03:40:51.214 INFO:tasks.workunit.client.0.smithi094.stdout:8/997: fdatasync d18/d3c/d76/f9e 0 2024-04-06T03:40:51.214 INFO:tasks.workunit.client.0.smithi094.stdout:8/998: chown d18/d5c/de3 2912 1 2024-04-06T03:40:51.216 INFO:tasks.workunit.client.0.smithi094.stdout:8/999: dread d18/d2c/d37/d115/f137 [0,4194304] 0 2024-04-06T03:40:51.217 INFO:tasks.workunit.client.0.smithi094.stdout:6/933: creat dc/d3a/d4a/d54/dde/d50/f143 x:0 0 0 2024-04-06T03:40:51.217 INFO:tasks.workunit.client.0.smithi094.stdout:6/934: fsync dc/d3a/d4a/d54/dde/d50/fbf 0 2024-04-06T03:40:51.218 INFO:tasks.workunit.client.0.smithi094.stdout:5/957: mknod db/d19/d21/dce/c14f 0 2024-04-06T03:40:51.218 INFO:tasks.workunit.client.0.smithi094.stdout:5/958: chown db/fa8 6 1 2024-04-06T03:40:51.219 INFO:tasks.workunit.client.0.smithi094.stdout:5/959: truncate db/d19/dc1/dd4/d77/fbb 4722841 0 2024-04-06T03:40:51.221 INFO:tasks.workunit.client.0.smithi094.stdout:2/925: dread d0/dd/d3d/d49/f53 [0,4194304] 0 2024-04-06T03:40:51.221 INFO:tasks.workunit.client.0.smithi094.stdout:2/926: creat d0/d8/d1d/d77/f13a x:0 0 0 2024-04-06T03:40:51.221 INFO:tasks.workunit.client.0.smithi094.stdout:2/927: fsync d0/dd/d19/fb8 0 2024-04-06T03:40:51.221 INFO:tasks.workunit.client.0.smithi094.stdout:2/928: stat d0/d8/f33 0 2024-04-06T03:40:51.226 INFO:tasks.workunit.client.0.smithi094.stdout:6/935: dread dc/f10f [0,4194304] 0 2024-04-06T03:40:51.227 INFO:tasks.workunit.client.0.smithi094.stdout:6/936: fsync dc/d3a/d4a/f56 0 2024-04-06T03:40:51.228 INFO:tasks.workunit.client.0.smithi094.stdout:6/937: mknod dc/dd3/d4e/d13e/d88/da3/db9/c144 0 2024-04-06T03:40:51.231 INFO:tasks.workunit.client.0.smithi094.stdout:6/938: dread dc/d3a/d4a/d54/d71/fcb [0,4194304] 0 2024-04-06T03:40:51.231 INFO:tasks.workunit.client.0.smithi094.stdout:6/939: dread - dc/dd3/fb4 zero size 2024-04-06T03:40:51.240 INFO:tasks.workunit.client.0.smithi094.stdout:5/960: dwrite db/d80/deb/f120 [0,4194304] 0 2024-04-06T03:40:51.246 INFO:tasks.workunit.client.0.smithi094.stdout:6/940: dwrite dc/f10f [0,4194304] 0 2024-04-06T03:40:51.247 INFO:tasks.workunit.client.0.smithi094.stdout:6/941: fsync dc/dd3/d24/fbe 0 2024-04-06T03:40:51.248 INFO:tasks.workunit.client.0.smithi094.stdout:6/942: mkdir dc/d3a/df4/df6/d7b/d145 0 2024-04-06T03:40:51.250 INFO:tasks.workunit.client.0.smithi094.stdout:6/943: mkdir dc/dd8/d146 0 2024-04-06T03:40:51.252 INFO:tasks.workunit.client.0.smithi094.stdout:6/944: unlink dc/c13 0 2024-04-06T03:40:51.252 INFO:tasks.workunit.client.0.smithi094.stdout:5/961: dwrite db/d19/d30/fb8 [0,4194304] 0 2024-04-06T03:40:51.253 INFO:tasks.workunit.client.0.smithi094.stdout:6/945: creat dc/dd3/d4e/d13e/d88/da3/f147 x:0 0 0 2024-04-06T03:40:51.253 INFO:tasks.workunit.client.0.smithi094.stdout:6/946: creat dc/d3a/d4a/d54/dde/d50/f148 x:0 0 0 2024-04-06T03:40:51.256 INFO:tasks.workunit.client.0.smithi094.stdout:5/962: creat db/d19/d21/d11e/d28/d85/dd5/f150 x:0 0 0 2024-04-06T03:40:51.256 INFO:tasks.workunit.client.0.smithi094.stdout:5/963: truncate db/d19/dc1/dd4/fe9 230903 0 2024-04-06T03:40:51.257 INFO:tasks.workunit.client.0.smithi094.stdout:5/964: symlink db/d19/d21/d11e/d28/dec/l151 0 2024-04-06T03:40:51.259 INFO:tasks.workunit.client.0.smithi094.stdout:5/965: link db/f76 db/d19/f152 0 2024-04-06T03:40:51.263 INFO:tasks.workunit.client.0.smithi094.stdout:6/947: read dc/dd3/d24/fe8 [1440482,129923] 0 2024-04-06T03:40:51.263 INFO:tasks.workunit.client.0.smithi094.stdout:5/966: dread db/d19/d30/d83/fa1 [0,4194304] 0 2024-04-06T03:40:51.287 INFO:tasks.workunit.client.0.smithi094.stdout:6/948: dwrite dc/dd3/d4e/d13e/d87/f8a [0,4194304] 0 2024-04-06T03:40:51.287 INFO:tasks.workunit.client.0.smithi094.stdout:6/949: write dc/d3a/d4a/d54/dde/d50/d61/f114 [481284,94732] 0 2024-04-06T03:40:51.287 INFO:tasks.workunit.client.0.smithi094.stdout:6/950: dread - dc/d3a/df4/df6/d7b/dc6/feb zero size 2024-04-06T03:40:51.287 INFO:tasks.workunit.client.0.smithi094.stdout:5/967: dwrite db/d19/d2c/d6d/fcb [0,4194304] 0 2024-04-06T03:40:51.287 INFO:tasks.workunit.client.0.smithi094.stdout:5/968: chown db/d7c/d8b/c99 416283548 1 2024-04-06T03:40:51.289 INFO:tasks.workunit.client.0.smithi094.stdout:5/969: link db/d19/d21/d11e/d28/dec/l151 db/d19/d21/d11e/l153 0 2024-04-06T03:40:51.291 INFO:tasks.workunit.client.0.smithi094.stdout:6/951: rename dc/d3a/df4/df6/df2/lff to dc/l149 0 2024-04-06T03:40:51.293 INFO:tasks.workunit.client.0.smithi094.stdout:5/970: creat db/d19/d21/d11e/d28/d14d/f154 x:0 0 0 2024-04-06T03:40:51.293 INFO:tasks.workunit.client.0.smithi094.stdout:5/971: creat db/d19/d2c/f155 x:0 0 0 2024-04-06T03:40:51.308 INFO:tasks.workunit.client.0.smithi094.stdout:6/952: dwrite dc/dd3/d24/fbe [0,4194304] 0 2024-04-06T03:40:51.308 INFO:tasks.workunit.client.0.smithi094.stdout:6/953: dread - dc/d3a/d4a/f92 zero size 2024-04-06T03:40:51.309 INFO:tasks.workunit.client.0.smithi094.stdout:6/954: write dc/d3a/d4a/d54/d71/fcb [1834071,104114] 0 2024-04-06T03:40:51.314 INFO:tasks.workunit.client.0.smithi094.stdout:5/972: dwrite db/d19/d2c/d6d/f115 [0,4194304] 0 2024-04-06T03:40:51.331 INFO:tasks.workunit.client.0.smithi094.stdout:5/973: dwrite db/d19/dc1/dd4/d77/fb1 [0,4194304] 0 2024-04-06T03:40:51.412 INFO:tasks.workunit.client.0.smithi094.stdout:6/955: getdents dc/dd3/d4e/d13e/d88/da3/db9 0 2024-04-06T03:40:51.433 INFO:tasks.workunit.client.0.smithi094.stdout:5/974: creat db/d19/dc1/dd4/f156 x:0 0 0 2024-04-06T03:40:51.434 INFO:tasks.workunit.client.0.smithi094.stdout:6/956: mknod dc/d3a/d7f/dd4/c14a 0 2024-04-06T03:40:51.437 INFO:tasks.workunit.client.0.smithi094.stdout:5/975: rename db/d19/d21/d11e/d28/dec/c10a to db/d19/d21/d11e/d28/d85/dd5/dfa/d141/c157 0 2024-04-06T03:40:51.439 INFO:tasks.workunit.client.0.smithi094.stdout:6/957: dread dc/dd3/f31 [0,4194304] 0 2024-04-06T03:40:51.439 INFO:tasks.workunit.client.0.smithi094.stdout:6/958: fsync dc/dd3/d24/de4/fe6 0 2024-04-06T03:40:51.439 INFO:tasks.workunit.client.0.smithi094.stdout:5/976: rename db/d19/d30/d83/fa1 to db/d19/d21/d11e/d28/d14d/f158 0 2024-04-06T03:40:51.441 INFO:tasks.workunit.client.0.smithi094.stdout:6/959: rename dc/dd3/d24/fe8 to dc/dab/d138/f14b 0 2024-04-06T03:40:51.443 INFO:tasks.workunit.client.0.smithi094.stdout:6/960: mknod dc/d3a/d120/c14c 0 2024-04-06T03:40:51.446 INFO:tasks.workunit.client.0.smithi094.stdout:6/961: unlink dc/l1a 0 2024-04-06T03:40:51.446 INFO:tasks.workunit.client.0.smithi094.stdout:6/962: rmdir dc/d3a/d4a/d54/d71/d103 39 2024-04-06T03:40:51.447 INFO:tasks.workunit.client.0.smithi094.stdout:6/963: creat dc/dd8/d146/f14d x:0 0 0 2024-04-06T03:40:51.449 INFO:tasks.workunit.client.0.smithi094.stdout:6/964: creat dc/dd3/d24/f14e x:0 0 0 2024-04-06T03:40:51.449 INFO:tasks.workunit.client.0.smithi094.stdout:6/965: dread - dc/dd3/d4e/ffd zero size 2024-04-06T03:40:51.449 INFO:tasks.workunit.client.0.smithi094.stdout:6/966: fsync dc/d3a/d4a/d54/fc7 0 2024-04-06T03:40:51.449 INFO:tasks.workunit.client.0.smithi094.stdout:5/977: dread db/d19/d2c/d6d/fcb [0,4194304] 0 2024-04-06T03:40:51.449 INFO:tasks.workunit.client.0.smithi094.stdout:5/978: write db/d19/d21/fda [247975,7580] 0 2024-04-06T03:40:51.450 INFO:tasks.workunit.client.0.smithi094.stdout:6/967: creat dc/d3a/d4a/d54/dde/d50/de0/f14f x:0 0 0 2024-04-06T03:40:51.450 INFO:tasks.workunit.client.0.smithi094.stdout:6/968: stat dc/d3a/d7f/dd9 0 2024-04-06T03:40:51.451 INFO:tasks.workunit.client.0.smithi094.stdout:5/979: mkdir db/d19/d21/d12f/d159 0 2024-04-06T03:40:51.453 INFO:tasks.workunit.client.0.smithi094.stdout:6/969: write dc/d3a/d4a/d54/fc7 [325118,112457] 0 2024-04-06T03:40:51.476 INFO:tasks.workunit.client.0.smithi094.stdout:6/970: dwrite dc/d3a/d4a/d54/dde/d50/d61/ff8 [0,4194304] 0 2024-04-06T03:40:51.477 INFO:tasks.workunit.client.0.smithi094.stdout:5/980: dwrite db/d19/d21/d11e/d28/d14d/f154 [0,4194304] 0 2024-04-06T03:40:51.486 INFO:tasks.workunit.client.0.smithi094.stdout:5/981: creat db/d19/d30/f15a x:0 0 0 2024-04-06T03:40:51.488 INFO:tasks.workunit.client.0.smithi094.stdout:6/971: dwrite dc/d3a/d7f/fc0 [0,4194304] 0 2024-04-06T03:40:51.491 INFO:tasks.workunit.client.0.smithi094.stdout:5/982: mkdir db/d19/d30/d15b 0 2024-04-06T03:40:51.493 INFO:tasks.workunit.client.0.smithi094.stdout:6/972: fsync dc/d3a/d4a/d54/dde/d50/d61/f6c 0 2024-04-06T03:40:51.497 INFO:tasks.workunit.client.0.smithi094.stdout:6/973: unlink dc/dd3/d24/c119 0 2024-04-06T03:40:51.503 INFO:tasks.workunit.client.0.smithi094.stdout:5/983: dwrite db/d19/d21/d11e/f53 [0,4194304] 0 2024-04-06T03:40:51.503 INFO:tasks.workunit.client.0.smithi094.stdout:5/984: chown db/d19/d21/d11e/f72 0 1 2024-04-06T03:40:51.503 INFO:tasks.workunit.client.0.smithi094.stdout:5/985: dread - db/d19/f134 zero size 2024-04-06T03:40:51.511 INFO:tasks.workunit.client.0.smithi094.stdout:5/986: write db/d19/d2c/d6d/fcb [2594289,105906] 0 2024-04-06T03:40:51.517 INFO:tasks.workunit.client.0.smithi094.stdout:6/974: dwrite dc/d3a/d4a/d54/d71/f127 [0,4194304] 0 2024-04-06T03:40:51.523 INFO:tasks.workunit.client.0.smithi094.stdout:5/987: dwrite db/d19/d2c/f73 [8388608,4194304] 0 2024-04-06T03:40:51.525 INFO:tasks.workunit.client.0.smithi094.stdout:5/988: rename db/d19/d2c/ce2 to db/d19/d2c/d6d/de6/d119/c15c 0 2024-04-06T03:40:51.525 INFO:tasks.workunit.client.0.smithi094.stdout:5/989: chown db/d19/d21/d12f/f146 67360 1 2024-04-06T03:40:51.531 INFO:tasks.workunit.client.0.smithi094.stdout:6/975: dwrite dc/d3a/d4a/d54/f58 [0,4194304] 0 2024-04-06T03:40:51.531 INFO:tasks.workunit.client.0.smithi094.stdout:6/976: creat dc/d3a/df4/df6/df2/f150 x:0 0 0 2024-04-06T03:40:51.531 INFO:tasks.workunit.client.0.smithi094.stdout:6/977: write dc/dd3/f52 [3890094,111050] 0 2024-04-06T03:40:51.532 INFO:tasks.workunit.client.0.smithi094.stdout:7/889: sync 2024-04-06T03:40:51.532 INFO:tasks.workunit.client.0.smithi094.stdout:7/890: creat d2/d6/dc/dd/df/d95/dc9/f123 x:0 0 0 2024-04-06T03:40:51.532 INFO:tasks.workunit.client.0.smithi094.stdout:7/891: fsync d2/d6/dc/d12/d3a/f116 0 2024-04-06T03:40:51.537 INFO:tasks.workunit.client.0.smithi094.stdout:6/978: mknod dc/dd3/d4e/d13e/d88/da3/dc4/c151 0 2024-04-06T03:40:51.537 INFO:tasks.workunit.client.0.smithi094.stdout:6/979: chown dc/d3a/d4a/d54/lf7 218701596 1 2024-04-06T03:40:51.547 INFO:tasks.workunit.client.0.smithi094.stdout:2/929: sync 2024-04-06T03:40:51.548 INFO:tasks.workunit.client.0.smithi094.stdout:3/993: sync 2024-04-06T03:40:51.548 INFO:tasks.workunit.client.0.smithi094.stdout:3/994: truncate d9/d10/d9b/dee/f151 284889 0 2024-04-06T03:40:51.548 INFO:tasks.workunit.client.0.smithi094.stdout:3/995: write d9/d20/d76/d6b/f12e [284028,15219] 0 2024-04-06T03:40:51.562 INFO:tasks.workunit.client.0.smithi094.stdout:2/930: dwrite d0/d8/d1d/d77/f132 [0,4194304] 0 2024-04-06T03:40:51.562 INFO:tasks.workunit.client.0.smithi094.stdout:2/931: dread - d0/d8/d1d/d77/f13a zero size 2024-04-06T03:40:51.562 INFO:tasks.workunit.client.0.smithi094.stdout:2/932: stat d0/dd/c84 0 2024-04-06T03:40:51.562 INFO:tasks.workunit.client.0.smithi094.stdout:2/933: fdatasync d0/dd/d19/dbe/fe9 0 2024-04-06T03:40:51.564 INFO:tasks.workunit.client.0.smithi094.stdout:2/934: link d0/dd/d19/dbe/lb2 d0/dd/d1b/de2/de7/l13b 0 2024-04-06T03:40:51.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:51 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:40:51.649 INFO:tasks.workunit.client.0.smithi094.stdout:6/980: dwrite dc/d3a/df4/df6/d7b/dc6/f122 [4194304,4194304] 0 2024-04-06T03:40:51.652 INFO:tasks.workunit.client.0.smithi094.stdout:7/892: dwrite d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:51.653 INFO:tasks.workunit.client.0.smithi094.stdout:6/981: symlink dc/d11/l152 0 2024-04-06T03:40:51.655 INFO:tasks.workunit.client.0.smithi094.stdout:6/982: creat dc/f153 x:0 0 0 2024-04-06T03:40:51.656 INFO:tasks.workunit.client.0.smithi094.stdout:6/983: mknod dc/dd3/d4e/d13e/d87/c154 0 2024-04-06T03:40:51.659 INFO:tasks.workunit.client.0.smithi094.stdout:2/935: rmdir d0 39 2024-04-06T03:40:51.659 INFO:tasks.workunit.client.0.smithi094.stdout:2/936: chown d0/dd/d1b/d72/d8f/lc7 0 1 2024-04-06T03:40:51.660 INFO:tasks.workunit.client.0.smithi094.stdout:7/893: read d2/d6/dc/dd/fae [3957267,84176] 0 2024-04-06T03:40:51.660 INFO:tasks.workunit.client.0.smithi094.stdout:7/894: write d2/d6/dc/d12/f5f [1586528,111901] 0 2024-04-06T03:40:51.664 INFO:tasks.workunit.client.0.smithi094.stdout:7/895: rename d2/f11 to d2/d6/dc/d12/d3a/dcc/f124 0 2024-04-06T03:40:51.665 INFO:tasks.workunit.client.0.smithi094.stdout:7/896: write d2/d6/dc/d12/d29/f76 [993601,52328] 0 2024-04-06T03:40:51.666 INFO:tasks.workunit.client.0.smithi094.stdout:6/984: dread dc/d3a/d4a/d54/d71/f127 [0,4194304] 0 2024-04-06T03:40:51.666 INFO:tasks.workunit.client.0.smithi094.stdout:2/937: unlink d0/d8/d4f/l5e 0 2024-04-06T03:40:51.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:51 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:40:51.668 INFO:tasks.workunit.client.0.smithi094.stdout:7/897: symlink d2/d6/dc/d12/d29/daa/l125 0 2024-04-06T03:40:51.668 INFO:tasks.workunit.client.0.smithi094.stdout:7/898: getdents d2/d6/dc/dd/dd2 0 2024-04-06T03:40:51.669 INFO:tasks.workunit.client.0.smithi094.stdout:6/985: readlink dc/l33 0 2024-04-06T03:40:51.669 INFO:tasks.workunit.client.0.smithi094.stdout:6/986: stat dc/d3a/df4/f125 0 2024-04-06T03:40:51.669 INFO:tasks.workunit.client.0.smithi094.stdout:6/987: stat dc/dd3/fb4 0 2024-04-06T03:40:51.669 INFO:tasks.workunit.client.0.smithi094.stdout:6/988: chown dc/dd3/fb4 404 1 2024-04-06T03:40:51.674 INFO:tasks.workunit.client.0.smithi094.stdout:7/899: getdents d2/d6/dc/d12/d29/d3f/dff 0 2024-04-06T03:40:51.676 INFO:tasks.workunit.client.0.smithi094.stdout:6/989: mkdir dc/dd3/d4e/d13e/d88/d141/d155 0 2024-04-06T03:40:51.678 INFO:tasks.workunit.client.0.smithi094.stdout:7/900: getdents d2/d6/dc/d12/d9a/da0/dd4 0 2024-04-06T03:40:51.680 INFO:tasks.workunit.client.0.smithi094.stdout:6/990: truncate dc/d3a/df4/df6/fa5 1493394 0 2024-04-06T03:40:51.681 INFO:tasks.workunit.client.0.smithi094.stdout:7/901: rmdir d2/d6/dac 39 2024-04-06T03:40:51.683 INFO:tasks.workunit.client.0.smithi094.stdout:6/991: creat dc/d3a/df4/f156 x:0 0 0 2024-04-06T03:40:51.683 INFO:tasks.workunit.client.0.smithi094.stdout:6/992: creat dc/dd3/d24/f157 x:0 0 0 2024-04-06T03:40:51.688 INFO:tasks.workunit.client.0.smithi094.stdout:7/902: mknod d2/d6/dc/d12/d29/d3f/dce/c126 0 2024-04-06T03:40:51.688 INFO:tasks.workunit.client.0.smithi094.stdout:6/993: rmdir dc/d11/d97 39 2024-04-06T03:40:51.691 INFO:tasks.workunit.client.0.smithi094.stdout:7/903: unlink d2/d6/d93/cf3 0 2024-04-06T03:40:51.693 INFO:tasks.workunit.client.0.smithi094.stdout:6/994: rename dc/d3a/d4a/d54/dde/d50/d61/f13b to dc/dd3/d4e/d13e/d88/da3/dc4/f158 0 2024-04-06T03:40:51.694 INFO:tasks.workunit.client.0.smithi094.stdout:2/938: dwrite d0/dd/d1b/f12f [0,4194304] 0 2024-04-06T03:40:51.697 INFO:tasks.workunit.client.0.smithi094.stdout:6/995: symlink dc/d3a/df4/df6/d7b/dc6/d13c/l159 0 2024-04-06T03:40:51.700 INFO:tasks.workunit.client.0.smithi094.stdout:2/939: getdents d0/d8/d1d 0 2024-04-06T03:40:51.702 INFO:tasks.workunit.client.0.smithi094.stdout:6/996: creat dc/d3a/d4a/d54/dde/d50/daf/d12a/f15a x:0 0 0 2024-04-06T03:40:51.702 INFO:tasks.workunit.client.0.smithi094.stdout:6/997: fdatasync dc/dd3/d4e/fd7 0 2024-04-06T03:40:51.702 INFO:tasks.workunit.client.0.smithi094.stdout:6/998: dread - dc/d3a/d4a/d54/dde/d50/de0/f14f zero size 2024-04-06T03:40:51.702 INFO:tasks.workunit.client.0.smithi094.stdout:6/999: chown dc/d3a/d7f/dd4/c14a 3835 1 2024-04-06T03:40:51.707 INFO:tasks.workunit.client.0.smithi094.stdout:7/904: dwrite d2/d6/dc/d12/d3a/fe8 [0,4194304] 0 2024-04-06T03:40:51.711 INFO:tasks.workunit.client.0.smithi094.stdout:7/905: symlink d2/d6/dc/dd/df/d36/l127 0 2024-04-06T03:40:51.711 INFO:tasks.workunit.client.0.smithi094.stdout:7/906: fsync d2/d6/dc/d12/d3a/dcc/f122 0 2024-04-06T03:40:51.713 INFO:tasks.workunit.client.0.smithi094.stdout:7/907: rename d2/c4c to d2/d6/dc/dd/df/d95/dc9/d57/c128 0 2024-04-06T03:40:51.714 INFO:tasks.workunit.client.0.smithi094.stdout:2/940: dwrite d0/dd/d1b/f105 [0,4194304] 0 2024-04-06T03:40:51.715 INFO:tasks.workunit.client.0.smithi094.stdout:7/908: write d2/f23 [4502813,4428] 0 2024-04-06T03:40:51.715 INFO:tasks.workunit.client.0.smithi094.stdout:7/909: write d2/d6/dc/dd/f3b [1888744,66136] 0 2024-04-06T03:40:51.715 INFO:tasks.workunit.client.0.smithi094.stdout:7/910: chown d2/d6/dc/d12/d9a/fd3 761388 1 2024-04-06T03:40:51.718 INFO:tasks.workunit.client.0.smithi094.stdout:7/911: creat d2/d6/dc/dd/df/d95/f129 x:0 0 0 2024-04-06T03:40:51.719 INFO:tasks.workunit.client.0.smithi094.stdout:2/941: dread d0/dd/d3d/f100 [0,4194304] 0 2024-04-06T03:40:51.719 INFO:tasks.workunit.client.0.smithi094.stdout:2/942: dread - d0/dd/da4/fcb zero size 2024-04-06T03:40:51.725 INFO:tasks.workunit.client.0.smithi094.stdout:7/912: dread d2/d6/f22 [0,4194304] 0 2024-04-06T03:40:51.730 INFO:tasks.workunit.client.0.smithi094.stdout:7/913: truncate d2/d6/dc/dd/df/f2d 1419861 0 2024-04-06T03:40:51.742 INFO:tasks.workunit.client.0.smithi094.stdout:7/914: fsync d2/d6/dc/f74 0 2024-04-06T03:40:51.745 INFO:tasks.workunit.client.0.smithi094.stdout:7/915: mkdir d2/d6/dc/d12a 0 2024-04-06T03:40:51.747 INFO:tasks.workunit.client.0.smithi094.stdout:7/916: symlink d2/d6/dc/d12/d9a/dbd/l12b 0 2024-04-06T03:40:51.749 INFO:tasks.workunit.client.0.smithi094.stdout:7/917: creat d2/d6/dac/f12c x:0 0 0 2024-04-06T03:40:51.750 INFO:tasks.workunit.client.0.smithi094.stdout:7/918: creat d2/d6/dc/d12/d29/d3f/dce/f12d x:0 0 0 2024-04-06T03:40:51.751 INFO:tasks.workunit.client.0.smithi094.stdout:7/919: symlink d2/d6/dac/ddd/l12e 0 2024-04-06T03:40:51.753 INFO:tasks.workunit.client.0.smithi094.stdout:7/920: unlink d2/d6/dc/d12/d9a/da0/dd4/db1/fb2 0 2024-04-06T03:40:51.754 INFO:tasks.workunit.client.0.smithi094.stdout:7/921: mknod d2/d6/dc/dd/df/d36/c12f 0 2024-04-06T03:40:51.756 INFO:tasks.workunit.client.0.smithi094.stdout:7/922: symlink d2/d6/dc/dd/df/d36/l130 0 2024-04-06T03:40:51.756 INFO:tasks.workunit.client.0.smithi094.stdout:7/923: dread - d2/d6/dc/dd/df/d95/dc9/f115 zero size 2024-04-06T03:40:51.764 INFO:tasks.workunit.client.0.smithi094.stdout:7/924: dread d2/d6/dc/d12/d29/d3f/fee [0,4194304] 0 2024-04-06T03:40:51.765 INFO:tasks.workunit.client.0.smithi094.stdout:7/925: truncate d2/d6/f30 2223069 0 2024-04-06T03:40:51.775 INFO:tasks.workunit.client.0.smithi094.stdout:7/926: dread d2/d6/dc/d12/d9a/fd3 [0,4194304] 0 2024-04-06T03:40:51.777 INFO:tasks.workunit.client.0.smithi094.stdout:7/927: link d2/d6/dc/dd/df/d95/dc9/d57/c6d d2/d6/dc/d12/d9a/da0/c131 0 2024-04-06T03:40:51.778 INFO:tasks.workunit.client.0.smithi094.stdout:7/928: mknod d2/d6/dc/d12/c132 0 2024-04-06T03:40:51.780 INFO:tasks.workunit.client.0.smithi094.stdout:7/929: symlink d2/d6/dc/dd/df/d36/d79/d11f/l133 0 2024-04-06T03:40:51.780 INFO:tasks.workunit.client.0.smithi094.stdout:7/930: fdatasync d2/d6/dac/f12c 0 2024-04-06T03:40:51.780 INFO:tasks.workunit.client.0.smithi094.stdout:7/931: stat d2/d6/dc/dd/df/d95/dc9/f123 0 2024-04-06T03:40:51.810 INFO:tasks.workunit.client.0.smithi094.stdout:7/932: dwrite d2/d6/dc/f2b [0,4194304] 0 2024-04-06T03:40:51.815 INFO:tasks.workunit.client.0.smithi094.stdout:5/990: sync 2024-04-06T03:40:51.815 INFO:tasks.workunit.client.0.smithi094.stdout:3/996: sync 2024-04-06T03:40:51.815 INFO:tasks.workunit.client.0.smithi094.stdout:5/991: creat db/d19/dc1/dd4/dcf/f15d x:0 0 0 2024-04-06T03:40:51.816 INFO:tasks.workunit.client.0.smithi094.stdout:5/992: chown db/d19/d30/l6b 4285 1 2024-04-06T03:40:51.816 INFO:tasks.workunit.client.0.smithi094.stdout:7/933: dread d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:51.816 INFO:tasks.workunit.client.0.smithi094.stdout:7/934: readlink d2/d6/dc/dd/df/d36/d79/l110 0 2024-04-06T03:40:51.816 INFO:tasks.workunit.client.0.smithi094.stdout:7/935: read - d2/d6/dc/d12/d3a/f116 zero size 2024-04-06T03:40:51.824 INFO:tasks.workunit.client.0.smithi094.stdout:7/936: rmdir d2/d6/dc/dd/df/d95/d118 0 2024-04-06T03:40:51.826 INFO:tasks.workunit.client.0.smithi094.stdout:5/993: dwrite db/d7c/d98/f9f [4194304,4194304] 0 2024-04-06T03:40:51.829 INFO:tasks.workunit.client.0.smithi094.stdout:3/997: mkdir d9/d20/da6/de9/d164 0 2024-04-06T03:40:51.834 INFO:tasks.workunit.client.0.smithi094.stdout:3/998: mknod d9/d9d/dc1/c165 0 2024-04-06T03:40:51.840 INFO:tasks.workunit.client.0.smithi094.stdout:7/937: dwrite d2/d6/dc/d12/f78 [0,4194304] 0 2024-04-06T03:40:51.849 INFO:tasks.workunit.client.0.smithi094.stdout:5/994: dwrite db/d19/d21/d11e/d28/d85/dd5/f150 [0,4194304] 0 2024-04-06T03:40:51.850 INFO:tasks.workunit.client.0.smithi094.stdout:5/995: write db/d19/d21/f26 [2299285,12305] 0 2024-04-06T03:40:51.855 INFO:tasks.workunit.client.0.smithi094.stdout:7/938: dwrite d2/d6/dc/dd/f28 [0,4194304] 0 2024-04-06T03:40:51.856 INFO:tasks.workunit.client.0.smithi094.stdout:3/999: dwrite d9/d20/d76/ff7 [0,4194304] 0 2024-04-06T03:40:51.857 INFO:tasks.workunit.client.0.smithi094.stdout:7/939: dread d2/d6/dc/d12/d9a/dbd/fe3 [0,4194304] 0 2024-04-06T03:40:51.857 INFO:tasks.workunit.client.0.smithi094.stdout:7/940: chown d2/d6/dc/dd/df/d36/l127 72 1 2024-04-06T03:40:51.860 INFO:tasks.workunit.client.0.smithi094.stdout:5/996: rmdir db/d19/dc1/ddf 39 2024-04-06T03:40:51.860 INFO:tasks.workunit.client.0.smithi094.stdout:5/997: chown db/d19/d21/f49 5 1 2024-04-06T03:40:51.862 INFO:tasks.workunit.client.0.smithi094.stdout:7/941: mknod d2/d6/dc/d12/d29/d3f/db9/c134 0 2024-04-06T03:40:51.862 INFO:tasks.workunit.client.0.smithi094.stdout:7/942: write d2/d6/dc/d12/d29/f89 [500127,63582] 0 2024-04-06T03:40:51.864 INFO:tasks.workunit.client.0.smithi094.stdout:5/998: creat db/d19/d21/d11e/d28/dec/f15e x:0 0 0 2024-04-06T03:40:51.867 INFO:tasks.workunit.client.0.smithi094.stdout:7/943: symlink d2/d6/dc/dd/d90/l135 0 2024-04-06T03:40:51.869 INFO:tasks.workunit.client.0.smithi094.stdout:7/944: link d2/d6/dc/dd/l1b d2/d6/dc/dd/df/d95/dc9/d57/l136 0 2024-04-06T03:40:51.871 INFO:tasks.workunit.client.0.smithi094.stdout:7/945: mknod d2/d6/dc/d12/d9a/dbd/c137 0 2024-04-06T03:40:51.872 INFO:tasks.workunit.client.0.smithi094.stdout:7/946: truncate d2/d6/dc/d12/d29/d3f/dce/f10e 243985 0 2024-04-06T03:40:51.874 INFO:tasks.workunit.client.0.smithi094.stdout:7/947: read d2/d6/dc/dd/df/d36/f120 [95892,82134] 0 2024-04-06T03:40:51.876 INFO:tasks.workunit.client.0.smithi094.stdout:7/948: mknod d2/d6/dc/d12/d61/c138 0 2024-04-06T03:40:51.876 INFO:tasks.workunit.client.0.smithi094.stdout:7/949: write d2/d6/f8 [805852,41924] 0 2024-04-06T03:40:51.878 INFO:tasks.workunit.client.0.smithi094.stdout:7/950: rename d2/d6/dc/dd/df/d36/f60 to d2/d6/dc/d12/d9a/da0/dd4/f139 0 2024-04-06T03:40:51.878 INFO:tasks.workunit.client.0.smithi094.stdout:7/951: chown d2/d6/dc/d12/d9a/da0 186720 1 2024-04-06T03:40:51.879 INFO:tasks.workunit.client.0.smithi094.stdout:5/999: dwrite db/d19/dc1/dd4/dcf/f15d [0,4194304] 0 2024-04-06T03:40:51.880 INFO:tasks.workunit.client.0.smithi094.stdout:7/952: truncate d2/d6/dc/dd/df/d95/dc9/fbc 1215338 0 2024-04-06T03:40:51.880 INFO:tasks.workunit.client.0.smithi094.stdout:7/953: fdatasync d2/d6/dc/dd/f3b 0 2024-04-06T03:40:51.893 INFO:tasks.workunit.client.0.smithi094.stdout:7/954: mknod d2/d6/dc/d12/d3a/dcc/dde/c13a 0 2024-04-06T03:40:51.896 INFO:tasks.workunit.client.0.smithi094.stdout:2/943: sync 2024-04-06T03:40:51.897 INFO:tasks.workunit.client.0.smithi094.stdout:2/944: symlink d0/dd/d3d/d49/l13c 0 2024-04-06T03:40:51.901 INFO:tasks.workunit.client.0.smithi094.stdout:2/945: symlink d0/dd/d19/d31/d67/l13d 0 2024-04-06T03:40:51.903 INFO:tasks.workunit.client.0.smithi094.stdout:2/946: symlink d0/d8/d1d/l13e 0 2024-04-06T03:40:51.903 INFO:tasks.workunit.client.0.smithi094.stdout:2/947: chown d0/dd/d1b/d72/f11a 57911160 1 2024-04-06T03:40:51.903 INFO:tasks.workunit.client.0.smithi094.stdout:2/948: truncate d0/dd/d1b/de2/de7/f116 1063993 0 2024-04-06T03:40:51.905 INFO:tasks.workunit.client.0.smithi094.stdout:2/949: creat d0/dd/d1b/d72/da8/dd2/f13f x:0 0 0 2024-04-06T03:40:51.907 INFO:tasks.workunit.client.0.smithi094.stdout:2/950: truncate d0/dd/d19/d31/f7d 2858820 0 2024-04-06T03:40:51.907 INFO:tasks.workunit.client.0.smithi094.stdout:2/951: truncate d0/dd/d19/fb8 884819 0 2024-04-06T03:40:51.907 INFO:tasks.workunit.client.0.smithi094.stdout:2/952: fsync d0/d8/d1d/f79 0 2024-04-06T03:40:51.908 INFO:tasks.workunit.client.0.smithi094.stdout:7/955: dwrite d2/d6/dc/d12/f78 [4194304,4194304] 0 2024-04-06T03:40:51.908 INFO:tasks.workunit.client.0.smithi094.stdout:7/956: stat d2/d6/dc/d12/d9a/ff1 0 2024-04-06T03:40:51.909 INFO:tasks.workunit.client.0.smithi094.stdout:2/953: mkdir d0/dd/d1b/d72/df3/d140 0 2024-04-06T03:40:51.911 INFO:tasks.workunit.client.0.smithi094.stdout:7/957: stat d2/d6/dc/d12/d9a/c108 0 2024-04-06T03:40:51.912 INFO:tasks.workunit.client.0.smithi094.stdout:7/958: fsync d2/fec 0 2024-04-06T03:40:51.917 INFO:tasks.workunit.client.0.smithi094.stdout:2/954: write d0/dd/f62 [2143109,76164] 0 2024-04-06T03:40:51.919 INFO:tasks.workunit.client.0.smithi094.stdout:2/955: rename d0/d8/d27/l36 to d0/dd/d3d/d49/l141 0 2024-04-06T03:40:51.921 INFO:tasks.workunit.client.0.smithi094.stdout:2/956: mkdir d0/d8/d1d/d142 0 2024-04-06T03:40:51.924 INFO:tasks.workunit.client.0.smithi094.stdout:2/957: creat d0/d8/d1d/d77/f143 x:0 0 0 2024-04-06T03:40:51.924 INFO:tasks.workunit.client.0.smithi094.stdout:2/958: write d0/dd/d1b/f1c [613475,61244] 0 2024-04-06T03:40:51.927 INFO:tasks.workunit.client.0.smithi094.stdout:2/959: mkdir d0/dd/d19/d31/d67/d144 0 2024-04-06T03:40:51.935 INFO:tasks.workunit.client.0.smithi094.stdout:2/960: dread d0/dd/d1b/f105 [0,4194304] 0 2024-04-06T03:40:51.935 INFO:tasks.workunit.client.0.smithi094.stdout:2/961: chown d0/dd/d3d/f100 0 1 2024-04-06T03:40:51.935 INFO:tasks.workunit.client.0.smithi094.stdout:2/962: write d0/d18/f12c [384461,60783] 0 2024-04-06T03:40:51.949 INFO:tasks.workunit.client.0.smithi094.stdout:2/963: dwrite d0/dd/d19/d31/f3a [0,4194304] 0 2024-04-06T03:40:51.949 INFO:tasks.workunit.client.0.smithi094.stdout:2/964: read - d0/dd/d19/d31/f103 zero size 2024-04-06T03:40:51.950 INFO:tasks.workunit.client.0.smithi094.stdout:2/965: symlink d0/d8/l145 0 2024-04-06T03:40:51.950 INFO:tasks.workunit.client.0.smithi094.stdout:2/966: fdatasync d0/d8/d27/fb4 0 2024-04-06T03:40:51.950 INFO:tasks.workunit.client.0.smithi094.stdout:2/967: fsync d0/dd/d19/d31/f7d 0 2024-04-06T03:40:51.950 INFO:tasks.workunit.client.0.smithi094.stdout:2/968: chown d0/d8/d4f/d136/la7 61217223 1 2024-04-06T03:40:51.963 INFO:tasks.workunit.client.0.smithi094.stdout:2/969: dwrite d0/dd/d1b/d72/d8f/f106 [0,4194304] 0 2024-04-06T03:40:51.963 INFO:tasks.workunit.client.0.smithi094.stdout:2/970: chown d0/d8/d1d/c114 41867700 1 2024-04-06T03:40:51.963 INFO:tasks.workunit.client.0.smithi094.stdout:2/971: getdents d0/dd/d1b/d72/da8/d125 0 2024-04-06T03:40:51.963 INFO:tasks.workunit.client.0.smithi094.stdout:2/972: fsync d0/dd/d19/d31/d67/d98/ddc/ff8 0 2024-04-06T03:40:51.972 INFO:tasks.workunit.client.0.smithi094.stdout:2/973: write d0/d8/d1d/d77/fa6 [884987,1036] 0 2024-04-06T03:40:51.978 INFO:tasks.workunit.client.0.smithi094.stdout:2/974: read d0/d18/f22 [7163855,18914] 0 2024-04-06T03:40:51.990 INFO:tasks.workunit.client.0.smithi094.stdout:2/975: dwrite d0/d8/ff6 [0,4194304] 0 2024-04-06T03:40:51.996 INFO:tasks.workunit.client.0.smithi094.stdout:2/976: symlink d0/dd/d1b/de2/de7/l146 0 2024-04-06T03:40:51.996 INFO:tasks.workunit.client.0.smithi094.stdout:2/977: truncate d0/dd/d1b/de2/f126 878349 0 2024-04-06T03:40:52.183 INFO:tasks.workunit.client.0.smithi094.stdout:7/959: sync 2024-04-06T03:40:52.184 INFO:tasks.workunit.client.0.smithi094.stdout:7/960: chown d2/d6/dc/d12 726990 1 2024-04-06T03:40:52.185 INFO:tasks.workunit.client.0.smithi094.stdout:7/961: rename d2/d6/dc/d12/d61/lcf to d2/d6/d93/l13b 0 2024-04-06T03:40:52.186 INFO:tasks.workunit.client.0.smithi094.stdout:7/962: rmdir d2/d6/dc/dd/df/d36/d79 39 2024-04-06T03:40:52.190 INFO:tasks.workunit.client.0.smithi094.stdout:7/963: unlink d2/d6/dc/dd/c20 0 2024-04-06T03:40:52.190 INFO:tasks.workunit.client.0.smithi094.stdout:7/964: creat d2/d6/f13c x:0 0 0 2024-04-06T03:40:52.193 INFO:tasks.workunit.client.0.smithi094.stdout:7/965: dread d2/d6/dc/dd/df/f1a [0,4194304] 0 2024-04-06T03:40:52.196 INFO:tasks.workunit.client.0.smithi094.stdout:7/966: dread d2/d6/f9 [0,4194304] 0 2024-04-06T03:40:52.198 INFO:tasks.workunit.client.0.smithi094.stdout:7/967: rmdir d2/d6/dc/d12/d9a/dbf 39 2024-04-06T03:40:52.202 INFO:tasks.workunit.client.0.smithi094.stdout:7/968: rename d2/d6/dc/d12/d29/d3f/fee to d2/d6/dc/d12/d29/d3f/dff/f13d 0 2024-04-06T03:40:52.213 INFO:tasks.workunit.client.0.smithi094.stdout:7/969: dwrite d2/d6/dc/dd/df/d95/dc9/fb0 [0,4194304] 0 2024-04-06T03:40:52.217 INFO:tasks.workunit.client.0.smithi094.stdout:7/970: creat d2/d6/dc/d12/f13e x:0 0 0 2024-04-06T03:40:52.217 INFO:tasks.workunit.client.0.smithi094.stdout:7/971: readlink d2/d6/dc/dd/df/l9f 0 2024-04-06T03:40:52.219 INFO:tasks.workunit.client.0.smithi094.stdout:7/972: mknod d2/d6/dc/c13f 0 2024-04-06T03:40:52.219 INFO:tasks.workunit.client.0.smithi094.stdout:7/973: readlink d2/d6/l40 0 2024-04-06T03:40:52.233 INFO:tasks.workunit.client.0.smithi094.stdout:7/974: dwrite d2/d6/dc/d12/d9a/da0/dd4/f9c [4194304,4194304] 0 2024-04-06T03:40:52.236 INFO:tasks.workunit.client.0.smithi094.stdout:7/975: creat d2/d6/dc/d12/d3a/dcc/dde/f140 x:0 0 0 2024-04-06T03:40:52.237 INFO:tasks.workunit.client.0.smithi094.stdout:7/976: creat d2/d6/dc/d12/d29/daa/f141 x:0 0 0 2024-04-06T03:40:52.239 INFO:tasks.workunit.client.0.smithi094.stdout:7/977: unlink d2/d6/d93/l11c 0 2024-04-06T03:40:52.239 INFO:tasks.workunit.client.0.smithi094.stdout:7/978: dread - d2/d6/dc/d12/d3a/dcc/dde/f140 zero size 2024-04-06T03:40:52.240 INFO:tasks.workunit.client.0.smithi094.stdout:7/979: mkdir d2/d6/dc/d12a/d142 0 2024-04-06T03:40:52.244 INFO:tasks.workunit.client.0.smithi094.stdout:7/980: link d2/d6/dc/d12/d29/ccd d2/d6/dc/d12/d29/d3f/dce/dfe/c143 0 2024-04-06T03:40:52.244 INFO:tasks.workunit.client.0.smithi094.stdout:7/981: truncate d2/d6/dc/d12/d29/d3f/dce/f12d 471730 0 2024-04-06T03:40:52.245 INFO:tasks.workunit.client.0.smithi094.stdout:7/982: mknod d2/d6/dc/d12/d29/daa/c144 0 2024-04-06T03:40:52.246 INFO:tasks.workunit.client.0.smithi094.stdout:7/983: creat d2/d6/dc/dd/df/d36/d79/d11f/f145 x:0 0 0 2024-04-06T03:40:52.248 INFO:tasks.workunit.client.0.smithi094.stdout:7/984: creat d2/d6/dc/d12/d29/d3f/db9/f146 x:0 0 0 2024-04-06T03:40:52.253 INFO:tasks.workunit.client.0.smithi094.stdout:7/985: write d2/d6/dc/d12/d61/f83 [2305596,34182] 0 2024-04-06T03:40:52.253 INFO:tasks.workunit.client.0.smithi094.stdout:7/986: dread - d2/d6/dc/d12/d3a/f116 zero size 2024-04-06T03:40:52.264 INFO:tasks.workunit.client.0.smithi094.stdout:7/987: dwrite d2/d6/dc/dd/df/d95/dc9/f10d [0,4194304] 0 2024-04-06T03:40:52.270 INFO:tasks.workunit.client.0.smithi094.stdout:7/988: dread d2/d6/dc/d12/d29/f76 [0,4194304] 0 2024-04-06T03:40:52.270 INFO:tasks.workunit.client.0.smithi094.stdout:7/989: chown d2/d6/dc/d12/d61/lea 2336 1 2024-04-06T03:40:52.270 INFO:tasks.workunit.client.0.smithi094.stdout:7/990: stat d2/d6/dc/dd/df/d95 0 2024-04-06T03:40:52.272 INFO:tasks.workunit.client.0.smithi094.stdout:7/991: mknod d2/d6/d93/c147 0 2024-04-06T03:40:52.281 INFO:tasks.workunit.client.0.smithi094.stdout:7/992: dread d2/d6/dc/d12/d29/f89 [0,4194304] 0 2024-04-06T03:40:52.282 INFO:tasks.workunit.client.0.smithi094.stdout:7/993: rmdir d2/d6/dc 39 2024-04-06T03:40:52.283 INFO:tasks.workunit.client.0.smithi094.stdout:7/994: creat d2/d6/dc/d12a/f148 x:0 0 0 2024-04-06T03:40:52.283 INFO:tasks.workunit.client.0.smithi094.stdout:7/995: chown d2/d6/dc/d12/d3a/dcc/dde/db7/ld8 2503405 1 2024-04-06T03:40:52.290 INFO:tasks.workunit.client.0.smithi094.stdout:2/978: sync 2024-04-06T03:40:52.292 INFO:tasks.workunit.client.0.smithi094.stdout:2/979: creat d0/dd/d19/d31/dcc/f147 x:0 0 0 2024-04-06T03:40:52.292 INFO:tasks.workunit.client.0.smithi094.stdout:2/980: chown d0/d8/d4f/d136/ded/f101 910292 1 2024-04-06T03:40:52.293 INFO:tasks.workunit.client.0.smithi094.stdout:2/981: symlink d0/dd/d19/d31/d67/l148 0 2024-04-06T03:40:52.294 INFO:tasks.workunit.client.0.smithi094.stdout:7/996: dwrite d2/d6/dc/d12/d29/f76 [4194304,4194304] 0 2024-04-06T03:40:52.299 INFO:tasks.workunit.client.0.smithi094.stdout:7/997: symlink d2/d6/dc/dd/df/d95/dc9/d57/l149 0 2024-04-06T03:40:52.301 INFO:tasks.workunit.client.0.smithi094.stdout:2/982: symlink d0/d18/l149 0 2024-04-06T03:40:52.303 INFO:tasks.workunit.client.0.smithi094.stdout:7/998: link d2/d6/dc/d12/d29/d3f/db9/cf2 d2/d6/dc/dd/df/c14a 0 2024-04-06T03:40:52.322 INFO:tasks.workunit.client.0.smithi094.stdout:7/999: dwrite d2/d6/dc/d12/d29/daa/f141 [0,4194304] 0 2024-04-06T03:40:52.322 INFO:tasks.workunit.client.0.smithi094.stdout:2/983: dwrite d0/dd/d1b/d72/da8/dd2/f13f [0,4194304] 0 2024-04-06T03:40:52.323 INFO:tasks.workunit.client.0.smithi094.stdout:2/984: stat d0/dd/d1b/d72/df3 0 2024-04-06T03:40:52.328 INFO:tasks.workunit.client.0.smithi094.stdout:2/985: unlink d0/dd/d3d/d49/lb1 0 2024-04-06T03:40:52.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/986: dwrite d0/dd/d19/d31/dcc/ffd [0,4194304] 0 2024-04-06T03:40:52.344 INFO:tasks.workunit.client.0.smithi094.stdout:2/987: write d0/dd/d19/d31/dcc/f147 [902777,12626] 0 2024-04-06T03:40:52.348 INFO:tasks.workunit.client.0.smithi094.stdout:2/988: dread d0/d18/fd6 [0,4194304] 0 2024-04-06T03:40:52.350 INFO:tasks.workunit.client.0.smithi094.stdout:2/989: symlink d0/dd/d19/d31/d67/d144/l14a 0 2024-04-06T03:40:52.364 INFO:tasks.workunit.client.0.smithi094.stdout:2/990: dwrite d0/f112 [12582912,4194304] 0 2024-04-06T03:40:52.367 INFO:tasks.workunit.client.0.smithi094.stdout:2/991: truncate d0/d8/d1d/fc0 2830612 0 2024-04-06T03:40:52.373 INFO:tasks.workunit.client.0.smithi094.stdout:2/992: dread d0/dd/d1b/d72/d8f/f106 [0,4194304] 0 2024-04-06T03:40:52.374 INFO:tasks.workunit.client.0.smithi094.stdout:2/993: creat d0/dd/d19/dbe/f14b x:0 0 0 2024-04-06T03:40:52.593 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:52 smithi094 ceph-mon[29713]: pgmap v26: 65 pgs: 65 active+clean; 4.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 296 MiB/s rd, 372 MiB/s wr, 552 op/s 2024-04-06T03:40:52.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:52 smithi136 ceph-mon[36436]: pgmap v26: 65 pgs: 65 active+clean; 4.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 296 MiB/s rd, 372 MiB/s wr, 552 op/s 2024-04-06T03:40:52.615 INFO:tasks.workunit.client.0.smithi094.stdout:2/994: sync 2024-04-06T03:40:52.628 INFO:tasks.workunit.client.0.smithi094.stdout:2/995: dwrite d0/d18/fd6 [0,4194304] 0 2024-04-06T03:40:52.633 INFO:tasks.workunit.client.0.smithi094.stdout:2/996: dread d0/d18/f22 [4194304,4194304] 0 2024-04-06T03:40:52.878 INFO:tasks.workunit.client.0.smithi094.stdout:2/997: sync 2024-04-06T03:40:52.880 INFO:tasks.workunit.client.0.smithi094.stdout:2/998: mkdir d0/d8/d27/d2b/d14c 0 2024-04-06T03:40:52.880 INFO:tasks.workunit.client.0.smithi094.stdout:2/999: write d0/d8/d4f/dab/ffe [418264,71890] 0 2024-04-06T03:40:52.882 INFO:tasks.workunit.client.0.smithi094.stderr:+ rm -rf -- ./tmp.xr8h8PEbww 2024-04-06T03:40:54.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.845+0000 7fb5a0273700 1 -- 172.21.15.94:0/24213839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5980fcf00 msgr2=0x7fb5980fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:54.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.845+0000 7fb5a0273700 1 --2- 172.21.15.94:0/24213839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5980fcf00 0x7fb5980fd310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb594004940 tx=0x7fb594009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:54.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.845+0000 7fb5a0273700 1 -- 172.21.15.94:0/24213839 shutdown_connections 2024-04-06T03:40:54.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.845+0000 7fb5a0273700 1 --2- 172.21.15.94:0/24213839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5980fe070 0x7fb5980fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:54.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.845+0000 7fb5a0273700 1 --2- 172.21.15.94:0/24213839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5980fcf00 0x7fb5980fd310 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:54.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.845+0000 7fb5a0273700 1 -- 172.21.15.94:0/24213839 >> 172.21.15.94:0/24213839 conn(0x7fb5980f85a0 msgr2=0x7fb5980fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.845+0000 7fb5a0273700 1 -- 172.21.15.94:0/24213839 shutdown_connections 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.845+0000 7fb5a0273700 1 -- 172.21.15.94:0/24213839 wait complete. 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb5a0273700 1 Processor -- start 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb5a0273700 1 -- start start 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb5a0273700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5980fcf00 0x7fb5981140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb5a0273700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5980fe070 0x7fb598114600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb5a0273700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb598114c00 con 0x7fb5980fcf00 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb5a0273700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb598115440 con 0x7fb5980fe070 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb59d80e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5980fe070 0x7fb598114600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb59e00f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5980fcf00 0x7fb5981140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb59d80e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5980fe070 0x7fb598114600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:53152/0 (socket says 172.21.15.94:53152) 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.846+0000 7fb59d80e700 1 -- 172.21.15.94:0/98008091 learned_addr learned my addr 172.21.15.94:0/98008091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb59e00f700 1 -- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5980fe070 msgr2=0x7fb598114600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb59e00f700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5980fe070 0x7fb598114600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:54.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb59e00f700 1 -- 172.21.15.94:0/98008091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb594005040 con 0x7fb5980fcf00 2024-04-06T03:40:54.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb59d80e700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5980fe070 0x7fb598114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:40:54.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb59e00f700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5980fcf00 0x7fb5981140c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb59400b3b0 tx=0x7fb594009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:54.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb58effd700 1 -- 172.21.15.94:0/98008091 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb59400e040 con 0x7fb5980fcf00 2024-04-06T03:40:54.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5981156c0 con 0x7fb5980fcf00 2024-04-06T03:40:54.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb58effd700 1 -- 172.21.15.94:0/98008091 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb59400bbf0 con 0x7fb5980fcf00 2024-04-06T03:40:54.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb58effd700 1 -- 172.21.15.94:0/98008091 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb594013660 con 0x7fb5980fcf00 2024-04-06T03:40:54.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.847+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb598115c00 con 0x7fb5980fcf00 2024-04-06T03:40:54.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.849+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb598101a00 con 0x7fb5980fcf00 2024-04-06T03:40:54.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.850+0000 7fb58effd700 1 -- 172.21.15.94:0/98008091 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7fb59400bd60 con 0x7fb5980fcf00 2024-04-06T03:40:54.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.850+0000 7fb58effd700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fb58406b610 0x7fb58406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:54.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.850+0000 7fb58effd700 1 -- 172.21.15.94:0/98008091 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fb59408b350 con 0x7fb5980fcf00 2024-04-06T03:40:54.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.850+0000 7fb59d80e700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fb58406b610 0x7fb58406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:54.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.851+0000 7fb59d80e700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fb58406b610 0x7fb58406dac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb5880041d0 tx=0x7fb58800b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:54.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:54.856+0000 7fb58effd700 1 -- 172.21.15.94:0/98008091 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fb594058d70 con 0x7fb5980fcf00 2024-04-06T03:40:55.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.151+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb598115e70 con 0x7fb58406b610 2024-04-06T03:40:55.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.153+0000 7fb58effd700 1 -- 172.21.15.94:0/98008091 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fb598115e70 con 0x7fb58406b610 2024-04-06T03:40:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fb58406b610 msgr2=0x7fb58406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fb58406b610 0x7fb58406dac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb5880041d0 tx=0x7fb58800b040 comp rx=0 tx=0).stop 2024-04-06T03:40:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5980fcf00 msgr2=0x7fb5981140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5980fcf00 0x7fb5981140c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb59400b3b0 tx=0x7fb594009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 shutdown_connections 2024-04-06T03:40:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb5980fcf00 0x7fb5981140c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fb58406b610 0x7fb58406dac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 --2- 172.21.15.94:0/98008091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5980fe070 0x7fb598114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 >> 172.21.15.94:0/98008091 conn(0x7fb5980f85a0 msgr2=0x7fb5980f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 shutdown_connections 2024-04-06T03:40:55.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.156+0000 7fb5a0273700 1 -- 172.21.15.94:0/98008091 wait complete. 2024-04-06T03:40:55.171 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:40:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.296+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/298092488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f80fcfb0 msgr2=0x7fe6f80fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:55.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.296+0000 7fe6ffa9c700 1 --2- 172.21.15.94:0/298092488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f80fcfb0 0x7fe6f80fd420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009ce0 tx=0x7fe6f0004d10 comp rx=0 tx=0).stop 2024-04-06T03:40:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.297+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/298092488 shutdown_connections 2024-04-06T03:40:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.297+0000 7fe6ffa9c700 1 --2- 172.21.15.94:0/298092488 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f80fcfb0 0x7fe6f80fd420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.297+0000 7fe6ffa9c700 1 --2- 172.21.15.94:0/298092488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f80fbd00 0x7fe6f80fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.297+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/298092488 >> 172.21.15.94:0/298092488 conn(0x7fe6f80f7480 msgr2=0x7fe6f80f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.300+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/298092488 shutdown_connections 2024-04-06T03:40:55.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.300+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/298092488 wait complete. 2024-04-06T03:40:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.301+0000 7fe6ffa9c700 1 Processor -- start 2024-04-06T03:40:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.301+0000 7fe6ffa9c700 1 -- start start 2024-04-06T03:40:55.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.301+0000 7fe6ffa9c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f80fbd00 0x7fe6f810c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.301+0000 7fe6ffa9c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f80fcfb0 0x7fe6f810cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.302+0000 7fe6ffa9c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f810d3c0 con 0x7fe6f80fbd00 2024-04-06T03:40:55.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.302+0000 7fe6ffa9c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f810d530 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.302+0000 7fe6fd838700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f80fbd00 0x7fe6f810c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.302+0000 7fe6fd037700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f80fcfb0 0x7fe6f810cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.302+0000 7fe6fd037700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f80fcfb0 0x7fe6f810cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:53172/0 (socket says 172.21.15.94:53172) 2024-04-06T03:40:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.302+0000 7fe6fd037700 1 -- 172.21.15.94:0/3567254811 learned_addr learned my addr 172.21.15.94:0/3567254811 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.303+0000 7fe6fd037700 1 -- 172.21.15.94:0/3567254811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f80fbd00 msgr2=0x7fe6f810c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.303+0000 7fe6fd037700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f80fbd00 0x7fe6f810c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.303+0000 7fe6fd037700 1 -- 172.21.15.94:0/3567254811 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0009940 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.303+0000 7fe6fd037700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f80fcfb0 0x7fe6f810cdc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0004e60 tx=0x7fe6f0004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.303+0000 7fe6ee7fc700 1 -- 172.21.15.94:0/3567254811 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f000e040 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.303+0000 7fe6ee7fc700 1 -- 172.21.15.94:0/3567254811 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6f001cb20 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.303+0000 7fe6ee7fc700 1 -- 172.21.15.94:0/3567254811 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f0012af0 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.304+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f8066290 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.304+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f8066850 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.307+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f804c710 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.307+0000 7fe6ee7fc700 1 -- 172.21.15.94:0/3567254811 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7fe6f001c650 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.308+0000 7fe6ee7fc700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe6e406b720 0x7fe6e406dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:55.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.308+0000 7fe6ee7fc700 1 -- 172.21.15.94:0/3567254811 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fe6f008c230 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.308+0000 7fe6fd838700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe6e406b720 0x7fe6e406dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:55.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.309+0000 7fe6fd838700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe6e406b720 0x7fe6e406dbd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009b80 tx=0x7fe6f400b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:55.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.317+0000 7fe6ee7fc700 1 -- 172.21.15.94:0/3567254811 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fe6f0059b30 con 0x7fe6f80fcfb0 2024-04-06T03:40:55.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:55 smithi094 ceph-mon[29713]: pgmap v27: 65 pgs: 65 active+clean; 4.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 180 MiB/s rd, 233 MiB/s wr, 345 op/s 2024-04-06T03:40:55.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:55 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:55.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:55 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:55.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:55 smithi136 ceph-mon[36436]: pgmap v27: 65 pgs: 65 active+clean; 4.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 180 MiB/s rd, 233 MiB/s wr, 345 op/s 2024-04-06T03:40:55.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:55 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:55.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:55 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:40:55.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.682+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6f8066420 con 0x7fe6e406b720 2024-04-06T03:40:55.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.684+0000 7fe6ee7fc700 1 -- 172.21.15.94:0/3567254811 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fe6f8066420 con 0x7fe6e406b720 2024-04-06T03:40:55.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe6e406b720 msgr2=0x7fe6e406dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:55.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe6e406b720 0x7fe6e406dbd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009b80 tx=0x7fe6f400b040 comp rx=0 tx=0).stop 2024-04-06T03:40:55.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f80fcfb0 msgr2=0x7fe6f810cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:55.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f80fcfb0 0x7fe6f810cdc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0004e60 tx=0x7fe6f0004d10 comp rx=0 tx=0).stop 2024-04-06T03:40:55.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 shutdown_connections 2024-04-06T03:40:55.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6f80fbd00 0x7fe6f810c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe6e406b720 0x7fe6e406dbd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 --2- 172.21.15.94:0/3567254811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f80fcfb0 0x7fe6f810cdc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 >> 172.21.15.94:0/3567254811 conn(0x7fe6f80f7480 msgr2=0x7fe6f80f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:55.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 shutdown_connections 2024-04-06T03:40:55.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.687+0000 7fe6ffa9c700 1 -- 172.21.15.94:0/3567254811 wait complete. 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.803+0000 7f52584fd700 1 -- 172.21.15.94:0/334447160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 msgr2=0x7f5250102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.803+0000 7f52584fd700 1 --2- 172.21.15.94:0/334447160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 0x7f5250102320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5240004940 tx=0x7f5240009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.803+0000 7f52584fd700 1 -- 172.21.15.94:0/334447160 shutdown_connections 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.803+0000 7f52584fd700 1 --2- 172.21.15.94:0/334447160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 0x7f5250102320 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.803+0000 7f52584fd700 1 --2- 172.21.15.94:0/334447160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52500662e0 0x7f52500ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.803+0000 7f52584fd700 1 -- 172.21.15.94:0/334447160 >> 172.21.15.94:0/334447160 conn(0x7f52500f7430 msgr2=0x7f52500f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.803+0000 7f52584fd700 1 -- 172.21.15.94:0/334447160 shutdown_connections 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.803+0000 7f52584fd700 1 -- 172.21.15.94:0/334447160 wait complete. 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f52584fd700 1 Processor -- start 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f52584fd700 1 -- start start 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f52584fd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52500662e0 0x7f525010da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:55.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f52584fd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 0x7f525010dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f52584fd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525010e5a0 con 0x7f52500fff00 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f52584fd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525010edf0 con 0x7f52500662e0 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f5255a98700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 0x7f525010dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f5255a98700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 0x7f525010dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59896/0 (socket says 172.21.15.94:59896) 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f5255a98700 1 -- 172.21.15.94:0/619690628 learned_addr learned my addr 172.21.15.94:0/619690628 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.804+0000 7f5256299700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52500662e0 0x7f525010da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.805+0000 7f5255a98700 1 -- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52500662e0 msgr2=0x7f525010da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.805+0000 7f5255a98700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52500662e0 0x7f525010da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.805+0000 7f5255a98700 1 -- 172.21.15.94:0/619690628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5240005040 con 0x7f52500fff00 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.805+0000 7f5255a98700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 0x7f525010dfa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f524000b3b0 tx=0x7f5240009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.805+0000 7f5246ffd700 1 -- 172.21.15.94:0/619690628 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f524000e040 con 0x7f52500fff00 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.805+0000 7f5246ffd700 1 -- 172.21.15.94:0/619690628 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f524000bbf0 con 0x7f52500fff00 2024-04-06T03:40:55.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.805+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525010f070 con 0x7f52500fff00 2024-04-06T03:40:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.806+0000 7f5246ffd700 1 -- 172.21.15.94:0/619690628 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f524001c630 con 0x7f52500fff00 2024-04-06T03:40:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.806+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525010f5b0 con 0x7f52500fff00 2024-04-06T03:40:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.807+0000 7f5246ffd700 1 -- 172.21.15.94:0/619690628 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f52400123c0 con 0x7f52500fff00 2024-04-06T03:40:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.807+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525005f5f0 con 0x7f52500fff00 2024-04-06T03:40:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.807+0000 7f5246ffd700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f523c06b540 0x7f523c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.807+0000 7f5246ffd700 1 -- 172.21.15.94:0/619690628 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f524008ccf0 con 0x7f52500fff00 2024-04-06T03:40:55.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.807+0000 7f5256299700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f523c06b540 0x7f523c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:55.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.808+0000 7f5256299700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f523c06b540 0x7f523c06d9f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f524c0041d0 tx=0x7f524c0043c0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:55.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:55.816+0000 7f5246ffd700 1 -- 172.21.15.94:0/619690628 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f5240090060 con 0x7f52500fff00 2024-04-06T03:40:56.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.139+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f525010f930 con 0x7f523c06b540 2024-04-06T03:40:56.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.149+0000 7f5246ffd700 1 -- 172.21.15.94:0/619690628 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f525010f930 con 0x7f523c06b540 2024-04-06T03:40:56.152 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:40:56.152 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 starting - - - - 2024-04-06T03:40:56.152 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (8m) 29s ago 8m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (7m) 41s ago 7m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (6m) 29s ago 8m 86.4M - 9.4.7 954c08fa6188 78ad0d4cf52b 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (2m) 29s ago 2m 12.9M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (3m) 29s ago 3m 157M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (3m) 41s ago 3m 13.0M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (2m) 41s ago 2m 15.4M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:9283 running (9m) 29s ago 9m 337M - 17.2.7-819-g6699495c ec300eaa768b d83dfb730a39 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283 running (70s) 41s ago 7m 488M - 19.0.0-2027-g9f192350 a5511c7deba9 7f3bea4f4172 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (9m) 29s ago 9m 48.7M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (7m) 41s ago 7m 38.5M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 starting - - - - 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (7m) 41s ago 7m 17.2M - 1.5.0 0da6a335fe13 e5324db1ea82 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (6m) 29s ago 6m 104M 1557M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:40:56.153 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (6m) 29s ago 6m 113M 1557M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:40:56.154 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (5m) 29s ago 5m 102M 1557M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:40:56.154 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (5m) 41s ago 5m 111M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:40:56.154 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (4m) 41s ago 4m 106M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:40:56.154 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (4m) 41s ago 4m 104M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:40:56.154 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (6m) 29s ago 7m 55.7M - 2.43.0 a07b618ecd1d 82a078c9e43c 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f523c06b540 msgr2=0x7f523c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f523c06b540 0x7f523c06d9f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f524c0041d0 tx=0x7f524c0043c0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 msgr2=0x7f525010dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 0x7f525010dfa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f524000b3b0 tx=0x7f5240009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 shutdown_connections 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f523c06b540 0x7f523c06d9f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52500662e0 0x7f525010da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 --2- 172.21.15.94:0/619690628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52500fff00 0x7f525010dfa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 >> 172.21.15.94:0/619690628 conn(0x7f52500f7430 msgr2=0x7f52500f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.151+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 shutdown_connections 2024-04-06T03:40:56.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.152+0000 7f52584fd700 1 -- 172.21.15.94:0/619690628 wait complete. 2024-04-06T03:40:56.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.317+0000 7f07083f3700 1 -- 172.21.15.94:0/3724988181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006fda0 msgr2=0x7f070006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:56.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.317+0000 7f07083f3700 1 --2- 172.21.15.94:0/3724988181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006fda0 0x7f070006e2b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f06f0004940 tx=0x7f06f0009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:56.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.318+0000 7f07083f3700 1 -- 172.21.15.94:0/3724988181 shutdown_connections 2024-04-06T03:40:56.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.318+0000 7f07083f3700 1 --2- 172.21.15.94:0/3724988181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070006e7f0 0x7f070006ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.318+0000 7f07083f3700 1 --2- 172.21.15.94:0/3724988181 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006fda0 0x7f070006e2b0 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f06f0004940 tx=0x7f06f0009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:56.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.318+0000 7f07083f3700 1 -- 172.21.15.94:0/3724988181 >> 172.21.15.94:0/3724988181 conn(0x7f0700069de0 msgr2=0x7f070006c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:56.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.321+0000 7f07083f3700 1 -- 172.21.15.94:0/3724988181 shutdown_connections 2024-04-06T03:40:56.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.321+0000 7f07083f3700 1 -- 172.21.15.94:0/3724988181 wait complete. 2024-04-06T03:40:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.321+0000 7f07083f3700 1 Processor -- start 2024-04-06T03:40:56.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.322+0000 7f07083f3700 1 -- start start 2024-04-06T03:40:56.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.322+0000 7f07083f3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006e7f0 0x7f0700080960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:56.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.322+0000 7f07083f3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070007f010 0x7f070007f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:56.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.322+0000 7f07083f3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0700080ef0 con 0x7f070006e7f0 2024-04-06T03:40:56.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.322+0000 7f07083f3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0700081030 con 0x7f070007f010 2024-04-06T03:40:56.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.323+0000 7f070598e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070007f010 0x7f070007f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:56.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.323+0000 7f070618f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006e7f0 0x7f0700080960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:56.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.323+0000 7f070618f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006e7f0 0x7f0700080960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59922/0 (socket says 172.21.15.94:59922) 2024-04-06T03:40:56.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.323+0000 7f070618f700 1 -- 172.21.15.94:0/2980618169 learned_addr learned my addr 172.21.15.94:0/2980618169 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:56.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.323+0000 7f070618f700 1 -- 172.21.15.94:0/2980618169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070007f010 msgr2=0x7f070007f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:56.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.323+0000 7f070618f700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070007f010 0x7f070007f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.323+0000 7f070618f700 1 -- 172.21.15.94:0/2980618169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f0005040 con 0x7f070006e7f0 2024-04-06T03:40:56.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.323+0000 7f070618f700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006e7f0 0x7f0700080960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f06f0004940 tx=0x7f06f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:56.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.326+0000 7f06f6ffd700 1 -- 172.21.15.94:0/2980618169 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06f000e030 con 0x7f070006e7f0 2024-04-06T03:40:56.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.326+0000 7f06f6ffd700 1 -- 172.21.15.94:0/2980618169 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06f0007e40 con 0x7f070006e7f0 2024-04-06T03:40:56.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.326+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070007fa80 con 0x7f070006e7f0 2024-04-06T03:40:56.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.326+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0700133800 con 0x7f070006e7f0 2024-04-06T03:40:56.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.327+0000 7f06f6ffd700 1 -- 172.21.15.94:0/2980618169 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06f001c810 con 0x7f070006e7f0 2024-04-06T03:40:56.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.329+0000 7f06f6ffd700 1 -- 172.21.15.94:0/2980618169 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f06f001c970 con 0x7f070006e7f0 2024-04-06T03:40:56.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.329+0000 7f06f6ffd700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f06ec06b720 0x7f06ec06dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:56.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.329+0000 7f06f6ffd700 1 -- 172.21.15.94:0/2980618169 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f06f008b770 con 0x7f070006e7f0 2024-04-06T03:40:56.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.330+0000 7f070598e700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f06ec06b720 0x7f06ec06dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:56.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.330+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e40052e0 con 0x7f070006e7f0 2024-04-06T03:40:56.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.330+0000 7f070598e700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f06ec06b720 0x7f06ec06dbd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009fd0 tx=0x7f06fc009f60 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:56.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.337+0000 7f06f6ffd700 1 -- 172.21.15.94:0/2980618169 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f06f0058cc0 con 0x7f070006e7f0 2024-04-06T03:40:56.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:56 smithi094 ceph-mon[29713]: Reconfiguring grafana.smithi094 (dependencies changed)... 2024-04-06T03:40:56.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:56 smithi094 ceph-mon[29713]: Reconfiguring daemon grafana.smithi094 on smithi094 2024-04-06T03:40:56.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:56 smithi094 ceph-mon[29713]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:56.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:56 smithi136 ceph-mon[36436]: Reconfiguring grafana.smithi094 (dependencies changed)... 2024-04-06T03:40:56.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:56 smithi136 ceph-mon[36436]: Reconfiguring daemon grafana.smithi094 on smithi094 2024-04-06T03:40:56.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:56 smithi136 ceph-mon[36436]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:56.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.751+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f06e4005860 con 0x7f070006e7f0 2024-04-06T03:40:56.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.751+0000 7f06f6ffd700 1 -- 172.21.15.94:0/2980618169 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f06f0058cc0 con 0x7f070006e7f0 2024-04-06T03:40:56.754 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:40:56.754 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 1, 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 1 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 13, 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 1 2024-04-06T03:40:56.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:40:56.756 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:40:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.754+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f06ec06b720 msgr2=0x7f06ec06dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.754+0000 7f07083f3700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f06ec06b720 0x7f06ec06dbd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009fd0 tx=0x7f06fc009f60 comp rx=0 tx=0).stop 2024-04-06T03:40:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.754+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006e7f0 msgr2=0x7f0700080960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.754+0000 7f07083f3700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006e7f0 0x7f0700080960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f06f0004940 tx=0x7f06f0009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.754+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 shutdown_connections 2024-04-06T03:40:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.754+0000 7f07083f3700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f070006e7f0 0x7f0700080960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.754+0000 7f07083f3700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f06ec06b720 0x7f06ec06dbd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.754+0000 7f07083f3700 1 --2- 172.21.15.94:0/2980618169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070007f010 0x7f070007f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.754+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 >> 172.21.15.94:0/2980618169 conn(0x7f0700069de0 msgr2=0x7f070006b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.755+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 shutdown_connections 2024-04-06T03:40:56.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.755+0000 7f07083f3700 1 -- 172.21.15.94:0/2980618169 wait complete. 2024-04-06T03:40:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.892+0000 7f76fd6f9700 1 -- 172.21.15.94:0/208886506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f80fff20 msgr2=0x7f76f8102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.892+0000 7f76fd6f9700 1 --2- 172.21.15.94:0/208886506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f80fff20 0x7f76f8102340 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f76ec004940 tx=0x7f76ec009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.892+0000 7f76fd6f9700 1 -- 172.21.15.94:0/208886506 shutdown_connections 2024-04-06T03:40:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.892+0000 7f76fd6f9700 1 --2- 172.21.15.94:0/208886506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f80fff20 0x7f76f8102340 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.892+0000 7f76fd6f9700 1 --2- 172.21.15.94:0/208886506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76f8066b70 0x7f76f80ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.892+0000 7f76fd6f9700 1 -- 172.21.15.94:0/208886506 >> 172.21.15.94:0/208886506 conn(0x7f76f80f7450 msgr2=0x7f76f80f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:56.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.892+0000 7f76fd6f9700 1 -- 172.21.15.94:0/208886506 shutdown_connections 2024-04-06T03:40:56.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.892+0000 7f76fd6f9700 1 -- 172.21.15.94:0/208886506 wait complete. 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.893+0000 7f76fd6f9700 1 Processor -- start 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.893+0000 7f76fd6f9700 1 -- start start 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.893+0000 7f76fd6f9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76f8066b70 0x7f76f810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.893+0000 7f76fd6f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f80fff20 0x7f76f810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.893+0000 7f76fd6f9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f810e570 con 0x7f76f80fff20 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.893+0000 7f76fd6f9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f810ed70 con 0x7f76f8066b70 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.894+0000 7f76f6ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76f8066b70 0x7f76f810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.894+0000 7f76f6ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76f8066b70 0x7f76f810da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:53214/0 (socket says 172.21.15.94:53214) 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.894+0000 7f76f6ffd700 1 -- 172.21.15.94:0/1960050036 learned_addr learned my addr 172.21.15.94:0/1960050036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:56.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.894+0000 7f76f67fc700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f80fff20 0x7f76f810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.894+0000 7f76f6ffd700 1 -- 172.21.15.94:0/1960050036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f80fff20 msgr2=0x7f76f810df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.894+0000 7f76f6ffd700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f80fff20 0x7f76f810df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.894+0000 7f76f6ffd700 1 -- 172.21.15.94:0/1960050036 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec005040 con 0x7f76f8066b70 2024-04-06T03:40:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.894+0000 7f76f6ffd700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76f8066b70 0x7f76f810da30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f76e800eb40 tx=0x7f76e800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:56.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.895+0000 7f76df7fe700 1 -- 172.21.15.94:0/1960050036 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76e8007cc0 con 0x7f76f8066b70 2024-04-06T03:40:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.895+0000 7f76df7fe700 1 -- 172.21.15.94:0/1960050036 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76e800f040 con 0x7f76f8066b70 2024-04-06T03:40:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.895+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f810eff0 con 0x7f76f8066b70 2024-04-06T03:40:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.895+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f810f530 con 0x7f76f8066b70 2024-04-06T03:40:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.895+0000 7f76df7fe700 1 -- 172.21.15.94:0/1960050036 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76e80095e0 con 0x7f76f8066b70 2024-04-06T03:40:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.896+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e40052e0 con 0x7f76f8066b70 2024-04-06T03:40:56.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.897+0000 7f76df7fe700 1 -- 172.21.15.94:0/1960050036 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f76e8009740 con 0x7f76f8066b70 2024-04-06T03:40:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.897+0000 7f76df7fe700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f76e006b540 0x7f76e006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.897+0000 7f76df7fe700 1 -- 172.21.15.94:0/1960050036 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f76e8089af0 con 0x7f76f8066b70 2024-04-06T03:40:56.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.897+0000 7f76f67fc700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f76e006b540 0x7f76e006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:56.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.898+0000 7f76f67fc700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f76e006b540 0x7f76e006d9f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00b0d0 tx=0x7f76ec009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:56.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:56.902+0000 7f76df7fe700 1 -- 172.21.15.94:0/1960050036 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f76e804fb80 con 0x7f76f8066b70 2024-04-06T03:40:57.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.396+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f76e4005860 con 0x7f76f8066b70 2024-04-06T03:40:57.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.398+0000 7f76df7fe700 1 -- 172.21.15.94:0/1960050036 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7f76e8019720 con 0x7f76f8066b70 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:40:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:40:57.402 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:40:57.403 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:40:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.401+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f76e006b540 msgr2=0x7f76e006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.401+0000 7f76fd6f9700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f76e006b540 0x7f76e006d9f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00b0d0 tx=0x7f76ec009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.401+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76f8066b70 msgr2=0x7f76f810da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:57.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.401+0000 7f76fd6f9700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76f8066b70 0x7f76f810da30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f76e800eb40 tx=0x7f76e800b040 comp rx=0 tx=0).stop 2024-04-06T03:40:57.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.402+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 shutdown_connections 2024-04-06T03:40:57.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.402+0000 7f76fd6f9700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f76e006b540 0x7f76e006d9f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:57.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.402+0000 7f76fd6f9700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76f8066b70 0x7f76f810da30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:57.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.402+0000 7f76fd6f9700 1 --2- 172.21.15.94:0/1960050036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f80fff20 0x7f76f810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:57.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.402+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 >> 172.21.15.94:0/1960050036 conn(0x7f76f80f7450 msgr2=0x7f76f80f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:57.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.403+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 shutdown_connections 2024-04-06T03:40:57.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.403+0000 7f76fd6f9700 1 -- 172.21.15.94:0/1960050036 wait complete. 2024-04-06T03:40:57.407 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:40:57.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:57 smithi094 ceph-mon[29713]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:57.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:57 smithi094 ceph-mon[29713]: pgmap v28: 65 pgs: 65 active+clean; 4.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 180 MiB/s rd, 233 MiB/s wr, 346 op/s 2024-04-06T03:40:57.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:57 smithi094 ceph-mon[29713]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:57.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:57 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2980618169' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:40:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.551+0000 7f63aed12700 1 -- 172.21.15.94:0/2933402359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a80fff70 msgr2=0x7f63a8102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.551+0000 7f63aed12700 1 --2- 172.21.15.94:0/2933402359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a80fff70 0x7f63a8102350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6394004940 tx=0x7f6394009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.551+0000 7f63aed12700 1 -- 172.21.15.94:0/2933402359 shutdown_connections 2024-04-06T03:40:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.551+0000 7f63aed12700 1 --2- 172.21.15.94:0/2933402359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a80fff70 0x7f63a8102350 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.551+0000 7f63aed12700 1 --2- 172.21.15.94:0/2933402359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8066b10 0x7f63a80ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.551+0000 7f63aed12700 1 -- 172.21.15.94:0/2933402359 >> 172.21.15.94:0/2933402359 conn(0x7f63a80f7440 msgr2=0x7f63a80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.552+0000 7f63aed12700 1 -- 172.21.15.94:0/2933402359 shutdown_connections 2024-04-06T03:40:57.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.552+0000 7f63aed12700 1 -- 172.21.15.94:0/2933402359 wait complete. 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.552+0000 7f63aed12700 1 Processor -- start 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.552+0000 7f63aed12700 1 -- start start 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63aed12700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a8066b10 0x7f63a810da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63aed12700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a80fff70 0x7f63a810df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63aed12700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a810e560 con 0x7f63a8066b10 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63aed12700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a810ed60 con 0x7f63a80fff70 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63acaae700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a8066b10 0x7f63a810da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63acaae700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a8066b10 0x7f63a810da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38874/0 (socket says 172.21.15.94:38874) 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63acaae700 1 -- 172.21.15.94:0/2814552794 learned_addr learned my addr 172.21.15.94:0/2814552794 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:57.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63a7fff700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a80fff70 0x7f63a810df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63acaae700 1 -- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a80fff70 msgr2=0x7f63a810df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63acaae700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a80fff70 0x7f63a810df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.553+0000 7f63acaae700 1 -- 172.21.15.94:0/2814552794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394005040 con 0x7f63a8066b10 2024-04-06T03:40:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.554+0000 7f63acaae700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a8066b10 0x7f63a810da20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f639c00eb70 tx=0x7f639c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.554+0000 7f63a57fa700 1 -- 172.21.15.94:0/2814552794 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f639c007cc0 con 0x7f63a8066b10 2024-04-06T03:40:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.554+0000 7f63a57fa700 1 -- 172.21.15.94:0/2814552794 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f639c00f040 con 0x7f63a8066b10 2024-04-06T03:40:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.554+0000 7f63a57fa700 1 -- 172.21.15.94:0/2814552794 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f639c00cb20 con 0x7f63a8066b10 2024-04-06T03:40:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.554+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a810efe0 con 0x7f63a8066b10 2024-04-06T03:40:57.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.554+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a810f4d0 con 0x7f63a8066b10 2024-04-06T03:40:57.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.555+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a805f5f0 con 0x7f63a8066b10 2024-04-06T03:40:57.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.556+0000 7f63a57fa700 1 -- 172.21.15.94:0/2814552794 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f639c00cc80 con 0x7f63a8066b10 2024-04-06T03:40:57.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.556+0000 7f63a57fa700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f639806b540 0x7f639806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:57.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.556+0000 7f63a57fa700 1 -- 172.21.15.94:0/2814552794 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f639c089af0 con 0x7f63a8066b10 2024-04-06T03:40:57.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.556+0000 7f63a7fff700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f639806b540 0x7f639806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:57.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.557+0000 7f63a7fff700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f639806b540 0x7f639806d9f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f639400b0d0 tx=0x7f6394009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:57.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.561+0000 7f63a57fa700 1 -- 172.21.15.94:0/2814552794 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f639c04fb80 con 0x7f63a8066b10 2024-04-06T03:40:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:57 smithi136 ceph-mon[36436]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:57 smithi136 ceph-mon[36436]: pgmap v28: 65 pgs: 65 active+clean; 4.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 180 MiB/s rd, 233 MiB/s wr, 346 op/s 2024-04-06T03:40:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:57 smithi136 ceph-mon[36436]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:57 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2980618169' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:40:57.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.888+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63a810f8b0 con 0x7f639806b540 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.889+0000 7f63a57fa700 1 -- 172.21.15.94:0/2814552794 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f63a810f8b0 con 0x7f639806b540 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "1/21 daemons upgraded", 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "", 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:40:57.892 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:40:57.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f639806b540 msgr2=0x7f639806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:57.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f639806b540 0x7f639806d9f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f639400b0d0 tx=0x7f6394009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:57.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a8066b10 msgr2=0x7f63a810da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:57.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a8066b10 0x7f63a810da20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f639c00eb70 tx=0x7f639c00b040 comp rx=0 tx=0).stop 2024-04-06T03:40:57.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 shutdown_connections 2024-04-06T03:40:57.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63a8066b10 0x7f63a810da20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:57.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f639806b540 0x7f639806d9f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:57.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 --2- 172.21.15.94:0/2814552794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a80fff70 0x7f63a810df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:57.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 >> 172.21.15.94:0/2814552794 conn(0x7f63a80f7440 msgr2=0x7f63a80f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:57.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 shutdown_connections 2024-04-06T03:40:57.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:57.892+0000 7f63aed12700 1 -- 172.21.15.94:0/2814552794 wait complete. 2024-04-06T03:40:58.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.041+0000 7f7d2793c700 1 -- 172.21.15.94:0/889522642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 msgr2=0x7f7d201026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.041+0000 7f7d2793c700 1 --2- 172.21.15.94:0/889522642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 0x7f7d201026c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7d10004940 tx=0x7f7d10009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.041+0000 7f7d2793c700 1 -- 172.21.15.94:0/889522642 shutdown_connections 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.041+0000 7f7d2793c700 1 --2- 172.21.15.94:0/889522642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 0x7f7d201026c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.041+0000 7f7d2793c700 1 --2- 172.21.15.94:0/889522642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d200fd9c0 0x7f7d200ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.041+0000 7f7d2793c700 1 -- 172.21.15.94:0/889522642 >> 172.21.15.94:0/889522642 conn(0x7f7d200f7480 msgr2=0x7f7d200f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.041+0000 7f7d2793c700 1 -- 172.21.15.94:0/889522642 shutdown_connections 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.041+0000 7f7d2793c700 1 -- 172.21.15.94:0/889522642 wait complete. 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d2793c700 1 Processor -- start 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d2793c700 1 -- start start 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d2793c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d200fd9c0 0x7f7d2010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d2793c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 0x7f7d2010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:58.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d2793c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2010e570 con 0x7f7d201002e0 2024-04-06T03:40:58.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d2793c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2010edc0 con 0x7f7d200fd9c0 2024-04-06T03:40:58.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d24ed7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 0x7f7d2010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:58.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d24ed7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 0x7f7d2010df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38904/0 (socket says 172.21.15.94:38904) 2024-04-06T03:40:58.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d24ed7700 1 -- 172.21.15.94:0/373892717 learned_addr learned my addr 172.21.15.94:0/373892717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:40:58.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d256d8700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d200fd9c0 0x7f7d2010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:58.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d24ed7700 1 -- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d200fd9c0 msgr2=0x7f7d2010da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:58.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d24ed7700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d200fd9c0 0x7f7d2010da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:58.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.042+0000 7f7d24ed7700 1 -- 172.21.15.94:0/373892717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10005040 con 0x7f7d201002e0 2024-04-06T03:40:58.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.043+0000 7f7d24ed7700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 0x7f7d2010df70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000b3b0 tx=0x7f7d10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:58.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.043+0000 7f7d15ffb700 1 -- 172.21.15.94:0/373892717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d1000e040 con 0x7f7d201002e0 2024-04-06T03:40:58.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.043+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2010f040 con 0x7f7d201002e0 2024-04-06T03:40:58.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.043+0000 7f7d15ffb700 1 -- 172.21.15.94:0/373892717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d1000bcc0 con 0x7f7d201002e0 2024-04-06T03:40:58.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.043+0000 7f7d15ffb700 1 -- 172.21.15.94:0/373892717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d1001c630 con 0x7f7d201002e0 2024-04-06T03:40:58.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.044+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2010f580 con 0x7f7d201002e0 2024-04-06T03:40:58.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.044+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2005f5f0 con 0x7f7d201002e0 2024-04-06T03:40:58.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.045+0000 7f7d15ffb700 1 -- 172.21.15.94:0/373892717 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f7d100139f0 con 0x7f7d201002e0 2024-04-06T03:40:58.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.046+0000 7f7d15ffb700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7d0c06f950 0x7f7d0c071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:40:58.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.046+0000 7f7d15ffb700 1 -- 172.21.15.94:0/373892717 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f7d1008acf0 con 0x7f7d201002e0 2024-04-06T03:40:58.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.046+0000 7f7d256d8700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7d0c06f950 0x7f7d0c071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:40:58.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.047+0000 7f7d256d8700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7d0c06f950 0x7f7d0c071e00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d20066960 tx=0x7f7d1c009ee0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:40:58.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.052+0000 7f7d15ffb700 1 -- 172.21.15.94:0/373892717 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f7d10058760 con 0x7f7d201002e0 2024-04-06T03:40:58.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.514+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7d20105990 con 0x7f7d201002e0 2024-04-06T03:40:58.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:58 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/1960050036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:40:58.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:58 smithi094 ceph-mon[29713]: from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:58.517 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:58 smithi094 ceph-mon[29713]: pgmap v29: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 182 MiB/s rd, 236 MiB/s wr, 438 op/s 2024-04-06T03:40:58.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.514+0000 7f7d15ffb700 1 -- 172.21.15.94:0/373892717 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7d100582f0 con 0x7f7d201002e0 2024-04-06T03:40:58.518 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-04-06T03:40:58.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7d0c06f950 msgr2=0x7f7d0c071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:58.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7d0c06f950 0x7f7d0c071e00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d20066960 tx=0x7f7d1c009ee0 comp rx=0 tx=0).stop 2024-04-06T03:40:58.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 msgr2=0x7f7d2010df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:40:58.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 0x7f7d2010df70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000b3b0 tx=0x7f7d10009e30 comp rx=0 tx=0).stop 2024-04-06T03:40:58.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 shutdown_connections 2024-04-06T03:40:58.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7d0c06f950 0x7f7d0c071e00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:58.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d200fd9c0 0x7f7d2010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:58.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 --2- 172.21.15.94:0/373892717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d201002e0 0x7f7d2010df70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:40:58.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 >> 172.21.15.94:0/373892717 conn(0x7f7d200f7480 msgr2=0x7f7d200f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:40:58.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 shutdown_connections 2024-04-06T03:40:58.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:40:58.517+0000 7f7d2793c700 1 -- 172.21.15.94:0/373892717 wait complete. 2024-04-06T03:40:58.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:58 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/1960050036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:40:58.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:58 smithi136 ceph-mon[36436]: from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:40:58.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:58 smithi136 ceph-mon[36436]: pgmap v29: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 182 MiB/s rd, 236 MiB/s wr, 438 op/s 2024-04-06T03:40:59.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:40:59 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/373892717' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:40:59.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:40:59 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/373892717' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:41:00.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:00 smithi136 ceph-mon[36436]: pgmap v30: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 78 MiB/s rd, 104 MiB/s wr, 247 op/s 2024-04-06T03:41:00.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:00 smithi094 ceph-mon[29713]: pgmap v30: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 78 MiB/s rd, 104 MiB/s wr, 247 op/s 2024-04-06T03:41:03.356 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:03 smithi094 ceph-mon[29713]: pgmap v31: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 78 MiB/s rd, 105 MiB/s wr, 324 op/s 2024-04-06T03:41:03.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:03 smithi136 ceph-mon[36436]: pgmap v31: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 78 MiB/s rd, 105 MiB/s wr, 324 op/s 2024-04-06T03:41:05.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:05 smithi094 ceph-mon[29713]: pgmap v32: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-04-06T03:41:05.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:05 smithi136 ceph-mon[36436]: pgmap v32: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-04-06T03:41:06.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:06 smithi094 ceph-mon[29713]: pgmap v33: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-04-06T03:41:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:06 smithi136 ceph-mon[36436]: pgmap v33: 65 pgs: 65 active+clean; 2.4 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.2 MiB/s wr, 170 op/s 2024-04-06T03:41:07.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:07 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:41:07.505 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-06T03:41:07.505 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-06T03:41:07.571 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:07 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:41:08.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:08 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:08.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:08 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:08.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:08 smithi136 ceph-mon[36436]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-04-06T03:41:08.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:08 smithi136 ceph-mon[36436]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-04-06T03:41:08.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:08 smithi136 ceph-mon[36436]: pgmap v34: 65 pgs: 65 active+clean; 1.2 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-04-06T03:41:08.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:08 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:08.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:08 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:08.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:08 smithi094 ceph-mon[29713]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-04-06T03:41:08.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:08 smithi094 ceph-mon[29713]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-04-06T03:41:08.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:08 smithi094 ceph-mon[29713]: pgmap v34: 65 pgs: 65 active+clean; 1.2 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 249 op/s 2024-04-06T03:41:10.675 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-06T03:41:10.676 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-06T03:41:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:11 smithi094 ceph-mon[29713]: pgmap v35: 65 pgs: 65 active+clean; 1.2 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-06T03:41:11.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:11 smithi136 ceph-mon[36436]: pgmap v35: 65 pgs: 65 active+clean; 1.2 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-06T03:41:13.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:13 smithi094 ceph-mon[29713]: pgmap v36: 65 pgs: 65 active+clean; 293 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.8 MiB/s wr, 270 op/s 2024-04-06T03:41:13.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:13 smithi136 ceph-mon[36436]: pgmap v36: 65 pgs: 65 active+clean; 293 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.8 MiB/s wr, 270 op/s 2024-04-06T03:41:14.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:14 smithi136 ceph-mon[36436]: pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-04-06T03:41:14.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:14 smithi094 ceph-mon[29713]: pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-04-06T03:41:17.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:17 smithi094 ceph-mon[29713]: pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-04-06T03:41:17.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:17 smithi136 ceph-mon[36436]: pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-04-06T03:41:19.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:19 smithi094 ceph-mon[29713]: pgmap v39: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 335 op/s 2024-04-06T03:41:19.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:19 smithi136 ceph-mon[36436]: pgmap v39: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 2.0 MiB/s wr, 335 op/s 2024-04-06T03:41:20.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:20 smithi136 ceph-mon[36436]: pgmap v40: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-04-06T03:41:20.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:20 smithi094 ceph-mon[29713]: pgmap v40: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-04-06T03:41:21.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:21 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:41:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:21 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:41:22.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:22 smithi136 ceph-mon[36436]: pgmap v41: 65 pgs: 65 active+clean; 296 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-04-06T03:41:22.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:22 smithi094 ceph-mon[29713]: pgmap v41: 65 pgs: 65 active+clean; 296 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-04-06T03:41:25.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:24 smithi136 ceph-mon[36436]: pgmap v42: 65 pgs: 65 active+clean; 296 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-06T03:41:25.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:24 smithi094 ceph-mon[29713]: pgmap v42: 65 pgs: 65 active+clean; 296 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-06T03:41:27.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:27 smithi094 ceph-mon[29713]: pgmap v43: 65 pgs: 65 active+clean; 296 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-06T03:41:27.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:27 smithi136 ceph-mon[36436]: pgmap v43: 65 pgs: 65 active+clean; 296 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-06T03:41:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.650+0000 7fe609824700 1 -- 172.21.15.94:0/3425793623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040fce80 msgr2=0x7fe6040fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.650+0000 7fe609824700 1 --2- 172.21.15.94:0/3425793623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040fce80 0x7fe6040fd290 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4004940 tx=0x7fe5f4009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 -- 172.21.15.94:0/3425793623 shutdown_connections 2024-04-06T03:41:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 --2- 172.21.15.94:0/3425793623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6040fdff0 0x7fe6040fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 --2- 172.21.15.94:0/3425793623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040fce80 0x7fe6040fd290 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:28.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 -- 172.21.15.94:0/3425793623 >> 172.21.15.94:0/3425793623 conn(0x7fe6040f8580 msgr2=0x7fe6040fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 -- 172.21.15.94:0/3425793623 shutdown_connections 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 -- 172.21.15.94:0/3425793623 wait complete. 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 Processor -- start 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 -- start start 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6040fce80 0x7fe60410da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040fdff0 0x7fe60410dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60410e630 con 0x7fe6040fdff0 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.651+0000 7fe609824700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60410ee20 con 0x7fe6040fce80 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe602ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6040fce80 0x7fe60410da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe6027fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040fdff0 0x7fe60410dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe602ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6040fce80 0x7fe60410da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:49058/0 (socket says 172.21.15.94:49058) 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe602ffd700 1 -- 172.21.15.94:0/409997662 learned_addr learned my addr 172.21.15.94:0/409997662 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe602ffd700 1 -- 172.21.15.94:0/409997662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040fdff0 msgr2=0x7fe60410dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe602ffd700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040fdff0 0x7fe60410dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe602ffd700 1 -- 172.21.15.94:0/409997662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4005040 con 0x7fe6040fce80 2024-04-06T03:41:28.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe602ffd700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6040fce80 0x7fe60410da60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400b3b0 tx=0x7fe5f4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:28.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe5eb7fe700 1 -- 172.21.15.94:0/409997662 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5f400e040 con 0x7fe6040fce80 2024-04-06T03:41:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60410f0a0 con 0x7fe6040fce80 2024-04-06T03:41:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe5eb7fe700 1 -- 172.21.15.94:0/409997662 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5f400bbf0 con 0x7fe6040fce80 2024-04-06T03:41:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.652+0000 7fe5eb7fe700 1 -- 172.21.15.94:0/409997662 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5f401c630 con 0x7fe6040fce80 2024-04-06T03:41:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.653+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60410f610 con 0x7fe6040fce80 2024-04-06T03:41:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.654+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60405f5f0 con 0x7fe6040fce80 2024-04-06T03:41:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.654+0000 7fe5eb7fe700 1 -- 172.21.15.94:0/409997662 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7fe5f4026440 con 0x7fe6040fce80 2024-04-06T03:41:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.655+0000 7fe5eb7fe700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe5ec06b540 0x7fe5ec06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.655+0000 7fe5eb7fe700 1 -- 172.21.15.94:0/409997662 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fe5f408acd0 con 0x7fe6040fce80 2024-04-06T03:41:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.655+0000 7fe6027fc700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe5ec06b540 0x7fe5ec06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.656+0000 7fe6027fc700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe5ec06b540 0x7fe5ec06d9f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009bf0 tx=0x7fe5f80091a0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:28.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.660+0000 7fe5eb7fe700 1 -- 172.21.15.94:0/409997662 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fe5f4058780 con 0x7fe6040fce80 2024-04-06T03:41:28.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.979+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe60410f8f0 con 0x7fe5ec06b540 2024-04-06T03:41:28.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.981+0000 7fe5eb7fe700 1 -- 172.21.15.94:0/409997662 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fe60410f8f0 con 0x7fe5ec06b540 2024-04-06T03:41:28.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe5ec06b540 msgr2=0x7fe5ec06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:28.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe5ec06b540 0x7fe5ec06d9f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009bf0 tx=0x7fe5f80091a0 comp rx=0 tx=0).stop 2024-04-06T03:41:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6040fce80 msgr2=0x7fe60410da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6040fce80 0x7fe60410da60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400b3b0 tx=0x7fe5f4009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 shutdown_connections 2024-04-06T03:41:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe5ec06b540 0x7fe5ec06d9f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6040fce80 0x7fe60410da60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 --2- 172.21.15.94:0/409997662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6040fdff0 0x7fe60410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 >> 172.21.15.94:0/409997662 conn(0x7fe6040f8580 msgr2=0x7fe6040f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 shutdown_connections 2024-04-06T03:41:28.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:28.985+0000 7fe609824700 1 -- 172.21.15.94:0/409997662 wait complete. 2024-04-06T03:41:28.999 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.125+0000 7efe81249700 1 -- 172.21.15.94:0/2283043637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 msgr2=0x7efe7c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.125+0000 7efe81249700 1 --2- 172.21.15.94:0/2283043637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 0x7efe7c102320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efe70004940 tx=0x7efe70009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.125+0000 7efe81249700 1 -- 172.21.15.94:0/2283043637 shutdown_connections 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.125+0000 7efe81249700 1 --2- 172.21.15.94:0/2283043637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 0x7efe7c102320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.125+0000 7efe81249700 1 --2- 172.21.15.94:0/2283043637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe7c0662e0 0x7efe7c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.125+0000 7efe81249700 1 -- 172.21.15.94:0/2283043637 >> 172.21.15.94:0/2283043637 conn(0x7efe7c0f7430 msgr2=0x7efe7c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.125+0000 7efe81249700 1 -- 172.21.15.94:0/2283043637 shutdown_connections 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.125+0000 7efe81249700 1 -- 172.21.15.94:0/2283043637 wait complete. 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe81249700 1 Processor -- start 2024-04-06T03:41:29.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe81249700 1 -- start start 2024-04-06T03:41:29.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe81249700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe7c0662e0 0x7efe7c10da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:29.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe81249700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 0x7efe7c10dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:29.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe81249700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c10e5a0 con 0x7efe7c0fff00 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe81249700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c10edf0 con 0x7efe7c0662e0 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe7a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 0x7efe7c10dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe7a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 0x7efe7c10dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45280/0 (socket says 172.21.15.94:45280) 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe7a59c700 1 -- 172.21.15.94:0/406482221 learned_addr learned my addr 172.21.15.94:0/406482221 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe7ad9d700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe7c0662e0 0x7efe7c10da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe7a59c700 1 -- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe7c0662e0 msgr2=0x7efe7c10da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.126+0000 7efe7a59c700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe7c0662e0 0x7efe7c10da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.127+0000 7efe7a59c700 1 -- 172.21.15.94:0/406482221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe70005040 con 0x7efe7c0fff00 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.127+0000 7efe7a59c700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 0x7efe7c10dfa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efe7000b970 tx=0x7efe70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.127+0000 7efe637fe700 1 -- 172.21.15.94:0/406482221 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe7000e040 con 0x7efe7c0fff00 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.127+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c10f070 con 0x7efe7c0fff00 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.127+0000 7efe637fe700 1 -- 172.21.15.94:0/406482221 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe7000bbf0 con 0x7efe7c0fff00 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.127+0000 7efe637fe700 1 -- 172.21.15.94:0/406482221 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe7001c630 con 0x7efe7c0fff00 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.127+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c10f5b0 con 0x7efe7c0fff00 2024-04-06T03:41:29.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.128+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c05f5f0 con 0x7efe7c0fff00 2024-04-06T03:41:29.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.129+0000 7efe637fe700 1 -- 172.21.15.94:0/406482221 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7efe7000bd60 con 0x7efe7c0fff00 2024-04-06T03:41:29.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.129+0000 7efe637fe700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7efe6406b540 0x7efe6406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:29.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.129+0000 7efe637fe700 1 -- 172.21.15.94:0/406482221 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7efe7008b420 con 0x7efe7c0fff00 2024-04-06T03:41:29.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.129+0000 7efe7ad9d700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7efe6406b540 0x7efe6406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:29.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.130+0000 7efe7ad9d700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7efe6406b540 0x7efe6406d9f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe7c066020 tx=0x7efe6c00b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:29.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.134+0000 7efe637fe700 1 -- 172.21.15.94:0/406482221 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7efe70058d40 con 0x7efe7c0fff00 2024-04-06T03:41:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.355+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe7c10f890 con 0x7efe6406b540 2024-04-06T03:41:29.359 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:29 smithi094 ceph-mon[29713]: pgmap v44: 65 pgs: 65 active+clean; 302 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-04-06T03:41:29.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.357+0000 7efe637fe700 1 -- 172.21.15.94:0/406482221 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7efe7c10f890 con 0x7efe6406b540 2024-04-06T03:41:29.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7efe6406b540 msgr2=0x7efe6406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7efe6406b540 0x7efe6406d9f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe7c066020 tx=0x7efe6c00b040 comp rx=0 tx=0).stop 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 msgr2=0x7efe7c10dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 0x7efe7c10dfa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efe7000b970 tx=0x7efe70009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 shutdown_connections 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7efe6406b540 0x7efe6406d9f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe7c0662e0 0x7efe7c10da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 --2- 172.21.15.94:0/406482221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe7c0fff00 0x7efe7c10dfa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 >> 172.21.15.94:0/406482221 conn(0x7efe7c0f7430 msgr2=0x7efe7c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.359+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 shutdown_connections 2024-04-06T03:41:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.360+0000 7efe81249700 1 -- 172.21.15.94:0/406482221 wait complete. 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.475+0000 7f25568cb700 1 -- 172.21.15.94:0/3985674856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fdff0 msgr2=0x7f25500fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.475+0000 7f25568cb700 1 --2- 172.21.15.94:0/3985674856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fdff0 0x7f25500fe460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2544004940 tx=0x7f2544009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.475+0000 7f25568cb700 1 -- 172.21.15.94:0/3985674856 shutdown_connections 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.475+0000 7f25568cb700 1 --2- 172.21.15.94:0/3985674856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fdff0 0x7f25500fe460 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.475+0000 7f25568cb700 1 --2- 172.21.15.94:0/3985674856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25500fce80 0x7f25500fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.475+0000 7f25568cb700 1 -- 172.21.15.94:0/3985674856 >> 172.21.15.94:0/3985674856 conn(0x7f25500f8540 msgr2=0x7f25500fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.475+0000 7f25568cb700 1 -- 172.21.15.94:0/3985674856 shutdown_connections 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.475+0000 7f25568cb700 1 -- 172.21.15.94:0/3985674856 wait complete. 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.475+0000 7f25568cb700 1 Processor -- start 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f25568cb700 1 -- start start 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f25568cb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fce80 0x7f255010da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f25568cb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25500fdff0 0x7f255010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:29.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f25568cb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255010e560 con 0x7f25500fce80 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f25568cb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255010edb0 con 0x7f25500fdff0 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f254ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fce80 0x7f255010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f254ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fce80 0x7f255010da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45304/0 (socket says 172.21.15.94:45304) 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f254ffff700 1 -- 172.21.15.94:0/2130577532 learned_addr learned my addr 172.21.15.94:0/2130577532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f254f7fe700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25500fdff0 0x7f255010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f254ffff700 1 -- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25500fdff0 msgr2=0x7f255010df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f254ffff700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25500fdff0 0x7f255010df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f254ffff700 1 -- 172.21.15.94:0/2130577532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2544005040 con 0x7f25500fce80 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f254f7fe700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25500fdff0 0x7f255010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.476+0000 7f254ffff700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fce80 0x7f255010da20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2540002a50 tx=0x7f25400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.477+0000 7f254cff9700 1 -- 172.21.15.94:0/2130577532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2540007cc0 con 0x7f25500fce80 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.477+0000 7f254cff9700 1 -- 172.21.15.94:0/2130577532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f254000f040 con 0x7f25500fce80 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.477+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255010f090 con 0x7f25500fce80 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.477+0000 7f254cff9700 1 -- 172.21.15.94:0/2130577532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2540009640 con 0x7f25500fce80 2024-04-06T03:41:29.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.477+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255010f5d0 con 0x7f25500fce80 2024-04-06T03:41:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.478+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255005f5f0 con 0x7f25500fce80 2024-04-06T03:41:29.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.479+0000 7f254cff9700 1 -- 172.21.15.94:0/2130577532 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f25400097a0 con 0x7f25500fce80 2024-04-06T03:41:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.479+0000 7f254cff9700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f253806b540 0x7f253806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.479+0000 7f254cff9700 1 -- 172.21.15.94:0/2130577532 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f2540089f30 con 0x7f25500fce80 2024-04-06T03:41:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.479+0000 7f254f7fe700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f253806b540 0x7f253806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:29.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.480+0000 7f254f7fe700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f253806b540 0x7f253806d9f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f254400b970 tx=0x7f2544009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:29.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.487+0000 7f254cff9700 1 -- 172.21.15.94:0/2130577532 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f2540057840 con 0x7f25500fce80 2024-04-06T03:41:29.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:29 smithi136 ceph-mon[36436]: pgmap v44: 65 pgs: 65 active+clean; 302 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.5 MiB/s wr, 311 op/s 2024-04-06T03:41:29.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.715+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f255010f960 con 0x7f253806b540 2024-04-06T03:41:29.738 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 starting - - - - 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (9m) 63s ago 9m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (8m) 75s ago 8m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 starting - - - - 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (3m) 63s ago 3m 12.9M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (3m) 63s ago 3m 157M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (3m) 75s ago 3m 13.0M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (3m) 75s ago 3m 15.4M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:9283 running (10m) 63s ago 10m 337M - 17.2.7-819-g6699495c ec300eaa768b d83dfb730a39 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283 running (104s) 75s ago 7m 488M - 19.0.0-2027-g9f192350 a5511c7deba9 7f3bea4f4172 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (10m) 63s ago 10m 48.7M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (7m) 75s ago 7m 38.5M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 starting - - - - 2024-04-06T03:41:29.739 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (8m) 75s ago 8m 17.2M - 1.5.0 0da6a335fe13 e5324db1ea82 2024-04-06T03:41:29.740 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7m) 63s ago 7m 104M 1557M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:41:29.740 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (6m) 63s ago 6m 113M 1557M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:41:29.740 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (6m) 63s ago 6m 102M 1557M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:41:29.740 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (5m) 75s ago 5m 111M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:41:29.740 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (5m) 75s ago 5m 106M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:41:29.740 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (5m) 75s ago 4m 104M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:41:29.740 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (7m) 63s ago 8m 55.7M - 2.43.0 a07b618ecd1d 82a078c9e43c 2024-04-06T03:41:29.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.735+0000 7f254cff9700 1 -- 172.21.15.94:0/2130577532 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f255010f960 con 0x7f253806b540 2024-04-06T03:41:29.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.738+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f253806b540 msgr2=0x7f253806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.738+0000 7f25568cb700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f253806b540 0x7f253806d9f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f254400b970 tx=0x7f2544009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:29.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.738+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fce80 msgr2=0x7f255010da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.738+0000 7f25568cb700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fce80 0x7f255010da20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2540002a50 tx=0x7f25400046b0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.739+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 shutdown_connections 2024-04-06T03:41:29.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.739+0000 7f25568cb700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25500fce80 0x7f255010da20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.739+0000 7f25568cb700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f253806b540 0x7f253806d9f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.739+0000 7f25568cb700 1 --2- 172.21.15.94:0/2130577532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25500fdff0 0x7f255010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.739+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 >> 172.21.15.94:0/2130577532 conn(0x7f25500f8540 msgr2=0x7f25500f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:29.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.740+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 shutdown_connections 2024-04-06T03:41:29.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.740+0000 7f25568cb700 1 -- 172.21.15.94:0/2130577532 wait complete. 2024-04-06T03:41:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.849+0000 7f0a4a836700 1 -- 172.21.15.94:0/3817039856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 msgr2=0x7f0a440fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.849+0000 7f0a4a836700 1 --2- 172.21.15.94:0/3817039856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 0x7f0a440fc0f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0a34004940 tx=0x7f0a34009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.849+0000 7f0a4a836700 1 -- 172.21.15.94:0/3817039856 shutdown_connections 2024-04-06T03:41:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.849+0000 7f0a4a836700 1 --2- 172.21.15.94:0/3817039856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a440fcf90 0x7f0a440fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.849+0000 7f0a4a836700 1 --2- 172.21.15.94:0/3817039856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 0x7f0a440fc0f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.849+0000 7f0a4a836700 1 -- 172.21.15.94:0/3817039856 >> 172.21.15.94:0/3817039856 conn(0x7f0a440f7480 msgr2=0x7f0a440f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.849+0000 7f0a4a836700 1 -- 172.21.15.94:0/3817039856 shutdown_connections 2024-04-06T03:41:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.849+0000 7f0a4a836700 1 -- 172.21.15.94:0/3817039856 wait complete. 2024-04-06T03:41:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a4a836700 1 Processor -- start 2024-04-06T03:41:29.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a4a836700 1 -- start start 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a4a836700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 0x7f0a441106d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a4a836700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a440fcf90 0x7f0a44112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a4a836700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a44066c40 con 0x7f0a440fbce0 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a4a836700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a44066d80 con 0x7f0a440fcf90 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a43fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 0x7f0a441106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a43fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 0x7f0a441106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45336/0 (socket says 172.21.15.94:45336) 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a43fff700 1 -- 172.21.15.94:0/4155221904 learned_addr learned my addr 172.21.15.94:0/4155221904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.850+0000 7f0a437fe700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a440fcf90 0x7f0a44112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.851+0000 7f0a43fff700 1 -- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a440fcf90 msgr2=0x7f0a44112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.851+0000 7f0a43fff700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a440fcf90 0x7f0a44112c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.851+0000 7f0a43fff700 1 -- 172.21.15.94:0/4155221904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a34005040 con 0x7f0a440fbce0 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.851+0000 7f0a43fff700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 0x7f0a441106d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009150 tx=0x7f0a34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.851+0000 7f0a40ff9700 1 -- 172.21.15.94:0/4155221904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a3400e040 con 0x7f0a440fbce0 2024-04-06T03:41:29.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.851+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a441131c0 con 0x7f0a440fbce0 2024-04-06T03:41:29.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.851+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a44113780 con 0x7f0a440fbce0 2024-04-06T03:41:29.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.851+0000 7f0a40ff9700 1 -- 172.21.15.94:0/4155221904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a3400bbf0 con 0x7f0a440fbce0 2024-04-06T03:41:29.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.851+0000 7f0a40ff9700 1 -- 172.21.15.94:0/4155221904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a3401c630 con 0x7f0a440fbce0 2024-04-06T03:41:29.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.852+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a300052e0 con 0x7f0a440fbce0 2024-04-06T03:41:29.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.853+0000 7f0a40ff9700 1 -- 172.21.15.94:0/4155221904 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f0a3400bd60 con 0x7f0a440fbce0 2024-04-06T03:41:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.854+0000 7f0a40ff9700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0a2c06b540 0x7f0a2c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.854+0000 7f0a40ff9700 1 -- 172.21.15.94:0/4155221904 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f0a3408be20 con 0x7f0a440fbce0 2024-04-06T03:41:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.854+0000 7f0a437fe700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0a2c06b540 0x7f0a2c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:29.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.854+0000 7f0a437fe700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0a2c06b540 0x7f0a2c06d9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a38004f40 tx=0x7f0a3800b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:29.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:29.861+0000 7f0a40ff9700 1 -- 172.21.15.94:0/4155221904 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f0a34055cb0 con 0x7f0a440fbce0 2024-04-06T03:41:30.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.165+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0a30005860 con 0x7f0a440fbce0 2024-04-06T03:41:30.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.166+0000 7f0a40ff9700 1 -- 172.21.15.94:0/4155221904 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f0a34013e60 con 0x7f0a440fbce0 2024-04-06T03:41:30.170 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:41:30.170 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:41:30.170 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:41:30.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:41:30.170 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:41:30.170 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 1, 2024-04-06T03:41:30.170 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 1 2024-04-06T03:41:30.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 13, 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 1 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:41:30.171 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:41:30.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.169+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0a2c06b540 msgr2=0x7f0a2c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.169+0000 7f0a4a836700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0a2c06b540 0x7f0a2c06d9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a38004f40 tx=0x7f0a3800b040 comp rx=0 tx=0).stop 2024-04-06T03:41:30.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.169+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 msgr2=0x7f0a441106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.169+0000 7f0a4a836700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 0x7f0a441106d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009150 tx=0x7f0a34009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.169+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 shutdown_connections 2024-04-06T03:41:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.169+0000 7f0a4a836700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a440fbce0 0x7f0a441106d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.169+0000 7f0a4a836700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f0a2c06b540 0x7f0a2c06d9f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.169+0000 7f0a4a836700 1 --2- 172.21.15.94:0/4155221904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a440fcf90 0x7f0a44112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.169+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 >> 172.21.15.94:0/4155221904 conn(0x7f0a440f7480 msgr2=0x7f0a440f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.170+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 shutdown_connections 2024-04-06T03:41:30.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.170+0000 7f0a4a836700 1 -- 172.21.15.94:0/4155221904 wait complete. 2024-04-06T03:41:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.311+0000 7f46d548e700 1 -- 172.21.15.94:0/324170343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 msgr2=0x7f46d00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.311+0000 7f46d548e700 1 --2- 172.21.15.94:0/324170343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 0x7f46d00fc0e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f46c0004940 tx=0x7f46c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.311+0000 7f46d548e700 1 -- 172.21.15.94:0/324170343 shutdown_connections 2024-04-06T03:41:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.311+0000 7f46d548e700 1 --2- 172.21.15.94:0/324170343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46d00fcf80 0x7f46d00fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.311+0000 7f46d548e700 1 --2- 172.21.15.94:0/324170343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 0x7f46d00fc0e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.311+0000 7f46d548e700 1 -- 172.21.15.94:0/324170343 >> 172.21.15.94:0/324170343 conn(0x7f46d00f7430 msgr2=0x7f46d00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.311+0000 7f46d548e700 1 -- 172.21.15.94:0/324170343 shutdown_connections 2024-04-06T03:41:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.311+0000 7f46d548e700 1 -- 172.21.15.94:0/324170343 wait complete. 2024-04-06T03:41:30.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46d548e700 1 Processor -- start 2024-04-06T03:41:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46d548e700 1 -- start start 2024-04-06T03:41:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46d548e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 0x7f46d010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46d548e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46d00fcf80 0x7f46d010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46d548e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d010e540 con 0x7f46d00fbcd0 2024-04-06T03:41:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46d548e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d010ed90 con 0x7f46d00fcf80 2024-04-06T03:41:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46ceffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 0x7f46d010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46ce7fc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46d00fcf80 0x7f46d010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46ceffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 0x7f46d010da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45352/0 (socket says 172.21.15.94:45352) 2024-04-06T03:41:30.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.312+0000 7f46ceffd700 1 -- 172.21.15.94:0/3630456033 learned_addr learned my addr 172.21.15.94:0/3630456033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:41:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.313+0000 7f46ceffd700 1 -- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46d00fcf80 msgr2=0x7f46d010df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.313+0000 7f46ceffd700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46d00fcf80 0x7f46d010df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.313+0000 7f46ceffd700 1 -- 172.21.15.94:0/3630456033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0005040 con 0x7f46d00fbcd0 2024-04-06T03:41:30.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.313+0000 7f46ce7fc700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46d00fcf80 0x7f46d010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:41:30.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.313+0000 7f46ceffd700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 0x7f46d010da00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009150 tx=0x7f46c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:30.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.313+0000 7f46b77fe700 1 -- 172.21.15.94:0/3630456033 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46c000e040 con 0x7f46d00fbcd0 2024-04-06T03:41:30.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.313+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d010f010 con 0x7f46d00fbcd0 2024-04-06T03:41:30.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.313+0000 7f46b77fe700 1 -- 172.21.15.94:0/3630456033 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46c000bbf0 con 0x7f46d00fbcd0 2024-04-06T03:41:30.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.313+0000 7f46b77fe700 1 -- 172.21.15.94:0/3630456033 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46c001c630 con 0x7f46d00fbcd0 2024-04-06T03:41:30.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.314+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d010f550 con 0x7f46d00fbcd0 2024-04-06T03:41:30.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.314+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d005f5f0 con 0x7f46d00fbcd0 2024-04-06T03:41:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.315+0000 7f46b77fe700 1 -- 172.21.15.94:0/3630456033 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f46c000bd60 con 0x7f46d00fbcd0 2024-04-06T03:41:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.316+0000 7f46b77fe700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f46b806b540 0x7f46b806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.316+0000 7f46b77fe700 1 -- 172.21.15.94:0/3630456033 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f46c008b350 con 0x7f46d00fbcd0 2024-04-06T03:41:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.316+0000 7f46ce7fc700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f46b806b540 0x7f46b806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:30.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.317+0000 7f46ce7fc700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f46b806b540 0x7f46b806d9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f46c4004fd0 tx=0x7f46c40043c0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:30.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.324+0000 7f46b77fe700 1 -- 172.21.15.94:0/3630456033 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f46c0058d10 con 0x7f46d00fbcd0 2024-04-06T03:41:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:30 smithi094 ceph-mon[29713]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:41:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:30 smithi094 ceph-mon[29713]: from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:41:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:30 smithi094 ceph-mon[29713]: from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:41:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:30 smithi094 ceph-mon[29713]: pgmap v45: 65 pgs: 65 active+clean; 302 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2024-04-06T03:41:30.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:30 smithi136 ceph-mon[36436]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:41:30.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:30 smithi136 ceph-mon[36436]: from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:41:30.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:30 smithi136 ceph-mon[36436]: from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:41:30.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:30 smithi136 ceph-mon[36436]: pgmap v45: 65 pgs: 65 active+clean; 302 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2024-04-06T03:41:30.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.624+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f46d010f920 con 0x7f46d00fbcd0 2024-04-06T03:41:30.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.625+0000 7f46b77fe700 1 -- 172.21.15.94:0/3630456033 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7f46c0013e60 con 0x7f46d00fbcd0 2024-04-06T03:41:30.628 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:41:30.628 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:41:30.628 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:41:30.628 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:41:30.629 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:41:30.630 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:41:30.631 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:41:30.631 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:41:30.631 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:41:30.631 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:41:30.631 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f46b806b540 msgr2=0x7f46b806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f46b806b540 0x7f46b806d9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f46c4004fd0 tx=0x7f46c40043c0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 msgr2=0x7f46d010da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 0x7f46d010da00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009150 tx=0x7f46c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 shutdown_connections 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46d00fbcd0 0x7f46d010da00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f46b806b540 0x7f46b806d9f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 --2- 172.21.15.94:0/3630456033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46d00fcf80 0x7f46d010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 >> 172.21.15.94:0/3630456033 conn(0x7f46d00f7430 msgr2=0x7f46d00f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 shutdown_connections 2024-04-06T03:41:30.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.628+0000 7f46d548e700 1 -- 172.21.15.94:0/3630456033 wait complete. 2024-04-06T03:41:30.633 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:41:30.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.745+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/3526796379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fe400 msgr2=0x7f7eec0fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.745+0000 7f7ef1d0a700 1 --2- 172.21.15.94:0/3526796379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fe400 0x7f7eec0fe810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7edc004940 tx=0x7f7edc009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:30.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.745+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/3526796379 shutdown_connections 2024-04-06T03:41:30.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.745+0000 7f7ef1d0a700 1 --2- 172.21.15.94:0/3526796379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7eec0fed50 0x7f7eec0fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.745+0000 7f7ef1d0a700 1 --2- 172.21.15.94:0/3526796379 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fe400 0x7f7eec0fe810 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.745+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/3526796379 >> 172.21.15.94:0/3526796379 conn(0x7f7eec0f6ba0 msgr2=0x7f7eec0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:30.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.745+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/3526796379 shutdown_connections 2024-04-06T03:41:30.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.745+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/3526796379 wait complete. 2024-04-06T03:41:30.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.746+0000 7f7ef1d0a700 1 Processor -- start 2024-04-06T03:41:30.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.746+0000 7f7ef1d0a700 1 -- start start 2024-04-06T03:41:30.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.746+0000 7f7ef1d0a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7eec0fe400 0x7f7eec10d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:30.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.746+0000 7f7ef1d0a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fed50 0x7f7eec10df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:30.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.746+0000 7f7ef1d0a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec10e530 con 0x7f7eec0fed50 2024-04-06T03:41:30.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.746+0000 7f7ef1d0a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec10ed80 con 0x7f7eec0fe400 2024-04-06T03:41:30.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7eeaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fed50 0x7f7eec10df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:30.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7eeaffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fed50 0x7f7eec10df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45368/0 (socket says 172.21.15.94:45368) 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7eeaffd700 1 -- 172.21.15.94:0/917833150 learned_addr learned my addr 172.21.15.94:0/917833150 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7eeb7fe700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7eec0fe400 0x7f7eec10d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7eeaffd700 1 -- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7eec0fe400 msgr2=0x7f7eec10d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7eeaffd700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7eec0fe400 0x7f7eec10d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7eeaffd700 1 -- 172.21.15.94:0/917833150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc005040 con 0x7f7eec0fed50 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7eeaffd700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fed50 0x7f7eec10df30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0002a50 tx=0x7f7ee00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7ed3fff700 1 -- 172.21.15.94:0/917833150 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ee0007cc0 con 0x7f7eec0fed50 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec10f060 con 0x7f7eec0fed50 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec10f5a0 con 0x7f7eec0fed50 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7ed3fff700 1 -- 172.21.15.94:0/917833150 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ee000f040 con 0x7f7eec0fed50 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.747+0000 7f7ed3fff700 1 -- 172.21.15.94:0/917833150 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ee000da30 con 0x7f7eec0fed50 2024-04-06T03:41:30.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.748+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed80052e0 con 0x7f7eec0fed50 2024-04-06T03:41:30.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.749+0000 7f7ed3fff700 1 -- 172.21.15.94:0/917833150 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f7ee000dc70 con 0x7f7eec0fed50 2024-04-06T03:41:30.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.750+0000 7f7ed3fff700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7ed406b540 0x7f7ed406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:30.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.750+0000 7f7ed3fff700 1 -- 172.21.15.94:0/917833150 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f7ee0089cf0 con 0x7f7eec0fed50 2024-04-06T03:41:30.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.750+0000 7f7eeb7fe700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7ed406b540 0x7f7ed406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:30.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.750+0000 7f7eeb7fe700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7ed406b540 0x7f7ed406d9f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00b970 tx=0x7f7edc009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:30.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.755+0000 7f7ed3fff700 1 -- 172.21.15.94:0/917833150 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f7ee0057610 con 0x7f7eec0fed50 2024-04-06T03:41:30.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.986+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ed8000bf0 con 0x7f7ed406b540 2024-04-06T03:41:30.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.987+0000 7f7ed3fff700 1 -- 172.21.15.94:0/917833150 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f7ed8000bf0 con 0x7f7ed406b540 2024-04-06T03:41:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:41:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:41:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:41:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:41:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-04-06T03:41:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "1/21 daemons upgraded", 2024-04-06T03:41:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "", 2024-04-06T03:41:30.991 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:41:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:41:30.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.989+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7ed406b540 msgr2=0x7f7ed406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.989+0000 7f7ef1d0a700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7ed406b540 0x7f7ed406d9f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00b970 tx=0x7f7edc009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:30.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.989+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fed50 msgr2=0x7f7eec10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:30.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.989+0000 7f7ef1d0a700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fed50 0x7f7eec10df30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0002a50 tx=0x7f7ee00046b0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.990+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 shutdown_connections 2024-04-06T03:41:30.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.990+0000 7f7ef1d0a700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f7ed406b540 0x7f7ed406d9f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.990+0000 7f7ef1d0a700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7eec0fe400 0x7f7eec10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.990+0000 7f7ef1d0a700 1 --2- 172.21.15.94:0/917833150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7eec0fed50 0x7f7eec10df30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:30.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.990+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 >> 172.21.15.94:0/917833150 conn(0x7f7eec0f6ba0 msgr2=0x7f7eec0f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:30.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.990+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 shutdown_connections 2024-04-06T03:41:30.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:30.990+0000 7f7ef1d0a700 1 -- 172.21.15.94:0/917833150 wait complete. 2024-04-06T03:41:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.108+0000 7f94dde80700 1 -- 172.21.15.94:0/1201865748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8070870 msgr2=0x7f94d8070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.108+0000 7f94dde80700 1 --2- 172.21.15.94:0/1201865748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8070870 0x7f94d8070ce0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f94cc004940 tx=0x7f94cc009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.108+0000 7f94dde80700 1 -- 172.21.15.94:0/1201865748 shutdown_connections 2024-04-06T03:41:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.108+0000 7f94dde80700 1 --2- 172.21.15.94:0/1201865748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8070870 0x7f94d8070ce0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.108+0000 7f94dde80700 1 --2- 172.21.15.94:0/1201865748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d8071e20 0x7f94d8070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.108+0000 7f94dde80700 1 -- 172.21.15.94:0/1201865748 >> 172.21.15.94:0/1201865748 conn(0x7f94d80f8550 msgr2=0x7f94d80fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:31.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.109+0000 7f94dde80700 1 -- 172.21.15.94:0/1201865748 shutdown_connections 2024-04-06T03:41:31.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.109+0000 7f94dde80700 1 -- 172.21.15.94:0/1201865748 wait complete. 2024-04-06T03:41:31.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.109+0000 7f94dde80700 1 Processor -- start 2024-04-06T03:41:31.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.109+0000 7f94dde80700 1 -- start start 2024-04-06T03:41:31.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94dde80700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d8070870 0x7f94d810fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:31.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94dde80700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8071e20 0x7f94d8110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:31.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94dde80700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8110770 con 0x7f94d8071e20 2024-04-06T03:41:31.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94dde80700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8110fc0 con 0x7f94d8070870 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94d6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8071e20 0x7f94d8110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94d6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8071e20 0x7f94d8110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45386/0 (socket says 172.21.15.94:45386) 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94d6ffd700 1 -- 172.21.15.94:0/3096307925 learned_addr learned my addr 172.21.15.94:0/3096307925 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94d77fe700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d8070870 0x7f94d810fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94d6ffd700 1 -- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d8070870 msgr2=0x7f94d810fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94d6ffd700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d8070870 0x7f94d810fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94d6ffd700 1 -- 172.21.15.94:0/3096307925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc005040 con 0x7f94d8071e20 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94d6ffd700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8071e20 0x7f94d8110170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00b3b0 tx=0x7f94cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94bffff700 1 -- 172.21.15.94:0/3096307925 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94cc00e040 con 0x7f94d8071e20 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.110+0000 7f94bffff700 1 -- 172.21.15.94:0/3096307925 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94cc00bbf0 con 0x7f94d8071e20 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.111+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d8111240 con 0x7f94d8071e20 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.111+0000 7f94bffff700 1 -- 172.21.15.94:0/3096307925 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94cc01c630 con 0x7f94d8071e20 2024-04-06T03:41:31.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.111+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d8111780 con 0x7f94d8071e20 2024-04-06T03:41:31.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.112+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c40052e0 con 0x7f94d8071e20 2024-04-06T03:41:31.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.113+0000 7f94bffff700 1 -- 172.21.15.94:0/3096307925 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 89201+0+0 (secure 0 0 0) 0x7f94cc00bd60 con 0x7f94d8071e20 2024-04-06T03:41:31.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.113+0000 7f94bffff700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f94c006b540 0x7f94c006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:41:31.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.113+0000 7f94bffff700 1 -- 172.21.15.94:0/3096307925 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f94cc08aab0 con 0x7f94d8071e20 2024-04-06T03:41:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.113+0000 7f94d77fe700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f94c006b540 0x7f94c006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:41:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.114+0000 7f94d77fe700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f94c006b540 0x7f94c006d9f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94d8071b60 tx=0x7f94c800b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:41:31.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.122+0000 7f94bffff700 1 -- 172.21.15.94:0/3096307925 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f94cc058590 con 0x7f94d8071e20 2024-04-06T03:41:31.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:31 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/4155221904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:41:31.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:31 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3630456033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:41:31.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:31 smithi094 ceph-mon[29713]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:41:31.564 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-04-06T03:41:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.560+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f94c4005860 con 0x7f94d8071e20 2024-04-06T03:41:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.560+0000 7f94bffff700 1 -- 172.21.15.94:0/3096307925 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f94cc058120 con 0x7f94d8071e20 2024-04-06T03:41:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f94c006b540 msgr2=0x7f94c006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f94c006b540 0x7f94c006d9f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94d8071b60 tx=0x7f94c800b040 comp rx=0 tx=0).stop 2024-04-06T03:41:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8071e20 msgr2=0x7f94d8110170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:41:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8071e20 0x7f94d8110170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00b3b0 tx=0x7f94cc009e30 comp rx=0 tx=0).stop 2024-04-06T03:41:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 shutdown_connections 2024-04-06T03:41:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f94c006b540 0x7f94c006d9f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d8070870 0x7f94d810fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:31.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 --2- 172.21.15.94:0/3096307925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94d8071e20 0x7f94d8110170 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:41:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 >> 172.21.15.94:0/3096307925 conn(0x7f94d80f8550 msgr2=0x7f94d80f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:41:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 shutdown_connections 2024-04-06T03:41:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:41:31.563+0000 7f94dde80700 1 -- 172.21.15.94:0/3096307925 wait complete. 2024-04-06T03:41:31.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:31 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/4155221904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:41:31.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:31 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3630456033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:41:31.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:31 smithi136 ceph-mon[36436]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:41:32.467 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:32 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3096307925' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:41:32.467 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:32 smithi094 ceph-mon[29713]: pgmap v46: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 2.7 MiB/s wr, 282 op/s 2024-04-06T03:41:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:32 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3096307925' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:41:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:32 smithi136 ceph-mon[36436]: pgmap v46: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 2.7 MiB/s wr, 282 op/s 2024-04-06T03:41:35.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:35 smithi094 ceph-mon[29713]: pgmap v47: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-06T03:41:35.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:35 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:35.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:35 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:35.591 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:35 smithi136 ceph-mon[36436]: pgmap v47: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-06T03:41:35.591 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:35 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:35.591 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:35 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:36 smithi136 ceph-mon[36436]: Reconfiguring node-exporter.smithi136 (dependencies changed)... 2024-04-06T03:41:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:36 smithi136 ceph-mon[36436]: Reconfiguring daemon node-exporter.smithi136 on smithi136 2024-04-06T03:41:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:36 smithi136 ceph-mon[36436]: pgmap v48: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-06T03:41:36.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:36 smithi094 ceph-mon[29713]: Reconfiguring node-exporter.smithi136 (dependencies changed)... 2024-04-06T03:41:36.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:36 smithi094 ceph-mon[29713]: Reconfiguring daemon node-exporter.smithi136 on smithi136 2024-04-06T03:41:36.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:36 smithi094 ceph-mon[29713]: pgmap v48: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-04-06T03:41:37.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:37 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:41:37.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:37 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:41:38.539 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:38.539 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:38.539 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:41:38.539 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:41:38.539 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:41:38.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:41:38.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:41:38.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:41:38.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:41:38.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: Upgrade: Need to upgrade myself (mgr.smithi136.jagvwt) 2024-04-06T03:41:38.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: Upgrade: Need to upgrade myself (mgr.smithi136.jagvwt) 2024-04-06T03:41:38.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:38 smithi094 ceph-mon[29713]: pgmap v49: 65 pgs: 65 active+clean; 319 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.5 MiB/s wr, 318 op/s 2024-04-06T03:41:38.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:38.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: Upgrade: Need to upgrade myself (mgr.smithi136.jagvwt) 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: Upgrade: Need to upgrade myself (mgr.smithi136.jagvwt) 2024-04-06T03:41:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:38 smithi136 ceph-mon[36436]: pgmap v49: 65 pgs: 65 active+clean; 319 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.5 MiB/s wr, 318 op/s 2024-04-06T03:41:39.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:39 smithi094 ceph-mon[29713]: mgrmap e26: smithi136.jagvwt(active, since 92s), standbys: smithi094.purlzp 2024-04-06T03:41:39.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:39 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:39.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:39 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.purlzp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:41:39.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:39 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:41:39.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:39 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:41:39.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:39 smithi136 ceph-mon[36436]: mgrmap e26: smithi136.jagvwt(active, since 92s), standbys: smithi094.purlzp 2024-04-06T03:41:39.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:39 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:39.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:39 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.purlzp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:41:39.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:39 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:41:39.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:39 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:41:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:40 smithi136 ceph-mon[36436]: Upgrade: Updating mgr.smithi094.purlzp 2024-04-06T03:41:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:40 smithi136 ceph-mon[36436]: Deploying daemon mgr.smithi094.purlzp on smithi094 2024-04-06T03:41:40.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:40 smithi136 ceph-mon[36436]: pgmap v50: 65 pgs: 65 active+clean; 319 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-04-06T03:41:40.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:40 smithi094 ceph-mon[29713]: Upgrade: Updating mgr.smithi094.purlzp 2024-04-06T03:41:40.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:40 smithi094 ceph-mon[29713]: Deploying daemon mgr.smithi094.purlzp on smithi094 2024-04-06T03:41:40.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:40 smithi094 ceph-mon[29713]: pgmap v50: 65 pgs: 65 active+clean; 319 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-04-06T03:41:42.492 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:42 smithi094 ceph-mon[29713]: pgmap v51: 65 pgs: 65 active+clean; 318 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 2.6 MiB/s wr, 387 op/s 2024-04-06T03:41:42.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:42 smithi136 ceph-mon[36436]: pgmap v51: 65 pgs: 65 active+clean; 318 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 2.6 MiB/s wr, 387 op/s 2024-04-06T03:41:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:44 smithi136 ceph-mon[36436]: pgmap v52: 65 pgs: 65 active+clean; 318 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-04-06T03:41:44.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:44 smithi094 ceph-mon[29713]: pgmap v52: 65 pgs: 65 active+clean; 318 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-04-06T03:41:46.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:46 smithi136 ceph-mon[36436]: pgmap v53: 65 pgs: 65 active+clean; 318 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-04-06T03:41:46.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:46 smithi094 ceph-mon[29713]: pgmap v53: 65 pgs: 65 active+clean; 318 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 274 op/s 2024-04-06T03:41:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:48 smithi094 ceph-mon[29713]: pgmap v54: 65 pgs: 65 active+clean; 312 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.5 MiB/s wr, 852 op/s 2024-04-06T03:41:48.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:48 smithi136 ceph-mon[36436]: pgmap v54: 65 pgs: 65 active+clean; 312 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.5 MiB/s wr, 852 op/s 2024-04-06T03:41:50.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:50 smithi136 ceph-mon[36436]: pgmap v55: 65 pgs: 65 active+clean; 312 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 32 KiB/s rd, 1.7 MiB/s wr, 757 op/s 2024-04-06T03:41:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:50 smithi094 ceph-mon[29713]: pgmap v55: 65 pgs: 65 active+clean; 312 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 32 KiB/s rd, 1.7 MiB/s wr, 757 op/s 2024-04-06T03:41:52.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:52 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:52.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:52 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:52.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:52 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:41:52.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:52 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:41:52.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:52 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:52.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:52 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:52.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:52 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:41:52.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:52 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:41:53.248 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:53 smithi094 ceph-mon[29713]: pgmap v56: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 1.26k op/s 2024-04-06T03:41:53.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:53 smithi136 ceph-mon[36436]: pgmap v56: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 2.4 MiB/s wr, 1.26k op/s 2024-04-06T03:41:55.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:55 smithi094 ceph-mon[29713]: pgmap v57: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 1.08k op/s 2024-04-06T03:41:55.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:55 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:55.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:55 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:55.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:55 smithi136 ceph-mon[36436]: pgmap v57: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 1.08k op/s 2024-04-06T03:41:55.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:55 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:55.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:55 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:41:57.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:57 smithi136 ceph-mon[36436]: pgmap v58: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 1.08k op/s 2024-04-06T03:41:57.406 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:57 smithi094 ceph-mon[29713]: pgmap v58: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 1.08k op/s 2024-04-06T03:41:59.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:41:59 smithi094 ceph-mon[29713]: pgmap v59: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.3 MiB/s wr, 1.30k op/s 2024-04-06T03:41:59.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:41:59 smithi136 ceph-mon[36436]: pgmap v59: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.3 MiB/s wr, 1.30k op/s 2024-04-06T03:42:00.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:00 smithi094 ceph-mon[29713]: Standby manager daemon smithi094.purlzp restarted 2024-04-06T03:42:00.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:00 smithi094 ceph-mon[29713]: Standby manager daemon smithi094.purlzp started 2024-04-06T03:42:00.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:00 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.purlzp/crt"}]: dispatch 2024-04-06T03:42:00.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:00 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:42:00.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:00 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.purlzp/key"}]: dispatch 2024-04-06T03:42:00.157 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:00 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:42:00.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:00 smithi136 ceph-mon[36436]: Standby manager daemon smithi094.purlzp restarted 2024-04-06T03:42:00.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:00 smithi136 ceph-mon[36436]: Standby manager daemon smithi094.purlzp started 2024-04-06T03:42:00.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:00 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.purlzp/crt"}]: dispatch 2024-04-06T03:42:00.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:00 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:42:00.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:00 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi094.purlzp/key"}]: dispatch 2024-04-06T03:42:00.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:00 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:42:01.283 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:01 smithi094 ceph-mon[29713]: pgmap v60: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 727 op/s 2024-04-06T03:42:01.283 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:01 smithi094 ceph-mon[29713]: mgrmap e27: smithi136.jagvwt(active, since 114s), standbys: smithi094.purlzp 2024-04-06T03:42:01.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:01 smithi136 ceph-mon[36436]: pgmap v60: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 727 op/s 2024-04-06T03:42:01.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:01 smithi136 ceph-mon[36436]: mgrmap e27: smithi136.jagvwt(active, since 114s), standbys: smithi094.purlzp 2024-04-06T03:42:01.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.736+0000 7fc913748700 1 -- 172.21.15.94:0/575364934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90c0fbcd0 msgr2=0x7fc90c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:01.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.736+0000 7fc913748700 1 --2- 172.21.15.94:0/575364934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90c0fbcd0 0x7fc90c0fc0e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc908004940 tx=0x7fc908009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:01.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.736+0000 7fc913748700 1 -- 172.21.15.94:0/575364934 shutdown_connections 2024-04-06T03:42:01.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.736+0000 7fc913748700 1 --2- 172.21.15.94:0/575364934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc90c0fcf80 0x7fc90c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:01.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.736+0000 7fc913748700 1 --2- 172.21.15.94:0/575364934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90c0fbcd0 0x7fc90c0fc0e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:01.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.736+0000 7fc913748700 1 -- 172.21.15.94:0/575364934 >> 172.21.15.94:0/575364934 conn(0x7fc90c0f7430 msgr2=0x7fc90c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:01.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.736+0000 7fc913748700 1 -- 172.21.15.94:0/575364934 shutdown_connections 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.736+0000 7fc913748700 1 -- 172.21.15.94:0/575364934 wait complete. 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.737+0000 7fc913748700 1 Processor -- start 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.737+0000 7fc913748700 1 -- start start 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.737+0000 7fc913748700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc90c0fbcd0 0x7fc90c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc913748700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90c0fcf80 0x7fc90c06e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc913748700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c06fdd0 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc913748700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c06ff40 con 0x7fc90c0fcf80 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc9114e4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc90c0fbcd0 0x7fc90c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc9114e4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc90c0fbcd0 0x7fc90c06e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48158/0 (socket says 172.21.15.94:48158) 2024-04-06T03:42:01.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc9114e4700 1 -- 172.21.15.94:0/3819816809 learned_addr learned my addr 172.21.15.94:0/3819816809 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:01.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc910ce3700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90c0fcf80 0x7fc90c06e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:01.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc9114e4700 1 -- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90c0fcf80 msgr2=0x7fc90c06e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:01.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc9114e4700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90c0fcf80 0x7fc90c06e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:01.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc9114e4700 1 -- 172.21.15.94:0/3819816809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc908005040 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.738+0000 7fc910ce3700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90c0fcf80 0x7fc90c06e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:42:01.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.739+0000 7fc9114e4700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc90c0fbcd0 0x7fc90c06e3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc90800b0d0 tx=0x7fc908009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.739+0000 7fc901ffb700 1 -- 172.21.15.94:0/3819816809 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc90800e030 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.739+0000 7fc901ffb700 1 -- 172.21.15.94:0/3819816809 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc908013d00 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.739+0000 7fc901ffb700 1 -- 172.21.15.94:0/3819816809 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc908022630 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.739+0000 7fc913748700 1 -- 172.21.15.94:0/3819816809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c06ee40 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.740+0000 7fc913748700 1 -- 172.21.15.94:0/3819816809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c066210 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.741+0000 7fc8f77fe700 1 -- 172.21.15.94:0/3819816809 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec0052b0 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.742+0000 7fc901ffb700 1 -- 172.21.15.94:0/3819816809 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 91575+0+0 (secure 0 0 0) 0x7fc90801aa30 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.742+0000 7fc901ffb700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc8f806d5f0 0x7fc8f806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:01.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.743+0000 7fc901ffb700 1 -- 172.21.15.94:0/3819816809 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc908090320 con 0x7fc90c0fbcd0 2024-04-06T03:42:01.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.743+0000 7fc910ce3700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc8f806d5f0 0x7fc8f806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:01.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.744+0000 7fc910ce3700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc8f806d5f0 0x7fc8f806faa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009fd0 tx=0x7fc8fc00b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:01.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:01.753+0000 7fc901ffb700 1 -- 172.21.15.94:0/3819816809 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fc90805ddc0 con 0x7fc90c0fbcd0 2024-04-06T03:42:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.062+0000 7fc8f77fe700 1 -- 172.21.15.94:0/3819816809 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8ec000bc0 con 0x7fc8f806d5f0 2024-04-06T03:42:02.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.064+0000 7fc901ffb700 1 -- 172.21.15.94:0/3819816809 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc8ec000bc0 con 0x7fc8f806d5f0 2024-04-06T03:42:02.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 -- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc8f806d5f0 msgr2=0x7fc8f806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc8f806d5f0 0x7fc8f806faa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009fd0 tx=0x7fc8fc00b040 comp rx=0 tx=0).stop 2024-04-06T03:42:02.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 -- 172.21.15.94:0/3819816809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc90c0fbcd0 msgr2=0x7fc90c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc90c0fbcd0 0x7fc90c06e3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc90800b0d0 tx=0x7fc908009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:02.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 -- 172.21.15.94:0/3819816809 shutdown_connections 2024-04-06T03:42:02.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc90c0fbcd0 0x7fc90c06e3c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc8f806d5f0 0x7fc8f806faa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 --2- 172.21.15.94:0/3819816809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc90c0fcf80 0x7fc90c06e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 -- 172.21.15.94:0/3819816809 >> 172.21.15.94:0/3819816809 conn(0x7fc90c0f7430 msgr2=0x7fc90c0f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:02.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 -- 172.21.15.94:0/3819816809 shutdown_connections 2024-04-06T03:42:02.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.066+0000 7fc8f77fe700 1 -- 172.21.15.94:0/3819816809 wait complete. 2024-04-06T03:42:02.079 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:42:02.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.206+0000 7fe55b929700 1 -- 172.21.15.94:0/765284370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5540fed90 msgr2=0x7fe5540faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.206+0000 7fe55b929700 1 --2- 172.21.15.94:0/765284370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5540fed90 0x7fe5540faed0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe548004940 tx=0x7fe548009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:02.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.206+0000 7fe55b929700 1 -- 172.21.15.94:0/765284370 shutdown_connections 2024-04-06T03:42:02.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.206+0000 7fe55b929700 1 --2- 172.21.15.94:0/765284370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5540fed90 0x7fe5540faed0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.206+0000 7fe55b929700 1 --2- 172.21.15.94:0/765284370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5540fe440 0x7fe5540fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.206+0000 7fe55b929700 1 -- 172.21.15.94:0/765284370 >> 172.21.15.94:0/765284370 conn(0x7fe5540f6bc0 msgr2=0x7fe5540f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:02.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.206+0000 7fe55b929700 1 -- 172.21.15.94:0/765284370 shutdown_connections 2024-04-06T03:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.206+0000 7fe55b929700 1 -- 172.21.15.94:0/765284370 wait complete. 2024-04-06T03:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe55b929700 1 Processor -- start 2024-04-06T03:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe55b929700 1 -- start start 2024-04-06T03:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe55b929700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5540fe440 0x7fe55410d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe55b929700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5540fed90 0x7fe55410ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe55b929700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55410e3f0 con 0x7fe5540fed90 2024-04-06T03:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe55b929700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55410e560 con 0x7fe5540fe440 2024-04-06T03:42:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe558ec4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5540fed90 0x7fe55410ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:02.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe5596c5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5540fe440 0x7fe55410d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:02.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe5596c5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5540fe440 0x7fe55410d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:44642/0 (socket says 172.21.15.94:44642) 2024-04-06T03:42:02.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.207+0000 7fe5596c5700 1 -- 172.21.15.94:0/2976838637 learned_addr learned my addr 172.21.15.94:0/2976838637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:02.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.208+0000 7fe5596c5700 1 -- 172.21.15.94:0/2976838637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5540fed90 msgr2=0x7fe55410ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.208+0000 7fe5596c5700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5540fed90 0x7fe55410ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.208+0000 7fe5596c5700 1 -- 172.21.15.94:0/2976838637 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe550005040 con 0x7fe5540fe440 2024-04-06T03:42:02.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.208+0000 7fe5596c5700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5540fe440 0x7fe55410d8b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5500096d0 tx=0x7fe550009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:02.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.208+0000 7fe545ffb700 1 -- 172.21.15.94:0/2976838637 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe55000e840 con 0x7fe5540fe440 2024-04-06T03:42:02.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.208+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe548005040 con 0x7fe5540fe440 2024-04-06T03:42:02.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.208+0000 7fe545ffb700 1 -- 172.21.15.94:0/2976838637 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe55000f040 con 0x7fe5540fe440 2024-04-06T03:42:02.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.208+0000 7fe545ffb700 1 -- 172.21.15.94:0/2976838637 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe550013dc0 con 0x7fe5540fe440 2024-04-06T03:42:02.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.209+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5541132c0 con 0x7fe5540fe440 2024-04-06T03:42:02.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.209+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55404c710 con 0x7fe5540fe440 2024-04-06T03:42:02.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.211+0000 7fe545ffb700 1 -- 172.21.15.94:0/2976838637 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 27) v1 ==== 91575+0+0 (secure 0 0 0) 0x7fe55000e9a0 con 0x7fe5540fe440 2024-04-06T03:42:02.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.211+0000 7fe545ffb700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe54006d5a0 0x7fe54006fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:02.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.211+0000 7fe545ffb700 1 -- 172.21.15.94:0/2976838637 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fe55008bfe0 con 0x7fe5540fe440 2024-04-06T03:42:02.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.211+0000 7fe558ec4700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe54006d5a0 0x7fe54006fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:02.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.212+0000 7fe558ec4700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe54006d5a0 0x7fe54006fa50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe548009fd0 tx=0x7fe548009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.216+0000 7fe545ffb700 1 -- 172.21.15.94:0/2976838637 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fe550059b20 con 0x7fe5540fe440 2024-04-06T03:42:02.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.515+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe554102490 con 0x7fe54006d5a0 2024-04-06T03:42:02.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.516+0000 7fe545ffb700 1 -- 172.21.15.94:0/2976838637 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe554102490 con 0x7fe54006d5a0 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.518+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe54006d5a0 msgr2=0x7fe54006fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.518+0000 7fe55b929700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe54006d5a0 0x7fe54006fa50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe548009fd0 tx=0x7fe548009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.518+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5540fe440 msgr2=0x7fe55410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.519+0000 7fe55b929700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5540fe440 0x7fe55410d8b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5500096d0 tx=0x7fe550009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.519+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 shutdown_connections 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.519+0000 7fe55b929700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fe54006d5a0 0x7fe54006fa50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.519+0000 7fe55b929700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5540fe440 0x7fe55410d8b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.519+0000 7fe55b929700 1 --2- 172.21.15.94:0/2976838637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5540fed90 0x7fe55410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.519+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 >> 172.21.15.94:0/2976838637 conn(0x7fe5540f6bc0 msgr2=0x7fe5540f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.519+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 shutdown_connections 2024-04-06T03:42:02.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.519+0000 7fe55b929700 1 -- 172.21.15.94:0/2976838637 wait complete. 2024-04-06T03:42:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.649+0000 7f595b0ff700 1 -- 172.21.15.94:0/3696702359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 msgr2=0x7f5954103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.649+0000 7f595b0ff700 1 --2- 172.21.15.94:0/3696702359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 0x7f5954103800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5948004880 tx=0x7f5948009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.649+0000 7f595b0ff700 1 -- 172.21.15.94:0/3696702359 shutdown_connections 2024-04-06T03:42:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.649+0000 7f595b0ff700 1 --2- 172.21.15.94:0/3696702359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59541046a0 0x7f5954104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.649+0000 7f595b0ff700 1 --2- 172.21.15.94:0/3696702359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 0x7f5954103800 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.649+0000 7f595b0ff700 1 -- 172.21.15.94:0/3696702359 >> 172.21.15.94:0/3696702359 conn(0x7f5954072560 msgr2=0x7f59540749b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.650+0000 7f595b0ff700 1 -- 172.21.15.94:0/3696702359 shutdown_connections 2024-04-06T03:42:02.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.650+0000 7f595b0ff700 1 -- 172.21.15.94:0/3696702359 wait complete. 2024-04-06T03:42:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.651+0000 7f595b0ff700 1 Processor -- start 2024-04-06T03:42:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.651+0000 7f595b0ff700 1 -- start start 2024-04-06T03:42:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.651+0000 7f595b0ff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 0x7f5954114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.651+0000 7f595b0ff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59541046a0 0x7f5954114550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.651+0000 7f595b0ff700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5954114b50 con 0x7f59541046a0 2024-04-06T03:42:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.651+0000 7f595b0ff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541153a0 con 0x7f59541033f0 2024-04-06T03:42:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.652+0000 7f5958e9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 0x7f5954114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.652+0000 7f5958e9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 0x7f5954114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:44656/0 (socket says 172.21.15.94:44656) 2024-04-06T03:42:02.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.652+0000 7f5958e9b700 1 -- 172.21.15.94:0/4217809332 learned_addr learned my addr 172.21.15.94:0/4217809332 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.652+0000 7f5953fff700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59541046a0 0x7f5954114550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.652+0000 7f5958e9b700 1 -- 172.21.15.94:0/4217809332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59541046a0 msgr2=0x7f5954114550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.652+0000 7f5958e9b700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59541046a0 0x7f5954114550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.652+0000 7f5958e9b700 1 -- 172.21.15.94:0/4217809332 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948005040 con 0x7f59541033f0 2024-04-06T03:42:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.653+0000 7f5958e9b700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 0x7f5954114010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5948008d00 tx=0x7f5948009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.653+0000 7f59517fa700 1 -- 172.21.15.94:0/4217809332 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594800e030 con 0x7f59541033f0 2024-04-06T03:42:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.653+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5954115620 con 0x7f59541033f0 2024-04-06T03:42:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.653+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5954115b60 con 0x7f59541033f0 2024-04-06T03:42:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.653+0000 7f59517fa700 1 -- 172.21.15.94:0/4217809332 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5948013e60 con 0x7f59541033f0 2024-04-06T03:42:02.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.653+0000 7f59517fa700 1 -- 172.21.15.94:0/4217809332 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59480123c0 con 0x7f59541033f0 2024-04-06T03:42:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.656+0000 7f59517fa700 1 -- 172.21.15.94:0/4217809332 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 27) v1 ==== 91575+0+0 (secure 0 0 0) 0x7f594801c400 con 0x7f59541033f0 2024-04-06T03:42:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.656+0000 7f59517fa700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f593c06d680 0x7f593c06fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.657+0000 7f59517fa700 1 -- 172.21.15.94:0/4217809332 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f594808e380 con 0x7f59541033f0 2024-04-06T03:42:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.657+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59400052e0 con 0x7f59541033f0 2024-04-06T03:42:02.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.657+0000 7f5953fff700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f593c06d680 0x7f593c06fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:02.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.658+0000 7f5953fff700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f593c06d680 0x7f593c06fb30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59440046c0 tx=0x7f594400b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:02.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.666+0000 7f59517fa700 1 -- 172.21.15.94:0/4217809332 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f594805bea0 con 0x7f59541033f0 2024-04-06T03:42:02.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:02 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:02.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:02 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:02.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:02 smithi094 ceph-mon[29713]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:02.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:02 smithi094 ceph-mon[29713]: pgmap v61: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 2.1 MiB/s wr, 1.14k op/s 2024-04-06T03:42:02.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.917+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5940000bf0 con 0x7f593c06d680 2024-04-06T03:42:02.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.927+0000 7f59517fa700 1 -- 172.21.15.94:0/4217809332 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5940000bf0 con 0x7f593c06d680 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (68s) 1s ago 9m 16.5M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (9m) 1s ago 9m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (8m) 8s ago 8m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 1s ago 9m - - 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (4m) 1s ago 4m 13.5M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (4m) 1s ago 4m 249M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (4m) 8s ago 4m 13.5M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (4m) 8s ago 4m 15.9M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (12s) 1s ago 10m 66.3M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283 running (2m) 8s ago 8m 506M - 19.0.0-2027-g9f192350 a5511c7deba9 7f3bea4f4172 2024-04-06T03:42:02.931 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (10m) 1s ago 10m 47.4M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (8m) 8s ago 8m 36.5M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (88s) 1s ago 9m 8749k - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (25s) 8s ago 8m 8464k - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7m) 1s ago 7m 215M 1557M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (7m) 1s ago 7m 233M 1557M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (6m) 1s ago 6m 195M 1557M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (6m) 8s ago 6m 255M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (5m) 8s ago 5m 226M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (5m) 8s ago 5m 211M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:42:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (30s) 1s ago 8m 45.8M - 2.43.0 a07b618ecd1d 51f6660abb0d 2024-04-06T03:42:02.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.929+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f593c06d680 msgr2=0x7f593c06fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.929+0000 7f595b0ff700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f593c06d680 0x7f593c06fb30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59440046c0 tx=0x7f594400b040 comp rx=0 tx=0).stop 2024-04-06T03:42:02.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.929+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 msgr2=0x7f5954114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:02.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.929+0000 7f595b0ff700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 0x7f5954114010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5948008d00 tx=0x7f5948009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:02.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.930+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 shutdown_connections 2024-04-06T03:42:02.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.930+0000 7f595b0ff700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f593c06d680 0x7f593c06fb30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.930+0000 7f595b0ff700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59541033f0 0x7f5954114010 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.930+0000 7f595b0ff700 1 --2- 172.21.15.94:0/4217809332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59541046a0 0x7f5954114550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:02.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.930+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 >> 172.21.15.94:0/4217809332 conn(0x7f5954072560 msgr2=0x7f5954073c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:02.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.930+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 shutdown_connections 2024-04-06T03:42:02.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:02.930+0000 7f595b0ff700 1 -- 172.21.15.94:0/4217809332 wait complete. 2024-04-06T03:42:03.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.034+0000 7fa72f48a700 1 -- 172.21.15.94:0/2945203113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 msgr2=0x7fa7280fd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:03.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.034+0000 7fa72f48a700 1 --2- 172.21.15.94:0/2945203113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 0x7fa7280fd050 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa718004940 tx=0x7fa718009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:03.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.034+0000 7fa72f48a700 1 -- 172.21.15.94:0/2945203113 shutdown_connections 2024-04-06T03:42:03.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.034+0000 7fa72f48a700 1 --2- 172.21.15.94:0/2945203113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 0x7fa7280fd050 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.034+0000 7fa72f48a700 1 --2- 172.21.15.94:0/2945203113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7280fc290 0x7fa7280fc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.034+0000 7fa72f48a700 1 -- 172.21.15.94:0/2945203113 >> 172.21.15.94:0/2945203113 conn(0x7fa7280f6ba0 msgr2=0x7fa7280f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:03.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.034+0000 7fa72f48a700 1 -- 172.21.15.94:0/2945203113 shutdown_connections 2024-04-06T03:42:03.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.034+0000 7fa72f48a700 1 -- 172.21.15.94:0/2945203113 wait complete. 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72f48a700 1 Processor -- start 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72f48a700 1 -- start start 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72f48a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7280fc290 0x7fa728110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72f48a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 0x7fa728112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72f48a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280fbce0 con 0x7fa7280fcbe0 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72f48a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280fbe20 con 0x7fa7280fc290 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72ca25700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 0x7fa728112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72ca25700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 0x7fa728112b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48214/0 (socket says 172.21.15.94:48214) 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72ca25700 1 -- 172.21.15.94:0/3900949823 learned_addr learned my addr 172.21.15.94:0/3900949823 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.035+0000 7fa72d226700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7280fc290 0x7fa728110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.036+0000 7fa72ca25700 1 -- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7280fc290 msgr2=0x7fa728110640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.036+0000 7fa72ca25700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7280fc290 0x7fa728110640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.036+0000 7fa72ca25700 1 -- 172.21.15.94:0/3900949823 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa718005040 con 0x7fa7280fcbe0 2024-04-06T03:42:03.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.036+0000 7fa72ca25700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 0x7fa728112b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa718013870 tx=0x7fa718009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:03.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.036+0000 7fa71dffb700 1 -- 172.21.15.94:0/3900949823 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa718021070 con 0x7fa7280fcbe0 2024-04-06T03:42:03.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.036+0000 7fa71dffb700 1 -- 172.21.15.94:0/3900949823 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa718009430 con 0x7fa7280fcbe0 2024-04-06T03:42:03.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.036+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa728113130 con 0x7fa7280fcbe0 2024-04-06T03:42:03.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.036+0000 7fa71dffb700 1 -- 172.21.15.94:0/3900949823 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa71800e070 con 0x7fa7280fcbe0 2024-04-06T03:42:03.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.036+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa728113670 con 0x7fa7280fcbe0 2024-04-06T03:42:03.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.037+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72805f5f0 con 0x7fa7280fcbe0 2024-04-06T03:42:03.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.038+0000 7fa71dffb700 1 -- 172.21.15.94:0/3900949823 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 91575+0+0 (secure 0 0 0) 0x7fa71801c750 con 0x7fa7280fcbe0 2024-04-06T03:42:03.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.038+0000 7fa71dffb700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fa71406d530 0x7fa71406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:03.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.038+0000 7fa71dffb700 1 -- 172.21.15.94:0/3900949823 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa71808dca0 con 0x7fa7280fcbe0 2024-04-06T03:42:03.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.039+0000 7fa72d226700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fa71406d530 0x7fa71406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:03.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.039+0000 7fa72d226700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fa71406d530 0x7fa71406f9e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa7280fb960 tx=0x7fa72400b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:03.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.043+0000 7fa71dffb700 1 -- 172.21.15.94:0/3900949823 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fa718057c10 con 0x7fa7280fcbe0 2024-04-06T03:42:03.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:02 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:03.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:02 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:03.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:02 smithi136 ceph-mon[36436]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:03.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:02 smithi136 ceph-mon[36436]: pgmap v61: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 2.1 MiB/s wr, 1.14k op/s 2024-04-06T03:42:03.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.430+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa7281047b0 con 0x7fa7280fcbe0 2024-04-06T03:42:03.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.431+0000 7fa71dffb700 1 -- 172.21.15.94:0/3900949823 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fa71805b2d0 con 0x7fa7280fcbe0 2024-04-06T03:42:03.434 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:42:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:42:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:42:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:42:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 12, 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:42:03.435 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.433+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fa71406d530 msgr2=0x7fa71406f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.433+0000 7fa72f48a700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fa71406d530 0x7fa71406f9e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa7280fb960 tx=0x7fa72400b040 comp rx=0 tx=0).stop 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.433+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 msgr2=0x7fa728112b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.433+0000 7fa72f48a700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 0x7fa728112b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa718013870 tx=0x7fa718009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.434+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 shutdown_connections 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.434+0000 7fa72f48a700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fa71406d530 0x7fa71406f9e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.434+0000 7fa72f48a700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7280fc290 0x7fa728110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.434+0000 7fa72f48a700 1 --2- 172.21.15.94:0/3900949823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7280fcbe0 0x7fa728112b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.434+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 >> 172.21.15.94:0/3900949823 conn(0x7fa7280f6ba0 msgr2=0x7fa7280f81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:03.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.434+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 shutdown_connections 2024-04-06T03:42:03.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.434+0000 7fa72f48a700 1 -- 172.21.15.94:0/3900949823 wait complete. 2024-04-06T03:42:03.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.549+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/1482526846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd180fcf80 msgr2=0x7fbd180fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:03.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.549+0000 7fbd1cdf6700 1 --2- 172.21.15.94:0/1482526846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd180fcf80 0x7fbd180fd3f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c004940 tx=0x7fbd0c009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:03.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.549+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/1482526846 shutdown_connections 2024-04-06T03:42:03.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.549+0000 7fbd1cdf6700 1 --2- 172.21.15.94:0/1482526846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd180fcf80 0x7fbd180fd3f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.549+0000 7fbd1cdf6700 1 --2- 172.21.15.94:0/1482526846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd180fbcd0 0x7fbd180fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.549+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/1482526846 >> 172.21.15.94:0/1482526846 conn(0x7fbd180f7430 msgr2=0x7fbd180f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:03.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.549+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/1482526846 shutdown_connections 2024-04-06T03:42:03.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.549+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/1482526846 wait complete. 2024-04-06T03:42:03.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.550+0000 7fbd1cdf6700 1 Processor -- start 2024-04-06T03:42:03.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.550+0000 7fbd1cdf6700 1 -- start start 2024-04-06T03:42:03.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.550+0000 7fbd1cdf6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd180fbcd0 0x7fbd1810c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:03.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.550+0000 7fbd1cdf6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd180fcf80 0x7fbd1810ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:03.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.550+0000 7fbd1cdf6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1810d440 con 0x7fbd180fcf80 2024-04-06T03:42:03.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.550+0000 7fbd1cdf6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1810dc90 con 0x7fbd180fbcd0 2024-04-06T03:42:03.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.550+0000 7fbd1659c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd180fbcd0 0x7fbd1810c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.550+0000 7fbd1659c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd180fbcd0 0x7fbd1810c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:44668/0 (socket says 172.21.15.94:44668) 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.550+0000 7fbd1659c700 1 -- 172.21.15.94:0/3013833909 learned_addr learned my addr 172.21.15.94:0/3013833909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd15d9b700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd180fcf80 0x7fbd1810ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd1659c700 1 -- 172.21.15.94:0/3013833909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd180fcf80 msgr2=0x7fbd1810ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd1659c700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd180fcf80 0x7fbd1810ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd1659c700 1 -- 172.21.15.94:0/3013833909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c005040 con 0x7fbd180fbcd0 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd1659c700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd180fbcd0 0x7fbd1810c900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800c830 tx=0x7fbd08004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd06ffd700 1 -- 172.21.15.94:0/3013833909 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd080106f0 con 0x7fbd180fbcd0 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1810df70 con 0x7fbd180fbcd0 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd06ffd700 1 -- 172.21.15.94:0/3013833909 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd08010d20 con 0x7fbd180fbcd0 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd06ffd700 1 -- 172.21.15.94:0/3013833909 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd080113a0 con 0x7fbd180fbcd0 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.551+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1810e4b0 con 0x7fbd180fbcd0 2024-04-06T03:42:03.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.552+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1805f5f0 con 0x7fbd180fbcd0 2024-04-06T03:42:03.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.553+0000 7fbd06ffd700 1 -- 172.21.15.94:0/3013833909 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 27) v1 ==== 91575+0+0 (secure 0 0 0) 0x7fbd08012400 con 0x7fbd180fbcd0 2024-04-06T03:42:03.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.554+0000 7fbd06ffd700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fbd0006f7b0 0x7fbd00071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:03.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.554+0000 7fbd06ffd700 1 -- 172.21.15.94:0/3013833909 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fbd0808ca40 con 0x7fbd180fbcd0 2024-04-06T03:42:03.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.554+0000 7fbd15d9b700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fbd0006f7b0 0x7fbd00071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:03.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.554+0000 7fbd15d9b700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fbd0006f7b0 0x7fbd00071c60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b970 tx=0x7fbd0c009e30 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:03.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.558+0000 7fbd06ffd700 1 -- 172.21.15.94:0/3013833909 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fbd0805a4e0 con 0x7fbd180fbcd0 2024-04-06T03:42:03.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.943+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd1810e900 con 0x7fbd180fbcd0 2024-04-06T03:42:03.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.943+0000 7fbd06ffd700 1 -- 172.21.15.94:0/3013833909 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7fbd0805a070 con 0x7fbd180fbcd0 2024-04-06T03:42:03.946 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:42:03.947 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:42:03.948 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:03.949 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:03.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fbd0006f7b0 msgr2=0x7fbd00071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:03.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fbd0006f7b0 0x7fbd00071c60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b970 tx=0x7fbd0c009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:03.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd180fbcd0 msgr2=0x7fbd1810c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:03.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd180fbcd0 0x7fbd1810c900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800c830 tx=0x7fbd08004470 comp rx=0 tx=0).stop 2024-04-06T03:42:03.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 shutdown_connections 2024-04-06T03:42:03.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fbd0006f7b0 0x7fbd00071c60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd180fbcd0 0x7fbd1810c900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 --2- 172.21.15.94:0/3013833909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd180fcf80 0x7fbd1810ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:03.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 >> 172.21.15.94:0/3013833909 conn(0x7fbd180f7430 msgr2=0x7fbd180f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:03.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 shutdown_connections 2024-04-06T03:42:03.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:03.945+0000 7fbd1cdf6700 1 -- 172.21.15.94:0/3013833909 wait complete. 2024-04-06T03:42:03.951 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:42:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:03 smithi136 ceph-mon[36436]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:03 smithi136 ceph-mon[36436]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:04.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:03 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3900949823' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:42:04.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.131+0000 7f9c983f7700 1 -- 172.21.15.94:0/3375357099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 msgr2=0x7f9c9006ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:04.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.131+0000 7f9c983f7700 1 --2- 172.21.15.94:0/3375357099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 0x7f9c9006ee10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009ce0 tx=0x7f9c88004d10 comp rx=0 tx=0).stop 2024-04-06T03:42:04.135 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:03 smithi094 ceph-mon[29713]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:04.135 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:03 smithi094 ceph-mon[29713]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:04.135 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:03 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3900949823' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:42:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.132+0000 7f9c983f7700 1 -- 172.21.15.94:0/3375357099 shutdown_connections 2024-04-06T03:42:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.132+0000 7f9c983f7700 1 --2- 172.21.15.94:0/3375357099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 0x7f9c9006ee10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.132+0000 7f9c983f7700 1 --2- 172.21.15.94:0/3375357099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c9006f9d0 0x7f9c9006e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:04.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.132+0000 7f9c983f7700 1 -- 172.21.15.94:0/3375357099 >> 172.21.15.94:0/3375357099 conn(0x7f9c90069de0 msgr2=0x7f9c9006c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.136+0000 7f9c983f7700 1 -- 172.21.15.94:0/3375357099 shutdown_connections 2024-04-06T03:42:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.137+0000 7f9c983f7700 1 -- 172.21.15.94:0/3375357099 wait complete. 2024-04-06T03:42:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.137+0000 7f9c983f7700 1 Processor -- start 2024-04-06T03:42:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.137+0000 7f9c983f7700 1 -- start start 2024-04-06T03:42:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.137+0000 7f9c983f7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 0x7f9c90113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.137+0000 7f9c983f7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c9006f9d0 0x7f9c901144d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.137+0000 7f9c983f7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90114ad0 con 0x7f9c9006e9a0 2024-04-06T03:42:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.137+0000 7f9c983f7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90114c40 con 0x7f9c9006f9d0 2024-04-06T03:42:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.138+0000 7f9c95992700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c9006f9d0 0x7f9c901144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.138+0000 7f9c96193700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 0x7f9c90113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.138+0000 7f9c96193700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 0x7f9c90113f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48246/0 (socket says 172.21.15.94:48246) 2024-04-06T03:42:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.138+0000 7f9c96193700 1 -- 172.21.15.94:0/2675601000 learned_addr learned my addr 172.21.15.94:0/2675601000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.138+0000 7f9c96193700 1 -- 172.21.15.94:0/2675601000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c9006f9d0 msgr2=0x7f9c901144d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.138+0000 7f9c96193700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c9006f9d0 0x7f9c901144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.138+0000 7f9c96193700 1 -- 172.21.15.94:0/2675601000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88009940 con 0x7f9c9006e9a0 2024-04-06T03:42:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.139+0000 7f9c96193700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 0x7f9c90113f90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c01fb70 tx=0x7f9c8c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.139+0000 7f9c86ffd700 1 -- 172.21.15.94:0/2675601000 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c8c018cd0 con 0x7f9c9006e9a0 2024-04-06T03:42:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.139+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c90119640 con 0x7f9c9006e9a0 2024-04-06T03:42:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.139+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c90119c00 con 0x7f9c9006e9a0 2024-04-06T03:42:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.140+0000 7f9c86ffd700 1 -- 172.21.15.94:0/2675601000 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c8c020040 con 0x7f9c9006e9a0 2024-04-06T03:42:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.140+0000 7f9c86ffd700 1 -- 172.21.15.94:0/2675601000 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c8c018cd0 con 0x7f9c9006e9a0 2024-04-06T03:42:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.142+0000 7f9c86ffd700 1 -- 172.21.15.94:0/2675601000 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 91575+0+0 (secure 0 0 0) 0x7f9c8c01a4f0 con 0x7f9c9006e9a0 2024-04-06T03:42:04.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.142+0000 7f9c86ffd700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f9c7c06d650 0x7f9c7c06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:04.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.143+0000 7f9c86ffd700 1 -- 172.21.15.94:0/2675601000 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f9c8c0700b0 con 0x7f9c9006e9a0 2024-04-06T03:42:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.143+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c740052e0 con 0x7f9c9006e9a0 2024-04-06T03:42:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.143+0000 7f9c95992700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f9c7c06d650 0x7f9c7c06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:04.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.143+0000 7f9c95992700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f9c7c06d650 0x7f9c7c06fb00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800b460 tx=0x7f9c88004d10 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:04.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.152+0000 7f9c86ffd700 1 -- 172.21.15.94:0/2675601000 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f9c8c068a40 con 0x7f9c9006e9a0 2024-04-06T03:42:04.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.499+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 --> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c74000bf0 con 0x7f9c7c06d650 2024-04-06T03:42:04.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.500+0000 7f9c86ffd700 1 -- 172.21.15.94:0/2675601000 <== mgr.24351 v2:172.21.15.136:6828/614857857 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9c74000bf0 con 0x7f9c7c06d650 2024-04-06T03:42:04.503 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "2/21 daemons upgraded", 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mgr daemons", 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:42:04.504 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:42:04.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.502+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f9c7c06d650 msgr2=0x7f9c7c06fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:04.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.502+0000 7f9c983f7700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f9c7c06d650 0x7f9c7c06fb00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800b460 tx=0x7f9c88004d10 comp rx=0 tx=0).stop 2024-04-06T03:42:04.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.502+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 msgr2=0x7f9c90113f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:04.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.502+0000 7f9c983f7700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 0x7f9c90113f90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c01fb70 tx=0x7f9c8c01c040 comp rx=0 tx=0).stop 2024-04-06T03:42:04.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.503+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 shutdown_connections 2024-04-06T03:42:04.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.503+0000 7f9c983f7700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c9006e9a0 0x7f9c90113f90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:04.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.503+0000 7f9c983f7700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7f9c7c06d650 0x7f9c7c06fb00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:04.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.503+0000 7f9c983f7700 1 --2- 172.21.15.94:0/2675601000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c9006f9d0 0x7f9c901144d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:04.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.503+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 >> 172.21.15.94:0/2675601000 conn(0x7f9c90069de0 msgr2=0x7f9c9006b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:04.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.503+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 shutdown_connections 2024-04-06T03:42:04.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.503+0000 7f9c983f7700 1 -- 172.21.15.94:0/2675601000 wait complete. 2024-04-06T03:42:04.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 -- 172.21.15.94:0/3029285496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 msgr2=0x7fc86c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 --2- 172.21.15.94:0/3029285496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 0x7fc86c0fd3f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc860004940 tx=0x7fc860009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 -- 172.21.15.94:0/3029285496 shutdown_connections 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 --2- 172.21.15.94:0/3029285496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 0x7fc86c0fd3f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 --2- 172.21.15.94:0/3029285496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc86c0fbcd0 0x7fc86c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 -- 172.21.15.94:0/3029285496 >> 172.21.15.94:0/3029285496 conn(0x7fc86c0f7430 msgr2=0x7fc86c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 -- 172.21.15.94:0/3029285496 shutdown_connections 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 -- 172.21.15.94:0/3029285496 wait complete. 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 Processor -- start 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.650+0000 7fc870cc2700 1 -- start start 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc870cc2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc86c0fbcd0 0x7fc86c112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc870cc2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 0x7fc86c115050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc870cc2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c066240 con 0x7fc86c0fcf80 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc870cc2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c066380 con 0x7fc86c0fbcd0 2024-04-06T03:42:04.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc869d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 0x7fc86c115050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc869d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 0x7fc86c115050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48276/0 (socket says 172.21.15.94:48276) 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc869d9b700 1 -- 172.21.15.94:0/3088878383 learned_addr learned my addr 172.21.15.94:0/3088878383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc86a59c700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc86c0fbcd0 0x7fc86c112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc869d9b700 1 -- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc86c0fbcd0 msgr2=0x7fc86c112b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc869d9b700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc86c0fbcd0 0x7fc86c112b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc869d9b700 1 -- 172.21.15.94:0/3088878383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860005040 con 0x7fc86c0fcf80 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc869d9b700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 0x7fc86c115050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc860013870 tx=0x7fc860009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc85affd700 1 -- 172.21.15.94:0/3088878383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc86000e070 con 0x7fc86c0fcf80 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c115590 con 0x7fc86c0fcf80 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc85affd700 1 -- 172.21.15.94:0/3088878383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc860013de0 con 0x7fc86c0fcf80 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.651+0000 7fc85affd700 1 -- 172.21.15.94:0/3088878383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc86001cea0 con 0x7fc86c0fcf80 2024-04-06T03:42:04.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.652+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c1158e0 con 0x7fc86c0fcf80 2024-04-06T03:42:04.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.653+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c105020 con 0x7fc86c0fcf80 2024-04-06T03:42:04.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.653+0000 7fc85affd700 1 -- 172.21.15.94:0/3088878383 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 91575+0+0 (secure 0 0 0) 0x7fc86001c690 con 0x7fc86c0fcf80 2024-04-06T03:42:04.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.654+0000 7fc85affd700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc85406d5b0 0x7fc85406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:04.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.654+0000 7fc85affd700 1 -- 172.21.15.94:0/3088878383 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc86008e040 con 0x7fc86c0fcf80 2024-04-06T03:42:04.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.654+0000 7fc86a59c700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc85406d5b0 0x7fc85406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:04.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.654+0000 7fc86a59c700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc85406d5b0 0x7fc85406fa60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc86c065fd0 tx=0x7fc85c00b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:04.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:04.662+0000 7fc85affd700 1 -- 172.21.15.94:0/3088878383 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fc8600580f0 con 0x7fc86c0fcf80 2024-04-06T03:42:05.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.008+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc86c05f5f0 con 0x7fc86c0fcf80 2024-04-06T03:42:05.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.008+0000 7fc85affd700 1 -- 172.21.15.94:0/3088878383 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc860004490 con 0x7fc86c0fcf80 2024-04-06T03:42:05.012 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-04-06T03:42:05.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc85406d5b0 msgr2=0x7fc85406fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc85406d5b0 0x7fc85406fa60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc86c065fd0 tx=0x7fc85c00b040 comp rx=0 tx=0).stop 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 msgr2=0x7fc86c115050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 0x7fc86c115050 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc860013870 tx=0x7fc860009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 shutdown_connections 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:6828/614857857,v1:172.21.15.136:6829/614857857] conn(0x7fc85406d5b0 0x7fc85406fa60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc86c0fbcd0 0x7fc86c112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 --2- 172.21.15.94:0/3088878383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc86c0fcf80 0x7fc86c115050 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 >> 172.21.15.94:0/3088878383 conn(0x7fc86c0f7430 msgr2=0x7fc86c0f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 shutdown_connections 2024-04-06T03:42:05.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:05.010+0000 7fc870cc2700 1 -- 172.21.15.94:0/3088878383 wait complete. 2024-04-06T03:42:05.101 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:04 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3013833909' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:42:05.102 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:04 smithi136 ceph-mon[36436]: pgmap v62: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 635 op/s 2024-04-06T03:42:05.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:04 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3013833909' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:42:05.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:04 smithi094 ceph-mon[29713]: pgmap v62: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 635 op/s 2024-04-06T03:42:06.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:05 smithi136 ceph-mon[36436]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:06.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:05 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3088878383' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:42:06.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:05 smithi094 ceph-mon[29713]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:06.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:05 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3088878383' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:42:07.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:06 smithi136 ceph-mon[36436]: pgmap v63: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 635 op/s 2024-04-06T03:42:07.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:42:07.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:07.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:06 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:07.137 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:06 smithi094 ceph-mon[29713]: pgmap v63: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 635 op/s 2024-04-06T03:42:07.137 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:42:07.137 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:07.137 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:06 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:08.502 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:08 smithi094 ceph-mon[29713]: pgmap v64: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 2.0 MiB/s wr, 1.11k op/s 2024-04-06T03:42:08.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:08 smithi136 ceph-mon[36436]: pgmap v64: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 2.0 MiB/s wr, 1.11k op/s 2024-04-06T03:42:10.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:10 smithi094 ceph-mon[29713]: pgmap v65: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 890 op/s 2024-04-06T03:42:10.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:10 smithi136 ceph-mon[36436]: pgmap v65: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 890 op/s 2024-04-06T03:42:12.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:12 smithi136 ceph-mon[36436]: pgmap v66: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.0 MiB/s wr, 1.12k op/s 2024-04-06T03:42:12.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:12 smithi094 ceph-mon[29713]: pgmap v66: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 2.0 MiB/s wr, 1.12k op/s 2024-04-06T03:42:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:42:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: pgmap v67: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.5 MiB/s wr, 772 op/s 2024-04-06T03:42:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mgr fail", "who": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: osdmap e47: 6 total, 6 up, 6 in 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd='[{"prefix": "mgr fail", "who": "smithi136.jagvwt"}]': finished 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: mgrmap e28: smithi094.purlzp(active, starting, since 0.00654271s) 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:42:14.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: Manager daemon smithi094.purlzp is now available 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:42:14.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: [06/Apr/2024:03:42:13] ENGINE Bus STARTING 2024-04-06T03:42:14.353 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:14 smithi136 ceph-mon[36436]: [06/Apr/2024:03:42:13] ENGINE Serving on https://172.21.15.94:7150 2024-04-06T03:42:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:42:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: pgmap v67: 65 pgs: 65 active+clean; 263 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.5 MiB/s wr, 772 op/s 2024-04-06T03:42:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "mgr fail", "who": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: osdmap e47: 6 total, 6 up, 6 in 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.24351 172.21.15.136:0/1836316826' entity='mgr.smithi136.jagvwt' cmd='[{"prefix": "mgr fail", "who": "smithi136.jagvwt"}]': finished 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: mgrmap e28: smithi094.purlzp(active, starting, since 0.00654271s) 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:42:14.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:42:14.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: Manager daemon smithi094.purlzp is now available 2024-04-06T03:42:14.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:14.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:42:14.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:42:14.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:42:14.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:42:14.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:42:14.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: [06/Apr/2024:03:42:13] ENGINE Bus STARTING 2024-04-06T03:42:14.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:14 smithi094 ceph-mon[29713]: [06/Apr/2024:03:42:13] ENGINE Serving on https://172.21.15.94:7150 2024-04-06T03:42:15.255 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:15 smithi136 ceph-mon[36436]: [06/Apr/2024:03:42:14] ENGINE Serving on http://172.21.15.94:8765 2024-04-06T03:42:15.255 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:15 smithi136 ceph-mon[36436]: [06/Apr/2024:03:42:14] ENGINE Bus STARTED 2024-04-06T03:42:15.255 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:15 smithi136 ceph-mon[36436]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-06T03:42:15.255 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:15 smithi136 ceph-mon[36436]: mgrmap e29: smithi094.purlzp(active, since 1.00978s) 2024-04-06T03:42:15.255 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:15 smithi136 ceph-mon[36436]: pgmap v3: 65 pgs: 65 active+clean; 251 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:15.255 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:15 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:15.255 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:15 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:15 smithi094 ceph-mon[29713]: [06/Apr/2024:03:42:14] ENGINE Serving on http://172.21.15.94:8765 2024-04-06T03:42:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:15 smithi094 ceph-mon[29713]: [06/Apr/2024:03:42:14] ENGINE Bus STARTED 2024-04-06T03:42:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:15 smithi094 ceph-mon[29713]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-06T03:42:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:15 smithi094 ceph-mon[29713]: mgrmap e29: smithi094.purlzp(active, since 1.00978s) 2024-04-06T03:42:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:15 smithi094 ceph-mon[29713]: pgmap v3: 65 pgs: 65 active+clean; 251 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:15 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:15 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:16.227 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:16 smithi094 ceph-mon[29713]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-06T03:42:16.227 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:16 smithi094 ceph-mon[29713]: Cluster is now healthy 2024-04-06T03:42:16.227 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:16 smithi094 ceph-mon[29713]: pgmap v4: 65 pgs: 65 active+clean; 251 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:16.301 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:16 smithi136 ceph-mon[36436]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-06T03:42:16.301 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:16 smithi136 ceph-mon[36436]: Cluster is now healthy 2024-04-06T03:42:16.301 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:16 smithi136 ceph-mon[36436]: pgmap v4: 65 pgs: 65 active+clean; 251 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:17.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:17 smithi136 ceph-mon[36436]: mgrmap e30: smithi094.purlzp(active, since 3s) 2024-04-06T03:42:17.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:17 smithi094 ceph-mon[29713]: mgrmap e30: smithi094.purlzp(active, since 3s) 2024-04-06T03:42:18.551 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:18 smithi094 ceph-mon[29713]: pgmap v5: 65 pgs: 65 active+clean; 251 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:18.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:18 smithi136 ceph-mon[36436]: pgmap v5: 65 pgs: 65 active+clean; 251 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:19.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:19 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:19.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:19 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:19.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:19 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:19.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:19 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:20.744 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:20 smithi094 ceph-mon[29713]: pgmap v6: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 448 KiB/s wr, 204 op/s 2024-04-06T03:42:20.745 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:20 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:20.745 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:20 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:20.750 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:20 smithi136 ceph-mon[36436]: pgmap v6: 65 pgs: 65 active+clean; 252 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 448 KiB/s wr, 204 op/s 2024-04-06T03:42:20.750 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:20 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:20.750 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:20 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:21.762 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:21 smithi136 ceph-mon[36436]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:42:21.762 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:21 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1180432416' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:42:21.762 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:21 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1180432416' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:42:21.762 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:21 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1180432416' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:42:21.763 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:21 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/1180432416' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:42:21.837 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:21 smithi094 ceph-mon[29713]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:42:21.837 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:21 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1180432416' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:42:21.837 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:21 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1180432416' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:42:21.837 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:21 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1180432416' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:42:21.837 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:21 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/1180432416' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:42:22.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:22 smithi136 ceph-mon[36436]: pgmap v7: 65 pgs: 65 active+clean; 242 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 1.1 MiB/s wr, 774 op/s 2024-04-06T03:42:22.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:22 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi136.jagvwt", "id": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:42:22.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:22 smithi136 ceph-mon[36436]: mgrmap e31: smithi094.purlzp(active, since 8s), standbys: smithi136.jagvwt 2024-04-06T03:42:22.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:22 smithi094 ceph-mon[29713]: pgmap v7: 65 pgs: 65 active+clean; 242 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 1.1 MiB/s wr, 774 op/s 2024-04-06T03:42:22.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:22 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi136.jagvwt", "id": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:42:22.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:22 smithi094 ceph-mon[29713]: mgrmap e31: smithi094.purlzp(active, since 8s), standbys: smithi136.jagvwt 2024-04-06T03:42:23.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:23 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:23.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:23 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:24.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:23 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:24.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:23 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:24.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:24 smithi094 ceph-mon[29713]: pgmap v8: 65 pgs: 65 active+clean; 242 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 38 KiB/s rd, 901 KiB/s wr, 602 op/s 2024-04-06T03:42:24.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:24 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:24.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:24 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:25.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:24 smithi136 ceph-mon[36436]: pgmap v8: 65 pgs: 65 active+clean; 242 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 38 KiB/s rd, 901 KiB/s wr, 602 op/s 2024-04-06T03:42:25.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:24 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:25.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:24 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:26.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:26 smithi094 ceph-mon[29713]: pgmap v9: 65 pgs: 65 active+clean; 224 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.4 MiB/s wr, 692 op/s 2024-04-06T03:42:26.958 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:26 smithi136 ceph-mon[36436]: pgmap v9: 65 pgs: 65 active+clean; 224 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.4 MiB/s wr, 692 op/s 2024-04-06T03:42:28.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:28 smithi094 ceph-mon[29713]: pgmap v10: 65 pgs: 65 active+clean; 224 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.3 MiB/s wr, 633 op/s 2024-04-06T03:42:28.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:28 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:42:28.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:28 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:28.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:28 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:28.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:28 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:28.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:28 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:42:29.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:28 smithi136 ceph-mon[36436]: pgmap v10: 65 pgs: 65 active+clean; 224 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.3 MiB/s wr, 633 op/s 2024-04-06T03:42:29.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:28 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:42:29.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:28 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:29.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:28 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:29.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:28 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:29.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:28 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:42:29.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:29.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:29 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:30.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:29 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:30.300 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-06T03:42:30.300 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-06T03:42:30.629 DEBUG:teuthology.parallel:result is None 2024-04-06T03:42:30.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:30 smithi094 ceph-mon[29713]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:42:30.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:30 smithi094 ceph-mon[29713]: pgmap v11: 65 pgs: 65 active+clean; 221 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 680 op/s 2024-04-06T03:42:30.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:30 smithi094 ceph-mon[29713]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:42:30.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:30 smithi094 ceph-mon[29713]: pgmap v12: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 66 KiB/s rd, 2.1 MiB/s wr, 863 op/s 2024-04-06T03:42:30.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:30 smithi094 ceph-mon[29713]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-06T03:42:30.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:30 smithi094 ceph-mon[29713]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-04-06T03:42:30.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:30 smithi094 ceph-mon[29713]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-04-06T03:42:31.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:30 smithi136 ceph-mon[36436]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:42:31.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:30 smithi136 ceph-mon[36436]: pgmap v11: 65 pgs: 65 active+clean; 221 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 680 op/s 2024-04-06T03:42:31.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:30 smithi136 ceph-mon[36436]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:42:31.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:30 smithi136 ceph-mon[36436]: pgmap v12: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 66 KiB/s rd, 2.1 MiB/s wr, 863 op/s 2024-04-06T03:42:31.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:30 smithi136 ceph-mon[36436]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-06T03:42:31.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:30 smithi136 ceph-mon[36436]: Reconfiguring prometheus.smithi094 (dependencies changed)... 2024-04-06T03:42:31.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:30 smithi136 ceph-mon[36436]: Reconfiguring daemon prometheus.smithi094 on smithi094 2024-04-06T03:42:32.358 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-06T03:42:32.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-06T03:42:32.713 DEBUG:teuthology.parallel:result is None 2024-04-06T03:42:32.714 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-06T03:42:32.779 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-06T03:42:32.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-06T03:42:32.811 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:32 smithi136 ceph-mon[36436]: pgmap v13: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 436 op/s 2024-04-06T03:42:32.818 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-06T03:42:32.818 DEBUG:teuthology.parallel:result is None 2024-04-06T03:42:32.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:32 smithi094 ceph-mon[29713]: pgmap v13: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 436 op/s 2024-04-06T03:42:34.712 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:34 smithi094 ceph-mon[29713]: pgmap v14: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 436 op/s 2024-04-06T03:42:35.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:34 smithi136 ceph-mon[36436]: pgmap v14: 65 pgs: 65 active+clean; 218 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.6 MiB/s wr, 436 op/s 2024-04-06T03:42:35.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.117+0000 7f174a82c700 1 -- 172.21.15.94:0/79252195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fe050 msgr2=0x7f17440fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.117+0000 7f174a82c700 1 --2- 172.21.15.94:0/79252195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fe050 0x7f17440fe4c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1738004940 tx=0x7f1738009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.117+0000 7f174a82c700 1 -- 172.21.15.94:0/79252195 shutdown_connections 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.117+0000 7f174a82c700 1 --2- 172.21.15.94:0/79252195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fe050 0x7f17440fe4c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.117+0000 7f174a82c700 1 --2- 172.21.15.94:0/79252195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17440fce50 0x7f17440fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.117+0000 7f174a82c700 1 -- 172.21.15.94:0/79252195 >> 172.21.15.94:0/79252195 conn(0x7f17440f8580 msgr2=0x7f17440fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.117+0000 7f174a82c700 1 -- 172.21.15.94:0/79252195 shutdown_connections 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.117+0000 7f174a82c700 1 -- 172.21.15.94:0/79252195 wait complete. 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f174a82c700 1 Processor -- start 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f174a82c700 1 -- start start 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f174a82c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fce50 0x7f174410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f174a82c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17440fe050 0x7f174410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f174a82c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174410e570 con 0x7f17440fce50 2024-04-06T03:42:35.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f174a82c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744112ec0 con 0x7f17440fe050 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f1743fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fce50 0x7f174410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f1743fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fce50 0x7f174410d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55168/0 (socket says 172.21.15.94:55168) 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f1743fff700 1 -- 172.21.15.94:0/1678441836 learned_addr learned my addr 172.21.15.94:0/1678441836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.118+0000 7f17437fe700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17440fe050 0x7f174410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.119+0000 7f1743fff700 1 -- 172.21.15.94:0/1678441836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17440fe050 msgr2=0x7f174410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.119+0000 7f1743fff700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17440fe050 0x7f174410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.119+0000 7f1743fff700 1 -- 172.21.15.94:0/1678441836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1738005040 con 0x7f17440fce50 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.119+0000 7f1743fff700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fce50 0x7f174410d9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f173400b790 tx=0x7f1734009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.119+0000 7f1740ff9700 1 -- 172.21.15.94:0/1678441836 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173400f6f0 con 0x7f17440fce50 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.119+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1744113120 con 0x7f17440fce50 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.119+0000 7f1740ff9700 1 -- 172.21.15.94:0/1678441836 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f173400fd20 con 0x7f17440fce50 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.119+0000 7f1740ff9700 1 -- 172.21.15.94:0/1678441836 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17340103a0 con 0x7f17440fce50 2024-04-06T03:42:35.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.119+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1744113690 con 0x7f17440fce50 2024-04-06T03:42:35.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.120+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174404c710 con 0x7f17440fce50 2024-04-06T03:42:35.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.121+0000 7f1740ff9700 1 -- 172.21.15.94:0/1678441836 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 31) v1 ==== 91675+0+0 (secure 0 0 0) 0x7f1734011400 con 0x7f17440fce50 2024-04-06T03:42:35.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.121+0000 7f1740ff9700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f172c06d6d0 0x7f172c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:35.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.121+0000 7f1740ff9700 1 -- 172.21.15.94:0/1678441836 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f173408c9c0 con 0x7f17440fce50 2024-04-06T03:42:35.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.121+0000 7f17437fe700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f172c06d6d0 0x7f172c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:35.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.122+0000 7f17437fe700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f172c06d6d0 0x7f172c06fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f173800b970 tx=0x7f1738009e30 comp rx=0 tx=0).ready entity=mgr.14734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:35.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.126+0000 7f1740ff9700 1 -- 172.21.15.94:0/1678441836 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f173405a390 con 0x7f17440fce50 2024-04-06T03:42:35.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.341+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 --> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1744102330 con 0x7f172c06d6d0 2024-04-06T03:42:35.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.343+0000 7f1740ff9700 1 -- 172.21.15.94:0/1678441836 <== mgr.14734 v2:172.21.15.94:6800/4259523022 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f1744102330 con 0x7f172c06d6d0 2024-04-06T03:42:35.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f172c06d6d0 msgr2=0x7f172c06fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f172c06d6d0 0x7f172c06fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f173800b970 tx=0x7f1738009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:35.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fce50 msgr2=0x7f174410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fce50 0x7f174410d9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f173400b790 tx=0x7f1734009220 comp rx=0 tx=0).stop 2024-04-06T03:42:35.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 shutdown_connections 2024-04-06T03:42:35.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f172c06d6d0 0x7f172c06fb80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17440fce50 0x7f174410d9a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 --2- 172.21.15.94:0/1678441836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17440fe050 0x7f174410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 >> 172.21.15.94:0/1678441836 conn(0x7f17440f8580 msgr2=0x7f17440f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:35.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 shutdown_connections 2024-04-06T03:42:35.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.345+0000 7f174a82c700 1 -- 172.21.15.94:0/1678441836 wait complete. 2024-04-06T03:42:35.359 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:42:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.456+0000 7fe5f908b700 1 -- 172.21.15.94:0/111345748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fe0d0 msgr2=0x7fe5f40fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.456+0000 7fe5f908b700 1 --2- 172.21.15.94:0/111345748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fe0d0 0x7fe5f40fe540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8004940 tx=0x7fe5e8009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.456+0000 7fe5f908b700 1 -- 172.21.15.94:0/111345748 shutdown_connections 2024-04-06T03:42:35.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.456+0000 7fe5f908b700 1 --2- 172.21.15.94:0/111345748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fe0d0 0x7fe5f40fe540 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.456+0000 7fe5f908b700 1 --2- 172.21.15.94:0/111345748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5f40fcee0 0x7fe5f40fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.456+0000 7fe5f908b700 1 -- 172.21.15.94:0/111345748 >> 172.21.15.94:0/111345748 conn(0x7fe5f40f8560 msgr2=0x7fe5f40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.456+0000 7fe5f908b700 1 -- 172.21.15.94:0/111345748 shutdown_connections 2024-04-06T03:42:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.456+0000 7fe5f908b700 1 -- 172.21.15.94:0/111345748 wait complete. 2024-04-06T03:42:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.456+0000 7fe5f908b700 1 Processor -- start 2024-04-06T03:42:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f908b700 1 -- start start 2024-04-06T03:42:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f908b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fcee0 0x7fe5f410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:35.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f908b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5f40fe0d0 0x7fe5f410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f908b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f410e550 con 0x7fe5f40fcee0 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f908b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f410ed50 con 0x7fe5f40fe0d0 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fcee0 0x7fe5f410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fcee0 0x7fe5f410da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55192/0 (socket says 172.21.15.94:55192) 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f2d9d700 1 -- 172.21.15.94:0/2751734726 learned_addr learned my addr 172.21.15.94:0/2751734726 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f259c700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5f40fe0d0 0x7fe5f410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f2d9d700 1 -- 172.21.15.94:0/2751734726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5f40fe0d0 msgr2=0x7fe5f410df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f2d9d700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5f40fe0d0 0x7fe5f410df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f2d9d700 1 -- 172.21.15.94:0/2751734726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8005040 con 0x7fe5f40fcee0 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f259c700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5f40fe0d0 0x7fe5f410df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f2d9d700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fcee0 0x7fe5f410da10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400eb70 tx=0x7fe5e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5db7fe700 1 -- 172.21.15.94:0/2751734726 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e4007cc0 con 0x7fe5f40fcee0 2024-04-06T03:42:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f410efd0 con 0x7fe5f40fcee0 2024-04-06T03:42:35.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5db7fe700 1 -- 172.21.15.94:0/2751734726 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5e400f040 con 0x7fe5f40fcee0 2024-04-06T03:42:35.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.457+0000 7fe5db7fe700 1 -- 172.21.15.94:0/2751734726 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e400cbd0 con 0x7fe5f40fcee0 2024-04-06T03:42:35.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.458+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f410f510 con 0x7fe5f40fcee0 2024-04-06T03:42:35.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.459+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f405f5f0 con 0x7fe5f40fcee0 2024-04-06T03:42:35.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.459+0000 7fe5db7fe700 1 -- 172.21.15.94:0/2751734726 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 31) v1 ==== 91675+0+0 (secure 0 0 0) 0x7fe5e400cd30 con 0x7fe5f40fcee0 2024-04-06T03:42:35.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.459+0000 7fe5db7fe700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fe5dc06d6c0 0x7fe5dc06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:35.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.459+0000 7fe5db7fe700 1 -- 172.21.15.94:0/2751734726 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fe5e408c7d0 con 0x7fe5f40fcee0 2024-04-06T03:42:35.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.460+0000 7fe5f259c700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fe5dc06d6c0 0x7fe5dc06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:35.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.460+0000 7fe5f259c700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fe5dc06d6c0 0x7fe5dc06fb70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009b30 tx=0x7fe5e8009e30 comp rx=0 tx=0).ready entity=mgr.14734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:35.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.464+0000 7fe5db7fe700 1 -- 172.21.15.94:0/2751734726 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fe5e405a0a0 con 0x7fe5f40fcee0 2024-04-06T03:42:35.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.687+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 --> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5f410f850 con 0x7fe5dc06d6c0 2024-04-06T03:42:35.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.689+0000 7fe5db7fe700 1 -- 172.21.15.94:0/2751734726 <== mgr.14734 v2:172.21.15.94:6800/4259523022 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fe5f410f850 con 0x7fe5dc06d6c0 2024-04-06T03:42:35.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fe5dc06d6c0 msgr2=0x7fe5dc06fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fe5dc06d6c0 0x7fe5dc06fb70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009b30 tx=0x7fe5e8009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:35.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fcee0 msgr2=0x7fe5f410da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fcee0 0x7fe5f410da10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400eb70 tx=0x7fe5e400b040 comp rx=0 tx=0).stop 2024-04-06T03:42:35.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 shutdown_connections 2024-04-06T03:42:35.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fe5dc06d6c0 0x7fe5dc06fb70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5f40fcee0 0x7fe5f410da10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 --2- 172.21.15.94:0/2751734726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5f40fe0d0 0x7fe5f410df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 >> 172.21.15.94:0/2751734726 conn(0x7fe5f40f8560 msgr2=0x7fe5f40f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:35.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 shutdown_connections 2024-04-06T03:42:35.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.691+0000 7fe5f908b700 1 -- 172.21.15.94:0/2751734726 wait complete. 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.793+0000 7f9616cd7700 1 -- 172.21.15.94:0/1179493753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fedd0 msgr2=0x7f96100faf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.793+0000 7f9616cd7700 1 --2- 172.21.15.94:0/1179493753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fedd0 0x7f96100faf10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9600009ce0 tx=0x7f9600004d10 comp rx=0 tx=0).stop 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.793+0000 7f9616cd7700 1 -- 172.21.15.94:0/1179493753 shutdown_connections 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.793+0000 7f9616cd7700 1 --2- 172.21.15.94:0/1179493753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fedd0 0x7f96100faf10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.793+0000 7f9616cd7700 1 --2- 172.21.15.94:0/1179493753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96100fe480 0x7f96100fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.793+0000 7f9616cd7700 1 -- 172.21.15.94:0/1179493753 >> 172.21.15.94:0/1179493753 conn(0x7f96100f6ba0 msgr2=0x7f96100f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.793+0000 7f9616cd7700 1 -- 172.21.15.94:0/1179493753 shutdown_connections 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.793+0000 7f9616cd7700 1 -- 172.21.15.94:0/1179493753 wait complete. 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f9616cd7700 1 Processor -- start 2024-04-06T03:42:35.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f9616cd7700 1 -- start start 2024-04-06T03:42:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f9616cd7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fe480 0x7f961010db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f9616cd7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96100fedd0 0x7f961010e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f9616cd7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961010e760 con 0x7f96100fe480 2024-04-06T03:42:35.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f9616cd7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961010ef50 con 0x7f96100fedd0 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f9614a73700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fe480 0x7f961010db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f9614a73700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fe480 0x7f961010db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55214/0 (socket says 172.21.15.94:55214) 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f9614a73700 1 -- 172.21.15.94:0/992451064 learned_addr learned my addr 172.21.15.94:0/992451064 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.794+0000 7f960ffff700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96100fedd0 0x7f961010e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.795+0000 7f9614a73700 1 -- 172.21.15.94:0/992451064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96100fedd0 msgr2=0x7f961010e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.795+0000 7f9614a73700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96100fedd0 0x7f961010e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.795+0000 7f9614a73700 1 -- 172.21.15.94:0/992451064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600009940 con 0x7f96100fe480 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.795+0000 7f9614a73700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fe480 0x7f961010db90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f960800b790 tx=0x7f9608009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.795+0000 7f960d7fa700 1 -- 172.21.15.94:0/992451064 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f960800f770 con 0x7f96100fe480 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.795+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961010f230 con 0x7f96100fe480 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.795+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961010f7f0 con 0x7f96100fe480 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.795+0000 7f960d7fa700 1 -- 172.21.15.94:0/992451064 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f960800fda0 con 0x7f96100fe480 2024-04-06T03:42:35.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.795+0000 7f960d7fa700 1 -- 172.21.15.94:0/992451064 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f960800d310 con 0x7f96100fe480 2024-04-06T03:42:35.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.796+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961005f5f0 con 0x7f96100fe480 2024-04-06T03:42:35.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.796+0000 7f960d7fa700 1 -- 172.21.15.94:0/992451064 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 31) v1 ==== 91675+0+0 (secure 0 0 0) 0x7f9608010420 con 0x7f96100fe480 2024-04-06T03:42:35.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.797+0000 7f960d7fa700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f95fc06d6c0 0x7f95fc06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:35.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.797+0000 7f960d7fa700 1 -- 172.21.15.94:0/992451064 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f960808cc00 con 0x7f96100fe480 2024-04-06T03:42:35.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.797+0000 7f960ffff700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f95fc06d6c0 0x7f95fc06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:35.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.797+0000 7f960ffff700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f95fc06d6c0 0x7f95fc06fb70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96000095a0 tx=0x7f9600004d10 comp rx=0 tx=0).ready entity=mgr.14734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:35.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:35.801+0000 7f960d7fa700 1 -- 172.21.15.94:0/992451064 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f9608092020 con 0x7f96100fe480 2024-04-06T03:42:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.020+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 --> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f961010fb30 con 0x7f95fc06d6c0 2024-04-06T03:42:36.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.030+0000 7f960d7fa700 1 -- 172.21.15.94:0/992451064 <== mgr.14734 v2:172.21.15.94:6800/4259523022 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f961010fb30 con 0x7f95fc06d6c0 2024-04-06T03:42:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:42:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (101s) 13s ago 9m 19.0M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (10m) 13s ago 10m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (9m) 17s ago 9m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 13s ago 9m - - 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (4m) 13s ago 4m 13.7M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (4m) 13s ago 4m 224M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (4m) 17s ago 4m 13.7M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (4m) 17s ago 4m 16.1M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (45s) 13s ago 11m 478M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283 running (2m) 17s ago 9m 79.9M - 19.0.0-2027-g9f192350 a5511c7deba9 7f3bea4f4172 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (11m) 13s ago 11m 51.7M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (8m) 17s ago 8m 41.6M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (2m) 13s ago 10m 8208k - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:42:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (59s) 17s ago 9m 9.91M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:42:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (8m) 13s ago 8m 235M 1557M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:42:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (7m) 13s ago 7m 251M 1557M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:42:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (7m) 13s ago 7m 215M 1557M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:42:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (6m) 17s ago 6m 281M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:42:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (6m) 17s ago 6m 247M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:42:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (6m) 17s ago 6m 229M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:42:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (63s) 13s ago 9m 46.1M - 2.43.0 a07b618ecd1d 51f6660abb0d 2024-04-06T03:42:36.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f95fc06d6c0 msgr2=0x7f95fc06fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f95fc06d6c0 0x7f95fc06fb70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96000095a0 tx=0x7f9600004d10 comp rx=0 tx=0).stop 2024-04-06T03:42:36.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fe480 msgr2=0x7f961010db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fe480 0x7f961010db90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f960800b790 tx=0x7f9608009250 comp rx=0 tx=0).stop 2024-04-06T03:42:36.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 shutdown_connections 2024-04-06T03:42:36.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f95fc06d6c0 0x7f95fc06fb70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96100fe480 0x7f961010db90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 --2- 172.21.15.94:0/992451064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96100fedd0 0x7f961010e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 >> 172.21.15.94:0/992451064 conn(0x7f96100f6ba0 msgr2=0x7f96100f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 shutdown_connections 2024-04-06T03:42:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.032+0000 7f9616cd7700 1 -- 172.21.15.94:0/992451064 wait complete. 2024-04-06T03:42:36.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.133+0000 7f7ee1f02700 1 -- 172.21.15.94:0/1734670970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fcf80 msgr2=0x7f7edc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.133+0000 7f7ee1f02700 1 --2- 172.21.15.94:0/1734670970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fcf80 0x7f7edc0fd3f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4004940 tx=0x7f7ec4009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:36.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.133+0000 7f7ee1f02700 1 -- 172.21.15.94:0/1734670970 shutdown_connections 2024-04-06T03:42:36.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.133+0000 7f7ee1f02700 1 --2- 172.21.15.94:0/1734670970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fcf80 0x7f7edc0fd3f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.133+0000 7f7ee1f02700 1 --2- 172.21.15.94:0/1734670970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7edc0fbcd0 0x7f7edc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.133+0000 7f7ee1f02700 1 -- 172.21.15.94:0/1734670970 >> 172.21.15.94:0/1734670970 conn(0x7f7edc0f7430 msgr2=0x7f7edc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:36.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.133+0000 7f7ee1f02700 1 -- 172.21.15.94:0/1734670970 shutdown_connections 2024-04-06T03:42:36.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.133+0000 7f7ee1f02700 1 -- 172.21.15.94:0/1734670970 wait complete. 2024-04-06T03:42:36.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.133+0000 7f7ee1f02700 1 Processor -- start 2024-04-06T03:42:36.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7ee1f02700 1 -- start start 2024-04-06T03:42:36.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7ee1f02700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fbcd0 0x7f7edc112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:36.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7ee1f02700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7edc0fcf80 0x7f7edc1150b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:36.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7ee1f02700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc066240 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7ee1f02700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc066380 con 0x7f7edc0fcf80 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7edb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fbcd0 0x7f7edc112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7edb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fbcd0 0x7f7edc112b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55234/0 (socket says 172.21.15.94:55234) 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7edb7fe700 1 -- 172.21.15.94:0/3555350057 learned_addr learned my addr 172.21.15.94:0/3555350057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7edb7fe700 1 -- 172.21.15.94:0/3555350057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7edc0fcf80 msgr2=0x7f7edc1150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7edb7fe700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7edc0fcf80 0x7f7edc1150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7edb7fe700 1 -- 172.21.15.94:0/3555350057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec4005040 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7edb7fe700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fbcd0 0x7f7edc112b60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc002a50 tx=0x7f7ecc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7ed8ff9700 1 -- 172.21.15.94:0/3555350057 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ecc007cc0 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7ed8ff9700 1 -- 172.21.15.94:0/3555350057 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ecc00f040 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc1155f0 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.134+0000 7f7ed8ff9700 1 -- 172.21.15.94:0/3555350057 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ecc00db70 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.135+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc115a10 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.136+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc05f5f0 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.136+0000 7f7ed8ff9700 1 -- 172.21.15.94:0/3555350057 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 31) v1 ==== 91675+0+0 (secure 0 0 0) 0x7f7ecc00dd10 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.136+0000 7f7ed8ff9700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f7ec806d6c0 0x7f7ec806fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:36.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.136+0000 7f7ed8ff9700 1 -- 172.21.15.94:0/3555350057 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f7ecc08c4d0 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.137+0000 7f7ed2ffd700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f7ec806d6c0 0x7f7ec806fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:36.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.137+0000 7f7ed2ffd700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f7ec806d6c0 0x7f7ec806fb70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4004f90 tx=0x7f7ec4009e30 comp rx=0 tx=0).ready entity=mgr.14734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:36.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.141+0000 7f7ed8ff9700 1 -- 172.21.15.94:0/3555350057 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f7ecc056140 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.502+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7edc1047e0 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.502+0000 7f7ed8ff9700 1 -- 172.21.15.94:0/3555350057 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f7ecc059800 con 0x7f7edc0fbcd0 2024-04-06T03:42:36.506 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:42:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:42:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2 2024-04-06T03:42:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:42:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:42:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 12, 2024-04-06T03:42:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:42:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:42:36.508 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:42:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f7ec806d6c0 msgr2=0x7f7ec806fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f7ec806d6c0 0x7f7ec806fb70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4004f90 tx=0x7f7ec4009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fbcd0 msgr2=0x7f7edc112b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fbcd0 0x7f7edc112b60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc002a50 tx=0x7f7ecc0046b0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 shutdown_connections 2024-04-06T03:42:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f7ec806d6c0 0x7f7ec806fb70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7edc0fbcd0 0x7f7edc112b60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 --2- 172.21.15.94:0/3555350057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7edc0fcf80 0x7f7edc1150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 >> 172.21.15.94:0/3555350057 conn(0x7f7edc0f7430 msgr2=0x7f7edc0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 shutdown_connections 2024-04-06T03:42:36.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.505+0000 7f7ee1f02700 1 -- 172.21.15.94:0/3555350057 wait complete. 2024-04-06T03:42:36.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.618+0000 7fde5d5ce700 1 -- 172.21.15.94:0/3094369541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde58066ef0 msgr2=0x7fde580ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.618+0000 7fde5d5ce700 1 --2- 172.21.15.94:0/3094369541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde58066ef0 0x7fde580ffa40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fde48004940 tx=0x7fde48009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:36.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.618+0000 7fde5d5ce700 1 -- 172.21.15.94:0/3094369541 shutdown_connections 2024-04-06T03:42:36.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.618+0000 7fde5d5ce700 1 --2- 172.21.15.94:0/3094369541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde580fff80 0x7fde58102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.618+0000 7fde5d5ce700 1 --2- 172.21.15.94:0/3094369541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde58066ef0 0x7fde580ffa40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.618+0000 7fde5d5ce700 1 -- 172.21.15.94:0/3094369541 >> 172.21.15.94:0/3094369541 conn(0x7fde580f7430 msgr2=0x7fde580f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:36.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.618+0000 7fde5d5ce700 1 -- 172.21.15.94:0/3094369541 shutdown_connections 2024-04-06T03:42:36.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.618+0000 7fde5d5ce700 1 -- 172.21.15.94:0/3094369541 wait complete. 2024-04-06T03:42:36.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde5d5ce700 1 Processor -- start 2024-04-06T03:42:36.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde5d5ce700 1 -- start start 2024-04-06T03:42:36.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde5d5ce700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde58066ef0 0x7fde5810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:36.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde5d5ce700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde580fff80 0x7fde5810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:36.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde5d5ce700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5810e4b0 con 0x7fde580fff80 2024-04-06T03:42:36.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde5d5ce700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5810e620 con 0x7fde58066ef0 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde567fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde580fff80 0x7fde5810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde567fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde580fff80 0x7fde5810deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55262/0 (socket says 172.21.15.94:55262) 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde567fc700 1 -- 172.21.15.94:0/2071569709 learned_addr learned my addr 172.21.15.94:0/2071569709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde56ffd700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde58066ef0 0x7fde5810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde567fc700 1 -- 172.21.15.94:0/2071569709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde58066ef0 msgr2=0x7fde5810d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde567fc700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde58066ef0 0x7fde5810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.619+0000 7fde567fc700 1 -- 172.21.15.94:0/2071569709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48005040 con 0x7fde580fff80 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.620+0000 7fde567fc700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde580fff80 0x7fde5810deb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00c830 tx=0x7fde4c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.620+0000 7fde3f7fe700 1 -- 172.21.15.94:0/2071569709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde4c00a760 con 0x7fde580fff80 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.620+0000 7fde3f7fe700 1 -- 172.21.15.94:0/2071569709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde4c00ad90 con 0x7fde580fff80 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.620+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde58113080 con 0x7fde580fff80 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.620+0000 7fde3f7fe700 1 -- 172.21.15.94:0/2071569709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde4c0113a0 con 0x7fde580fff80 2024-04-06T03:42:36.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.620+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde58113640 con 0x7fde580fff80 2024-04-06T03:42:36.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.621+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5804c710 con 0x7fde580fff80 2024-04-06T03:42:36.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.621+0000 7fde3f7fe700 1 -- 172.21.15.94:0/2071569709 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 31) v1 ==== 91675+0+0 (secure 0 0 0) 0x7fde4c012400 con 0x7fde580fff80 2024-04-06T03:42:36.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.622+0000 7fde3f7fe700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fde4006d6d0 0x7fde4006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:36.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.622+0000 7fde3f7fe700 1 -- 172.21.15.94:0/2071569709 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fde4c08cc00 con 0x7fde580fff80 2024-04-06T03:42:36.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.622+0000 7fde56ffd700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fde4006d6d0 0x7fde4006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:36.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.622+0000 7fde56ffd700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fde4006d6d0 0x7fde4006fb80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fde4800b970 tx=0x7fde48009e30 comp rx=0 tx=0).ready entity=mgr.14734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:36.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.626+0000 7fde3f7fe700 1 -- 172.21.15.94:0/2071569709 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fde4c092020 con 0x7fde580fff80 2024-04-06T03:42:36.783 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:36 smithi094 ceph-mon[29713]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:36.783 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:36 smithi094 ceph-mon[29713]: pgmap v15: 65 pgs: 65 active+clean; 203 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 350 op/s 2024-04-06T03:42:36.783 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:36 smithi094 ceph-mon[29713]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:36.783 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:36 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/3555350057' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:42:36.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.877+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fde58113b40 con 0x7fde580fff80 2024-04-06T03:42:36.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.877+0000 7fde3f7fe700 1 -- 172.21.15.94:0/2071569709 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7fde4c05a5d0 con 0x7fde580fff80 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:42:36.881 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:42:36.882 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:42:36.883 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:42:36.884 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:42:36.884 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:36.884 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:36.884 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fde4006d6d0 msgr2=0x7fde4006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fde4006d6d0 0x7fde4006fb80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fde4800b970 tx=0x7fde48009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde580fff80 msgr2=0x7fde5810deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde580fff80 0x7fde5810deb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00c830 tx=0x7fde4c004470 comp rx=0 tx=0).stop 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 shutdown_connections 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde58066ef0 0x7fde5810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fde4006d6d0 0x7fde4006fb80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 --2- 172.21.15.94:0/2071569709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde580fff80 0x7fde5810deb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 >> 172.21.15.94:0/2071569709 conn(0x7fde580f7430 msgr2=0x7fde580f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 shutdown_connections 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.879+0000 7fde5d5ce700 1 -- 172.21.15.94:0/2071569709 wait complete. 2024-04-06T03:42:36.885 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:42:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.997+0000 7f8497f1f700 1 -- 172.21.15.94:0/1933842161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fbcd0 msgr2=0x7f84900fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.997+0000 7f8497f1f700 1 --2- 172.21.15.94:0/1933842161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fbcd0 0x7f84900fc0e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8480004940 tx=0x7f8480009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.997+0000 7f8497f1f700 1 -- 172.21.15.94:0/1933842161 shutdown_connections 2024-04-06T03:42:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.997+0000 7f8497f1f700 1 --2- 172.21.15.94:0/1933842161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84900fcf80 0x7f84900fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.997+0000 7f8497f1f700 1 --2- 172.21.15.94:0/1933842161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fbcd0 0x7f84900fc0e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.997+0000 7f8497f1f700 1 -- 172.21.15.94:0/1933842161 >> 172.21.15.94:0/1933842161 conn(0x7f84900f7430 msgr2=0x7f84900f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.997+0000 7f8497f1f700 1 -- 172.21.15.94:0/1933842161 shutdown_connections 2024-04-06T03:42:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.997+0000 7f8497f1f700 1 -- 172.21.15.94:0/1933842161 wait complete. 2024-04-06T03:42:37.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.997+0000 7f8497f1f700 1 Processor -- start 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f8497f1f700 1 -- start start 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f8497f1f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84900fbcd0 0x7f849010da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f8497f1f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fcf80 0x7f849010dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f8497f1f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849010e660 con 0x7f84900fcf80 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f8497f1f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849010ee00 con 0x7f84900fbcd0 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84954ba700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fcf80 0x7f849010dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84954ba700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fcf80 0x7f849010dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55266/0 (socket says 172.21.15.94:55266) 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84954ba700 1 -- 172.21.15.94:0/3442657638 learned_addr learned my addr 172.21.15.94:0/3442657638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f8495cbb700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84900fbcd0 0x7f849010da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84954ba700 1 -- 172.21.15.94:0/3442657638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84900fbcd0 msgr2=0x7f849010da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84954ba700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84900fbcd0 0x7f849010da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84954ba700 1 -- 172.21.15.94:0/3442657638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480005040 con 0x7f84900fcf80 2024-04-06T03:42:37.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84954ba700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fcf80 0x7f849010dfd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f848c00b790 tx=0x7f848c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84867fc700 1 -- 172.21.15.94:0/3442657638 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f848c00f770 con 0x7f84900fcf80 2024-04-06T03:42:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84867fc700 1 -- 172.21.15.94:0/3442657638 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f848c00fda0 con 0x7f84900fcf80 2024-04-06T03:42:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f84867fc700 1 -- 172.21.15.94:0/3442657638 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f848c00d310 con 0x7f84900fcf80 2024-04-06T03:42:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.998+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849010f0e0 con 0x7f84900fcf80 2024-04-06T03:42:37.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.999+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849010f6a0 con 0x7f84900fcf80 2024-04-06T03:42:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:36.999+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8490101b10 con 0x7f84900fcf80 2024-04-06T03:42:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.000+0000 7f84867fc700 1 -- 172.21.15.94:0/3442657638 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 31) v1 ==== 91675+0+0 (secure 0 0 0) 0x7f848c010420 con 0x7f84900fcf80 2024-04-06T03:42:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.000+0000 7f84867fc700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f847c06d6c0 0x7f847c06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.000+0000 7f84867fc700 1 -- 172.21.15.94:0/3442657638 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f848c08cd20 con 0x7f84900fcf80 2024-04-06T03:42:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.000+0000 7f8495cbb700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f847c06d6c0 0x7f847c06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:37.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.001+0000 7f8495cbb700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f847c06d6c0 0x7f847c06fb70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8480000c00 tx=0x7f8480009e30 comp rx=0 tx=0).ready entity=mgr.14734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:37.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.005+0000 7f84867fc700 1 -- 172.21.15.94:0/3442657638 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7f848c05a650 con 0x7f84900fcf80 2024-04-06T03:42:37.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:36 smithi136 ceph-mon[36436]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:37.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:36 smithi136 ceph-mon[36436]: pgmap v15: 65 pgs: 65 active+clean; 203 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 350 op/s 2024-04-06T03:42:37.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:36 smithi136 ceph-mon[36436]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:37.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:36 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/3555350057' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:42:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.240+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 --> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f849010f920 con 0x7f847c06d6c0 2024-04-06T03:42:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.242+0000 7f84867fc700 1 -- 172.21.15.94:0/3442657638 <== mgr.14734 v2:172.21.15.94:6800/4259523022 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f849010f920 con 0x7f847c06d6c0 2024-04-06T03:42:37.246 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:42:37.246 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:42:37.247 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:42:37.247 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:42:37.247 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:42:37.247 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-04-06T03:42:37.247 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:42:37.247 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "2/21 daemons upgraded", 2024-04-06T03:42:37.247 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mgr daemons", 2024-04-06T03:42:37.247 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:42:37.247 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:42:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f847c06d6c0 msgr2=0x7f847c06fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f847c06d6c0 0x7f847c06fb70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8480000c00 tx=0x7f8480009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fcf80 msgr2=0x7f849010dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fcf80 0x7f849010dfd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f848c00b790 tx=0x7f848c009250 comp rx=0 tx=0).stop 2024-04-06T03:42:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 shutdown_connections 2024-04-06T03:42:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84900fbcd0 0x7f849010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7f847c06d6c0 0x7f847c06fb70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 --2- 172.21.15.94:0/3442657638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84900fcf80 0x7f849010dfd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 >> 172.21.15.94:0/3442657638 conn(0x7f84900f7430 msgr2=0x7f8490065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 shutdown_connections 2024-04-06T03:42:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.244+0000 7f8497f1f700 1 -- 172.21.15.94:0/3442657638 wait complete. 2024-04-06T03:42:37.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.344+0000 7fdd8deb5700 1 -- 172.21.15.94:0/2872201587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 msgr2=0x7fdd880fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:37.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.344+0000 7fdd8deb5700 1 --2- 172.21.15.94:0/2872201587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 0x7fdd880fd420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c004940 tx=0x7fdd7c009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:37.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.344+0000 7fdd8deb5700 1 -- 172.21.15.94:0/2872201587 shutdown_connections 2024-04-06T03:42:37.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.344+0000 7fdd8deb5700 1 --2- 172.21.15.94:0/2872201587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 0x7fdd880fd420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.344+0000 7fdd8deb5700 1 --2- 172.21.15.94:0/2872201587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd880fbd00 0x7fdd880fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.344+0000 7fdd8deb5700 1 -- 172.21.15.94:0/2872201587 >> 172.21.15.94:0/2872201587 conn(0x7fdd880f7480 msgr2=0x7fdd880f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:37.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.344+0000 7fdd8deb5700 1 -- 172.21.15.94:0/2872201587 shutdown_connections 2024-04-06T03:42:37.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.344+0000 7fdd8deb5700 1 -- 172.21.15.94:0/2872201587 wait complete. 2024-04-06T03:42:37.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd8deb5700 1 Processor -- start 2024-04-06T03:42:37.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd8deb5700 1 -- start start 2024-04-06T03:42:37.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd8deb5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd880fbd00 0x7fdd880ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:37.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd8deb5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 0x7fdd880fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:37.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd8deb5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd881001a0 con 0x7fdd880fcfb0 2024-04-06T03:42:37.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd8deb5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880fe830 con 0x7fdd880fbd00 2024-04-06T03:42:37.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd86ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 0x7fdd880fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:37.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd86ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 0x7fdd880fe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55282/0 (socket says 172.21.15.94:55282) 2024-04-06T03:42:37.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd86ffd700 1 -- 172.21.15.94:0/4291032125 learned_addr learned my addr 172.21.15.94:0/4291032125 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:42:37.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.345+0000 7fdd877fe700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd880fbd00 0x7fdd880ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.346+0000 7fdd86ffd700 1 -- 172.21.15.94:0/4291032125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd880fbd00 msgr2=0x7fdd880ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.346+0000 7fdd86ffd700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd880fbd00 0x7fdd880ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.346+0000 7fdd86ffd700 1 -- 172.21.15.94:0/4291032125 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c005040 con 0x7fdd880fcfb0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.346+0000 7fdd86ffd700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 0x7fdd880fe2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009150 tx=0x7fdd7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.346+0000 7fdd6ffff700 1 -- 172.21.15.94:0/4291032125 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd7c00e040 con 0x7fdd880fcfb0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.346+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd880feab0 con 0x7fdd880fcfb0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.346+0000 7fdd6ffff700 1 -- 172.21.15.94:0/4291032125 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd7c00bbf0 con 0x7fdd880fcfb0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.346+0000 7fdd6ffff700 1 -- 172.21.15.94:0/4291032125 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd7c01c630 con 0x7fdd880fcfb0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.346+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd880ff070 con 0x7fdd880fcfb0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.347+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8804c710 con 0x7fdd880fcfb0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.347+0000 7fdd6ffff700 1 -- 172.21.15.94:0/4291032125 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 31) v1 ==== 91675+0+0 (secure 0 0 0) 0x7fdd7c00bd60 con 0x7fdd880fcfb0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.348+0000 7fdd6ffff700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fdd7006d6d0 0x7fdd7006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.348+0000 7fdd877fe700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fdd7006d6d0 0x7fdd7006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:42:37.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.348+0000 7fdd6ffff700 1 -- 172.21.15.94:0/4291032125 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fdd7c08e0e0 con 0x7fdd880fcfb0 2024-04-06T03:42:37.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.348+0000 7fdd877fe700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fdd7006d6d0 0x7fdd7006fb80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd78004f40 tx=0x7fdd7800b040 comp rx=0 tx=0).ready entity=mgr.14734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:42:37.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.352+0000 7fdd6ffff700 1 -- 172.21.15.94:0/4291032125 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183140 (secure 0 0 0) 0x7fdd7c05b900 con 0x7fdd880fcfb0 2024-04-06T03:42:37.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.630+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdd88101a70 con 0x7fdd880fcfb0 2024-04-06T03:42:37.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.630+0000 7fdd6ffff700 1 -- 172.21.15.94:0/4291032125 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+157 (secure 0 0 0) 0x7fdd7c013e60 con 0x7fdd880fcfb0 2024-04-06T03:42:37.633 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:42:37.634 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:42:37.634 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:42:37.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fdd7006d6d0 msgr2=0x7fdd7006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:37.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fdd7006d6d0 0x7fdd7006fb80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd78004f40 tx=0x7fdd7800b040 comp rx=0 tx=0).stop 2024-04-06T03:42:37.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 msgr2=0x7fdd880fe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:42:37.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 0x7fdd880fe2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009150 tx=0x7fdd7c009e30 comp rx=0 tx=0).stop 2024-04-06T03:42:37.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 shutdown_connections 2024-04-06T03:42:37.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd880fbd00 0x7fdd880ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:6800/4259523022,v1:172.21.15.94:6801/4259523022] conn(0x7fdd7006d6d0 0x7fdd7006fb80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 --2- 172.21.15.94:0/4291032125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd880fcfb0 0x7fdd880fe2f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:42:37.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 >> 172.21.15.94:0/4291032125 conn(0x7fdd880f7480 msgr2=0x7fdd880667a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:42:37.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 shutdown_connections 2024-04-06T03:42:37.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:42:37.632+0000 7fdd8deb5700 1 -- 172.21.15.94:0/4291032125 wait complete. 2024-04-06T03:42:37.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:37 smithi094 ceph-mon[29713]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:37.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:37 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/2071569709' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:42:37.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:37 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:37.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:37 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:37.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:37 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:42:37.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:37 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:37.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:37 smithi094 ceph-mon[29713]: from='client.? 172.21.15.94:0/4291032125' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:42:37.968 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:37 smithi136 ceph-mon[36436]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:37.968 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:37 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/2071569709' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:42:37.968 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:37 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:37.968 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:37 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:37.968 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:37 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:42:37.968 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:37 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:37.968 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:37 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/4291032125' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:42:38.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:38 smithi094 ceph-mon[29713]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:42:38.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:38 smithi094 ceph-mon[29713]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:38.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:38 smithi094 ceph-mon[29713]: pgmap v16: 65 pgs: 65 active+clean; 203 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 350 op/s 2024-04-06T03:42:39.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:38 smithi136 ceph-mon[36436]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:42:39.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:38 smithi136 ceph-mon[36436]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:42:39.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:38 smithi136 ceph-mon[36436]: pgmap v16: 65 pgs: 65 active+clean; 203 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 350 op/s 2024-04-06T03:42:39.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:39 smithi136 ceph-mon[36436]: Upgrade: Updating mgr.smithi136.jagvwt 2024-04-06T03:42:39.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:39 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:42:39.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:39 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:42:39.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:39 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:42:39.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:39 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:39.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:39 smithi136 ceph-mon[36436]: Deploying daemon mgr.smithi136.jagvwt on smithi136 2024-04-06T03:42:40.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:39 smithi094 ceph-mon[29713]: Upgrade: Updating mgr.smithi136.jagvwt 2024-04-06T03:42:40.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:39 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:42:40.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:39 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:42:40.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:39 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:42:40.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:39 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:40.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:39 smithi094 ceph-mon[29713]: Deploying daemon mgr.smithi136.jagvwt on smithi136 2024-04-06T03:42:41.078 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:40 smithi136 ceph-mon[36436]: pgmap v17: 65 pgs: 65 active+clean; 194 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 969 KiB/s wr, 320 op/s 2024-04-06T03:42:41.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:40 smithi094 ceph-mon[29713]: pgmap v17: 65 pgs: 65 active+clean; 194 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 969 KiB/s wr, 320 op/s 2024-04-06T03:42:42.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:42 smithi136 ceph-mon[36436]: pgmap v18: 65 pgs: 65 active+clean; 194 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 460 KiB/s wr, 129 op/s 2024-04-06T03:42:42.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:42 smithi094 ceph-mon[29713]: pgmap v18: 65 pgs: 65 active+clean; 194 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 460 KiB/s wr, 129 op/s 2024-04-06T03:42:44.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:44 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:44.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:44 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:44.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:44 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:44.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:44 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:44.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:44 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:42:44.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:44 smithi136 ceph-mon[36436]: pgmap v19: 65 pgs: 65 active+clean; 194 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 460 KiB/s wr, 129 op/s 2024-04-06T03:42:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:44 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:44.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:44 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:44.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:44 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:44.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:44 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:44.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:44 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:42:44.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:44 smithi094 ceph-mon[29713]: pgmap v19: 65 pgs: 65 active+clean; 194 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 460 KiB/s wr, 129 op/s 2024-04-06T03:42:46.650 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:46 smithi094 ceph-mon[29713]: pgmap v20: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 460 KiB/s wr, 130 op/s 2024-04-06T03:42:46.650 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:46 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:46.651 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:46 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:47.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:46 smithi136 ceph-mon[36436]: pgmap v20: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 11 KiB/s rd, 460 KiB/s wr, 130 op/s 2024-04-06T03:42:47.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:46 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:47.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:46 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:48.739 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:48 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:48.740 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:48 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:48.740 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:48 smithi094 ceph-mon[29713]: pgmap v21: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-04-06T03:42:48.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:48 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:48.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:48 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:48.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:48 smithi136 ceph-mon[36436]: pgmap v21: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-04-06T03:42:50.297 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:50 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:50.298 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:50 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:50.298 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:50 smithi136 ceph-mon[36436]: pgmap v22: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-04-06T03:42:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:50 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:50.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:50 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:50.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:50 smithi094 ceph-mon[29713]: pgmap v22: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 15 KiB/s wr, 22 op/s 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: Standby manager daemon smithi136.jagvwt restarted 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/2711085450' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/2711085450' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/2711085450' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/2711085450' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: pgmap v23: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 178 B/s wr, 0 op/s 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: pgmap v24: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 216 B/s wr, 0 op/s 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.purlzp"}]: dispatch 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.purlzp"}]: dispatch 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.purlzp"}]': finished 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi136.jagvwt"}]: dispatch 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi136.jagvwt"}]: dispatch 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi136.jagvwt"}]': finished 2024-04-06T03:42:51.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:51 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: Standby manager daemon smithi136.jagvwt restarted 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/2711085450' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/2711085450' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/2711085450' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.? 172.21.15.136:0/2711085450' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: pgmap v23: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 178 B/s wr, 0 op/s 2024-04-06T03:42:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: pgmap v24: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 216 B/s wr, 0 op/s 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.purlzp"}]: dispatch 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.purlzp"}]: dispatch 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi094.purlzp"}]': finished 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi136.jagvwt"}]: dispatch 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi136.jagvwt"}]: dispatch 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi136.jagvwt"}]': finished 2024-04-06T03:42:51.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:51 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-06T03:42:52.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:52 smithi094 ceph-mon[29713]: Upgrade: Setting container_image for all mgr 2024-04-06T03:42:52.422 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:52 smithi094 ceph-mon[29713]: mgrmap e32: smithi094.purlzp(active, since 37s), standbys: smithi136.jagvwt 2024-04-06T03:42:52.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:52 smithi136 ceph-mon[36436]: Upgrade: Setting container_image for all mgr 2024-04-06T03:42:52.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:52 smithi136 ceph-mon[36436]: mgrmap e32: smithi094.purlzp(active, since 37s), standbys: smithi136.jagvwt 2024-04-06T03:42:53.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:53 smithi094 ceph-mon[29713]: Upgrade: Updating mon.smithi094 2024-04-06T03:42:53.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:53 smithi094 ceph-mon[29713]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:53.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:53 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:42:53.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:53 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:42:53.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:53 smithi094 ceph-mon[29713]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:53.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:53 smithi094 ceph-mon[29713]: Deploying daemon mon.smithi094 on smithi094 2024-04-06T03:42:53.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:53 smithi094 ceph-mon[29713]: pgmap v25: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 216 B/s wr, 0 op/s 2024-04-06T03:42:54.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:53 smithi136 ceph-mon[36436]: Upgrade: Updating mon.smithi094 2024-04-06T03:42:54.101 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:53 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='mgr.smithi094.purlzp' 2024-04-06T03:42:54.101 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:53 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:42:54.101 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:53 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:42:54.101 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:53 smithi136 ceph-mon[36436]: from='mgr.14734 172.21.15.94:0/1303136507' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:42:54.101 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:53 smithi136 ceph-mon[36436]: Deploying daemon mon.smithi094 on smithi094 2024-04-06T03:42:54.101 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:53 smithi136 ceph-mon[36436]: pgmap v25: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 216 B/s wr, 0 op/s 2024-04-06T03:42:54.970 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:54 smithi094 systemd[1]: Stopping Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:42:55.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:54 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[29709]: 2024-04-06T03:42:54.966+0000 7f452e24b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi094 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:42:55.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:54 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[29709]: 2024-04-06T03:42:54.966+0000 7f452e24b700 -1 mon.smithi094@0(leader) e2 *** Got Signal Terminated *** 2024-04-06T03:42:55.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:55 smithi094 podman[93886]: 2024-04-06 03:42:55.212783659 +0000 UTC m=+0.352626819 container died b36acaaa3d783580f1ff821eeee34f0fdfc5c5cdadf892a57a0130bd9739a6a6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, RELEASE=quincy-6699495, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-06T03:42:55.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:55 smithi094 podman[93886]: 2024-04-06 03:42:55.575169553 +0000 UTC m=+0.715012721 container cleanup b36acaaa3d783580f1ff821eeee34f0fdfc5c5cdadf892a57a0130bd9739a6a6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=quincy-6699495, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-06T03:42:55.888 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:55 smithi094 bash[93886]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094 2024-04-06T03:42:56.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:55 smithi094 podman[93900]: 2024-04-06 03:42:55.885208411 +0000 UTC m=+0.669103464 container remove b36acaaa3d783580f1ff821eeee34f0fdfc5c5cdadf892a57a0130bd9739a6a6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=quincy-6699495, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:42:56.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094.service: Deactivated successfully. 2024-04-06T03:42:56.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 systemd[1]: Stopped Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:42:56.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094.service: Consumed 10.926s CPU time. 2024-04-06T03:42:56.589 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 systemd[1]: Starting Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:42:56.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 podman[94034]: 2024-04-06 03:42:56.348582486 +0000 UTC m=+0.017945422 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:42:56.590 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 podman[94034]: 2024-04-06 03:42:56.462005417 +0000 UTC m=+0.131368341 container create 02f9626f65141f687879b102d918bfd3bc3788255fa35fc4015ba2554477628d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-9f19235, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-06T03:42:57.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 podman[94034]: 2024-04-06 03:42:56.681467242 +0000 UTC m=+0.350830166 container init 02f9626f65141f687879b102d918bfd3bc3788255fa35fc4015ba2554477628d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-06T03:42:57.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 podman[94034]: 2024-04-06 03:42:56.684568007 +0000 UTC m=+0.353930931 container start 02f9626f65141f687879b102d918bfd3bc3788255fa35fc4015ba2554477628d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, ceph=True) 2024-04-06T03:42:57.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: set uid:gid to 167:167 (ceph:ceph) 2024-04-06T03:42:57.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev), process ceph-mon, pid 2 2024-04-06T03:42:57.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: pidfile_write: ignore empty --pid-file 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: load: jerasure load: lrc 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: RocksDB version: 7.9.2 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Git sha 0 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Compile date 2024-04-05 08:39:05 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: DB SUMMARY 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: DB Session ID: BSA0HZFHQ4UHJFYYNS5D 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: CURRENT file: CURRENT 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: IDENTITY file: IDENTITY 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: MANIFEST file: MANIFEST-000015 size: 972 Bytes 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi094/store.db dir, Total Num: 1, files: 000030.sst 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi094/store.db: 000028.log size: 1862414 ; 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.error_if_exists: 0 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.create_if_missing: 0 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.paranoid_checks: 1 2024-04-06T03:42:57.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.env: 0x55b5494b1c60 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.fs: PosixFileSystem 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.info_log: 0x55b54aef99a0 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_file_opening_threads: 16 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.statistics: (nil) 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.use_fsync: 0 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_log_file_size: 0 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.keep_log_file_num: 1000 2024-04-06T03:42:57.171 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.recycle_log_file_num: 0 2024-04-06T03:42:57.172 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.allow_fallocate: 1 2024-04-06T03:42:57.172 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.allow_mmap_reads: 0 2024-04-06T03:42:57.172 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.allow_mmap_writes: 0 2024-04-06T03:42:57.172 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.use_direct_reads: 0 2024-04-06T03:42:57.172 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-06T03:42:57.172 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.create_missing_column_families: 0 2024-04-06T03:42:57.172 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.db_log_dir: 2024-04-06T03:42:57.172 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.wal_dir: 2024-04-06T03:42:57.173 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-06T03:42:57.173 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-06T03:42:57.173 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-06T03:42:57.173 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-06T03:42:57.173 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-06T03:42:57.173 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-06T03:42:57.173 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.advise_random_on_open: 1 2024-04-06T03:42:57.173 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.db_write_buffer_size: 0 2024-04-06T03:42:57.174 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.write_buffer_manager: 0x55b54aecc1e0 2024-04-06T03:42:57.174 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-06T03:42:57.174 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-06T03:42:57.174 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-06T03:42:57.174 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.rate_limiter: (nil) 2024-04-06T03:42:57.174 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-06T03:42:57.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.wal_recovery_mode: 2 2024-04-06T03:42:57.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.enable_thread_tracking: 0 2024-04-06T03:42:57.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.enable_pipelined_write: 0 2024-04-06T03:42:57.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.unordered_write: 0 2024-04-06T03:42:57.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-06T03:42:57.175 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-06T03:42:57.176 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-06T03:42:57.176 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-06T03:42:57.176 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.row_cache: None 2024-04-06T03:42:57.176 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.wal_filter: None 2024-04-06T03:42:57.178 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-06T03:42:57.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.allow_ingest_behind: 0 2024-04-06T03:42:57.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.two_write_queues: 0 2024-04-06T03:42:57.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.manual_wal_flush: 0 2024-04-06T03:42:57.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.wal_compression: 0 2024-04-06T03:42:57.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.atomic_flush: 0 2024-04-06T03:42:57.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-06T03:42:57.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-06T03:42:57.179 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-06T03:42:57.180 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.log_readahead_size: 0 2024-04-06T03:42:57.180 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-06T03:42:57.180 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.best_efforts_recovery: 0 2024-04-06T03:42:57.180 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-06T03:42:57.180 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-06T03:42:57.180 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.allow_data_in_errors: 0 2024-04-06T03:42:57.181 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.db_host_id: __hostname__ 2024-04-06T03:42:57.181 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-06T03:42:57.181 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_background_jobs: 2 2024-04-06T03:42:57.181 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_background_compactions: -1 2024-04-06T03:42:57.181 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_subcompactions: 1 2024-04-06T03:42:57.181 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-06T03:42:57.181 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-06T03:42:57.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-06T03:42:57.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_total_wal_size: 0 2024-04-06T03:42:57.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-06T03:42:57.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-06T03:42:57.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-06T03:42:57.182 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-06T03:42:57.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_open_files: -1 2024-04-06T03:42:57.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bytes_per_sync: 0 2024-04-06T03:42:57.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-06T03:42:57.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-06T03:42:57.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_readahead_size: 0 2024-04-06T03:42:57.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_background_flushes: -1 2024-04-06T03:42:57.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Compression algorithms supported: 2024-04-06T03:42:57.183 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: kZSTD supported: 0 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: kXpressCompression supported: 0 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: kLZ4HCCompression supported: 1 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: kZlibCompression supported: 1 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: kSnappyCompression supported: 1 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: kLZ4Compression supported: 1 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: kBZip2Compression supported: 0 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi094/store.db/MANIFEST-000015 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-06T03:42:57.184 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-06T03:42:57.185 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.merge_operator: 2024-04-06T03:42:57.185 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_filter: None 2024-04-06T03:42:57.185 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_filter_factory: None 2024-04-06T03:42:57.185 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.sst_partitioner_factory: None 2024-04-06T03:42:57.185 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-06T03:42:57.185 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-06T03:42:57.185 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b54aef9aa0) 2024-04-06T03:42:57.185 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-04-06T03:42:57.185 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_type: 0 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_block_index_type: 0 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_shortening: 1 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: checksum: 4 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: no_block_cache: 0 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache: 0x55b54aefd090 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_options: 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: capacity : 536870912 2024-04-06T03:42:57.186 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: num_shard_bits : 4 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: strict_capacity_limit : 0 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_cache_compressed: (nil) 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: persistent_cache: (nil) 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_size: 4096 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_size_deviation: 10 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_restart_interval: 16 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: index_block_restart_interval: 1 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: metadata_block_size: 4096 2024-04-06T03:42:57.187 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: partition_filters: 0 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: use_delta_encoding: 1 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: filter_policy: bloomfilter 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: whole_key_filtering: 1 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: verify_compression: 0 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: format_version: 5 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: enable_index_compression: 1 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: block_align: 0 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: max_auto_readahead_size: 262144 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: prepopulate_block_cache: 0 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: initial_auto_readahead_size: 8192 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-04-06T03:42:57.188 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.write_buffer_size: 33554432 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_write_buffer_number: 2 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression: NoCompression 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression: Disabled 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.prefix_extractor: nullptr 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.num_levels: 7 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-06T03:42:57.189 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression_opts.level: 32767 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression_opts.strategy: 0 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-06T03:42:57.190 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression_opts.enabled: false 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.target_file_size_base: 67108864 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-06T03:42:57.191 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-06T03:42:57.192 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-06T03:42:57.192 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-06T03:42:57.192 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-06T03:42:57.192 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-06T03:42:57.192 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-06T03:42:57.192 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-06T03:42:57.192 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-06T03:42:57.192 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.arena_block_size: 1048576 2024-04-06T03:42:57.192 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.disable_auto_compactions: 0 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.table_properties_collectors: 2024-04-06T03:42:57.193 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.inplace_update_support: 0 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.bloom_locality: 0 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.max_successive_merges: 0 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.paranoid_file_checks: 0 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.force_consistency_checks: 1 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.report_bg_io_stats: 0 2024-04-06T03:42:57.194 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.ttl: 2592000 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.enable_blob_files: false 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.min_blob_size: 0 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.blob_file_size: 268435456 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-06T03:42:57.195 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.blob_file_starting_level: 0 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi094/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10471, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5caf4c69-916a-4e4a-a013-4b0eaf3b20b8 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712374976713682, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712374976721400, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 1698372, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10472, "largest_seqno": 11261, "table_properties": {"data_size": 1693959, "index_size": 2439, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 965, "raw_key_size": 8989, "raw_average_key_size": 23, "raw_value_size": 1685860, "raw_average_value_size": 4471, "num_data_blocks": 111, "num_entries": 377, "num_filter_entries": 377, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712374976, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5caf4c69-916a-4e4a-a013-4b0eaf3b20b8", "db_session_id": "BSA0HZFHQ4UHJFYYNS5D", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712374976721505, "job": 1, "event": "recovery_finished"} 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi094/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-06T03:42:57.196 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b54afee000 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: DB pointer 0x55b54afd8000 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** DB Stats ** 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-06T03:42:57.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-06T03:42:57.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:42:57.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-06T03:42:57.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-06T03:42:57.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-06T03:42:57.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: L0 1/0 1.62 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 462.2 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-06T03:42:57.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: L6 1/0 5.75 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-06T03:42:57.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Sum 2/0 7.37 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 462.2 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-06T03:42:57.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 462.2 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 462.2 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-06T03:42:57.199 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Cumulative compaction: 0.00 GB write, 158.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Interval compaction: 0.00 GB write, 158.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Block cache BinnedLRUCache@0x55b54aefd090#2 capacity: 512.00 MB usage: 3.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.03 KB,0.000196695%) IndexBlock(1,2.58 KB,0.000491738%) Misc(1,0.00 KB,0%) 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: starting mon.smithi094 rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi094 fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: mon.smithi094@-1(???) e2 preinit fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: mon.smithi094@-1(???).mds e12 new map 2024-04-06T03:42:57.200 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: mon.smithi094@-1(???).mds e12 print_map 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: e12 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: legacy client fscid: 1 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Filesystem 'cephfs' (1) 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: fs_name cephfs 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: epoch 11 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: tableserver 0 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: root 0 2024-04-06T03:42:57.201 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: session_timeout 60 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: session_autoclose 300 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: max_file_size 1099511627776 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: max_xattr_size 65536 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: required_client_features {} 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: last_failure 0 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: last_failure_osd_epoch 0 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: max_mds 1 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: in 0 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: up {0=14460} 2024-04-06T03:42:57.202 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: failed 2024-04-06T03:42:57.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: damaged 2024-04-06T03:42:57.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: stopped 2024-04-06T03:42:57.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: data_pools [3] 2024-04-06T03:42:57.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: metadata_pool 2 2024-04-06T03:42:57.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: inline_data disabled 2024-04-06T03:42:57.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: balancer 2024-04-06T03:42:57.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: bal_rank_mask -1 2024-04-06T03:42:57.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: standby_count_wanted 1 2024-04-06T03:42:57.203 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: qdb_cluster leader: 0 members: 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: [mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: Standby daemons: 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: [mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: [mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout: [mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: mon.smithi094@-1(???).osd e47 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: mon.smithi094@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: mon.smithi094@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: mon.smithi094@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 ceph-mon[94053]: mon.smithi094@-1(???).paxosservice(auth 1..20) refresh upgraded, format 0 -> 3 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 bash[94034]: 02f9626f65141f687879b102d918bfd3bc3788255fa35fc4015ba2554477628d 2024-04-06T03:42:57.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:56 smithi094 systemd[1]: Started Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: pgmap v26: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: pgmap v27: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: mon.smithi094 calling monitor election 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: mon.smithi094 is new leader, mons smithi094,smithi136 in quorum (ranks 0,1) 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: monmap epoch 2 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: last_changed 2024-04-06T03:33:38.265304+0000 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: created 2024-04-06T03:31:02.176975+0000 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: min_mon_release 17 (quincy) 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: election_strategy: 1 2024-04-06T03:42:58.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-04-06T03:42:58.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.smithi136 2024-04-06T03:42:58.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:42:58.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: osdmap e47: 6 total, 6 up, 6 in 2024-04-06T03:42:58.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: mgrmap e32: smithi094.purlzp(active, since 44s), standbys: smithi136.jagvwt 2024-04-06T03:42:58.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:42:58.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:42:58.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:42:58.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: from='mgr.14734 ' entity='' 2024-04-06T03:42:58.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:42:58 smithi094 ceph-mon[94053]: mgrmap e33: smithi094.purlzp(active, since 44s), standbys: smithi136.jagvwt 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: pgmap v26: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: pgmap v27: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: mon.smithi094 calling monitor election 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: mon.smithi094 is new leader, mons smithi094,smithi136 in quorum (ranks 0,1) 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: monmap epoch 2 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: last_changed 2024-04-06T03:33:38.265304+0000 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: created 2024-04-06T03:31:02.176975+0000 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: min_mon_release 17 (quincy) 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: election_strategy: 1 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.smithi136 2024-04-06T03:42:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:42:58.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: osdmap e47: 6 total, 6 up, 6 in 2024-04-06T03:42:58.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: mgrmap e32: smithi094.purlzp(active, since 44s), standbys: smithi136.jagvwt 2024-04-06T03:42:58.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:42:58.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:42:58.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:42:58.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: from='mgr.14734 ' entity='' 2024-04-06T03:42:58.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:42:58 smithi136 ceph-mon[36436]: mgrmap e33: smithi094.purlzp(active, since 44s), standbys: smithi136.jagvwt 2024-04-06T03:43:05.288 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:05 smithi136 ceph-mon[36436]: Standby manager daemon smithi136.jagvwt restarted 2024-04-06T03:43:05.288 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:05 smithi136 ceph-mon[36436]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:43:05.288 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:05 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/228843963' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:43:05.288 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:05 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/228843963' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:43:05.288 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:05 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/228843963' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:43:05.289 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:05 smithi136 ceph-mon[36436]: from='mgr.? 172.21.15.136:0/228843963' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:43:05.358 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:05 smithi094 ceph-mon[94053]: Standby manager daemon smithi136.jagvwt restarted 2024-04-06T03:43:05.358 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:05 smithi094 ceph-mon[94053]: Standby manager daemon smithi136.jagvwt started 2024-04-06T03:43:05.358 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:05 smithi094 ceph-mon[94053]: from='mgr.? 172.21.15.136:0/228843963' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/crt"}]: dispatch 2024-04-06T03:43:05.359 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:05 smithi094 ceph-mon[94053]: from='mgr.? 172.21.15.136:0/228843963' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-06T03:43:05.359 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:05 smithi094 ceph-mon[94053]: from='mgr.? 172.21.15.136:0/228843963' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi136.jagvwt/key"}]: dispatch 2024-04-06T03:43:05.359 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:05 smithi094 ceph-mon[94053]: from='mgr.? 172.21.15.136:0/228843963' entity='mgr.smithi136.jagvwt' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-06T03:43:06.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: mgrmap e34: smithi094.purlzp(active, since 51s), standbys: smithi136.jagvwt 2024-04-06T03:43:06.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: Active manager daemon smithi094.purlzp restarted 2024-04-06T03:43:06.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: Activating manager daemon smithi094.purlzp 2024-04-06T03:43:06.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: osdmap e48: 6 total, 6 up, 6 in 2024-04-06T03:43:06.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: mgrmap e35: smithi094.purlzp(active, starting, since 0.0051882s), standbys: smithi136.jagvwt 2024-04-06T03:43:06.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:43:06.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:43:06.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:43:06.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi136.jagvwt", "id": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:43:06.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: Manager daemon smithi094.purlzp is now available 2024-04-06T03:43:06.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:43:06.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:43:06.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:43:06.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:43:06.438 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: mgrmap e34: smithi094.purlzp(active, since 51s), standbys: smithi136.jagvwt 2024-04-06T03:43:06.438 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: Active manager daemon smithi094.purlzp restarted 2024-04-06T03:43:06.438 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: Activating manager daemon smithi094.purlzp 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: osdmap e48: 6 total, 6 up, 6 in 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: mgrmap e35: smithi094.purlzp(active, starting, since 0.0051882s), standbys: smithi136.jagvwt 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi094.purlzp", "id": "smithi094.purlzp"}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr metadata", "who": "smithi136.jagvwt", "id": "smithi136.jagvwt"}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:43:06.439 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: Manager daemon smithi094.purlzp is now available 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/mirror_snapshot_schedule"}]: dispatch 2024-04-06T03:43:06.440 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:06 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi094.purlzp/trash_purge_schedule"}]: dispatch 2024-04-06T03:43:07.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:07 smithi136 ceph-mon[36436]: [06/Apr/2024:03:43:05] ENGINE Bus STARTING 2024-04-06T03:43:07.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:07 smithi136 ceph-mon[36436]: [06/Apr/2024:03:43:05] ENGINE Serving on http://172.21.15.94:8765 2024-04-06T03:43:07.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:07 smithi136 ceph-mon[36436]: [06/Apr/2024:03:43:06] ENGINE Serving on https://172.21.15.94:7150 2024-04-06T03:43:07.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:07 smithi136 ceph-mon[36436]: [06/Apr/2024:03:43:06] ENGINE Bus STARTED 2024-04-06T03:43:07.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:07 smithi136 ceph-mon[36436]: mgrmap e36: smithi094.purlzp(active, since 1.00894s), standbys: smithi136.jagvwt 2024-04-06T03:43:07.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:07 smithi136 ceph-mon[36436]: pgmap v3: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:07.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:07 smithi094 ceph-mon[94053]: [06/Apr/2024:03:43:05] ENGINE Bus STARTING 2024-04-06T03:43:07.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:07 smithi094 ceph-mon[94053]: [06/Apr/2024:03:43:05] ENGINE Serving on http://172.21.15.94:8765 2024-04-06T03:43:07.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:07 smithi094 ceph-mon[94053]: [06/Apr/2024:03:43:06] ENGINE Serving on https://172.21.15.94:7150 2024-04-06T03:43:07.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:07 smithi094 ceph-mon[94053]: [06/Apr/2024:03:43:06] ENGINE Bus STARTED 2024-04-06T03:43:07.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:07 smithi094 ceph-mon[94053]: mgrmap e36: smithi094.purlzp(active, since 1.00894s), standbys: smithi136.jagvwt 2024-04-06T03:43:07.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:07 smithi094 ceph-mon[94053]: pgmap v3: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.737+0000 7fdd74716700 1 -- 172.21.15.94:0/3249382444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 msgr2=0x7fdd6c0fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.737+0000 7fdd74716700 1 --2- 172.21.15.94:0/3249382444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 0x7fdd6c0fe4c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c004940 tx=0x7fdd5c009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.737+0000 7fdd74716700 1 -- 172.21.15.94:0/3249382444 shutdown_connections 2024-04-06T03:43:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.737+0000 7fdd74716700 1 --2- 172.21.15.94:0/3249382444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 0x7fdd6c0fe4c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.737+0000 7fdd74716700 1 --2- 172.21.15.94:0/3249382444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd6c0fce50 0x7fdd6c0fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.737+0000 7fdd74716700 1 -- 172.21.15.94:0/3249382444 >> 172.21.15.94:0/3249382444 conn(0x7fdd6c0f8560 msgr2=0x7fdd6c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.738+0000 7fdd74716700 1 -- 172.21.15.94:0/3249382444 shutdown_connections 2024-04-06T03:43:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.738+0000 7fdd74716700 1 -- 172.21.15.94:0/3249382444 wait complete. 2024-04-06T03:43:07.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.738+0000 7fdd74716700 1 Processor -- start 2024-04-06T03:43:07.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.738+0000 7fdd74716700 1 -- start start 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.738+0000 7fdd74716700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd6c0fce50 0x7fdd6c10d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.738+0000 7fdd74716700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 0x7fdd6c10de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.738+0000 7fdd74716700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c10e410 con 0x7fdd6c0fe050 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.738+0000 7fdd74716700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c10e580 con 0x7fdd6c0fce50 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd71cb1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 0x7fdd6c10de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd71cb1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 0x7fdd6c10de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48388/0 (socket says 172.21.15.94:48388) 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd71cb1700 1 -- 172.21.15.94:0/4294693667 learned_addr learned my addr 172.21.15.94:0/4294693667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd724b2700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd6c0fce50 0x7fdd6c10d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd71cb1700 1 -- 172.21.15.94:0/4294693667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd6c0fce50 msgr2=0x7fdd6c10d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd71cb1700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd6c0fce50 0x7fdd6c10d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd71cb1700 1 -- 172.21.15.94:0/4294693667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c005040 con 0x7fdd6c0fe050 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd71cb1700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 0x7fdd6c10de10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c013870 tx=0x7fdd5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd62ffd700 1 -- 172.21.15.94:0/4294693667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd5c021070 con 0x7fdd6c0fe050 2024-04-06T03:43:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd62ffd700 1 -- 172.21.15.94:0/4294693667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd5c009430 con 0x7fdd6c0fe050 2024-04-06T03:43:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c112f80 con 0x7fdd6c0fe050 2024-04-06T03:43:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd62ffd700 1 -- 172.21.15.94:0/4294693667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd5c00e070 con 0x7fdd6c0fe050 2024-04-06T03:43:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.739+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c113490 con 0x7fdd6c0fe050 2024-04-06T03:43:07.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.741+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c04c710 con 0x7fdd6c0fe050 2024-04-06T03:43:07.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.741+0000 7fdd62ffd700 1 -- 172.21.15.94:0/4294693667 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91611+0+0 (secure 0 0 0) 0x7fdd5c01c750 con 0x7fdd6c0fe050 2024-04-06T03:43:07.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.742+0000 7fdd62ffd700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fdd5806d960 0x7fdd5806fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:07.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.742+0000 7fdd62ffd700 1 -- 172.21.15.94:0/4294693667 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdd5c08e140 con 0x7fdd6c0fe050 2024-04-06T03:43:07.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.742+0000 7fdd724b2700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fdd5806d960 0x7fdd5806fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:07.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.742+0000 7fdd724b2700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fdd5806d960 0x7fdd5806fe10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009b80 tx=0x7fdd6800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.749+0000 7fdd62ffd700 1 -- 172.21.15.94:0/4294693667 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdd5c057a50 con 0x7fdd6c0fe050 2024-04-06T03:43:07.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.963+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd6c1136d0 con 0x7fdd5806d960 2024-04-06T03:43:07.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.965+0000 7fdd62ffd700 1 -- 172.21.15.94:0/4294693667 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fdd6c1136d0 con 0x7fdd5806d960 2024-04-06T03:43:07.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fdd5806d960 msgr2=0x7fdd5806fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:07.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fdd5806d960 0x7fdd5806fe10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009b80 tx=0x7fdd6800b040 comp rx=0 tx=0).stop 2024-04-06T03:43:07.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 msgr2=0x7fdd6c10de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:07.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 0x7fdd6c10de10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c013870 tx=0x7fdd5c009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:07.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 shutdown_connections 2024-04-06T03:43:07.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fdd5806d960 0x7fdd5806fe10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:07.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd6c0fce50 0x7fdd6c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:07.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 --2- 172.21.15.94:0/4294693667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c0fe050 0x7fdd6c10de10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:07.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 >> 172.21.15.94:0/4294693667 conn(0x7fdd6c0f8560 msgr2=0x7fdd6c0f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:07.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 shutdown_connections 2024-04-06T03:43:07.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:07.967+0000 7fdd74716700 1 -- 172.21.15.94:0/4294693667 wait complete. 2024-04-06T03:43:07.981 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:43:08.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.068+0000 7f7a4f89e700 1 -- 172.21.15.94:0/210134424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fcf90 msgr2=0x7f7a480fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.068+0000 7f7a4f89e700 1 --2- 172.21.15.94:0/210134424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fcf90 0x7f7a480fd400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a38004940 tx=0x7f7a38009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.068+0000 7f7a4f89e700 1 -- 172.21.15.94:0/210134424 shutdown_connections 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.068+0000 7f7a4f89e700 1 --2- 172.21.15.94:0/210134424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fcf90 0x7f7a480fd400 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.068+0000 7f7a4f89e700 1 --2- 172.21.15.94:0/210134424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a480fbce0 0x7f7a480fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.068+0000 7f7a4f89e700 1 -- 172.21.15.94:0/210134424 >> 172.21.15.94:0/210134424 conn(0x7f7a480f7440 msgr2=0x7f7a480f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4f89e700 1 -- 172.21.15.94:0/210134424 shutdown_connections 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4f89e700 1 -- 172.21.15.94:0/210134424 wait complete. 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4f89e700 1 Processor -- start 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4f89e700 1 -- start start 2024-04-06T03:43:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4f89e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fbce0 0x7f7a48066db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4f89e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a480fcf90 0x7f7a481126b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4f89e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48112bf0 con 0x7f7a480fbce0 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4f89e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48112d30 con 0x7f7a480fcf90 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4d63a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fbce0 0x7f7a48066db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4d63a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fbce0 0x7f7a48066db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48408/0 (socket says 172.21.15.94:48408) 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.069+0000 7f7a4d63a700 1 -- 172.21.15.94:0/1229033274 learned_addr learned my addr 172.21.15.94:0/1229033274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a4ce39700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a480fcf90 0x7f7a481126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a4d63a700 1 -- 172.21.15.94:0/1229033274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a480fcf90 msgr2=0x7f7a481126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a4d63a700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a480fcf90 0x7f7a481126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a4d63a700 1 -- 172.21.15.94:0/1229033274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38005040 con 0x7f7a480fbce0 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a4d63a700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fbce0 0x7f7a48066db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a44002a50 tx=0x7f7a440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a3dffb700 1 -- 172.21.15.94:0/1229033274 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a44007cc0 con 0x7f7a480fbce0 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a3dffb700 1 -- 172.21.15.94:0/1229033274 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a4400f040 con 0x7f7a480fbce0 2024-04-06T03:43:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a48112fb0 con 0x7f7a480fbce0 2024-04-06T03:43:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a3dffb700 1 -- 172.21.15.94:0/1229033274 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a44009640 con 0x7f7a480fbce0 2024-04-06T03:43:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.070+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a48113540 con 0x7f7a480fbce0 2024-04-06T03:43:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.071+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4804c710 con 0x7f7a480fbce0 2024-04-06T03:43:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.072+0000 7f7a3dffb700 1 -- 172.21.15.94:0/1229033274 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 36) v1 ==== 91611+0+0 (secure 0 0 0) 0x7f7a440097a0 con 0x7f7a480fbce0 2024-04-06T03:43:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.072+0000 7f7a3dffb700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a3406d9b0 0x7f7a3406fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.072+0000 7f7a3dffb700 1 -- 172.21.15.94:0/1229033274 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7a4408ccd0 con 0x7f7a480fbce0 2024-04-06T03:43:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.072+0000 7f7a4ce39700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a3406d9b0 0x7f7a3406fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.072+0000 7f7a4ce39700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a3406d9b0 0x7f7a3406fe60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800b970 tx=0x7f7a38009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:08.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.077+0000 7f7a3dffb700 1 -- 172.21.15.94:0/1229033274 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7a440565e0 con 0x7f7a480fbce0 2024-04-06T03:43:08.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.170+0000 7f7a3dffb700 1 -- 172.21.15.94:0/1229033274 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 37) v1 ==== 91905+0+0 (secure 0 0 0) 0x7f7a440569b0 con 0x7f7a480fbce0 2024-04-06T03:43:08.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.307+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a48113780 con 0x7f7a3406d9b0 2024-04-06T03:43:08.311 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:08 smithi094 ceph-mon[94053]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-06T03:43:08.311 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:08 smithi094 ceph-mon[94053]: Cluster is now healthy 2024-04-06T03:43:08.311 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:08 smithi094 ceph-mon[94053]: pgmap v4: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:08.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.309+0000 7f7a3dffb700 1 -- 172.21.15.94:0/1229033274 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f7a48113780 con 0x7f7a3406d9b0 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a3406d9b0 msgr2=0x7f7a3406fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a3406d9b0 0x7f7a3406fe60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800b970 tx=0x7f7a38009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fbce0 msgr2=0x7f7a48066db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fbce0 0x7f7a48066db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a44002a50 tx=0x7f7a440046b0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 shutdown_connections 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a480fbce0 0x7f7a48066db0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a3406d9b0 0x7f7a3406fe60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 --2- 172.21.15.94:0/1229033274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a480fcf90 0x7f7a481126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 >> 172.21.15.94:0/1229033274 conn(0x7f7a480f7440 msgr2=0x7f7a480f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 shutdown_connections 2024-04-06T03:43:08.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.311+0000 7f7a4f89e700 1 -- 172.21.15.94:0/1229033274 wait complete. 2024-04-06T03:43:08.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.411+0000 7fbd2c93e700 1 -- 172.21.15.94:0/3456982024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 msgr2=0x7fbd280fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.411+0000 7fbd2c93e700 1 --2- 172.21.15.94:0/3456982024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 0x7fbd280fe450 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c004940 tx=0x7fbd1c009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 -- 172.21.15.94:0/3456982024 shutdown_connections 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 --2- 172.21.15.94:0/3456982024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 0x7fbd280fe450 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 --2- 172.21.15.94:0/3456982024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd280fce70 0x7fbd280fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 -- 172.21.15.94:0/3456982024 >> 172.21.15.94:0/3456982024 conn(0x7fbd280f8550 msgr2=0x7fbd280fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 -- 172.21.15.94:0/3456982024 shutdown_connections 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 -- 172.21.15.94:0/3456982024 wait complete. 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 Processor -- start 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 -- start start 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd280fce70 0x7fbd2810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 0x7fbd2810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2810e460 con 0x7fbd280fdfe0 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2c93e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2810e5d0 con 0x7fbd280fce70 2024-04-06T03:43:08.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd25d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 0x7fbd2810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd25d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 0x7fbd2810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48428/0 (socket says 172.21.15.94:48428) 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd25d9b700 1 -- 172.21.15.94:0/1573844071 learned_addr learned my addr 172.21.15.94:0/1573844071 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.412+0000 7fbd2659c700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd280fce70 0x7fbd2810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.413+0000 7fbd25d9b700 1 -- 172.21.15.94:0/1573844071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd280fce70 msgr2=0x7fbd2810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.413+0000 7fbd25d9b700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd280fce70 0x7fbd2810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.413+0000 7fbd25d9b700 1 -- 172.21.15.94:0/1573844071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c005040 con 0x7fbd280fdfe0 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.413+0000 7fbd25d9b700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 0x7fbd2810de60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00b3b0 tx=0x7fbd1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.413+0000 7fbd16ffd700 1 -- 172.21.15.94:0/1573844071 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd1c00e040 con 0x7fbd280fdfe0 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.413+0000 7fbd16ffd700 1 -- 172.21.15.94:0/1573844071 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd1c00bbf0 con 0x7fbd280fdfe0 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.413+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd28112fd0 con 0x7fbd280fdfe0 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.413+0000 7fbd16ffd700 1 -- 172.21.15.94:0/1573844071 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd1c01c630 con 0x7fbd280fdfe0 2024-04-06T03:43:08.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.413+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd28113560 con 0x7fbd280fdfe0 2024-04-06T03:43:08.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.414+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2804c710 con 0x7fbd280fdfe0 2024-04-06T03:43:08.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.414+0000 7fbd16ffd700 1 -- 172.21.15.94:0/1573844071 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 37) v1 ==== 91905+0+0 (secure 0 0 0) 0x7fbd1c00bd60 con 0x7fbd280fdfe0 2024-04-06T03:43:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.415+0000 7fbd16ffd700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbd1006dd30 0x7fbd100701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.415+0000 7fbd16ffd700 1 -- 172.21.15.94:0/1573844071 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbd1c08e200 con 0x7fbd280fdfe0 2024-04-06T03:43:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.415+0000 7fbd2659c700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbd1006dd30 0x7fbd100701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:08.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.415+0000 7fbd2659c700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbd1006dd30 0x7fbd100701e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbd18004f40 tx=0x7fbd1800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:08.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.420+0000 7fbd16ffd700 1 -- 172.21.15.94:0/1573844071 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbd1c05b2a0 con 0x7fbd280fdfe0 2024-04-06T03:43:08.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:08 smithi136 ceph-mon[36436]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-06T03:43:08.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:08 smithi136 ceph-mon[36436]: Cluster is now healthy 2024-04-06T03:43:08.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:08 smithi136 ceph-mon[36436]: pgmap v4: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:08.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.628+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd281023a0 con 0x7fbd1006dd30 2024-04-06T03:43:08.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.638+0000 7fbd16ffd700 1 -- 172.21.15.94:0/1573844071 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbd281023a0 con 0x7fbd1006dd30 2024-04-06T03:43:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:43:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (2m) 19s ago 10m 19.0M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:43:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (10m) 19s ago 10m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:43:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (9m) 0s ago 9m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:43:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 19s ago 10m - - 2024-04-06T03:43:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (5m) 19s ago 5m 13.9M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:43:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (5m) 19s ago 5m 158M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:43:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (5m) 0s ago 5m 14.2M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:43:08.642 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (5m) 0s ago 5m 16.5M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (78s) 19s ago 11m 493M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (25s) 0s ago 9m 446M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (11m) 19s ago 11m 51.6M 2048M 17.2.7-819-g6699495c ec300eaa768b b36acaaa3d78 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (9m) 0s ago 9m 45.6M 2048M 17.2.7-819-g6699495c ec300eaa768b 9f20264e45e7 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (2m) 19s ago 10m 8980k - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (91s) 0s ago 9m 17.2M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (8m) 19s ago 8m 251M 1557M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (8m) 19s ago 8m 264M 1557M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (7m) 19s ago 7m 226M 1557M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (7m) 0s ago 7m 291M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:43:08.643 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (7m) 0s ago 7m 256M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:43:08.644 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (6m) 0s ago 6m 240M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:43:08.644 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (32s) 19s ago 10m 37.9M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:43:08.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbd1006dd30 msgr2=0x7fbd100701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbd1006dd30 0x7fbd100701e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbd18004f40 tx=0x7fbd1800b040 comp rx=0 tx=0).stop 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 msgr2=0x7fbd2810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 0x7fbd2810de60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00b3b0 tx=0x7fbd1c009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 shutdown_connections 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbd1006dd30 0x7fbd100701e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd280fce70 0x7fbd2810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 --2- 172.21.15.94:0/1573844071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd280fdfe0 0x7fbd2810de60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 >> 172.21.15.94:0/1573844071 conn(0x7fbd280f8550 msgr2=0x7fbd280f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 shutdown_connections 2024-04-06T03:43:08.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.640+0000 7fbd2c93e700 1 -- 172.21.15.94:0/1573844071 wait complete. 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.742+0000 7fc657183700 1 -- 172.21.15.94:0/385767003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 msgr2=0x7fc6500fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.742+0000 7fc657183700 1 --2- 172.21.15.94:0/385767003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 0x7fc6500fc0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc648004940 tx=0x7fc648009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.742+0000 7fc657183700 1 -- 172.21.15.94:0/385767003 shutdown_connections 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.742+0000 7fc657183700 1 --2- 172.21.15.94:0/385767003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6500fcf80 0x7fc6500fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.742+0000 7fc657183700 1 --2- 172.21.15.94:0/385767003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 0x7fc6500fc0e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.742+0000 7fc657183700 1 -- 172.21.15.94:0/385767003 >> 172.21.15.94:0/385767003 conn(0x7fc6500f7430 msgr2=0x7fc6500f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.742+0000 7fc657183700 1 -- 172.21.15.94:0/385767003 shutdown_connections 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.742+0000 7fc657183700 1 -- 172.21.15.94:0/385767003 wait complete. 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.742+0000 7fc657183700 1 Processor -- start 2024-04-06T03:43:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc657183700 1 -- start start 2024-04-06T03:43:08.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc657183700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 0x7fc6501106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:08.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc657183700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6500fcf80 0x7fc650112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:08.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc657183700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc650066350 con 0x7fc6500fbcd0 2024-04-06T03:43:08.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc657183700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc650066490 con 0x7fc6500fcf80 2024-04-06T03:43:08.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc654f1f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 0x7fc6501106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:08.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc654f1f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 0x7fc6501106b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48438/0 (socket says 172.21.15.94:48438) 2024-04-06T03:43:08.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc654f1f700 1 -- 172.21.15.94:0/33399477 learned_addr learned my addr 172.21.15.94:0/33399477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc64ffff700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6500fcf80 0x7fc650112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc654f1f700 1 -- 172.21.15.94:0/33399477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6500fcf80 msgr2=0x7fc650112c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc654f1f700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6500fcf80 0x7fc650112c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc654f1f700 1 -- 172.21.15.94:0/33399477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648005040 con 0x7fc6500fbcd0 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.743+0000 7fc654f1f700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 0x7fc6501106b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc648013870 tx=0x7fc648009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.744+0000 7fc64d7fa700 1 -- 172.21.15.94:0/33399477 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc648021070 con 0x7fc6500fbcd0 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.744+0000 7fc64d7fa700 1 -- 172.21.15.94:0/33399477 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc64801c400 con 0x7fc6500fbcd0 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.744+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc650113140 con 0x7fc6500fbcd0 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.744+0000 7fc64d7fa700 1 -- 172.21.15.94:0/33399477 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc64800e070 con 0x7fc6500fbcd0 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.744+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc650113680 con 0x7fc6500fbcd0 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.745+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65005f5f0 con 0x7fc6500fbcd0 2024-04-06T03:43:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.745+0000 7fc64d7fa700 1 -- 172.21.15.94:0/33399477 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 37) v1 ==== 91905+0+0 (secure 0 0 0) 0x7fc64801ca20 con 0x7fc6500fbcd0 2024-04-06T03:43:08.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.745+0000 7fc64d7fa700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc63c06dd40 0x7fc63c0701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:08.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.746+0000 7fc64d7fa700 1 -- 172.21.15.94:0/33399477 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc64808e6e0 con 0x7fc6500fbcd0 2024-04-06T03:43:08.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.746+0000 7fc64ffff700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc63c06dd40 0x7fc63c0701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:08.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.746+0000 7fc64ffff700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc63c06dd40 0x7fc63c0701f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc640001490 tx=0x7fc64000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:08.751+0000 7fc64d7fa700 1 -- 172.21.15.94:0/33399477 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc648057d90 con 0x7fc6500fbcd0 2024-04-06T03:43:09.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.046+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc6501047e0 con 0x7fc6500fbcd0 2024-04-06T03:43:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.046+0000 7fc64d7fa700 1 -- 172.21.15.94:0/33399477 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7fc648004490 con 0x7fc6500fbcd0 2024-04-06T03:43:09.050 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:43:09.050 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:43:09.050 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 1, 2024-04-06T03:43:09.050 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 1 2024-04-06T03:43:09.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:43:09.050 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:43:09.050 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 11, 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 3 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:43:09.051 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:43:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.048+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc63c06dd40 msgr2=0x7fc63c0701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.048+0000 7fc657183700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc63c06dd40 0x7fc63c0701f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc640001490 tx=0x7fc64000b040 comp rx=0 tx=0).stop 2024-04-06T03:43:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.049+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 msgr2=0x7fc6501106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.049+0000 7fc657183700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 0x7fc6501106b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc648013870 tx=0x7fc648009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.049+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 shutdown_connections 2024-04-06T03:43:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.049+0000 7fc657183700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500fbcd0 0x7fc6501106b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.049+0000 7fc657183700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc63c06dd40 0x7fc63c0701f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.049+0000 7fc657183700 1 --2- 172.21.15.94:0/33399477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6500fcf80 0x7fc650112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.049+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 >> 172.21.15.94:0/33399477 conn(0x7fc6500f7430 msgr2=0x7fc6500f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.049+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 shutdown_connections 2024-04-06T03:43:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.049+0000 7fc657183700 1 -- 172.21.15.94:0/33399477 wait complete. 2024-04-06T03:43:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.159+0000 7f7a9232b700 1 -- 172.21.15.94:0/1293320636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c0662e0 msgr2=0x7f7a8c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.159+0000 7f7a9232b700 1 --2- 172.21.15.94:0/1293320636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c0662e0 0x7f7a8c0ff9c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c004940 tx=0x7f7a7c009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.159+0000 7f7a9232b700 1 -- 172.21.15.94:0/1293320636 shutdown_connections 2024-04-06T03:43:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.159+0000 7f7a9232b700 1 --2- 172.21.15.94:0/1293320636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a8c0fff00 0x7f7a8c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.159+0000 7f7a9232b700 1 --2- 172.21.15.94:0/1293320636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c0662e0 0x7f7a8c0ff9c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.159+0000 7f7a9232b700 1 -- 172.21.15.94:0/1293320636 >> 172.21.15.94:0/1293320636 conn(0x7f7a8c0f7430 msgr2=0x7f7a8c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.159+0000 7f7a9232b700 1 -- 172.21.15.94:0/1293320636 shutdown_connections 2024-04-06T03:43:09.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.159+0000 7f7a9232b700 1 -- 172.21.15.94:0/1293320636 wait complete. 2024-04-06T03:43:09.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.160+0000 7f7a9232b700 1 Processor -- start 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.160+0000 7f7a9232b700 1 -- start start 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.160+0000 7f7a9232b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a8c0662e0 0x7f7a8c10fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.160+0000 7f7a9232b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c0fff00 0x7f7a8c1100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.160+0000 7f7a9232b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c1106f0 con 0x7f7a8c0fff00 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.160+0000 7f7a9232b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c110f40 con 0x7f7a8c0662e0 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a8bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a8c0662e0 0x7f7a8c10fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a8bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a8c0662e0 0x7f7a8c10fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:42972/0 (socket says 172.21.15.94:42972) 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a8bfff700 1 -- 172.21.15.94:0/762474054 learned_addr learned my addr 172.21.15.94:0/762474054 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a8b7fe700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c0fff00 0x7f7a8c1100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:09.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a8bfff700 1 -- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c0fff00 msgr2=0x7f7a8c1100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a8bfff700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c0fff00 0x7f7a8c1100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a8bfff700 1 -- 172.21.15.94:0/762474054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c005040 con 0x7f7a8c0662e0 2024-04-06T03:43:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a8b7fe700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c0fff00 0x7f7a8c1100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:43:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a8bfff700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a8c0662e0 0x7f7a8c10fbb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a88ff9700 1 -- 172.21.15.94:0/762474054 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7c00e030 con 0x7f7a8c0662e0 2024-04-06T03:43:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a88ff9700 1 -- 172.21.15.94:0/762474054 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a7c01c400 con 0x7f7a8c0662e0 2024-04-06T03:43:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c1111c0 con 0x7f7a8c0662e0 2024-04-06T03:43:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a88ff9700 1 -- 172.21.15.94:0/762474054 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7c00b650 con 0x7f7a8c0662e0 2024-04-06T03:43:09.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.161+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c111700 con 0x7f7a8c0662e0 2024-04-06T03:43:09.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.162+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8c05f5f0 con 0x7f7a8c0662e0 2024-04-06T03:43:09.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.163+0000 7f7a88ff9700 1 -- 172.21.15.94:0/762474054 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 37) v1 ==== 91897+0+0 (secure 0 0 0) 0x7f7a7c013af0 con 0x7f7a8c0662e0 2024-04-06T03:43:09.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.163+0000 7f7a88ff9700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a7406dd40 0x7f7a740701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:09.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.163+0000 7f7a88ff9700 1 -- 172.21.15.94:0/762474054 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7a7c08eff0 con 0x7f7a8c0662e0 2024-04-06T03:43:09.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.163+0000 7f7a8b7fe700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a7406dd40 0x7f7a740701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:09.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.164+0000 7f7a8b7fe700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a7406dd40 0x7f7a740701f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a80004600 tx=0x7f7a8000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:09.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.168+0000 7f7a88ff9700 1 -- 172.21.15.94:0/762474054 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7a7c05c010 con 0x7f7a8c0662e0 2024-04-06T03:43:09.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:09 smithi094 ceph-mon[94053]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:09.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:09 smithi094 ceph-mon[94053]: mgrmap e37: smithi094.purlzp(active, since 3s), standbys: smithi136.jagvwt 2024-04-06T03:43:09.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:09 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:09.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:09 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:09.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:09 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/33399477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:09.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.443+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7a8c1058c0 con 0x7f7a8c0662e0 2024-04-06T03:43:09.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.444+0000 7f7a88ff9700 1 -- 172.21.15.94:0/762474054 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1807 (secure 0 0 0) 0x7f7a7c05bba0 con 0x7f7a8c0662e0 2024-04-06T03:43:09.446 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:43:09.446 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:43:09.447 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:43:09.448 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:43:09.449 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:43:09.449 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:09.449 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:43:09.449 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:43:09.449 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:43:09.449 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:43:09.449 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:09.449 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:09.449 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a7406dd40 msgr2=0x7f7a740701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a7406dd40 0x7f7a740701f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a80004600 tx=0x7f7a8000b040 comp rx=0 tx=0).stop 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a8c0662e0 msgr2=0x7f7a8c10fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a8c0662e0 0x7f7a8c10fbb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 shutdown_connections 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7a7406dd40 0x7f7a740701f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a8c0662e0 0x7f7a8c10fbb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 --2- 172.21.15.94:0/762474054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a8c0fff00 0x7f7a8c1100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 >> 172.21.15.94:0/762474054 conn(0x7f7a8c0f7430 msgr2=0x7f7a8c0fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 shutdown_connections 2024-04-06T03:43:09.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.446+0000 7f7a9232b700 1 -- 172.21.15.94:0/762474054 wait complete. 2024-04-06T03:43:09.451 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.552+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2843734288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f94066f50 msgr2=0x7f0f940ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.552+0000 7f0f98aa4700 1 --2- 172.21.15.94:0/2843734288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f94066f50 0x7f0f940ff9c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f84004940 tx=0x7f0f84009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2843734288 shutdown_connections 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 --2- 172.21.15.94:0/2843734288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f940fff00 0x7f0f94102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 --2- 172.21.15.94:0/2843734288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f94066f50 0x7f0f940ff9c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2843734288 >> 172.21.15.94:0/2843734288 conn(0x7f0f940f7450 msgr2=0x7f0f940f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2843734288 shutdown_connections 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2843734288 wait complete. 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 Processor -- start 2024-04-06T03:43:09.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 -- start start 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f94066f50 0x7f0f9410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f940fff00 0x7f0f9410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9410e4a0 con 0x7f0f940fff00 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f98aa4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9410e610 con 0x7f0f94066f50 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f91d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f940fff00 0x7f0f9410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f91d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f940fff00 0x7f0f9410dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48470/0 (socket says 172.21.15.94:48470) 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f91d9b700 1 -- 172.21.15.94:0/2931852463 learned_addr learned my addr 172.21.15.94:0/2931852463 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.553+0000 7f0f9259c700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f94066f50 0x7f0f9410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.554+0000 7f0f91d9b700 1 -- 172.21.15.94:0/2931852463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f94066f50 msgr2=0x7f0f9410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.554+0000 7f0f91d9b700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f94066f50 0x7f0f9410d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.554+0000 7f0f91d9b700 1 -- 172.21.15.94:0/2931852463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f84005040 con 0x7f0f940fff00 2024-04-06T03:43:09.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.554+0000 7f0f91d9b700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f940fff00 0x7f0f9410dea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0f88002a50 tx=0x7f0f880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:09.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.554+0000 7f0f82ffd700 1 -- 172.21.15.94:0/2931852463 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f88007cc0 con 0x7f0f940fff00 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.554+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f94113070 con 0x7f0f940fff00 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.554+0000 7f0f82ffd700 1 -- 172.21.15.94:0/2931852463 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f8800f040 con 0x7f0f940fff00 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.554+0000 7f0f82ffd700 1 -- 172.21.15.94:0/2931852463 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f8800da30 con 0x7f0f940fff00 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.554+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f94113630 con 0x7f0f940fff00 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.555+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9404c710 con 0x7f0f940fff00 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.555+0000 7f0f82ffd700 1 -- 172.21.15.94:0/2931852463 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 37) v1 ==== 91905+0+0 (secure 0 0 0) 0x7f0f8800db90 con 0x7f0f940fff00 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.556+0000 7f0f82ffd700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0f7c06dd30 0x7f0f7c0701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.556+0000 7f0f9259c700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0f7c06dd30 0x7f0f7c0701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.556+0000 7f0f82ffd700 1 -- 172.21.15.94:0/2931852463 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0f8808d3d0 con 0x7f0f940fff00 2024-04-06T03:43:09.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.556+0000 7f0f9259c700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0f7c06dd30 0x7f0f7c0701e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0f84000c00 tx=0x7f0f84009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:09.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.561+0000 7f0f82ffd700 1 -- 172.21.15.94:0/2931852463 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0f8805a3f0 con 0x7f0f940fff00 2024-04-06T03:43:09.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:09 smithi136 ceph-mon[36436]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:09.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:09 smithi136 ceph-mon[36436]: mgrmap e37: smithi094.purlzp(active, since 3s), standbys: smithi136.jagvwt 2024-04-06T03:43:09.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:09 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:09.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:09 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:09.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:09 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/33399477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:09.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.777+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f940f8f40 con 0x7f0f7c06dd30 2024-04-06T03:43:09.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.778+0000 7f0f82ffd700 1 -- 172.21.15.94:0/2931852463 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f0f940f8f40 con 0x7f0f7c06dd30 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr" 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "2/21 daemons upgraded", 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "", 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:43:09.781 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0f7c06dd30 msgr2=0x7f0f7c0701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0f7c06dd30 0x7f0f7c0701e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0f84000c00 tx=0x7f0f84009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f940fff00 msgr2=0x7f0f9410dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f940fff00 0x7f0f9410dea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0f88002a50 tx=0x7f0f880046b0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 shutdown_connections 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0f7c06dd30 0x7f0f7c0701e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f94066f50 0x7f0f9410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 --2- 172.21.15.94:0/2931852463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f940fff00 0x7f0f9410dea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 >> 172.21.15.94:0/2931852463 conn(0x7f0f940f7450 msgr2=0x7f0f940f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 shutdown_connections 2024-04-06T03:43:09.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.780+0000 7f0f98aa4700 1 -- 172.21.15.94:0/2931852463 wait complete. 2024-04-06T03:43:09.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.881+0000 7f33010d2700 1 -- 172.21.15.94:0/3270276683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 msgr2=0x7f32fc0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.881+0000 7f33010d2700 1 --2- 172.21.15.94:0/3270276683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 0x7f32fc0fd3f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f32ec004940 tx=0x7f32ec009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:09.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.881+0000 7f33010d2700 1 -- 172.21.15.94:0/3270276683 shutdown_connections 2024-04-06T03:43:09.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.881+0000 7f33010d2700 1 --2- 172.21.15.94:0/3270276683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 0x7f32fc0fd3f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.881+0000 7f33010d2700 1 --2- 172.21.15.94:0/3270276683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32fc0fbcd0 0x7f32fc0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.881+0000 7f33010d2700 1 -- 172.21.15.94:0/3270276683 >> 172.21.15.94:0/3270276683 conn(0x7f32fc0f7430 msgr2=0x7f32fc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:09.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.881+0000 7f33010d2700 1 -- 172.21.15.94:0/3270276683 shutdown_connections 2024-04-06T03:43:09.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.881+0000 7f33010d2700 1 -- 172.21.15.94:0/3270276683 wait complete. 2024-04-06T03:43:09.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f33010d2700 1 Processor -- start 2024-04-06T03:43:09.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f33010d2700 1 -- start start 2024-04-06T03:43:09.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f33010d2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32fc0fbcd0 0x7f32fc10c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:09.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f33010d2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 0x7f32fc10ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:09.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f33010d2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc10d420 con 0x7f32fc0fcf80 2024-04-06T03:43:09.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f33010d2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc10dc70 con 0x7f32fc0fbcd0 2024-04-06T03:43:09.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f32fa59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 0x7f32fc10ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f32fa59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 0x7f32fc10ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48500/0 (socket says 172.21.15.94:48500) 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f32fa59c700 1 -- 172.21.15.94:0/459365860 learned_addr learned my addr 172.21.15.94:0/459365860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f32fad9d700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32fc0fbcd0 0x7f32fc10c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f32fa59c700 1 -- 172.21.15.94:0/459365860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32fc0fbcd0 msgr2=0x7f32fc10c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f32fa59c700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32fc0fbcd0 0x7f32fc10c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f32fa59c700 1 -- 172.21.15.94:0/459365860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec005040 con 0x7f32fc0fcf80 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f32fa59c700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 0x7f32fc10ce20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32ec013870 tx=0x7f32ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.882+0000 7f32e37fe700 1 -- 172.21.15.94:0/459365860 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32ec021070 con 0x7f32fc0fcf80 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.883+0000 7f32e37fe700 1 -- 172.21.15.94:0/459365860 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32ec009430 con 0x7f32fc0fcf80 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.883+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc10def0 con 0x7f32fc0fcf80 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.883+0000 7f32e37fe700 1 -- 172.21.15.94:0/459365860 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32ec00e070 con 0x7f32fc0fcf80 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.883+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc10e430 con 0x7f32fc0fcf80 2024-04-06T03:43:09.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.883+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc05f5f0 con 0x7f32fc0fcf80 2024-04-06T03:43:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.884+0000 7f32e37fe700 1 -- 172.21.15.94:0/459365860 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 37) v1 ==== 91905+0+0 (secure 0 0 0) 0x7f32ec01c750 con 0x7f32fc0fcf80 2024-04-06T03:43:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.885+0000 7f32e37fe700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f32e406dcd0 0x7f32e4070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.885+0000 7f32e37fe700 1 -- 172.21.15.94:0/459365860 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f32ec08ee30 con 0x7f32fc0fcf80 2024-04-06T03:43:09.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.885+0000 7f32fad9d700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f32e406dcd0 0x7f32e4070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:09.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.885+0000 7f32fad9d700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f32e406dcd0 0x7f32e4070180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009b80 tx=0x7f32f000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:09.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:09.889+0000 7f32e37fe700 1 -- 172.21.15.94:0/459365860 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f32ec05c010 con 0x7f32fc0fcf80 2024-04-06T03:43:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.180+0000 7f32e37fe700 1 -- 172.21.15.94:0/459365860 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f32ec01c400 con 0x7f32fc0fcf80 2024-04-06T03:43:10.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.203+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f32fc04bed0 con 0x7f32fc0fcf80 2024-04-06T03:43:10.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.203+0000 7f32e37fe700 1 -- 172.21.15.94:0/459365860 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f32ec07b0d0 con 0x7f32fc0fcf80 2024-04-06T03:43:10.205 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f32e406dcd0 msgr2=0x7f32e4070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f32e406dcd0 0x7f32e4070180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009b80 tx=0x7f32f000b040 comp rx=0 tx=0).stop 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 msgr2=0x7f32fc10ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 0x7f32fc10ce20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32ec013870 tx=0x7f32ec009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 shutdown_connections 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f32e406dcd0 0x7f32e4070180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32fc0fbcd0 0x7f32fc10c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 --2- 172.21.15.94:0/459365860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32fc0fcf80 0x7f32fc10ce20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 >> 172.21.15.94:0/459365860 conn(0x7f32fc0f7430 msgr2=0x7f32fc0f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 shutdown_connections 2024-04-06T03:43:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:10.205+0000 7f33010d2700 1 -- 172.21.15.94:0/459365860 wait complete. 2024-04-06T03:43:10.300 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:10 smithi136 ceph-mon[36436]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:10.300 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:10 smithi136 ceph-mon[36436]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:10.300 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:10 smithi136 ceph-mon[36436]: pgmap v5: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:10.300 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:10 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:10.301 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:10 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:10.301 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:10 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/762474054' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:43:10.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:10 smithi094 ceph-mon[94053]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:10.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:10 smithi094 ceph-mon[94053]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:10.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:10 smithi094 ceph-mon[94053]: pgmap v5: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:10.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:10.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:10.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:10 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/762474054' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:43:11.322 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:11 smithi136 ceph-mon[36436]: from='client.34148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:11.322 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:11 smithi136 ceph-mon[36436]: mgrmap e38: smithi094.purlzp(active, since 5s), standbys: smithi136.jagvwt 2024-04-06T03:43:11.322 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:11 smithi136 ceph-mon[36436]: from='client.? 172.21.15.94:0/459365860' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:43:11.322 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:11 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:11.323 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:11 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:11.430 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:11 smithi094 ceph-mon[94053]: from='client.34148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:11.430 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:11 smithi094 ceph-mon[94053]: mgrmap e38: smithi094.purlzp(active, since 5s), standbys: smithi136.jagvwt 2024-04-06T03:43:11.430 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:11 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/459365860' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:43:11.430 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:11.430 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:12.489 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:12 smithi094 ceph-mon[94053]: pgmap v6: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:12.489 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:12.489 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:12.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:12 smithi136 ceph-mon[36436]: pgmap v6: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:12.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:12 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:12.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:12 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:14.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:13 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:13 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:13 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:13 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:43:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:13 smithi136 ceph-mon[36436]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-06T03:43:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:13 smithi136 ceph-mon[36436]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-06T03:43:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:13 smithi136 ceph-mon[36436]: pgmap v7: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-06T03:43:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:13 smithi136 ceph-mon[36436]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:43:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:13 smithi136 ceph-mon[36436]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:43:14.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:13 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:13 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:13 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:13 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:43:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:13 smithi094 ceph-mon[94053]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-06T03:43:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:13 smithi094 ceph-mon[94053]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-06T03:43:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:13 smithi094 ceph-mon[94053]: pgmap v7: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-06T03:43:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:13 smithi094 ceph-mon[94053]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:43:14.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:13 smithi094 ceph-mon[94053]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.conf 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: pgmap v8: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:43:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:15.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:15 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-06T03:43:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: Updating smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: Updating smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: pgmap v8: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:15.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-06T03:43:16.315 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:16 smithi136 ceph-mon[36436]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-06T03:43:16.315 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:16 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:16.315 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:16 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:43:16.315 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:16 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:43:16.315 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:16 smithi136 ceph-mon[36436]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:16 smithi094 ceph-mon[94053]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-06T03:43:16.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:16.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:43:16.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:43:16.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:17.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:17 smithi094 ceph-mon[94053]: Upgrade: Updating mon.smithi136 2024-04-06T03:43:17.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:17 smithi094 ceph-mon[94053]: Deploying daemon mon.smithi136 on smithi136 2024-04-06T03:43:17.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:17 smithi094 ceph-mon[94053]: pgmap v9: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-06T03:43:17.280 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:17 smithi136 ceph-mon[36436]: Upgrade: Updating mon.smithi136 2024-04-06T03:43:17.280 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:17 smithi136 ceph-mon[36436]: Deploying daemon mon.smithi136 on smithi136 2024-04-06T03:43:17.280 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:17 smithi136 ceph-mon[36436]: pgmap v9: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-06T03:43:17.844 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:17 smithi136 systemd[1]: Stopping Ceph mon.smithi136 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:43:17.845 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:17 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136[36431]: 2024-04-06T03:43:17.788+0000 7fa75c613700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi136 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:43:17.845 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:17 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136[36431]: 2024-04-06T03:43:17.788+0000 7fa75c613700 -1 mon.smithi136@1(peon) e2 *** Got Signal Terminated *** 2024-04-06T03:43:17.845 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:17 smithi136 podman[81897]: 2024-04-06 03:43:17.840638368 +0000 UTC m=+0.145718912 container died 9f20264e45e783e2cbea5e1384eb88b9159e1748b340b8e0b8b49c582183dbaa (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=quincy-6699495, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:43:18.469 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:18 smithi136 podman[81897]: 2024-04-06 03:43:18.257218042 +0000 UTC m=+0.562298591 container cleanup 9f20264e45e783e2cbea5e1384eb88b9159e1748b340b8e0b8b49c582183dbaa (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, RELEASE=quincy-6699495, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-06T03:43:18.469 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:18 smithi136 bash[81897]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136 2024-04-06T03:43:18.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:18 smithi136 podman[81912]: 2024-04-06 03:43:18.46660534 +0000 UTC m=+0.624902485 container remove 9f20264e45e783e2cbea5e1384eb88b9159e1748b340b8e0b8b49c582183dbaa (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=quincy-6699495, GIT_BRANCH=HEAD) 2024-04-06T03:43:18.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:18 smithi136 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi136.service: Deactivated successfully. 2024-04-06T03:43:18.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:18 smithi136 systemd[1]: Stopped Ceph mon.smithi136 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:43:18.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:18 smithi136 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi136.service: Consumed 5.719s CPU time. 2024-04-06T03:43:19.219 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:18 smithi136 systemd[1]: Starting Ceph mon.smithi136 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:43:19.219 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 podman[82041]: 2024-04-06 03:43:18.990762841 +0000 UTC m=+0.018063490 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:43:19.219 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 podman[82041]: 2024-04-06 03:43:19.101054017 +0000 UTC m=+0.128354680 container create 46983eca715e307fac8e88e12b99cee98a2f947f2be64f0c4997abb73d432c8e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, io.buildah.version=1.33.5) 2024-04-06T03:43:19.481 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 podman[82041]: 2024-04-06 03:43:19.303833524 +0000 UTC m=+0.331134161 container init 46983eca715e307fac8e88e12b99cee98a2f947f2be64f0c4997abb73d432c8e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 podman[82041]: 2024-04-06 03:43:19.307372346 +0000 UTC m=+0.334672976 container start 46983eca715e307fac8e88e12b99cee98a2f947f2be64f0c4997abb73d432c8e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: set uid:gid to 167:167 (ceph:ceph) 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev), process ceph-mon, pid 2 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: pidfile_write: ignore empty --pid-file 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: load: jerasure load: lrc 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: RocksDB version: 7.9.2 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Git sha 0 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Compile date 2024-04-05 08:39:05 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: DB SUMMARY 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: DB Session ID: VVSY6DCSIUOG3W3QKHPN 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: CURRENT file: CURRENT 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: IDENTITY file: IDENTITY 2024-04-06T03:43:19.482 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: MANIFEST file: MANIFEST-000009 size: 897 Bytes 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi136/store.db dir, Total Num: 1, files: 000024.sst 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi136/store.db: 000022.log size: 5621110 ; 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.error_if_exists: 0 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.create_if_missing: 0 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.paranoid_checks: 1 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.env: 0x564acadeec60 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.fs: PosixFileSystem 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.info_log: 0x564acc0599a0 2024-04-06T03:43:19.483 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_file_opening_threads: 16 2024-04-06T03:43:19.485 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.statistics: (nil) 2024-04-06T03:43:19.485 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.use_fsync: 0 2024-04-06T03:43:19.485 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_log_file_size: 0 2024-04-06T03:43:19.485 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-06T03:43:19.485 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-06T03:43:19.485 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.keep_log_file_num: 1000 2024-04-06T03:43:19.485 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.recycle_log_file_num: 0 2024-04-06T03:43:19.485 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.allow_fallocate: 1 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.allow_mmap_reads: 0 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.allow_mmap_writes: 0 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.use_direct_reads: 0 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.create_missing_column_families: 0 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.db_log_dir: 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.wal_dir: 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-06T03:43:19.486 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.advise_random_on_open: 1 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.db_write_buffer_size: 0 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.write_buffer_manager: 0x564acc02c1e0 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.rate_limiter: (nil) 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.wal_recovery_mode: 2 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.enable_thread_tracking: 0 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.enable_pipelined_write: 0 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.unordered_write: 0 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-06T03:43:19.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.row_cache: None 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.wal_filter: None 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.allow_ingest_behind: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.two_write_queues: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.manual_wal_flush: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.wal_compression: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.atomic_flush: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.log_readahead_size: 0 2024-04-06T03:43:19.488 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.best_efforts_recovery: 0 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.allow_data_in_errors: 0 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.db_host_id: __hostname__ 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_background_jobs: 2 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_background_compactions: -1 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_subcompactions: 1 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_total_wal_size: 0 2024-04-06T03:43:19.489 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_open_files: -1 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bytes_per_sync: 0 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_readahead_size: 0 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_background_flushes: -1 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Compression algorithms supported: 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: kZSTD supported: 0 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: kXpressCompression supported: 0 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: kLZ4HCCompression supported: 1 2024-04-06T03:43:19.490 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: kZlibCompression supported: 1 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: kSnappyCompression supported: 1 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: kLZ4Compression supported: 1 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: kBZip2Compression supported: 0 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi136/store.db/MANIFEST-000009 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.merge_operator: 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_filter: None 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_filter_factory: None 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.sst_partitioner_factory: None 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-06T03:43:19.491 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564acc059aa0) 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: cache_index_and_filter_blocks: 1 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: pin_top_level_index_and_filter: 1 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: index_type: 0 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: data_block_index_type: 0 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: index_shortening: 1 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: checksum: 4 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: no_block_cache: 0 2024-04-06T03:43:19.492 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: block_cache: 0x564acc05d090 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: block_cache_name: BinnedLRUCache 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: block_cache_options: 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: capacity : 536870912 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: num_shard_bits : 4 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: strict_capacity_limit : 0 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: high_pri_pool_ratio: 0.000 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: block_cache_compressed: (nil) 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: persistent_cache: (nil) 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: block_size: 4096 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: block_size_deviation: 10 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: block_restart_interval: 16 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: index_block_restart_interval: 1 2024-04-06T03:43:19.493 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: metadata_block_size: 4096 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: partition_filters: 0 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: use_delta_encoding: 1 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: filter_policy: bloomfilter 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: whole_key_filtering: 1 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: verify_compression: 0 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: read_amp_bytes_per_bit: 0 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: format_version: 5 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: enable_index_compression: 1 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: block_align: 0 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: max_auto_readahead_size: 262144 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: prepopulate_block_cache: 0 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: initial_auto_readahead_size: 8192 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: num_file_reads_for_auto_readahead: 2 2024-04-06T03:43:19.494 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.write_buffer_size: 33554432 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_write_buffer_number: 2 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression: NoCompression 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression: Disabled 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.prefix_extractor: nullptr 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.num_levels: 7 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-06T03:43:19.495 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression_opts.level: 32767 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression_opts.strategy: 0 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression_opts.enabled: false 2024-04-06T03:43:19.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.target_file_size_base: 67108864 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-06T03:43:19.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.arena_block_size: 1048576 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.disable_auto_compactions: 0 2024-04-06T03:43:19.498 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.table_properties_collectors: 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.inplace_update_support: 0 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-06T03:43:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.bloom_locality: 0 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.max_successive_merges: 0 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.paranoid_file_checks: 0 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.force_consistency_checks: 1 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.report_bg_io_stats: 0 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.ttl: 2592000 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-06T03:43:19.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.enable_blob_files: false 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.min_blob_size: 0 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.blob_file_size: 268435456 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.blob_file_starting_level: 0 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 24.sst 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-06T03:43:19.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi136/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 26, last_sequence is 10600, log_number is 22,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 22 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 993e7db4-c4af-4990-bedf-4ed58dbc9202 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712374999337194, "job": 1, "event": "recovery_started", "wal_files": [22]} 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #22 mode 2 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712374999349015, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 27, "file_size": 3388635, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10605, "largest_seqno": 11965, "table_properties": {"data_size": 3382170, "index_size": 3910, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1541, "raw_key_size": 14160, "raw_average_key_size": 23, "raw_value_size": 3369405, "raw_average_value_size": 5653, "num_data_blocks": 178, "num_entries": 596, "num_filter_entries": 596, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712374999, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "993e7db4-c4af-4990-bedf-4ed58dbc9202", "db_session_id": "VVSY6DCSIUOG3W3QKHPN", "orig_file_number": 27, "seqno_to_time_mapping": "N/A"}} 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712374999349124, "job": 1, "event": "recovery_finished"} 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/version_set.cc:5047] Creating manifest 29 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi136/store.db/000022.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564acc14e000 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: DB pointer 0x564acc138000 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-06T03:43:19.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: ** DB Stats ** 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: ** Compaction Stats [default] ** 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: L0 1/0 3.23 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 705.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: L6 1/0 5.75 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-06T03:43:19.503 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Sum 2/0 8.98 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 705.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 705.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: ** Compaction Stats [default] ** 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 705.9 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-06T03:43:19.504 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Cumulative compaction: 0.00 GB write, 227.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Interval compaction: 0.00 GB write, 227.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Block cache BinnedLRUCache@0x564acc05d090#2 capacity: 512.00 MB usage: 5.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: starting mon.smithi136 rank 1 at public addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] at bind addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi136 fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: mon.smithi136@-1(???) e2 preinit fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: mon.smithi136@-1(???).mds e12 new map 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: mon.smithi136@-1(???).mds e12 print_map 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: e12 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: legacy client fscid: 1 2024-04-06T03:43:19.505 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Filesystem 'cephfs' (1) 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: fs_name cephfs 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: epoch 11 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: tableserver 0 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: root 0 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: session_timeout 60 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: session_autoclose 300 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: max_file_size 1099511627776 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: max_xattr_size 65536 2024-04-06T03:43:19.506 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: required_client_features {} 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: last_failure 0 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: last_failure_osd_epoch 0 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: max_mds 1 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: in 0 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: up {0=14460} 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: failed 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: damaged 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: stopped 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: data_pools [3] 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: metadata_pool 2 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: inline_data disabled 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: balancer 2024-04-06T03:43:19.507 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: bal_rank_mask -1 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: standby_count_wanted 1 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: qdb_cluster leader: 0 members: 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: [mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: Standby daemons: 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: [mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: [mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout: [mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:19.508 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: mon.smithi136@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-06T03:43:19.509 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: mon.smithi136@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-06T03:43:19.509 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: mon.smithi136@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-06T03:43:19.509 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: mon.smithi136@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-06T03:43:19.509 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 ceph-mon[82061]: mon.smithi136@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-06T03:43:19.509 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 bash[82041]: 46983eca715e307fac8e88e12b99cee98a2f947f2be64f0c4997abb73d432c8e 2024-04-06T03:43:19.509 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:19 smithi136 systemd[1]: Started Ceph mon.smithi136 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:43:20.751 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.smithi136 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: osdmap e48: 6 total, 6 up, 6 in 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: mgrmap e38: smithi094.purlzp(active, since 14s), standbys: smithi136.jagvwt 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: mon.smithi094 calling monitor election 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: mon.smithi136 calling monitor election 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: mon.smithi094 is new leader, mons smithi094,smithi136 in quorum (ranks 0,1) 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: monmap epoch 3 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:43:20.752 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: last_changed 2024-04-06T03:43:19.505310+0000 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: created 2024-04-06T03:31:02.176975+0000 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: min_mon_release 19 (squid) 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: election_strategy: 1 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.smithi136 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: osdmap e48: 6 total, 6 up, 6 in 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: mgrmap e38: smithi094.purlzp(active, since 14s), standbys: smithi136.jagvwt 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: pgmap v11: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-06T03:43:20.753 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:20.754 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:20.754 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:43:20.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.smithi136 2024-04-06T03:43:20.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:43:20.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: osdmap e48: 6 total, 6 up, 6 in 2024-04-06T03:43:20.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: mgrmap e38: smithi094.purlzp(active, since 14s), standbys: smithi136.jagvwt 2024-04-06T03:43:20.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi094"}]: dispatch 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mon metadata", "id": "smithi136"}]: dispatch 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: mon.smithi094 calling monitor election 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: mon.smithi136 calling monitor election 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: mon.smithi094 is new leader, mons smithi094,smithi136 in quorum (ranks 0,1) 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: monmap epoch 3 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: last_changed 2024-04-06T03:43:19.505310+0000 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: created 2024-04-06T03:31:02.176975+0000 2024-04-06T03:43:20.920 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: min_mon_release 19 (squid) 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: election_strategy: 1 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.smithi094 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.smithi136 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: osdmap e48: 6 total, 6 up, 6 in 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: mgrmap e38: smithi094.purlzp(active, since 14s), standbys: smithi136.jagvwt 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: pgmap v11: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-06T03:43:20.921 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:20.922 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:20.922 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:43:23.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:23 smithi136 ceph-mon[82061]: pgmap v12: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-06T03:43:23.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:23 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:23.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:23 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:23.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:23 smithi094 ceph-mon[94053]: pgmap v12: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-06T03:43:23.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:23 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:23.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:23 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:25.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:25 smithi136 ceph-mon[82061]: pgmap v13: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:25.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:25 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:25.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:25 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:25.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:25 smithi094 ceph-mon[94053]: pgmap v13: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:25.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:25 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:25.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:25 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:27 smithi094 ceph-mon[94053]: pgmap v14: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:27.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:27 smithi136 ceph-mon[82061]: pgmap v14: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:28.768 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:28.768 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:28.768 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:28.768 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:43:28.768 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: pgmap v15: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:28.768 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: pgmap v16: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:28.768 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:28.768 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: Reconfiguring mon.smithi094 (monmap changed)... 2024-04-06T03:43:28.768 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:43:28.769 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:43:28.769 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:28.769 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:28 smithi094 ceph-mon[94053]: Reconfiguring daemon mon.smithi094 on smithi094 2024-04-06T03:43:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:43:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: pgmap v15: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: pgmap v16: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: Reconfiguring mon.smithi094 (monmap changed)... 2024-04-06T03:43:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:43:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:43:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:28 smithi136 ceph-mon[82061]: Reconfiguring daemon mon.smithi094 on smithi094 2024-04-06T03:43:30.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:30 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:30.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:30 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:30.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:30 smithi136 ceph-mon[82061]: Reconfiguring mgr.smithi094.purlzp (monmap changed)... 2024-04-06T03:43:30.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:30 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.purlzp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:43:30.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:30 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:43:30.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:30 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:30.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:30 smithi136 ceph-mon[82061]: Reconfiguring daemon mgr.smithi094.purlzp on smithi094 2024-04-06T03:43:30.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:30 smithi136 ceph-mon[82061]: pgmap v17: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:30.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:30 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:30.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:30 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:30.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:30 smithi094 ceph-mon[94053]: Reconfiguring mgr.smithi094.purlzp (monmap changed)... 2024-04-06T03:43:30.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:30 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi094.purlzp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:43:30.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:30 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:43:30.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:30 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:30.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:30 smithi094 ceph-mon[94053]: Reconfiguring daemon mgr.smithi094.purlzp on smithi094 2024-04-06T03:43:30.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:30 smithi094 ceph-mon[94053]: pgmap v17: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:32.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:32 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:32.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:32 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:32.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:32 smithi094 ceph-mon[94053]: Reconfiguring crash.smithi094 (monmap changed)... 2024-04-06T03:43:32.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:32 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:43:32.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:32 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:32.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:32 smithi094 ceph-mon[94053]: Reconfiguring daemon crash.smithi094 on smithi094 2024-04-06T03:43:32.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:32 smithi094 ceph-mon[94053]: pgmap v18: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:32.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:32 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:32.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:32 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:32.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:32 smithi136 ceph-mon[82061]: Reconfiguring crash.smithi094 (monmap changed)... 2024-04-06T03:43:32.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:32 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:43:32.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:32 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:32.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:32 smithi136 ceph-mon[82061]: Reconfiguring daemon crash.smithi094 on smithi094 2024-04-06T03:43:32.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:32 smithi136 ceph-mon[82061]: pgmap v18: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:34.208 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:34.208 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:34.209 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:33 smithi094 ceph-mon[94053]: Reconfiguring osd.0 (monmap changed)... 2024-04-06T03:43:34.209 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-06T03:43:34.209 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:34.209 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:33 smithi094 ceph-mon[94053]: Reconfiguring daemon osd.0 on smithi094 2024-04-06T03:43:34.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:34.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:34.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:33 smithi136 ceph-mon[82061]: Reconfiguring osd.0 (monmap changed)... 2024-04-06T03:43:34.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-06T03:43:34.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:34.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:33 smithi136 ceph-mon[82061]: Reconfiguring daemon osd.0 on smithi094 2024-04-06T03:43:35.215 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:34 smithi094 ceph-mon[94053]: pgmap v19: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:35.216 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:35.216 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:35.216 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-06T03:43:35.216 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:35.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:34 smithi136 ceph-mon[82061]: pgmap v19: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:35.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:35.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:35.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-06T03:43:35.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:36.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:35 smithi094 ceph-mon[94053]: Reconfiguring osd.1 (monmap changed)... 2024-04-06T03:43:36.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:35 smithi094 ceph-mon[94053]: Reconfiguring daemon osd.1 on smithi094 2024-04-06T03:43:36.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:35 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:43:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:35 smithi136 ceph-mon[82061]: Reconfiguring osd.1 (monmap changed)... 2024-04-06T03:43:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:35 smithi136 ceph-mon[82061]: Reconfiguring daemon osd.1 on smithi094 2024-04-06T03:43:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:35 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:43:37.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:36 smithi094 ceph-mon[94053]: pgmap v20: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:37.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:36 smithi136 ceph-mon[82061]: pgmap v20: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:38.634 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:38 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:38.634 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:38 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:38.634 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:38 smithi094 ceph-mon[94053]: Reconfiguring osd.2 (monmap changed)... 2024-04-06T03:43:38.634 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:38 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-06T03:43:38.634 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:38 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:38.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:38 smithi094 ceph-mon[94053]: Reconfiguring daemon osd.2 on smithi094 2024-04-06T03:43:38.635 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:38 smithi094 ceph-mon[94053]: pgmap v21: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:38 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:38 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:38 smithi136 ceph-mon[82061]: Reconfiguring osd.2 (monmap changed)... 2024-04-06T03:43:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:38 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-06T03:43:38.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:38 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:38.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:38 smithi136 ceph-mon[82061]: Reconfiguring daemon osd.2 on smithi094 2024-04-06T03:43:38.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:38 smithi136 ceph-mon[82061]: pgmap v21: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.311+0000 7fcd0a909700 1 -- 172.21.15.94:0/3732594669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fe060 msgr2=0x7fcd040fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.311+0000 7fcd0a909700 1 --2- 172.21.15.94:0/3732594669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fe060 0x7fcd040fe4d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fccf8004940 tx=0x7fccf8009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:40.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.311+0000 7fcd0a909700 1 -- 172.21.15.94:0/3732594669 shutdown_connections 2024-04-06T03:43:40.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.311+0000 7fcd0a909700 1 --2- 172.21.15.94:0/3732594669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fe060 0x7fcd040fe4d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.311+0000 7fcd0a909700 1 --2- 172.21.15.94:0/3732594669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd040fcef0 0x7fcd040fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.311+0000 7fcd0a909700 1 -- 172.21.15.94:0/3732594669 >> 172.21.15.94:0/3732594669 conn(0x7fcd040f85b0 msgr2=0x7fcd040fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:40.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.311+0000 7fcd0a909700 1 -- 172.21.15.94:0/3732594669 shutdown_connections 2024-04-06T03:43:40.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.311+0000 7fcd0a909700 1 -- 172.21.15.94:0/3732594669 wait complete. 2024-04-06T03:43:40.316 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:40.316 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:40.317 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:40 smithi094 ceph-mon[94053]: Reconfiguring mds.cephfs.smithi094.qogyzw (monmap changed)... 2024-04-06T03:43:40.317 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.qogyzw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:43:40.317 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:40.317 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:40 smithi094 ceph-mon[94053]: Reconfiguring daemon mds.cephfs.smithi094.qogyzw on smithi094 2024-04-06T03:43:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.312+0000 7fcd0a909700 1 Processor -- start 2024-04-06T03:43:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.312+0000 7fcd0a909700 1 -- start start 2024-04-06T03:43:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.312+0000 7fcd0a909700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fcef0 0x7fcd04071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.312+0000 7fcd0a909700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd040fe060 0x7fcd04070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.312+0000 7fcd0a909700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040708c0 con 0x7fcd040fcef0 2024-04-06T03:43:40.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.312+0000 7fcd0a909700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04070a30 con 0x7fcd040fe060 2024-04-06T03:43:40.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd03fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fcef0 0x7fcd04071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:40.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd03fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fcef0 0x7fcd04071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49526/0 (socket says 172.21.15.94:49526) 2024-04-06T03:43:40.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd03fff700 1 -- 172.21.15.94:0/3110341927 learned_addr learned my addr 172.21.15.94:0/3110341927 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:40.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd037fe700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd040fe060 0x7fcd04070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:40.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd03fff700 1 -- 172.21.15.94:0/3110341927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd040fe060 msgr2=0x7fcd04070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd03fff700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd040fe060 0x7fcd04070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd03fff700 1 -- 172.21.15.94:0/3110341927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf8005040 con 0x7fcd040fcef0 2024-04-06T03:43:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd03fff700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fcef0 0x7fcd04071e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fccf4002a50 tx=0x7fccf40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd00ff9700 1 -- 172.21.15.94:0/3110341927 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf4007cc0 con 0x7fcd040fcef0 2024-04-06T03:43:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd00ff9700 1 -- 172.21.15.94:0/3110341927 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccf400f040 con 0x7fcd040fcef0 2024-04-06T03:43:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd00ff9700 1 -- 172.21.15.94:0/3110341927 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf4009640 con 0x7fcd040fcef0 2024-04-06T03:43:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd04070d10 con 0x7fcd040fcef0 2024-04-06T03:43:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.313+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd04115a70 con 0x7fcd040fcef0 2024-04-06T03:43:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.314+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0405f5f0 con 0x7fcd040fcef0 2024-04-06T03:43:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.315+0000 7fcd00ff9700 1 -- 172.21.15.94:0/3110341927 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7fccf40097c0 con 0x7fcd040fcef0 2024-04-06T03:43:40.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.316+0000 7fcd00ff9700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fccec06ddc0 0x7fccec070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:40.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.316+0000 7fcd00ff9700 1 -- 172.21.15.94:0/3110341927 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fccf408d4f0 con 0x7fcd040fcef0 2024-04-06T03:43:40.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.316+0000 7fcd037fe700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fccec06ddc0 0x7fccec070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:40.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.316+0000 7fcd037fe700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fccec06ddc0 0x7fccec070270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b3b0 tx=0x7fccf8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:40.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.322+0000 7fcd00ff9700 1 -- 172.21.15.94:0/3110341927 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fccf4056900 con 0x7fcd040fcef0 2024-04-06T03:43:40.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.541+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd041023c0 con 0x7fccec06ddc0 2024-04-06T03:43:40.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.542+0000 7fcd00ff9700 1 -- 172.21.15.94:0/3110341927 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fcd041023c0 con 0x7fccec06ddc0 2024-04-06T03:43:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.544+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fccec06ddc0 msgr2=0x7fccec070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.544+0000 7fcd0a909700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fccec06ddc0 0x7fccec070270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b3b0 tx=0x7fccf8009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.544+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fcef0 msgr2=0x7fcd04071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.544+0000 7fcd0a909700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fcef0 0x7fcd04071e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fccf4002a50 tx=0x7fccf40046b0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.545+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 shutdown_connections 2024-04-06T03:43:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.545+0000 7fcd0a909700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd040fcef0 0x7fcd04071e40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.545+0000 7fcd0a909700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fccec06ddc0 0x7fccec070270 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.545+0000 7fcd0a909700 1 --2- 172.21.15.94:0/3110341927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd040fe060 0x7fcd04070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.545+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 >> 172.21.15.94:0/3110341927 conn(0x7fcd040f85b0 msgr2=0x7fcd040f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.545+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 shutdown_connections 2024-04-06T03:43:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.545+0000 7fcd0a909700 1 -- 172.21.15.94:0/3110341927 wait complete. 2024-04-06T03:43:40.559 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:43:40.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:40 smithi136 ceph-mon[82061]: Reconfiguring mds.cephfs.smithi094.qogyzw (monmap changed)... 2024-04-06T03:43:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.qogyzw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:43:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:40 smithi136 ceph-mon[82061]: Reconfiguring daemon mds.cephfs.smithi094.qogyzw on smithi094 2024-04-06T03:43:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.647+0000 7fe4148e5700 1 -- 172.21.15.94:0/876823441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 msgr2=0x7fe4100fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.647+0000 7fe4148e5700 1 --2- 172.21.15.94:0/876823441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 0x7fe4100fb890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe408009ce0 tx=0x7fe408004d10 comp rx=0 tx=0).stop 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.647+0000 7fe4148e5700 1 -- 172.21.15.94:0/876823441 shutdown_connections 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.647+0000 7fe4148e5700 1 --2- 172.21.15.94:0/876823441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4100fc730 0x7fe4100fcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.647+0000 7fe4148e5700 1 --2- 172.21.15.94:0/876823441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 0x7fe4100fb890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.647+0000 7fe4148e5700 1 -- 172.21.15.94:0/876823441 >> 172.21.15.94:0/876823441 conn(0x7fe4100f6be0 msgr2=0x7fe4100f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.647+0000 7fe4148e5700 1 -- 172.21.15.94:0/876823441 shutdown_connections 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.647+0000 7fe4148e5700 1 -- 172.21.15.94:0/876823441 wait complete. 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.648+0000 7fe4148e5700 1 Processor -- start 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.648+0000 7fe4148e5700 1 -- start start 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.648+0000 7fe4148e5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 0x7fe4100ff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.648+0000 7fe4148e5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4100fc730 0x7fe4100fda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.648+0000 7fe4148e5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100fdfb0 con 0x7fe4100fb480 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.648+0000 7fe4148e5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100fe120 con 0x7fe4100fc730 2024-04-06T03:43:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.648+0000 7fe40e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 0x7fe4100ff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.648+0000 7fe40e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 0x7fe4100ff3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49540/0 (socket says 172.21.15.94:49540) 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.648+0000 7fe40e59c700 1 -- 172.21.15.94:0/4057364103 learned_addr learned my addr 172.21.15.94:0/4057364103 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe40e59c700 1 -- 172.21.15.94:0/4057364103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4100fc730 msgr2=0x7fe4100fda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe40dd9b700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4100fc730 0x7fe4100fda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe40e59c700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4100fc730 0x7fe4100fda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe40e59c700 1 -- 172.21.15.94:0/4057364103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009940 con 0x7fe4100fb480 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe40dd9b700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4100fc730 0x7fe4100fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe40e59c700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 0x7fe4100ff3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe408004740 tx=0x7fe408004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe3feffd700 1 -- 172.21.15.94:0/4057364103 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe40800e040 con 0x7fe4100fb480 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe3feffd700 1 -- 172.21.15.94:0/4057364103 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4080128e0 con 0x7fe4100fb480 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe3feffd700 1 -- 172.21.15.94:0/4057364103 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe40801ca80 con 0x7fe4100fb480 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4100fe3a0 con 0x7fe4100fb480 2024-04-06T03:43:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.649+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe410115b90 con 0x7fe4100fb480 2024-04-06T03:43:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.650+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41004c710 con 0x7fe4100fb480 2024-04-06T03:43:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.650+0000 7fe3feffd700 1 -- 172.21.15.94:0/4057364103 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7fe408012a50 con 0x7fe4100fb480 2024-04-06T03:43:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.651+0000 7fe3feffd700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe3f806ddd0 0x7fe3f8070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:40.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.651+0000 7fe3feffd700 1 -- 172.21.15.94:0/4057364103 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe40808f2e0 con 0x7fe4100fb480 2024-04-06T03:43:40.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.651+0000 7fe40dd9b700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe3f806ddd0 0x7fe3f8070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:40.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.651+0000 7fe40dd9b700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe3f806ddd0 0x7fe3f8070280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe404006fd0 tx=0x7fe404008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.655+0000 7fe3feffd700 1 -- 172.21.15.94:0/4057364103 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe40805c2a0 con 0x7fe4100fb480 2024-04-06T03:43:40.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.876+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe410100d30 con 0x7fe3f806ddd0 2024-04-06T03:43:40.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.877+0000 7fe3feffd700 1 -- 172.21.15.94:0/4057364103 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fe410100d30 con 0x7fe3f806ddd0 2024-04-06T03:43:40.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.879+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe3f806ddd0 msgr2=0x7fe3f8070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.879+0000 7fe4148e5700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe3f806ddd0 0x7fe3f8070280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe404006fd0 tx=0x7fe404008040 comp rx=0 tx=0).stop 2024-04-06T03:43:40.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.879+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 msgr2=0x7fe4100ff3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.879+0000 7fe4148e5700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 0x7fe4100ff3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe408004740 tx=0x7fe408004d10 comp rx=0 tx=0).stop 2024-04-06T03:43:40.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.879+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 shutdown_connections 2024-04-06T03:43:40.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.879+0000 7fe4148e5700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4100fb480 0x7fe4100ff3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.879+0000 7fe4148e5700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe3f806ddd0 0x7fe3f8070280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.879+0000 7fe4148e5700 1 --2- 172.21.15.94:0/4057364103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4100fc730 0x7fe4100fda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.879+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 >> 172.21.15.94:0/4057364103 conn(0x7fe4100f6be0 msgr2=0x7fe4100f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:40.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.880+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 shutdown_connections 2024-04-06T03:43:40.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.880+0000 7fe4148e5700 1 -- 172.21.15.94:0/4057364103 wait complete. 2024-04-06T03:43:40.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.984+0000 7f7f409d2700 1 -- 172.21.15.94:0/2952184759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fdff0 msgr2=0x7f7f3c0fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.984+0000 7f7f409d2700 1 --2- 172.21.15.94:0/2952184759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fdff0 0x7f7f3c0fe460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f30004940 tx=0x7f7f30009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:40.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.984+0000 7f7f409d2700 1 -- 172.21.15.94:0/2952184759 shutdown_connections 2024-04-06T03:43:40.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.984+0000 7f7f409d2700 1 --2- 172.21.15.94:0/2952184759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fdff0 0x7f7f3c0fe460 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.984+0000 7f7f409d2700 1 --2- 172.21.15.94:0/2952184759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f3c0fce80 0x7f7f3c0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.984+0000 7f7f409d2700 1 -- 172.21.15.94:0/2952184759 >> 172.21.15.94:0/2952184759 conn(0x7f7f3c0f8580 msgr2=0x7f7f3c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:40.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.984+0000 7f7f409d2700 1 -- 172.21.15.94:0/2952184759 shutdown_connections 2024-04-06T03:43:40.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.984+0000 7f7f409d2700 1 -- 172.21.15.94:0/2952184759 wait complete. 2024-04-06T03:43:40.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f409d2700 1 Processor -- start 2024-04-06T03:43:40.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f409d2700 1 -- start start 2024-04-06T03:43:40.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f409d2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fce80 0x7f7f3c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:40.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f409d2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f3c0fdff0 0x7f7f3c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:40.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f409d2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c10e600 con 0x7f7f3c0fce80 2024-04-06T03:43:40.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f409d2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c10edf0 con 0x7f7f3c0fdff0 2024-04-06T03:43:40.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f3a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fce80 0x7f7f3c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:40.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f3a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fce80 0x7f7f3c10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49554/0 (socket says 172.21.15.94:49554) 2024-04-06T03:43:40.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f3a59c700 1 -- 172.21.15.94:0/960802015 learned_addr learned my addr 172.21.15.94:0/960802015 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:40.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f39d9b700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f3c0fdff0 0x7f7f3c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f3a59c700 1 -- 172.21.15.94:0/960802015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f3c0fdff0 msgr2=0x7f7f3c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f3a59c700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f3c0fdff0 0x7f7f3c10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.985+0000 7f7f3a59c700 1 -- 172.21.15.94:0/960802015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f30005040 con 0x7f7f3c0fce80 2024-04-06T03:43:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.986+0000 7f7f3a59c700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fce80 0x7f7f3c10da30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00b790 tx=0x7f7f2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.986+0000 7f7f2affd700 1 -- 172.21.15.94:0/960802015 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f2c00f6f0 con 0x7f7f3c0fce80 2024-04-06T03:43:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.986+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3c10f0d0 con 0x7f7f3c0fce80 2024-04-06T03:43:40.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.986+0000 7f7f2affd700 1 -- 172.21.15.94:0/960802015 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f2c00fd20 con 0x7f7f3c0fce80 2024-04-06T03:43:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.986+0000 7f7f2affd700 1 -- 172.21.15.94:0/960802015 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f2c0103a0 con 0x7f7f3c0fce80 2024-04-06T03:43:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.986+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c10f690 con 0x7f7f3c0fce80 2024-04-06T03:43:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.987+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c05f5f0 con 0x7f7f3c0fce80 2024-04-06T03:43:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.987+0000 7f7f2affd700 1 -- 172.21.15.94:0/960802015 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f7f2c011400 con 0x7f7f3c0fce80 2024-04-06T03:43:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.988+0000 7f7f2affd700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7f2406ddc0 0x7f7f24070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.988+0000 7f7f2affd700 1 -- 172.21.15.94:0/960802015 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7f2c08d440 con 0x7f7f3c0fce80 2024-04-06T03:43:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.988+0000 7f7f39d9b700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7f2406ddc0 0x7f7f24070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:40.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.988+0000 7f7f39d9b700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7f2406ddc0 0x7f7f24070270 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b3b0 tx=0x7f7f30009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:40.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:40.992+0000 7f7f2affd700 1 -- 172.21.15.94:0/960802015 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7f2c05a510 con 0x7f7f3c0fce80 2024-04-06T03:43:41.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.208+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f3c10f9d0 con 0x7f7f2406ddc0 2024-04-06T03:43:41.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.219+0000 7f7f2affd700 1 -- 172.21.15.94:0/960802015 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f7f3c10f9d0 con 0x7f7f2406ddc0 2024-04-06T03:43:41.222 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (2m) 31s ago 10m 19.0M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (11m) 31s ago 11m 6660k - 17.2.7-819-g6699495c ec300eaa768b ec3c584ff1c5 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (10m) 18s ago 10m 6656k - 17.2.7-819-g6699495c ec300eaa768b f98224ba51dd 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 31s ago 10m - - 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (5m) 31s ago 5m 14.2M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (5m) 31s ago 5m 155M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (5m) 18s ago 5m 14.4M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (5m) 18s ago 5m 16.7M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (110s) 31s ago 12m 473M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (58s) 18s ago 10m 447M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (44s) 31s ago 12m 26.9M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (22s) 18s ago 10m 25.7M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (3m) 31s ago 11m 8086k - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:43:41.223 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (2m) 18s ago 10m 17.4M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:43:41.224 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (9m) 31s ago 9m 252M 1557M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:43:41.224 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (8m) 31s ago 8m 265M 1557M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:43:41.224 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (8m) 31s ago 8m 226M 1557M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:43:41.224 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (8m) 18s ago 8m 291M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:43:41.224 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (7m) 18s ago 7m 256M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:43:41.224 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (7m) 18s ago 7m 241M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:43:41.224 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (64s) 31s ago 10m 42.4M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7f2406ddc0 msgr2=0x7f7f24070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7f2406ddc0 0x7f7f24070270 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b3b0 tx=0x7f7f30009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fce80 msgr2=0x7f7f3c10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fce80 0x7f7f3c10da30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00b790 tx=0x7f7f2c009250 comp rx=0 tx=0).stop 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 shutdown_connections 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f3c0fce80 0x7f7f3c10da30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7f2406ddc0 0x7f7f24070270 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 --2- 172.21.15.94:0/960802015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f3c0fdff0 0x7f7f3c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 >> 172.21.15.94:0/960802015 conn(0x7f7f3c0f8580 msgr2=0x7f7f3c0f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 shutdown_connections 2024-04-06T03:43:41.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.221+0000 7f7f409d2700 1 -- 172.21.15.94:0/960802015 wait complete. 2024-04-06T03:43:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.326+0000 7fe69c579700 1 -- 172.21.15.94:0/752288801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 msgr2=0x7fe694102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.326+0000 7fe69c579700 1 --2- 172.21.15.94:0/752288801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 0x7fe694102380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe684004940 tx=0x7fe684009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:41.330 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:41 smithi094 ceph-mon[94053]: pgmap v22: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:41.330 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:41 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:41.330 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:41 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:41.330 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:41 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.kvxcqo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:43:41.330 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:41 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:41.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.326+0000 7fe69c579700 1 -- 172.21.15.94:0/752288801 shutdown_connections 2024-04-06T03:43:41.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.326+0000 7fe69c579700 1 --2- 172.21.15.94:0/752288801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 0x7fe694102380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.326+0000 7fe69c579700 1 --2- 172.21.15.94:0/752288801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6940fb200 0x7fe6940fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.326+0000 7fe69c579700 1 -- 172.21.15.94:0/752288801 >> 172.21.15.94:0/752288801 conn(0x7fe6940f6ba0 msgr2=0x7fe6940f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.326+0000 7fe69c579700 1 -- 172.21.15.94:0/752288801 shutdown_connections 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.326+0000 7fe69c579700 1 -- 172.21.15.94:0/752288801 wait complete. 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.327+0000 7fe69c579700 1 Processor -- start 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.327+0000 7fe69c579700 1 -- start start 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.327+0000 7fe69c579700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6940fb200 0x7fe69410da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.327+0000 7fe69c579700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 0x7fe69410df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.327+0000 7fe69c579700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69410e560 con 0x7fe6940fbb50 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.327+0000 7fe69c579700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69410edb0 con 0x7fe6940fb200 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe699b14700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 0x7fe69410df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe699b14700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 0x7fe69410df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49580/0 (socket says 172.21.15.94:49580) 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe699b14700 1 -- 172.21.15.94:0/183166671 learned_addr learned my addr 172.21.15.94:0/183166671 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe69a315700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6940fb200 0x7fe69410da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe699b14700 1 -- 172.21.15.94:0/183166671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6940fb200 msgr2=0x7fe69410da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe699b14700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6940fb200 0x7fe69410da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe699b14700 1 -- 172.21.15.94:0/183166671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe684005040 con 0x7fe6940fbb50 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe699b14700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 0x7fe69410df60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe68400b3b0 tx=0x7fe684009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe68affd700 1 -- 172.21.15.94:0/183166671 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe68400e040 con 0x7fe6940fbb50 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe68affd700 1 -- 172.21.15.94:0/183166671 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe68400bcc0 con 0x7fe6940fbb50 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69410f030 con 0x7fe6940fbb50 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe68affd700 1 -- 172.21.15.94:0/183166671 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe68401c630 con 0x7fe6940fbb50 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.328+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69410f570 con 0x7fe6940fbb50 2024-04-06T03:43:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.329+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69405f5f0 con 0x7fe6940fbb50 2024-04-06T03:43:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.330+0000 7fe68affd700 1 -- 172.21.15.94:0/183166671 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7fe684013980 con 0x7fe6940fbb50 2024-04-06T03:43:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.331+0000 7fe68affd700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe68006ddc0 0x7fe680070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.331+0000 7fe68affd700 1 -- 172.21.15.94:0/183166671 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe68408ebb0 con 0x7fe6940fbb50 2024-04-06T03:43:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.331+0000 7fe69a315700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe68006ddc0 0x7fe680070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.332+0000 7fe69a315700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe68006ddc0 0x7fe680070270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe690007fd0 tx=0x7fe690009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.335+0000 7fe68affd700 1 -- 172.21.15.94:0/183166671 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe68405bb70 con 0x7fe6940fbb50 2024-04-06T03:43:41.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:41 smithi136 ceph-mon[82061]: pgmap v22: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:41.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:41 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:41.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:41 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:41.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:41 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.kvxcqo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:43:41.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:41 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:41.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.639+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe6941058c0 con 0x7fe6940fbb50 2024-04-06T03:43:41.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.640+0000 7fe68affd700 1 -- 172.21.15.94:0/183166671 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fe68405b700 con 0x7fe6940fbb50 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:43:41.644 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:43:41.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:43:41.645 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:43:41.645 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 10, 2024-04-06T03:43:41.645 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 4 2024-04-06T03:43:41.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:43:41.645 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:43:41.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.642+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe68006ddc0 msgr2=0x7fe680070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:41.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.642+0000 7fe69c579700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe68006ddc0 0x7fe680070270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe690007fd0 tx=0x7fe690009040 comp rx=0 tx=0).stop 2024-04-06T03:43:41.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.642+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 msgr2=0x7fe69410df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:41.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.642+0000 7fe69c579700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 0x7fe69410df60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe68400b3b0 tx=0x7fe684009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:41.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.643+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 shutdown_connections 2024-04-06T03:43:41.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.643+0000 7fe69c579700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe68006ddc0 0x7fe680070270 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.643+0000 7fe69c579700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6940fb200 0x7fe69410da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.643+0000 7fe69c579700 1 --2- 172.21.15.94:0/183166671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6940fbb50 0x7fe69410df60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.643+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 >> 172.21.15.94:0/183166671 conn(0x7fe6940f6ba0 msgr2=0x7fe6941004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:41.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.643+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 shutdown_connections 2024-04-06T03:43:41.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.643+0000 7fe69c579700 1 -- 172.21.15.94:0/183166671 wait complete. 2024-04-06T03:43:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 -- 172.21.15.94:0/620154743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fbcd0 msgr2=0x7f83b40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 --2- 172.21.15.94:0/620154743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fbcd0 0x7f83b40fc0e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83ac004940 tx=0x7f83ac009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 -- 172.21.15.94:0/620154743 shutdown_connections 2024-04-06T03:43:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 --2- 172.21.15.94:0/620154743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b40fcf80 0x7f83b40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 --2- 172.21.15.94:0/620154743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fbcd0 0x7f83b40fc0e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 -- 172.21.15.94:0/620154743 >> 172.21.15.94:0/620154743 conn(0x7f83b40f7430 msgr2=0x7f83b40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:41.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 -- 172.21.15.94:0/620154743 shutdown_connections 2024-04-06T03:43:41.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 -- 172.21.15.94:0/620154743 wait complete. 2024-04-06T03:43:41.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 Processor -- start 2024-04-06T03:43:41.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.746+0000 7f83bc418700 1 -- start start 2024-04-06T03:43:41.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83bc418700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b40fbcd0 0x7f83b410da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:41.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83bc418700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fcf80 0x7f83b410dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:41.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83bc418700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b410e5b0 con 0x7f83b40fcf80 2024-04-06T03:43:41.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83bc418700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b410edb0 con 0x7f83b40fbcd0 2024-04-06T03:43:41.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b99b3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fcf80 0x7f83b410dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:41.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b99b3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fcf80 0x7f83b410dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49596/0 (socket says 172.21.15.94:49596) 2024-04-06T03:43:41.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b99b3700 1 -- 172.21.15.94:0/1754104421 learned_addr learned my addr 172.21.15.94:0/1754104421 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83ba1b4700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b40fbcd0 0x7f83b410da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b99b3700 1 -- 172.21.15.94:0/1754104421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b40fbcd0 msgr2=0x7f83b410da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b99b3700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b40fbcd0 0x7f83b410da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b99b3700 1 -- 172.21.15.94:0/1754104421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac005040 con 0x7f83b40fcf80 2024-04-06T03:43:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b99b3700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fcf80 0x7f83b410dfb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83a8001d80 tx=0x7f83a8009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b2ffd700 1 -- 172.21.15.94:0/1754104421 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83a8007d10 con 0x7f83b40fcf80 2024-04-06T03:43:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b410f090 con 0x7f83b40fcf80 2024-04-06T03:43:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b2ffd700 1 -- 172.21.15.94:0/1754104421 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83a8001e80 con 0x7f83b40fcf80 2024-04-06T03:43:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83b2ffd700 1 -- 172.21.15.94:0/1754104421 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83a8014640 con 0x7f83b40fcf80 2024-04-06T03:43:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.747+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b410f5d0 con 0x7f83b40fcf80 2024-04-06T03:43:41.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.748+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b4101b10 con 0x7f83b40fcf80 2024-04-06T03:43:41.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.749+0000 7f83b2ffd700 1 -- 172.21.15.94:0/1754104421 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f83a800e960 con 0x7f83b40fcf80 2024-04-06T03:43:41.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.749+0000 7f83b2ffd700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f839c06ddc0 0x7f839c070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:41.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.749+0000 7f83b2ffd700 1 -- 172.21.15.94:0/1754104421 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f83a808d3b0 con 0x7f83b40fcf80 2024-04-06T03:43:41.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.749+0000 7f83ba1b4700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f839c06ddc0 0x7f839c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:41.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.750+0000 7f83ba1b4700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f839c06ddc0 0x7f839c070270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00b970 tx=0x7f83ac009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:41.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.754+0000 7f83b2ffd700 1 -- 172.21.15.94:0/1754104421 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f83a805a4b0 con 0x7f83b40fcf80 2024-04-06T03:43:42.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.996+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f83b405f5f0 con 0x7f83b40fcf80 2024-04-06T03:43:42.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.996+0000 7f83b2ffd700 1 -- 172.21.15.94:0/1754104421 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f83a805a040 con 0x7f83b40fcf80 2024-04-06T03:43:42.000 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:43:42.000 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:43:42.000 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:43:42.001 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:43:42.001 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:43:42.001 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:43:42.001 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:43:42.001 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:43:42.001 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:43:42.001 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:43:42.001 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:43:42.001 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:43:42.002 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:42.003 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:43:42.004 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:43:42.004 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:43:42.004 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:43:42.004 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:42.004 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:42.005 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:43:42.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.998+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f839c06ddc0 msgr2=0x7f839c070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.998+0000 7f83bc418700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f839c06ddc0 0x7f839c070270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00b970 tx=0x7f83ac009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.999+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fcf80 msgr2=0x7f83b410dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.999+0000 7f83bc418700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fcf80 0x7f83b410dfb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83a8001d80 tx=0x7f83a8009ec0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.999+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 shutdown_connections 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.999+0000 7f83bc418700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f839c06ddc0 0x7f839c070270 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.999+0000 7f83bc418700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b40fbcd0 0x7f83b410da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.999+0000 7f83bc418700 1 --2- 172.21.15.94:0/1754104421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83b40fcf80 0x7f83b410dfb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.999+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 >> 172.21.15.94:0/1754104421 conn(0x7f83b40f7430 msgr2=0x7f83b40f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.999+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 shutdown_connections 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:41.999+0000 7f83bc418700 1 -- 172.21.15.94:0/1754104421 wait complete. 2024-04-06T03:43:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:43:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 -- 172.21.15.94:0/510237902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 msgr2=0x7f6d900fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 --2- 172.21.15.94:0/510237902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 0x7f6d900fd070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d78004940 tx=0x7f6d78009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 -- 172.21.15.94:0/510237902 shutdown_connections 2024-04-06T03:43:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 --2- 172.21.15.94:0/510237902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 0x7f6d900fd070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 --2- 172.21.15.94:0/510237902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d900fc2b0 0x7f6d900fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 -- 172.21.15.94:0/510237902 >> 172.21.15.94:0/510237902 conn(0x7f6d900f6ba0 msgr2=0x7f6d900f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:42.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 -- 172.21.15.94:0/510237902 shutdown_connections 2024-04-06T03:43:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 -- 172.21.15.94:0/510237902 wait complete. 2024-04-06T03:43:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 Processor -- start 2024-04-06T03:43:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.102+0000 7f6d9597e700 1 -- start start 2024-04-06T03:43:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d9597e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d900fc2b0 0x7f6d9010da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d9597e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 0x7f6d9010dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d9597e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9010e5c0 con 0x7f6d900fcc00 2024-04-06T03:43:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d9597e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9010edc0 con 0x7f6d900fc2b0 2024-04-06T03:43:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d867fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 0x7f6d9010dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:42.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d867fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 0x7f6d9010dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49622/0 (socket says 172.21.15.94:49622) 2024-04-06T03:43:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d867fc700 1 -- 172.21.15.94:0/3881973866 learned_addr learned my addr 172.21.15.94:0/3881973866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d8effd700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d900fc2b0 0x7f6d9010da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d867fc700 1 -- 172.21.15.94:0/3881973866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d900fc2b0 msgr2=0x7f6d9010da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d867fc700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d900fc2b0 0x7f6d9010da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d867fc700 1 -- 172.21.15.94:0/3881973866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78005040 con 0x7f6d900fcc00 2024-04-06T03:43:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d867fc700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 0x7f6d9010dfc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009150 tx=0x7f6d78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d87fff700 1 -- 172.21.15.94:0/3881973866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7800e040 con 0x7f6d900fcc00 2024-04-06T03:43:42.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9010f040 con 0x7f6d900fcc00 2024-04-06T03:43:42.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d87fff700 1 -- 172.21.15.94:0/3881973866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d7800bcc0 con 0x7f6d900fcc00 2024-04-06T03:43:42.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.103+0000 7f6d87fff700 1 -- 172.21.15.94:0/3881973866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7801c630 con 0x7f6d900fcc00 2024-04-06T03:43:42.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.104+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9010f580 con 0x7f6d900fcc00 2024-04-06T03:43:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.105+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d90101c00 con 0x7f6d900fcc00 2024-04-06T03:43:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.105+0000 7f6d87fff700 1 -- 172.21.15.94:0/3881973866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f6d78013420 con 0x7f6d900fcc00 2024-04-06T03:43:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.105+0000 7f6d87fff700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d7c06ddc0 0x7f6d7c070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.105+0000 7f6d8effd700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d7c06ddc0 0x7f6d7c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.105+0000 7f6d87fff700 1 -- 172.21.15.94:0/3881973866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6d7808e3f0 con 0x7f6d900fcc00 2024-04-06T03:43:42.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.106+0000 7f6d8effd700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d7c06ddc0 0x7f6d7c070270 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009fd0 tx=0x7f6d80009f00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:42.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.110+0000 7f6d87fff700 1 -- 172.21.15.94:0/3881973866 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6d7805b3b0 con 0x7f6d900fcc00 2024-04-06T03:43:42.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.327+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d90104960 con 0x7f6d7c06ddc0 2024-04-06T03:43:42.331 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:42 smithi094 ceph-mon[94053]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:42.331 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:42 smithi094 ceph-mon[94053]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:42.331 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:42 smithi094 ceph-mon[94053]: Reconfiguring mds.cephfs.smithi094.kvxcqo (monmap changed)... 2024-04-06T03:43:42.331 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:42 smithi094 ceph-mon[94053]: Reconfiguring daemon mds.cephfs.smithi094.kvxcqo on smithi094 2024-04-06T03:43:42.332 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:42 smithi094 ceph-mon[94053]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:42.332 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:42 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/183166671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:42.332 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:42 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1754104421' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:43:42.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.328+0000 7f6d87fff700 1 -- 172.21.15.94:0/3881973866 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f6d90104960 con 0x7f6d7c06ddc0 2024-04-06T03:43:42.332 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:43:42.332 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "4/21 daemons upgraded", 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mon daemons", 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:43:42.333 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d7c06ddc0 msgr2=0x7f6d7c070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d7c06ddc0 0x7f6d7c070270 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009fd0 tx=0x7f6d80009f00 comp rx=0 tx=0).stop 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 msgr2=0x7f6d9010dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 0x7f6d9010dfc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009150 tx=0x7f6d78009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 shutdown_connections 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d7c06ddc0 0x7f6d7c070270 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d900fc2b0 0x7f6d9010da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 --2- 172.21.15.94:0/3881973866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d900fcc00 0x7f6d9010dfc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 >> 172.21.15.94:0/3881973866 conn(0x7f6d900f6ba0 msgr2=0x7f6d900fb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 shutdown_connections 2024-04-06T03:43:42.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.331+0000 7f6d9597e700 1 -- 172.21.15.94:0/3881973866 wait complete. 2024-04-06T03:43:42.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.432+0000 7f583ccac700 1 -- 172.21.15.94:0/3658453590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58380fff30 msgr2=0x7f5838102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.432+0000 7f583ccac700 1 --2- 172.21.15.94:0/3658453590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58380fff30 0x7f5838102350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5820004940 tx=0x7f5820009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:42.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 -- 172.21.15.94:0/3658453590 shutdown_connections 2024-04-06T03:43:42.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 --2- 172.21.15.94:0/3658453590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58380fff30 0x7f5838102350 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 --2- 172.21.15.94:0/3658453590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5838066f90 0x7f58380ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 -- 172.21.15.94:0/3658453590 >> 172.21.15.94:0/3658453590 conn(0x7f58380f7480 msgr2=0x7f58380f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:42.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 -- 172.21.15.94:0/3658453590 shutdown_connections 2024-04-06T03:43:42.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 -- 172.21.15.94:0/3658453590 wait complete. 2024-04-06T03:43:42.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 Processor -- start 2024-04-06T03:43:42.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 -- start start 2024-04-06T03:43:42.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5838066f90 0x7f583810d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:42.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58380fff30 0x7f583810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:42.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583810e560 con 0x7f5838066f90 2024-04-06T03:43:42.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.433+0000 7f583ccac700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5838112f00 con 0x7f58380fff30 2024-04-06T03:43:42.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f583659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5838066f90 0x7f583810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:42.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f583659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5838066f90 0x7f583810d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49638/0 (socket says 172.21.15.94:49638) 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f583659c700 1 -- 172.21.15.94:0/731669210 learned_addr learned my addr 172.21.15.94:0/731669210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f582dd9b700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58380fff30 0x7f583810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f583659c700 1 -- 172.21.15.94:0/731669210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58380fff30 msgr2=0x7f583810ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f583659c700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58380fff30 0x7f583810ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f583659c700 1 -- 172.21.15.94:0/731669210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5820005040 con 0x7f5838066f90 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f583659c700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5838066f90 0x7f583810d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f582800b790 tx=0x7f5828009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f582f7fe700 1 -- 172.21.15.94:0/731669210 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582800f6f0 con 0x7f5838066f90 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5838113160 con 0x7f5838066f90 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f582f7fe700 1 -- 172.21.15.94:0/731669210 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f582800fd20 con 0x7f5838066f90 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f582f7fe700 1 -- 172.21.15.94:0/731669210 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58280103a0 con 0x7f5838066f90 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.434+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58381136d0 con 0x7f5838066f90 2024-04-06T03:43:42.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.435+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583804c710 con 0x7f5838066f90 2024-04-06T03:43:42.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.436+0000 7f582f7fe700 1 -- 172.21.15.94:0/731669210 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f5828011400 con 0x7f5838066f90 2024-04-06T03:43:42.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.436+0000 7f582f7fe700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f582406ddd0 0x7f5824070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:43:42.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.436+0000 7f582f7fe700 1 -- 172.21.15.94:0/731669210 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f582808d3e0 con 0x7f5838066f90 2024-04-06T03:43:42.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.436+0000 7f582dd9b700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f582406ddd0 0x7f5824070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:43:42.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.437+0000 7f582dd9b700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f582406ddd0 0x7f5824070280 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f582000b970 tx=0x7f5820009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:43:42.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.441+0000 7f582f7fe700 1 -- 172.21.15.94:0/731669210 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f582805a4b0 con 0x7f5838066f90 2024-04-06T03:43:42.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:42 smithi136 ceph-mon[82061]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:42.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:42 smithi136 ceph-mon[82061]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:42.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:42 smithi136 ceph-mon[82061]: Reconfiguring mds.cephfs.smithi094.kvxcqo (monmap changed)... 2024-04-06T03:43:42.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:42 smithi136 ceph-mon[82061]: Reconfiguring daemon mds.cephfs.smithi094.kvxcqo on smithi094 2024-04-06T03:43:42.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:42 smithi136 ceph-mon[82061]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:42.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:42 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/183166671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:42.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:42 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1754104421' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:43:42.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.725+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5838113b30 con 0x7f5838066f90 2024-04-06T03:43:42.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.725+0000 7f582f7fe700 1 -- 172.21.15.94:0/731669210 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+157 (secure 0 0 0) 0x7f582805a040 con 0x7f5838066f90 2024-04-06T03:43:42.729 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:43:42.729 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:43:42.729 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:43:42.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f582406ddd0 msgr2=0x7f5824070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f582406ddd0 0x7f5824070280 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f582000b970 tx=0x7f5820009e30 comp rx=0 tx=0).stop 2024-04-06T03:43:42.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5838066f90 msgr2=0x7f583810d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:43:42.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5838066f90 0x7f583810d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f582800b790 tx=0x7f5828009220 comp rx=0 tx=0).stop 2024-04-06T03:43:42.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 shutdown_connections 2024-04-06T03:43:42.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5838066f90 0x7f583810d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f582406ddd0 0x7f5824070280 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 --2- 172.21.15.94:0/731669210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58380fff30 0x7f583810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:43:42.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 >> 172.21.15.94:0/731669210 conn(0x7f58380f7480 msgr2=0x7f58380f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:43:42.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 shutdown_connections 2024-04-06T03:43:42.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:43:42.727+0000 7f583ccac700 1 -- 172.21.15.94:0/731669210 wait complete. 2024-04-06T03:43:43.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:43 smithi094 ceph-mon[94053]: pgmap v23: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:43.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:43 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:43.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:43 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:43.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:43 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi136", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:43:43.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:43 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:43.393 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:43 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/731669210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:43:43.410 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:43 smithi136 ceph-mon[82061]: pgmap v23: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:43.410 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:43 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:43.410 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:43 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:43.410 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:43 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi136", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:43:43.410 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:43 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:43.410 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:43 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/731669210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:43:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:44 smithi094 ceph-mon[94053]: from='client.34176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:44.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:44 smithi094 ceph-mon[94053]: Reconfiguring crash.smithi136 (monmap changed)... 2024-04-06T03:43:44.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:44 smithi094 ceph-mon[94053]: Reconfiguring daemon crash.smithi136 on smithi136 2024-04-06T03:43:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:44 smithi136 ceph-mon[82061]: from='client.34176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:43:44.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:44 smithi136 ceph-mon[82061]: Reconfiguring crash.smithi136 (monmap changed)... 2024-04-06T03:43:44.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:44 smithi136 ceph-mon[82061]: Reconfiguring daemon crash.smithi136 on smithi136 2024-04-06T03:43:45.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:45 smithi136 ceph-mon[82061]: pgmap v24: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:45.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:45 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:45.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:45 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:45.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:45 smithi136 ceph-mon[82061]: Reconfiguring mgr.smithi136.jagvwt (monmap changed)... 2024-04-06T03:43:45.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:45 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:43:45.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:45 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:43:45.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:45 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:45.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:45 smithi136 ceph-mon[82061]: Reconfiguring daemon mgr.smithi136.jagvwt on smithi136 2024-04-06T03:43:45.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:45 smithi094 ceph-mon[94053]: pgmap v24: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:45.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:45 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:45.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:45 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:45.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:45 smithi094 ceph-mon[94053]: Reconfiguring mgr.smithi136.jagvwt (monmap changed)... 2024-04-06T03:43:45.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:45 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi136.jagvwt", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-06T03:43:45.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:45 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-06T03:43:45.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:45 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:45.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:45 smithi094 ceph-mon[94053]: Reconfiguring daemon mgr.smithi136.jagvwt on smithi136 2024-04-06T03:43:46.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:46 smithi136 ceph-mon[82061]: pgmap v25: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:46.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:46 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:46.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:46 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:46.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:46 smithi136 ceph-mon[82061]: Reconfiguring mon.smithi136 (monmap changed)... 2024-04-06T03:43:46.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:46 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:43:46.854 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:46 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:43:46.854 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:46 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:46.854 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:46 smithi136 ceph-mon[82061]: Reconfiguring daemon mon.smithi136 on smithi136 2024-04-06T03:43:47.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:46 smithi094 ceph-mon[94053]: pgmap v25: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:47.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:46 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:47.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:46 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:47.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:46 smithi094 ceph-mon[94053]: Reconfiguring mon.smithi136 (monmap changed)... 2024-04-06T03:43:47.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:46 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-06T03:43:47.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:46 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-06T03:43:47.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:46 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:47.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:46 smithi094 ceph-mon[94053]: Reconfiguring daemon mon.smithi136 on smithi136 2024-04-06T03:43:48.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:48.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:48.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:48 smithi094 ceph-mon[94053]: Reconfiguring osd.3 (monmap changed)... 2024-04-06T03:43:48.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-06T03:43:48.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:48.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:48 smithi094 ceph-mon[94053]: Reconfiguring daemon osd.3 on smithi136 2024-04-06T03:43:48.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:48.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:48.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:48 smithi136 ceph-mon[82061]: Reconfiguring osd.3 (monmap changed)... 2024-04-06T03:43:48.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-06T03:43:48.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:48.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:48 smithi136 ceph-mon[82061]: Reconfiguring daemon osd.3 on smithi136 2024-04-06T03:43:49.544 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:49 smithi136 ceph-mon[82061]: pgmap v26: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:49.544 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:49 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:49.544 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:49 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:49.544 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:49 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-06T03:43:49.544 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:49 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:49.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:49 smithi094 ceph-mon[94053]: pgmap v26: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:49.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:49 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:49.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:49 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:49.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:49 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-06T03:43:49.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:49 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:50.467 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:50 smithi136 ceph-mon[82061]: Reconfiguring osd.4 (monmap changed)... 2024-04-06T03:43:50.468 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:50 smithi136 ceph-mon[82061]: Reconfiguring daemon osd.4 on smithi136 2024-04-06T03:43:50.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:50 smithi094 ceph-mon[94053]: Reconfiguring osd.4 (monmap changed)... 2024-04-06T03:43:50.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:50 smithi094 ceph-mon[94053]: Reconfiguring daemon osd.4 on smithi136 2024-04-06T03:43:51.496 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:51 smithi136 ceph-mon[82061]: pgmap v27: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:51.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:43:51.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:51.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:51.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-06T03:43:51.497 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:51.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:51 smithi094 ceph-mon[94053]: pgmap v27: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:43:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-06T03:43:51.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:52.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:52 smithi136 ceph-mon[82061]: Reconfiguring osd.5 (monmap changed)... 2024-04-06T03:43:52.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:52 smithi136 ceph-mon[82061]: Reconfiguring daemon osd.5 on smithi136 2024-04-06T03:43:52.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:52 smithi094 ceph-mon[94053]: Reconfiguring osd.5 (monmap changed)... 2024-04-06T03:43:52.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:52 smithi094 ceph-mon[94053]: Reconfiguring daemon osd.5 on smithi136 2024-04-06T03:43:53.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:53 smithi136 ceph-mon[82061]: pgmap v28: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:53.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:53 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:53.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:53 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:53.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:53 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.hbjgtw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:43:53.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:53 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:53.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:53 smithi094 ceph-mon[94053]: pgmap v28: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:53.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:53 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:53.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:53 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:53.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:53 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.hbjgtw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:43:53.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:53 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:54.559 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:54 smithi136 ceph-mon[82061]: Reconfiguring mds.cephfs.smithi136.hbjgtw (monmap changed)... 2024-04-06T03:43:54.560 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:54 smithi136 ceph-mon[82061]: Reconfiguring daemon mds.cephfs.smithi136.hbjgtw on smithi136 2024-04-06T03:43:54.560 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:54.560 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:54.560 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.trdonf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:43:54.560 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:54 smithi094 ceph-mon[94053]: Reconfiguring mds.cephfs.smithi136.hbjgtw (monmap changed)... 2024-04-06T03:43:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:54 smithi094 ceph-mon[94053]: Reconfiguring daemon mds.cephfs.smithi136.hbjgtw on smithi136 2024-04-06T03:43:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.trdonf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:43:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:55.540 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:55 smithi136 ceph-mon[82061]: pgmap v29: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:55.540 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:55 smithi136 ceph-mon[82061]: Reconfiguring mds.cephfs.smithi136.trdonf (monmap changed)... 2024-04-06T03:43:55.540 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:55 smithi136 ceph-mon[82061]: Reconfiguring daemon mds.cephfs.smithi136.trdonf on smithi136 2024-04-06T03:43:55.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:55 smithi094 ceph-mon[94053]: pgmap v29: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:55.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:55 smithi094 ceph-mon[94053]: Reconfiguring mds.cephfs.smithi136.trdonf (monmap changed)... 2024-04-06T03:43:55.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:55 smithi094 ceph-mon[94053]: Reconfiguring daemon mds.cephfs.smithi136.trdonf on smithi136 2024-04-06T03:43:56.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: pgmap v30: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:56.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:56.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:56.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:43:56.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:56.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:56.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all mon 2024-04-06T03:43:56.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:56.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi094"}]: dispatch 2024-04-06T03:43:56.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi094"}]': finished 2024-04-06T03:43:56.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi136"}]: dispatch 2024-04-06T03:43:56.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi136"}]': finished 2024-04-06T03:43:56.866 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: pgmap v30: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:56.866 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:56.866 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:56.867 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:43:56.867 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:56.867 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:43:56.867 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all mon 2024-04-06T03:43:56.867 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:56.867 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi094"}]: dispatch 2024-04-06T03:43:56.867 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi094"}]': finished 2024-04-06T03:43:56.867 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi136"}]: dispatch 2024-04-06T03:43:56.867 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi136"}]': finished 2024-04-06T03:43:58.371 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:58 smithi094 ceph-mon[94053]: Upgrade: Updating crash.smithi094 (1/2) 2024-04-06T03:43:58.371 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:58 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:58.371 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:58 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:43:58.371 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:58 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:58.372 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:58 smithi094 ceph-mon[94053]: Deploying daemon crash.smithi094 on smithi094 2024-04-06T03:43:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:58 smithi136 ceph-mon[82061]: Upgrade: Updating crash.smithi094 (1/2) 2024-04-06T03:43:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:58 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:43:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:58 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi094", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:43:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:58 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:43:58.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:58 smithi136 ceph-mon[82061]: Deploying daemon crash.smithi094 on smithi094 2024-04-06T03:43:59.232 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:43:59 smithi094 ceph-mon[94053]: pgmap v31: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:43:59.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:43:59 smithi136 ceph-mon[82061]: pgmap v31: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:01.407 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:01 smithi094 ceph-mon[94053]: pgmap v32: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:01.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:01 smithi136 ceph-mon[82061]: pgmap v32: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:03.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:02 smithi136 ceph-mon[82061]: pgmap v33: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:03.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:02 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:03.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:02 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:03.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:02 smithi094 ceph-mon[94053]: pgmap v33: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:03.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:02 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:03.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:02 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:04.785 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:04 smithi136 ceph-mon[82061]: pgmap v34: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:04.786 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:04 smithi136 ceph-mon[82061]: Upgrade: Updating crash.smithi136 (2/2) 2024-04-06T03:44:04.786 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:04 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:04.786 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:04 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi136", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:44:04.786 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:04 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:04.786 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:04 smithi136 ceph-mon[82061]: Deploying daemon crash.smithi136 on smithi136 2024-04-06T03:44:04.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:04 smithi094 ceph-mon[94053]: pgmap v34: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:04.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:04 smithi094 ceph-mon[94053]: Upgrade: Updating crash.smithi136 (2/2) 2024-04-06T03:44:04.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:04 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:04.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:04 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi136", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-06T03:44:04.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:04 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:04.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:04 smithi094 ceph-mon[94053]: Deploying daemon crash.smithi136 on smithi136 2024-04-06T03:44:05.632 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:05 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:44:05.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:05 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:44:06.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:06 smithi136 ceph-mon[82061]: pgmap v35: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:06.866 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:06 smithi094 ceph-mon[94053]: pgmap v35: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:08.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:08 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:08.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:08 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:08.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:08 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:44:08.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:08 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:08.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:08 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:08.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:08 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:44:09.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:09 smithi094 ceph-mon[94053]: pgmap v36: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:09.574 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:09 smithi136 ceph-mon[82061]: pgmap v36: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:11 smithi094 ceph-mon[94053]: pgmap v37: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:11.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:11.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:11.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:11 smithi136 ceph-mon[82061]: pgmap v37: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:11.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:11 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:11.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:11 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:12.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:12.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:12.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:12 smithi094 ceph-mon[94053]: pgmap v38: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:12.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:12.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:12.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:12 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:12.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:12 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:12.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:12 smithi136 ceph-mon[82061]: pgmap v38: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:12.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:12 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:12.502 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:12 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.833+0000 7ff9c6544700 1 -- 172.21.15.94:0/827005261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fcef0 msgr2=0x7ff9c00fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.833+0000 7ff9c6544700 1 --2- 172.21.15.94:0/827005261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fcef0 0x7ff9c00fd300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0004940 tx=0x7ff9b0009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.833+0000 7ff9c6544700 1 -- 172.21.15.94:0/827005261 shutdown_connections 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.833+0000 7ff9c6544700 1 --2- 172.21.15.94:0/827005261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9c00fe060 0x7ff9c00fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.833+0000 7ff9c6544700 1 --2- 172.21.15.94:0/827005261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fcef0 0x7ff9c00fd300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.833+0000 7ff9c6544700 1 -- 172.21.15.94:0/827005261 >> 172.21.15.94:0/827005261 conn(0x7ff9c00f85b0 msgr2=0x7ff9c00fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.833+0000 7ff9c6544700 1 -- 172.21.15.94:0/827005261 shutdown_connections 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.833+0000 7ff9c6544700 1 -- 172.21.15.94:0/827005261 wait complete. 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9c6544700 1 Processor -- start 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9c6544700 1 -- start start 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9c6544700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9c00fcef0 0x7ff9c010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:12.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9c6544700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fe060 0x7ff9c010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9c6544700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c010e4f0 con 0x7ff9c00fe060 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9c6544700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c010ed40 con 0x7ff9c00fcef0 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9b7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fe060 0x7ff9c010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9b7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fe060 0x7ff9c010def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33370/0 (socket says 172.21.15.94:33370) 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9b7fff700 1 -- 172.21.15.94:0/774839414 learned_addr learned my addr 172.21.15.94:0/774839414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9bffff700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9c00fcef0 0x7ff9c010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9b7fff700 1 -- 172.21.15.94:0/774839414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9c00fcef0 msgr2=0x7ff9c010d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9b7fff700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9c00fcef0 0x7ff9c010d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.834+0000 7ff9b7fff700 1 -- 172.21.15.94:0/774839414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0005040 con 0x7ff9c00fe060 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.835+0000 7ff9b7fff700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fe060 0x7ff9c010def0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800eb40 tx=0x7ff9a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.835+0000 7ff9bd7fa700 1 -- 172.21.15.94:0/774839414 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9a8007cc0 con 0x7ff9c00fe060 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.835+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c010efc0 con 0x7ff9c00fe060 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.835+0000 7ff9bd7fa700 1 -- 172.21.15.94:0/774839414 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9a800f040 con 0x7ff9c00fe060 2024-04-06T03:44:12.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.835+0000 7ff9bd7fa700 1 -- 172.21.15.94:0/774839414 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9a800ca10 con 0x7ff9c00fe060 2024-04-06T03:44:12.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.835+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c010f500 con 0x7ff9c00fe060 2024-04-06T03:44:12.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.836+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c005f5f0 con 0x7ff9c00fe060 2024-04-06T03:44:12.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.836+0000 7ff9bd7fa700 1 -- 172.21.15.94:0/774839414 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7ff9a800cc80 con 0x7ff9c00fe060 2024-04-06T03:44:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.837+0000 7ff9bd7fa700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff9ac06ddc0 0x7ff9ac070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.837+0000 7ff9bd7fa700 1 -- 172.21.15.94:0/774839414 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff9a808d310 con 0x7ff9c00fe060 2024-04-06T03:44:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.837+0000 7ff9bffff700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff9ac06ddc0 0x7ff9ac070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:12.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.837+0000 7ff9bffff700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff9ac06ddc0 0x7ff9ac070270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009b30 tx=0x7ff9b0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:12.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:12.842+0000 7ff9bd7fa700 1 -- 172.21.15.94:0/774839414 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff9a805a300 con 0x7ff9c00fe060 2024-04-06T03:44:13.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.056+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9c010f770 con 0x7ff9ac06ddc0 2024-04-06T03:44:13.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.057+0000 7ff9bd7fa700 1 -- 172.21.15.94:0/774839414 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7ff9c010f770 con 0x7ff9ac06ddc0 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff9ac06ddc0 msgr2=0x7ff9ac070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff9ac06ddc0 0x7ff9ac070270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009b30 tx=0x7ff9b0009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fe060 msgr2=0x7ff9c010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fe060 0x7ff9c010def0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800eb40 tx=0x7ff9a800b040 comp rx=0 tx=0).stop 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 shutdown_connections 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff9ac06ddc0 0x7ff9ac070270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9c00fcef0 0x7ff9c010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 --2- 172.21.15.94:0/774839414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9c00fe060 0x7ff9c010def0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 >> 172.21.15.94:0/774839414 conn(0x7ff9c00f85b0 msgr2=0x7ff9c00f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 shutdown_connections 2024-04-06T03:44:13.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.059+0000 7ff9c6544700 1 -- 172.21.15.94:0/774839414 wait complete. 2024-04-06T03:44:13.074 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:44:13.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.168+0000 7f9280335700 1 -- 172.21.15.94:0/3258457757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fdff0 msgr2=0x7f92780fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.168+0000 7f9280335700 1 --2- 172.21.15.94:0/3258457757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fdff0 0x7f92780fe460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9268004940 tx=0x7f9268009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:13.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 -- 172.21.15.94:0/3258457757 shutdown_connections 2024-04-06T03:44:13.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 --2- 172.21.15.94:0/3258457757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fdff0 0x7f92780fe460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 --2- 172.21.15.94:0/3258457757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92780fce80 0x7f92780fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 -- 172.21.15.94:0/3258457757 >> 172.21.15.94:0/3258457757 conn(0x7f92780f8580 msgr2=0x7f92780fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 -- 172.21.15.94:0/3258457757 shutdown_connections 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 -- 172.21.15.94:0/3258457757 wait complete. 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 Processor -- start 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 -- start start 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fce80 0x7f927810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92780fdff0 0x7f927810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.169+0000 7f9280335700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927810e600 con 0x7f92780fce80 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f9280335700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927810edf0 con 0x7f92780fdff0 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f927e0d1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fce80 0x7f927810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f927e0d1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fce80 0x7f927810da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33396/0 (socket says 172.21.15.94:33396) 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f927e0d1700 1 -- 172.21.15.94:0/1773133542 learned_addr learned my addr 172.21.15.94:0/1773133542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f927d8d0700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92780fdff0 0x7f927810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:13.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f927e0d1700 1 -- 172.21.15.94:0/1773133542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92780fdff0 msgr2=0x7f927810df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f927e0d1700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92780fdff0 0x7f927810df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f927e0d1700 1 -- 172.21.15.94:0/1773133542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268005040 con 0x7f92780fce80 2024-04-06T03:44:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f927e0d1700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fce80 0x7f927810da30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f927400b790 tx=0x7f9274009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f926effd700 1 -- 172.21.15.94:0/1773133542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927400f6f0 con 0x7f92780fce80 2024-04-06T03:44:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f926effd700 1 -- 172.21.15.94:0/1773133542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f927400fd20 con 0x7f92780fce80 2024-04-06T03:44:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927810f0d0 con 0x7f92780fce80 2024-04-06T03:44:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f926effd700 1 -- 172.21.15.94:0/1773133542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92740103a0 con 0x7f92780fce80 2024-04-06T03:44:13.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.170+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927810f640 con 0x7f92780fce80 2024-04-06T03:44:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.171+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927805f5f0 con 0x7f92780fce80 2024-04-06T03:44:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.172+0000 7f926effd700 1 -- 172.21.15.94:0/1773133542 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f9274011400 con 0x7f92780fce80 2024-04-06T03:44:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.172+0000 7f926effd700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f926406ddc0 0x7f9264070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.172+0000 7f926effd700 1 -- 172.21.15.94:0/1773133542 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f927408d440 con 0x7f92780fce80 2024-04-06T03:44:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.172+0000 7f927d8d0700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f926406ddc0 0x7f9264070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:13.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.173+0000 7f927d8d0700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f926406ddc0 0x7f9264070270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f926800b970 tx=0x7f9268009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:13.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.177+0000 7f926effd700 1 -- 172.21.15.94:0/1773133542 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f927405a510 con 0x7f92780fce80 2024-04-06T03:44:13.326 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:13 smithi094 ceph-mon[94053]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:13.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:13 smithi136 ceph-mon[82061]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.422+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f927810f9d0 con 0x7f926406ddc0 2024-04-06T03:44:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.423+0000 7f926effd700 1 -- 172.21.15.94:0/1773133542 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f927810f9d0 con 0x7f926406ddc0 2024-04-06T03:44:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f926406ddc0 msgr2=0x7f9264070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f926406ddc0 0x7f9264070270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f926800b970 tx=0x7f9268009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fce80 msgr2=0x7f927810da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fce80 0x7f927810da30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f927400b790 tx=0x7f9274009250 comp rx=0 tx=0).stop 2024-04-06T03:44:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 shutdown_connections 2024-04-06T03:44:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f92780fce80 0x7f927810da30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f926406ddc0 0x7f9264070270 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 --2- 172.21.15.94:0/1773133542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92780fdff0 0x7f927810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 >> 172.21.15.94:0/1773133542 conn(0x7f92780f8580 msgr2=0x7f92780f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 shutdown_connections 2024-04-06T03:44:13.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.426+0000 7f9280335700 1 -- 172.21.15.94:0/1773133542 wait complete. 2024-04-06T03:44:13.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.540+0000 7f5f32614700 1 -- 172.21.15.94:0/574089662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fdfe0 msgr2=0x7f5f2c0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.540+0000 7f5f32614700 1 --2- 172.21.15.94:0/574089662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fdfe0 0x7f5f2c0fe450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5f20004940 tx=0x7f5f20009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.540+0000 7f5f32614700 1 -- 172.21.15.94:0/574089662 shutdown_connections 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.540+0000 7f5f32614700 1 --2- 172.21.15.94:0/574089662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fdfe0 0x7f5f2c0fe450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.540+0000 7f5f32614700 1 --2- 172.21.15.94:0/574089662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f2c0fce70 0x7f5f2c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.540+0000 7f5f32614700 1 -- 172.21.15.94:0/574089662 >> 172.21.15.94:0/574089662 conn(0x7f5f2c0f8550 msgr2=0x7f5f2c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.540+0000 7f5f32614700 1 -- 172.21.15.94:0/574089662 shutdown_connections 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.540+0000 7f5f32614700 1 -- 172.21.15.94:0/574089662 wait complete. 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f32614700 1 Processor -- start 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f32614700 1 -- start start 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f32614700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fce70 0x7f5f2c10fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f32614700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f2c0fdfe0 0x7f5f2c110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f32614700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c110620 con 0x7f5f2c0fce70 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f32614700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c110790 con 0x7f5f2c0fdfe0 2024-04-06T03:44:13.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f2bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fce70 0x7f5f2c10fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f2bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fce70 0x7f5f2c10fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33422/0 (socket says 172.21.15.94:33422) 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f2bfff700 1 -- 172.21.15.94:0/1005654785 learned_addr learned my addr 172.21.15.94:0/1005654785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f2b7fe700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f2c0fdfe0 0x7f5f2c110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f2bfff700 1 -- 172.21.15.94:0/1005654785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f2c0fdfe0 msgr2=0x7f5f2c110020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.541+0000 7f5f2bfff700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f2c0fdfe0 0x7f5f2c110020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.542+0000 7f5f2bfff700 1 -- 172.21.15.94:0/1005654785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20005040 con 0x7f5f2c0fce70 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.542+0000 7f5f2bfff700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fce70 0x7f5f2c10fae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00eb40 tx=0x7f5f1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.542+0000 7f5f28ff9700 1 -- 172.21.15.94:0/1005654785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f1c007cc0 con 0x7f5f2c0fce70 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.542+0000 7f5f28ff9700 1 -- 172.21.15.94:0/1005654785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f1c00f040 con 0x7f5f2c0fce70 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.542+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c115190 con 0x7f5f2c0fce70 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.542+0000 7f5f28ff9700 1 -- 172.21.15.94:0/1005654785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f1c00ca10 con 0x7f5f2c0fce70 2024-04-06T03:44:13.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.542+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c1156d0 con 0x7f5f2c0fce70 2024-04-06T03:44:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.543+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c101970 con 0x7f5f2c0fce70 2024-04-06T03:44:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.543+0000 7f5f28ff9700 1 -- 172.21.15.94:0/1005654785 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f5f1c00cc80 con 0x7f5f2c0fce70 2024-04-06T03:44:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.544+0000 7f5f28ff9700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5f1406ddd0 0x7f5f14070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.544+0000 7f5f2b7fe700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5f1406ddd0 0x7f5f14070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:13.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.544+0000 7f5f28ff9700 1 -- 172.21.15.94:0/1005654785 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5f1c08d320 con 0x7f5f2c0fce70 2024-04-06T03:44:13.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.544+0000 7f5f2b7fe700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5f1406ddd0 0x7f5f14070280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009b30 tx=0x7f5f20009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:13.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.548+0000 7f5f28ff9700 1 -- 172.21.15.94:0/1005654785 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f1c05a310 con 0x7f5f2c0fce70 2024-04-06T03:44:13.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.757+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f2c02b340 con 0x7f5f1406ddd0 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.767+0000 7f5f28ff9700 1 -- 172.21.15.94:0/1005654785 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5f2c02b340 con 0x7f5f1406ddd0 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (3m) 1s ago 11m 25.5M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (12s) 1s ago 11m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (6s) 3s ago 10m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 1s ago 11m - - 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (6m) 1s ago 6m 14.6M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (6m) 1s ago 6m 154M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (6m) 3s ago 6m 14.6M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (6m) 3s ago 6m 17.0M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:44:13.771 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (2m) 1s ago 12m 498M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (91s) 3s ago 10m 449M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (77s) 1s ago 13m 41.6M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (54s) 3s ago 10m 36.7M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (3m) 1s ago 11m 15.4M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (2m) 3s ago 10m 17.3M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (9m) 1s ago 9m 253M 1557M 17.2.7-819-g6699495c ec300eaa768b ab97989ca463 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (9m) 1s ago 9m 266M 1557M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (8m) 1s ago 8m 226M 1557M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (8m) 3s ago 8m 292M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (8m) 3s ago 8m 257M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (7m) 3s ago 7m 241M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:44:13.772 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (97s) 1s ago 11m 47.2M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:44:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.769+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5f1406ddd0 msgr2=0x7f5f14070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.769+0000 7f5f32614700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5f1406ddd0 0x7f5f14070280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009b30 tx=0x7f5f20009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.769+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fce70 msgr2=0x7f5f2c10fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.769+0000 7f5f32614700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fce70 0x7f5f2c10fae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00eb40 tx=0x7f5f1c00b040 comp rx=0 tx=0).stop 2024-04-06T03:44:13.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.770+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 shutdown_connections 2024-04-06T03:44:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.770+0000 7f5f32614700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5f2c0fce70 0x7f5f2c10fae0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.770+0000 7f5f32614700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5f1406ddd0 0x7f5f14070280 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.770+0000 7f5f32614700 1 --2- 172.21.15.94:0/1005654785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f2c0fdfe0 0x7f5f2c110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.770+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 >> 172.21.15.94:0/1005654785 conn(0x7f5f2c0f8550 msgr2=0x7f5f2c0f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.770+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 shutdown_connections 2024-04-06T03:44:13.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.770+0000 7f5f32614700 1 -- 172.21.15.94:0/1005654785 wait complete. 2024-04-06T03:44:13.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.875+0000 7f421e6c9700 1 -- 172.21.15.94:0/2256113901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 msgr2=0x7f42180fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.875+0000 7f421e6c9700 1 --2- 172.21.15.94:0/2256113901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 0x7f42180fd400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f420c004940 tx=0x7f420c009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:13.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.875+0000 7f421e6c9700 1 -- 172.21.15.94:0/2256113901 shutdown_connections 2024-04-06T03:44:13.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.875+0000 7f421e6c9700 1 --2- 172.21.15.94:0/2256113901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 0x7f42180fd400 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.875+0000 7f421e6c9700 1 --2- 172.21.15.94:0/2256113901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42180fbce0 0x7f42180fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.875+0000 7f421e6c9700 1 -- 172.21.15.94:0/2256113901 >> 172.21.15.94:0/2256113901 conn(0x7f42180f7480 msgr2=0x7f42180f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:13.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.875+0000 7f421e6c9700 1 -- 172.21.15.94:0/2256113901 shutdown_connections 2024-04-06T03:44:13.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.875+0000 7f421e6c9700 1 -- 172.21.15.94:0/2256113901 wait complete. 2024-04-06T03:44:13.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f421e6c9700 1 Processor -- start 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f421e6c9700 1 -- start start 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f421e6c9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42180fbce0 0x7f42180ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f421e6c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 0x7f42180fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f421e6c9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180fe880 con 0x7f42180fcf90 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f421e6c9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180fe9f0 con 0x7f42180fbce0 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f42177fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 0x7f42180fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f42177fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 0x7f42180fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33440/0 (socket says 172.21.15.94:33440) 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f42177fe700 1 -- 172.21.15.94:0/1684832014 learned_addr learned my addr 172.21.15.94:0/1684832014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.876+0000 7f4217fff700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42180fbce0 0x7f42180ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.877+0000 7f42177fe700 1 -- 172.21.15.94:0/1684832014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42180fbce0 msgr2=0x7f42180ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.877+0000 7f42177fe700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42180fbce0 0x7f42180ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.877+0000 7f42177fe700 1 -- 172.21.15.94:0/1684832014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c005040 con 0x7f42180fcf90 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.877+0000 7f42177fe700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 0x7f42180fe2d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f420c013870 tx=0x7f420c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.877+0000 7f4214ff9700 1 -- 172.21.15.94:0/1684832014 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420c021070 con 0x7f42180fcf90 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.877+0000 7f4214ff9700 1 -- 172.21.15.94:0/1684832014 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f420c009430 con 0x7f42180fcf90 2024-04-06T03:44:13.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.877+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42180fec70 con 0x7f42180fcf90 2024-04-06T03:44:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.877+0000 7f4214ff9700 1 -- 172.21.15.94:0/1684832014 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420c00e070 con 0x7f42180fcf90 2024-04-06T03:44:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.877+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42181159c0 con 0x7f42180fcf90 2024-04-06T03:44:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.878+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4218101b20 con 0x7f42180fcf90 2024-04-06T03:44:13.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.879+0000 7f4214ff9700 1 -- 172.21.15.94:0/1684832014 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f420c013af0 con 0x7f42180fcf90 2024-04-06T03:44:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.879+0000 7f4214ff9700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f420006dd60 0x7f4200070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.879+0000 7f4214ff9700 1 -- 172.21.15.94:0/1684832014 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f420c08e850 con 0x7f42180fcf90 2024-04-06T03:44:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.879+0000 7f4217fff700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f420006dd60 0x7f4200070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:13.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.880+0000 7f4217fff700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f420006dd60 0x7f4200070210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42180ffb90 tx=0x7f420800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:13.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:13.884+0000 7f4214ff9700 1 -- 172.21.15.94:0/1684832014 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f420c057ce0 con 0x7f42180fcf90 2024-04-06T03:44:14.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.172+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f421805f5f0 con 0x7f42180fcf90 2024-04-06T03:44:14.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.173+0000 7f4214ff9700 1 -- 172.21.15.94:0/1684832014 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f420c004490 con 0x7f42180fcf90 2024-04-06T03:44:14.176 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:44:14.177 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 10, 2024-04-06T03:44:14.178 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 4 2024-04-06T03:44:14.178 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:44:14.178 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.176+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f420006dd60 msgr2=0x7f4200070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.176+0000 7f421e6c9700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f420006dd60 0x7f4200070210 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f42180ffb90 tx=0x7f420800b040 comp rx=0 tx=0).stop 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.176+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 msgr2=0x7f42180fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.176+0000 7f421e6c9700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 0x7f42180fe2d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f420c013870 tx=0x7f420c009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.176+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 shutdown_connections 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.176+0000 7f421e6c9700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f420006dd60 0x7f4200070210 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.176+0000 7f421e6c9700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42180fbce0 0x7f42180ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.176+0000 7f421e6c9700 1 --2- 172.21.15.94:0/1684832014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42180fcf90 0x7f42180fe2d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.176+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 >> 172.21.15.94:0/1684832014 conn(0x7f42180f7480 msgr2=0x7f42180663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.177+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 shutdown_connections 2024-04-06T03:44:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.177+0000 7f421e6c9700 1 -- 172.21.15.94:0/1684832014 wait complete. 2024-04-06T03:44:14.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.284+0000 7fbf0b714700 1 -- 172.21.15.94:0/4046176090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 msgr2=0x7fbf040fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.284+0000 7fbf0b714700 1 --2- 172.21.15.94:0/4046176090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 0x7fbf040fd3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbef4004940 tx=0x7fbef4009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:14.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.284+0000 7fbf0b714700 1 -- 172.21.15.94:0/4046176090 shutdown_connections 2024-04-06T03:44:14.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.284+0000 7fbf0b714700 1 --2- 172.21.15.94:0/4046176090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 0x7fbf040fd3f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.284+0000 7fbf0b714700 1 --2- 172.21.15.94:0/4046176090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf040fbcd0 0x7fbf040fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.284+0000 7fbf0b714700 1 -- 172.21.15.94:0/4046176090 >> 172.21.15.94:0/4046176090 conn(0x7fbf040f7450 msgr2=0x7fbf040f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:14.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.284+0000 7fbf0b714700 1 -- 172.21.15.94:0/4046176090 shutdown_connections 2024-04-06T03:44:14.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf0b714700 1 -- 172.21.15.94:0/4046176090 wait complete. 2024-04-06T03:44:14.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf0b714700 1 Processor -- start 2024-04-06T03:44:14.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf0b714700 1 -- start start 2024-04-06T03:44:14.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf0b714700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf040fbcd0 0x7fbf040ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:14.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf0b714700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 0x7fbf040fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf0b714700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf041001c0 con 0x7fbf040fcf80 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf0b714700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040fe800 con 0x7fbf040fbcd0 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf08caf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 0x7fbf040fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf08caf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 0x7fbf040fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33466/0 (socket says 172.21.15.94:33466) 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf08caf700 1 -- 172.21.15.94:0/2356749881 learned_addr learned my addr 172.21.15.94:0/2356749881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.285+0000 7fbf094b0700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf040fbcd0 0x7fbf040ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.286+0000 7fbf08caf700 1 -- 172.21.15.94:0/2356749881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf040fbcd0 msgr2=0x7fbf040ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.286+0000 7fbf08caf700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf040fbcd0 0x7fbf040ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.286+0000 7fbf08caf700 1 -- 172.21.15.94:0/2356749881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef4005040 con 0x7fbf040fcf80 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.286+0000 7fbf08caf700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 0x7fbf040fe2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbef400b3b0 tx=0x7fbef4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.286+0000 7fbef9ffb700 1 -- 172.21.15.94:0/2356749881 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbef400e040 con 0x7fbf040fcf80 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.286+0000 7fbef9ffb700 1 -- 172.21.15.94:0/2356749881 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbef400bcc0 con 0x7fbf040fcf80 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.286+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf040fea80 con 0x7fbf040fcf80 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.286+0000 7fbef9ffb700 1 -- 172.21.15.94:0/2356749881 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbef401c630 con 0x7fbf040fcf80 2024-04-06T03:44:14.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.286+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf040fefc0 con 0x7fbf040fcf80 2024-04-06T03:44:14.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.287+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf04101b10 con 0x7fbf040fcf80 2024-04-06T03:44:14.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.288+0000 7fbef9ffb700 1 -- 172.21.15.94:0/2356749881 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7fbef4026440 con 0x7fbf040fcf80 2024-04-06T03:44:14.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.288+0000 7fbef9ffb700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbef006ddd0 0x7fbef0070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:14.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.288+0000 7fbef9ffb700 1 -- 172.21.15.94:0/2356749881 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbef408e450 con 0x7fbf040fcf80 2024-04-06T03:44:14.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.288+0000 7fbf094b0700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbef006ddd0 0x7fbef0070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:14.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.289+0000 7fbf094b0700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbef006ddd0 0x7fbef0070280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbf00004990 tx=0x7fbf00009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:14.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.293+0000 7fbef9ffb700 1 -- 172.21.15.94:0/2356749881 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbef405b390 con 0x7fbf040fcf80 2024-04-06T03:44:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.548+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf0404c710 con 0x7fbf040fcf80 2024-04-06T03:44:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.548+0000 7fbef9ffb700 1 -- 172.21.15.94:0/2356749881 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fbef405af20 con 0x7fbf040fcf80 2024-04-06T03:44:14.552 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:44:14.552 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:44:14.552 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:44:14.553 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:44:14.554 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:44:14.555 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:44:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbef006ddd0 msgr2=0x7fbef0070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbef006ddd0 0x7fbef0070280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbf00004990 tx=0x7fbf00009ee0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 msgr2=0x7fbf040fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 0x7fbf040fe2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbef400b3b0 tx=0x7fbef4009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 shutdown_connections 2024-04-06T03:44:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbef006ddd0 0x7fbef0070280 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf040fbcd0 0x7fbf040ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 --2- 172.21.15.94:0/2356749881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf040fcf80 0x7fbf040fe2c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 >> 172.21.15.94:0/2356749881 conn(0x7fbf040f7450 msgr2=0x7fbf040667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 shutdown_connections 2024-04-06T03:44:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.551+0000 7fbf0b714700 1 -- 172.21.15.94:0/2356749881 wait complete. 2024-04-06T03:44:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:44:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.658+0000 7f49d1a6e700 1 -- 172.21.15.94:0/4248096114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 msgr2=0x7f49cc0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.658+0000 7f49d1a6e700 1 --2- 172.21.15.94:0/4248096114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 0x7f49cc0fc0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f49bc004940 tx=0x7f49bc009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.658+0000 7f49d1a6e700 1 -- 172.21.15.94:0/4248096114 shutdown_connections 2024-04-06T03:44:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.658+0000 7f49d1a6e700 1 --2- 172.21.15.94:0/4248096114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc0fcf80 0x7f49cc0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.658+0000 7f49d1a6e700 1 --2- 172.21.15.94:0/4248096114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 0x7f49cc0fc0e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.658+0000 7f49d1a6e700 1 -- 172.21.15.94:0/4248096114 >> 172.21.15.94:0/4248096114 conn(0x7f49cc0f7430 msgr2=0x7f49cc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.658+0000 7f49d1a6e700 1 -- 172.21.15.94:0/4248096114 shutdown_connections 2024-04-06T03:44:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.658+0000 7f49d1a6e700 1 -- 172.21.15.94:0/4248096114 wait complete. 2024-04-06T03:44:14.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49d1a6e700 1 Processor -- start 2024-04-06T03:44:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49d1a6e700 1 -- start start 2024-04-06T03:44:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49d1a6e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 0x7f49cc0661d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49d1a6e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc0fcf80 0x7f49cc112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49d1a6e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc112d60 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49d1a6e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc112ed0 con 0x7f49cc0fcf80 2024-04-06T03:44:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 0x7f49cc0661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49cb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 0x7f49cc0661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33492/0 (socket says 172.21.15.94:33492) 2024-04-06T03:44:14.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49cb7fe700 1 -- 172.21.15.94:0/1337409993 learned_addr learned my addr 172.21.15.94:0/1337409993 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49caffd700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc0fcf80 0x7f49cc112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49cb7fe700 1 -- 172.21.15.94:0/1337409993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc0fcf80 msgr2=0x7f49cc112730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49cb7fe700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc0fcf80 0x7f49cc112730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49cb7fe700 1 -- 172.21.15.94:0/1337409993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc005040 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.659+0000 7f49caffd700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc0fcf80 0x7f49cc112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.660+0000 7f49cb7fe700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 0x7f49cc0661d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00b3b0 tx=0x7f49bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.660+0000 7f49b3fff700 1 -- 172.21.15.94:0/1337409993 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49bc00e040 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.660+0000 7f49b3fff700 1 -- 172.21.15.94:0/1337409993 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49bc00bcc0 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.660+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc113150 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.660+0000 7f49b3fff700 1 -- 172.21.15.94:0/1337409993 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49bc01c630 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.660+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc113710 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.661+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc05f5f0 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.661+0000 7f49b3fff700 1 -- 172.21.15.94:0/1337409993 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f49bc013980 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.662+0000 7f49b3fff700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f49b406ddc0 0x7f49b4070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:14.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.662+0000 7f49caffd700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f49b406ddc0 0x7f49b4070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:14.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.662+0000 7f49b3fff700 1 -- 172.21.15.94:0/1337409993 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f49bc08e460 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.662+0000 7f49caffd700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f49b406ddc0 0x7f49b4070270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009fd0 tx=0x7f49c000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:14.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.666+0000 7f49b3fff700 1 -- 172.21.15.94:0/1337409993 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f49bc057870 con 0x7f49cc0fbcd0 2024-04-06T03:44:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.889+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49cc101290 con 0x7f49b406ddc0 2024-04-06T03:44:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.890+0000 7f49b3fff700 1 -- 172.21.15.94:0/1337409993 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f49cc101290 con 0x7f49b406ddc0 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:44:14.894 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "6/21 daemons upgraded", 2024-04-06T03:44:14.895 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading crash daemons", 2024-04-06T03:44:14.895 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:44:14.895 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:44:14.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f49b406ddc0 msgr2=0x7f49b4070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f49b406ddc0 0x7f49b4070270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009fd0 tx=0x7f49c000b040 comp rx=0 tx=0).stop 2024-04-06T03:44:14.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 msgr2=0x7f49cc0661d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:14.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 0x7f49cc0661d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00b3b0 tx=0x7f49bc009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:14.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 shutdown_connections 2024-04-06T03:44:14.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49cc0fbcd0 0x7f49cc0661d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f49b406ddc0 0x7f49b4070270 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 --2- 172.21.15.94:0/1337409993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc0fcf80 0x7f49cc112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 >> 172.21.15.94:0/1337409993 conn(0x7f49cc0f7430 msgr2=0x7f49cc0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 shutdown_connections 2024-04-06T03:44:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.893+0000 7f49d1a6e700 1 -- 172.21.15.94:0/1337409993 wait complete. 2024-04-06T03:44:15.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 -- 172.21.15.94:0/27216945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 msgr2=0x7f59080faf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:15.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 --2- 172.21.15.94:0/27216945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 0x7f59080faf80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f58f4004940 tx=0x7f58f4009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:15.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 -- 172.21.15.94:0/27216945 shutdown_connections 2024-04-06T03:44:15.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 --2- 172.21.15.94:0/27216945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 0x7f59080faf80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:15.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 --2- 172.21.15.94:0/27216945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59080fe4f0 0x7f59080fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:15.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 -- 172.21.15.94:0/27216945 >> 172.21.15.94:0/27216945 conn(0x7f59080f6bd0 msgr2=0x7f59080f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:15.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 -- 172.21.15.94:0/27216945 shutdown_connections 2024-04-06T03:44:15.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 -- 172.21.15.94:0/27216945 wait complete. 2024-04-06T03:44:15.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 Processor -- start 2024-04-06T03:44:15.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.996+0000 7f590ef94700 1 -- start start 2024-04-06T03:44:15.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f590ef94700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59080fe4f0 0x7f590810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:15.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f590ef94700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 0x7f590810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:15.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f590ef94700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590810e460 con 0x7f59080fee40 2024-04-06T03:44:15.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f590ef94700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590810e5d0 con 0x7f59080fe4f0 2024-04-06T03:44:15.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f5907fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 0x7f590810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:15.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f5907fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 0x7f590810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33520/0 (socket says 172.21.15.94:33520) 2024-04-06T03:44:15.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f5907fff700 1 -- 172.21.15.94:0/4213916049 learned_addr learned my addr 172.21.15.94:0/4213916049 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:15.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f590cd30700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59080fe4f0 0x7f590810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:15.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f5907fff700 1 -- 172.21.15.94:0/4213916049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59080fe4f0 msgr2=0x7f590810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:15.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f5907fff700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59080fe4f0 0x7f590810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:15.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f5907fff700 1 -- 172.21.15.94:0/4213916049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4005040 con 0x7f59080fee40 2024-04-06T03:44:15.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f5907fff700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 0x7f590810de60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009150 tx=0x7f58f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:15.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f59057fa700 1 -- 172.21.15.94:0/4213916049 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f400e040 con 0x7f59080fee40 2024-04-06T03:44:15.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f59057fa700 1 -- 172.21.15.94:0/4213916049 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58f400bbf0 con 0x7f59080fee40 2024-04-06T03:44:15.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5908112f60 con 0x7f59080fee40 2024-04-06T03:44:15.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f59057fa700 1 -- 172.21.15.94:0/4213916049 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f401c630 con 0x7f59080fee40 2024-04-06T03:44:15.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.997+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59081134f0 con 0x7f59080fee40 2024-04-06T03:44:15.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.998+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590804c710 con 0x7f59080fee40 2024-04-06T03:44:15.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.999+0000 7f59057fa700 1 -- 172.21.15.94:0/4213916049 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 38) v1 ==== 91961+0+0 (secure 0 0 0) 0x7f58f400bd60 con 0x7f59080fee40 2024-04-06T03:44:15.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.999+0000 7f59057fa700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f58f806ddd0 0x7f58f8070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:15.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.999+0000 7f59057fa700 1 -- 172.21.15.94:0/4213916049 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f58f408e920 con 0x7f59080fee40 2024-04-06T03:44:15.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:14.999+0000 7f590cd30700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f58f806ddd0 0x7f58f8070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:15.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.000+0000 7f590cd30700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f58f806ddd0 0x7f58f8070280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f58fc004f40 tx=0x7f58fc00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:15.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.004+0000 7f59057fa700 1 -- 172.21.15.94:0/4213916049 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f58f405b980 con 0x7f59080fee40 2024-04-06T03:44:15.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:14 smithi094 ceph-mon[94053]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:15.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:14 smithi094 ceph-mon[94053]: pgmap v39: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:15.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:14 smithi094 ceph-mon[94053]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:15.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:14 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:15.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:14 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:15.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:14 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1684832014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:15.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:14 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:15.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:14 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:15.170 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:14 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2356749881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:44:15.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.289+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5908105900 con 0x7f59080fee40 2024-04-06T03:44:15.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.290+0000 7f59057fa700 1 -- 172.21.15.94:0/4213916049 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+157 (secure 0 0 0) 0x7f58f4013e60 con 0x7f59080fee40 2024-04-06T03:44:15.293 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:44:15.293 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:44:15.293 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:44:15.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f58f806ddd0 msgr2=0x7f58f8070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:15.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f58f806ddd0 0x7f58f8070280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f58fc004f40 tx=0x7f58fc00b040 comp rx=0 tx=0).stop 2024-04-06T03:44:15.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 msgr2=0x7f590810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 0x7f590810de60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009150 tx=0x7f58f4009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 shutdown_connections 2024-04-06T03:44:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f58f806ddd0 0x7f58f8070280 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59080fe4f0 0x7f590810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 --2- 172.21.15.94:0/4213916049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59080fee40 0x7f590810de60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 >> 172.21.15.94:0/4213916049 conn(0x7f59080f6bd0 msgr2=0x7f59080f8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 shutdown_connections 2024-04-06T03:44:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:15.292+0000 7f590ef94700 1 -- 172.21.15.94:0/4213916049 wait complete. 2024-04-06T03:44:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:14 smithi136 ceph-mon[82061]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:15.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:14 smithi136 ceph-mon[82061]: pgmap v39: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:15.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:14 smithi136 ceph-mon[82061]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:15.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:14 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:15.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:14 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:15.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:14 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1684832014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:15.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:14 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:15.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:14 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:15.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:14 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2356749881' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:44:15.949 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:15 smithi094 ceph-mon[94053]: from='client.34204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:15.949 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:15 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/4213916049' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:44:16.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:15 smithi136 ceph-mon[82061]: from='client.34204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:16.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:15 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/4213916049' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:44:16.919 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:16 smithi094 ceph-mon[94053]: pgmap v40: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:17.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:16 smithi136 ceph-mon[82061]: pgmap v40: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:18.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:18.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:18.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:18.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:44:18.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: pgmap v41: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:18.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: pgmap v42: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:18.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:18.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:44:18.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:18.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:18.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:18.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all crash 2024-04-06T03:44:18.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:18.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi094"}]: dispatch 2024-04-06T03:44:18.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi094"}]': finished 2024-04-06T03:44:18.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi136"}]: dispatch 2024-04-06T03:44:18.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi136"}]': finished 2024-04-06T03:44:18.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-06T03:44:18.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-06T03:44:18.603 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:18 smithi136 ceph-mon[82061]: Upgrade: osd.0 is safe to restart 2024-04-06T03:44:18.662 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:18.662 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:18.662 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:18.662 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:44:18.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: pgmap v41: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:18.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: pgmap v42: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:18.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:18.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:44:18.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:18.663 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:18.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:18.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all crash 2024-04-06T03:44:18.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:18.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi094"}]: dispatch 2024-04-06T03:44:18.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi094"}]': finished 2024-04-06T03:44:18.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi136"}]: dispatch 2024-04-06T03:44:18.664 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi136"}]': finished 2024-04-06T03:44:18.665 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-06T03:44:18.665 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-06T03:44:18.665 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:18 smithi094 ceph-mon[94053]: Upgrade: osd.0 is safe to restart 2024-04-06T03:44:20.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:20 smithi094 ceph-mon[94053]: Upgrade: Updating osd.0 2024-04-06T03:44:20.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:20.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-06T03:44:20.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:20.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:20 smithi094 ceph-mon[94053]: Deploying daemon osd.0 on smithi094 2024-04-06T03:44:20.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:20 smithi136 ceph-mon[82061]: Upgrade: Updating osd.0 2024-04-06T03:44:20.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:20.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-06T03:44:20.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:20.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:20 smithi136 ceph-mon[82061]: Deploying daemon osd.0 on smithi094 2024-04-06T03:44:21.314 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:21 smithi094 ceph-mon[94053]: pgmap v43: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:21.314 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:44:21.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:21 smithi136 ceph-mon[82061]: pgmap v43: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:21.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:44:22.613 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:22 smithi094 systemd[1]: Stopping Ceph osd.0 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:44:22.918 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:22 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0[47717]: 2024-04-06T03:44:22.608+0000 7f28af315700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:44:22.919 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:22 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0[47717]: 2024-04-06T03:44:22.608+0000 7f28af315700 -1 osd.0 48 *** Got signal Terminated *** 2024-04-06T03:44:22.919 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:22 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0[47717]: 2024-04-06T03:44:22.608+0000 7f28af315700 -1 osd.0 48 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:44:23.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:23 smithi136 ceph-mon[82061]: pgmap v44: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:23.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:23 smithi136 ceph-mon[82061]: osd.0 marked itself down and dead 2024-04-06T03:44:23.370 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:23 smithi094 ceph-mon[94053]: pgmap v44: 65 pgs: 65 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:23.371 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:23 smithi094 ceph-mon[94053]: osd.0 marked itself down and dead 2024-04-06T03:44:23.918 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:23 smithi094 podman[103203]: 2024-04-06 03:44:23.712050582 +0000 UTC m=+1.228035682 container died ab97989ca46318d67a3d00238b80c73531651f23ae598da6f98442fe4f0c99cd (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=quincy-6699495) 2024-04-06T03:44:24.272 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:24 smithi094 ceph-mon[94053]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:44:24.273 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:24 smithi094 ceph-mon[94053]: osdmap e49: 6 total, 5 up, 6 in 2024-04-06T03:44:24.273 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:24 smithi094 podman[103203]: 2024-04-06 03:44:24.078130833 +0000 UTC m=+1.594115925 container cleanup ab97989ca46318d67a3d00238b80c73531651f23ae598da6f98442fe4f0c99cd (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, RELEASE=quincy-6699495, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-06T03:44:24.273 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:24 smithi094 bash[103203]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0 2024-04-06T03:44:24.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:24 smithi136 ceph-mon[82061]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:44:24.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:24 smithi136 ceph-mon[82061]: osdmap e49: 6 total, 5 up, 6 in 2024-04-06T03:44:24.894 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:24 smithi094 podman[103286]: 2024-04-06 03:44:24.538957607 +0000 UTC m=+0.929121940 container remove ab97989ca46318d67a3d00238b80c73531651f23ae598da6f98442fe4f0c99cd (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=quincy-6699495, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-06T03:44:25.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:25 smithi094 ceph-mon[94053]: pgmap v46: 65 pgs: 9 stale+active+clean, 56 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:25.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:25 smithi094 ceph-mon[94053]: osdmap e50: 6 total, 5 up, 6 in 2024-04-06T03:44:25.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:25 smithi136 ceph-mon[82061]: pgmap v46: 65 pgs: 9 stale+active+clean, 56 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:25.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:25 smithi136 ceph-mon[82061]: osdmap e50: 6 total, 5 up, 6 in 2024-04-06T03:44:25.740 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:25 smithi094 podman[103438]: 2024-04-06 03:44:25.383691271 +0000 UTC m=+0.018865085 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:44:25.741 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:25 smithi094 podman[103438]: 2024-04-06 03:44:25.493510062 +0000 UTC m=+0.128683866 container create 3b1ec57d0f8548f1f0c603b28a87e8e2219cfde3452d3e8ea8e3b22155b8548b (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9f19235) 2024-04-06T03:44:25.741 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:25 smithi094 podman[103438]: 2024-04-06 03:44:25.738323821 +0000 UTC m=+0.373497638 container init 3b1ec57d0f8548f1f0c603b28a87e8e2219cfde3452d3e8ea8e3b22155b8548b (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-06T03:44:26.032 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:25 smithi094 podman[103438]: 2024-04-06 03:44:25.74171293 +0000 UTC m=+0.376886736 container start 3b1ec57d0f8548f1f0c603b28a87e8e2219cfde3452d3e8ea8e3b22155b8548b (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-06T03:44:26.033 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:25 smithi094 podman[103438]: 2024-04-06 03:44:25.812627316 +0000 UTC m=+0.447801144 container attach 3b1ec57d0f8548f1f0c603b28a87e8e2219cfde3452d3e8ea8e3b22155b8548b (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-9f19235, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-06T03:44:26.335 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:26 smithi094 podman[103438]: 2024-04-06 03:44:26.030557326 +0000 UTC m=+0.665731135 container died 3b1ec57d0f8548f1f0c603b28a87e8e2219cfde3452d3e8ea8e3b22155b8548b (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-9f19235, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-06T03:44:26.598 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:26 smithi094 podman[103465]: 2024-04-06 03:44:26.574370694 +0000 UTC m=+0.536890754 container remove 3b1ec57d0f8548f1f0c603b28a87e8e2219cfde3452d3e8ea8e3b22155b8548b (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-06T03:44:26.867 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:26 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.0.service: Deactivated successfully. 2024-04-06T03:44:26.867 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:26 smithi094 systemd[1]: Stopped Ceph osd.0 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:44:26.867 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:26 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.0.service: Consumed 1min 4.259s CPU time. 2024-04-06T03:44:26.867 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:26 smithi094 systemd[1]: Starting Ceph osd.0 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:44:27.160 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:27 smithi094 ceph-mon[94053]: pgmap v48: 65 pgs: 9 stale+active+clean, 56 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:27.160 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 podman[103569]: 2024-04-06 03:44:26.918682238 +0000 UTC m=+0.017820860 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:44:27.160 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 podman[103569]: 2024-04-06 03:44:27.034429262 +0000 UTC m=+0.133567924 container create 624d676b2789eed540531cc8b9104e0d491cc9402d417171664fb1e924864fd1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, RELEASE=squid-9f19235, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-06T03:44:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:27 smithi136 ceph-mon[82061]: pgmap v48: 65 pgs: 9 stale+active+clean, 56 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:27.418 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 podman[103569]: 2024-04-06 03:44:27.286872596 +0000 UTC m=+0.386011225 container init 624d676b2789eed540531cc8b9104e0d491cc9402d417171664fb1e924864fd1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-06T03:44:27.419 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 podman[103569]: 2024-04-06 03:44:27.290416819 +0000 UTC m=+0.389555449 container start 624d676b2789eed540531cc8b9104e0d491cc9402d417171664fb1e924864fd1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-9f19235, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:44:27.419 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 podman[103569]: 2024-04-06 03:44:27.38664272 +0000 UTC m=+0.485781343 container attach 624d676b2789eed540531cc8b9104e0d491cc9402d417171664fb1e924864fd1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD) 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate[103584]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate[103584]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 bash[103569]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 bash[103569]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 bash[103569]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 bash[103569]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 bash[103569]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 bash[103569]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 bash[103569]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate[103584]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-06T03:44:28.051 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate[103584]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-06T03:44:28.052 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate[103584]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-06T03:44:28.052 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate[103584]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-06T03:44:28.052 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate[103584]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-06T03:44:28.052 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:27 smithi094 podman[103569]: 2024-04-06 03:44:27.750785291 +0000 UTC m=+0.849923910 container died 624d676b2789eed540531cc8b9104e0d491cc9402d417171664fb1e924864fd1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:44:28.351 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:28 smithi094 ceph-mon[94053]: Health check failed: Degraded data redundancy: 44/219 objects degraded (20.091%), 17 pgs degraded (PG_DEGRADED) 2024-04-06T03:44:28.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:28 smithi136 ceph-mon[82061]: Health check failed: Degraded data redundancy: 44/219 objects degraded (20.091%), 17 pgs degraded (PG_DEGRADED) 2024-04-06T03:44:28.602 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:28 smithi094 podman[103706]: 2024-04-06 03:44:28.348795721 +0000 UTC m=+0.593235893 container remove 624d676b2789eed540531cc8b9104e0d491cc9402d417171664fb1e924864fd1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-06T03:44:28.602 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:28 smithi094 podman[103769]: 2024-04-06 03:44:28.469182096 +0000 UTC m=+0.018272262 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:44:28.602 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:28 smithi094 podman[103769]: 2024-04-06 03:44:28.599892741 +0000 UTC m=+0.148982952 container create 38a0da79352f0159085a51d209f86b338f72ced672cf7a5ee5ef6cf18813bb2a (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-06T03:44:28.870 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:28 smithi094 podman[103769]: 2024-04-06 03:44:28.84429332 +0000 UTC m=+0.393383460 container init 38a0da79352f0159085a51d209f86b338f72ced672cf7a5ee5ef6cf18813bb2a (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-06T03:44:28.870 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:28 smithi094 podman[103769]: 2024-04-06 03:44:28.847440962 +0000 UTC m=+0.396531109 container start 38a0da79352f0159085a51d209f86b338f72ced672cf7a5ee5ef6cf18813bb2a (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-06T03:44:29.150 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:28 smithi094 ceph-osd[103788]: -- 172.21.15.94:0/2419253297 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a80a04b680 con 0x55a80a015000 2024-04-06T03:44:29.150 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:28 smithi094 bash[103769]: 38a0da79352f0159085a51d209f86b338f72ced672cf7a5ee5ef6cf18813bb2a 2024-04-06T03:44:29.150 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:28 smithi094 systemd[1]: Started Ceph osd.0 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:44:29.418 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:29 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0[103784]: 2024-04-06T03:44:29.390+0000 7f39480207c0 -1 Falling back to public interface 2024-04-06T03:44:29.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:29 smithi094 ceph-mon[94053]: pgmap v49: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:29.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:29 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:29.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:29 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:29.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:29 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:44:29.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:29 smithi136 ceph-mon[82061]: pgmap v49: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:29.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:29 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:29.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:29 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:29.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:29 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:44:31.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:31 smithi094 ceph-mon[94053]: pgmap v50: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:31.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:31 smithi136 ceph-mon[82061]: pgmap v50: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:33.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:33 smithi094 ceph-mon[94053]: pgmap v51: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:33 smithi136 ceph-mon[82061]: pgmap v51: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:34.842 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:34 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0[103784]: 2024-04-06T03:44:34.467+0000 7f39480207c0 -1 osd.0 48 log_to_monitors true 2024-04-06T03:44:35.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:34 smithi136 ceph-mon[82061]: pgmap v52: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:35.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:35.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:35.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:34 smithi136 ceph-mon[82061]: from='osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-06T03:44:35.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:34 smithi094 ceph-mon[94053]: pgmap v52: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:35.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:35.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:35.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:34 smithi094 ceph-mon[94053]: from='osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-06T03:44:35.987 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:44:35 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0[103784]: 2024-04-06T03:44:35.876+0000 7f3935470700 -1 osd.0 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:44:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:35 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:35 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:35 smithi136 ceph-mon[82061]: from='osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-06T03:44:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:35 smithi136 ceph-mon[82061]: osdmap e51: 6 total, 5 up, 6 in 2024-04-06T03:44:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:35 smithi136 ceph-mon[82061]: from='osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:44:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:35 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:35 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:44:36.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:35 smithi136 ceph-mon[82061]: from='osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004]' entity='osd.0' 2024-04-06T03:44:36.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:35 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:36.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:35 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:36.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:35 smithi094 ceph-mon[94053]: from='osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-06T03:44:36.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:35 smithi094 ceph-mon[94053]: osdmap e51: 6 total, 5 up, 6 in 2024-04-06T03:44:36.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:35 smithi094 ceph-mon[94053]: from='osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:44:36.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:35 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:36.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:35 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:44:36.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:35 smithi094 ceph-mon[94053]: from='osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004]' entity='osd.0' 2024-04-06T03:44:37.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:36 smithi094 ceph-mon[94053]: pgmap v54: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:37.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:36 smithi094 ceph-mon[94053]: mgrmap e39: smithi094.purlzp(active, since 91s), standbys: smithi136.jagvwt 2024-04-06T03:44:37.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:36 smithi094 ceph-mon[94053]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:44:37.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:36 smithi094 ceph-mon[94053]: osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004] boot 2024-04-06T03:44:37.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:36 smithi094 ceph-mon[94053]: osdmap e52: 6 total, 6 up, 6 in 2024-04-06T03:44:37.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:44:37.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:36 smithi136 ceph-mon[82061]: pgmap v54: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 44/219 objects degraded (20.091%) 2024-04-06T03:44:37.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:36 smithi136 ceph-mon[82061]: mgrmap e39: smithi094.purlzp(active, since 91s), standbys: smithi136.jagvwt 2024-04-06T03:44:37.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:36 smithi136 ceph-mon[82061]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:44:37.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:36 smithi136 ceph-mon[82061]: osd.0 [v2:172.21.15.94:6802/95929004,v1:172.21.15.94:6803/95929004] boot 2024-04-06T03:44:37.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:36 smithi136 ceph-mon[82061]: osdmap e52: 6 total, 6 up, 6 in 2024-04-06T03:44:37.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-06T03:44:39.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:38 smithi094 ceph-mon[94053]: pgmap v56: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 44/219 objects degraded (20.091%) 2024-04-06T03:44:39.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:38 smithi094 ceph-mon[94053]: osdmap e53: 6 total, 6 up, 6 in 2024-04-06T03:44:39.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:38 smithi136 ceph-mon[82061]: pgmap v56: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 44/219 objects degraded (20.091%) 2024-04-06T03:44:39.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:38 smithi136 ceph-mon[82061]: osdmap e53: 6 total, 6 up, 6 in 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: pgmap v58: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 531 B/s wr, 0 op/s; 44/219 objects degraded (20.091%) 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: pgmap v59: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 773 B/s wr, 0 op/s; 44/219 objects degraded (20.091%) 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:40.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:40.420 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-06T03:44:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:44:40.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: pgmap v58: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 531 B/s wr, 0 op/s; 44/219 objects degraded (20.091%) 2024-04-06T03:44:40.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: pgmap v59: 65 pgs: 17 active+undersized+degraded, 17 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 773 B/s wr, 0 op/s; 44/219 objects degraded (20.091%) 2024-04-06T03:44:40.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:40.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:44:40.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:40.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:40.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:40.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-06T03:44:41.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:41 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-06T03:44:41.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:41 smithi094 ceph-mon[94053]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-06T03:44:41.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:41 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-06T03:44:41.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:41 smithi136 ceph-mon[82061]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-06T03:44:42.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:42 smithi094 ceph-mon[94053]: pgmap v60: 65 pgs: 15 active+undersized+degraded, 10 active+undersized, 40 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 531 B/s wr, 0 op/s; 39/219 objects degraded (17.808%) 2024-04-06T03:44:42.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:42 smithi136 ceph-mon[82061]: pgmap v60: 65 pgs: 15 active+undersized+degraded, 10 active+undersized, 40 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 531 B/s wr, 0 op/s; 39/219 objects degraded (17.808%) 2024-04-06T03:44:43.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:43 smithi094 ceph-mon[94053]: Health check update: Degraded data redundancy: 39/219 objects degraded (17.808%), 15 pgs degraded (PG_DEGRADED) 2024-04-06T03:44:43.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:43 smithi136 ceph-mon[82061]: Health check update: Degraded data redundancy: 39/219 objects degraded (17.808%), 15 pgs degraded (PG_DEGRADED) 2024-04-06T03:44:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:44 smithi094 ceph-mon[94053]: pgmap v61: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:44.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:44 smithi094 ceph-mon[94053]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 39/219 objects degraded (17.808%), 15 pgs degraded) 2024-04-06T03:44:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:44 smithi136 ceph-mon[82061]: pgmap v61: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:44.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:44 smithi136 ceph-mon[82061]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 39/219 objects degraded (17.808%), 15 pgs degraded) 2024-04-06T03:44:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.407+0000 7f6c3de5c700 1 -- 172.21.15.94:0/1161061290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 msgr2=0x7f6c380fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.407+0000 7f6c3de5c700 1 --2- 172.21.15.94:0/1161061290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 0x7f6c380fd250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009ce0 tx=0x7f6c30004d10 comp rx=0 tx=0).stop 2024-04-06T03:44:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.407+0000 7f6c3de5c700 1 -- 172.21.15.94:0/1161061290 shutdown_connections 2024-04-06T03:44:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.407+0000 7f6c3de5c700 1 --2- 172.21.15.94:0/1161061290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c380fe040 0x7f6c380fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.407+0000 7f6c3de5c700 1 --2- 172.21.15.94:0/1161061290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 0x7f6c380fd250 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.407+0000 7f6c3de5c700 1 -- 172.21.15.94:0/1161061290 >> 172.21.15.94:0/1161061290 conn(0x7f6c380f8510 msgr2=0x7f6c380fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.407+0000 7f6c3de5c700 1 -- 172.21.15.94:0/1161061290 shutdown_connections 2024-04-06T03:44:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.408+0000 7f6c3de5c700 1 -- 172.21.15.94:0/1161061290 wait complete. 2024-04-06T03:44:45.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.408+0000 7f6c3de5c700 1 Processor -- start 2024-04-06T03:44:45.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.408+0000 7f6c3de5c700 1 -- start start 2024-04-06T03:44:45.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c3de5c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 0x7f6c3810dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:45.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c3de5c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c380fe040 0x7f6c3810e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:45.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c3de5c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3810e710 con 0x7f6c380fce40 2024-04-06T03:44:45.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c3de5c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3810ef60 con 0x7f6c380fe040 2024-04-06T03:44:45.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c377fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 0x7f6c3810dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c377fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 0x7f6c3810dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58414/0 (socket says 172.21.15.94:58414) 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c377fe700 1 -- 172.21.15.94:0/3880443623 learned_addr learned my addr 172.21.15.94:0/3880443623 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c36ffd700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c380fe040 0x7f6c3810e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c377fe700 1 -- 172.21.15.94:0/3880443623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c380fe040 msgr2=0x7f6c3810e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c377fe700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c380fe040 0x7f6c3810e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c377fe700 1 -- 172.21.15.94:0/3880443623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30009940 con 0x7f6c380fce40 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c36ffd700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c380fe040 0x7f6c3810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c377fe700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 0x7f6c3810dbd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009910 tx=0x7f6c30004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c2ffff700 1 -- 172.21.15.94:0/3880443623 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c3000e040 con 0x7f6c380fce40 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3810f1e0 con 0x7f6c380fce40 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c2ffff700 1 -- 172.21.15.94:0/3880443623 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c30008ed0 con 0x7f6c380fce40 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.409+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3810f720 con 0x7f6c380fce40 2024-04-06T03:44:45.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.410+0000 7f6c2ffff700 1 -- 172.21.15.94:0/3880443623 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c3001c800 con 0x7f6c380fce40 2024-04-06T03:44:45.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.411+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3805f5f0 con 0x7f6c380fce40 2024-04-06T03:44:45.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.411+0000 7f6c2ffff700 1 -- 172.21.15.94:0/3880443623 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f6c30012410 con 0x7f6c380fce40 2024-04-06T03:44:45.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.411+0000 7f6c2ffff700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6c2406dcb0 0x7f6c24070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:45.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.412+0000 7f6c2ffff700 1 -- 172.21.15.94:0/3880443623 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6c3008f0d0 con 0x7f6c380fce40 2024-04-06T03:44:45.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.412+0000 7f6c36ffd700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6c2406dcb0 0x7f6c24070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:45.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.412+0000 7f6c36ffd700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6c2406dcb0 0x7f6c24070160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c20007fd0 tx=0x7f6c20009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:45.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.416+0000 7f6c2ffff700 1 -- 172.21.15.94:0/3880443623 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6c3005c2b0 con 0x7f6c380fce40 2024-04-06T03:44:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.629+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c3810fb00 con 0x7f6c2406dcb0 2024-04-06T03:44:45.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.630+0000 7f6c2ffff700 1 -- 172.21.15.94:0/3880443623 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6c3810fb00 con 0x7f6c2406dcb0 2024-04-06T03:44:45.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.632+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6c2406dcb0 msgr2=0x7f6c24070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:45.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.632+0000 7f6c3de5c700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6c2406dcb0 0x7f6c24070160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c20007fd0 tx=0x7f6c20009040 comp rx=0 tx=0).stop 2024-04-06T03:44:45.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.632+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 msgr2=0x7f6c3810dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:45.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.632+0000 7f6c3de5c700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 0x7f6c3810dbd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009910 tx=0x7f6c30004d10 comp rx=0 tx=0).stop 2024-04-06T03:44:45.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.632+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 shutdown_connections 2024-04-06T03:44:45.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.632+0000 7f6c3de5c700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c380fce40 0x7f6c3810dbd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.632+0000 7f6c3de5c700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6c2406dcb0 0x7f6c24070160 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.632+0000 7f6c3de5c700 1 --2- 172.21.15.94:0/3880443623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c380fe040 0x7f6c3810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.632+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 >> 172.21.15.94:0/3880443623 conn(0x7f6c380f8510 msgr2=0x7f6c380f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:45.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.633+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 shutdown_connections 2024-04-06T03:44:45.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.633+0000 7f6c3de5c700 1 -- 172.21.15.94:0/3880443623 wait complete. 2024-04-06T03:44:45.646 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:44:45.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.736+0000 7f15a5d58700 1 -- 172.21.15.94:0/933911651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a0071ea0 msgr2=0x7f15a00703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.736+0000 7f15a5d58700 1 --2- 172.21.15.94:0/933911651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a0071ea0 0x7f15a00703b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1590004940 tx=0x7f1590009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.736+0000 7f15a5d58700 1 -- 172.21.15.94:0/933911651 shutdown_connections 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.736+0000 7f15a5d58700 1 --2- 172.21.15.94:0/933911651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15a00708f0 0x7f15a0070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.736+0000 7f15a5d58700 1 --2- 172.21.15.94:0/933911651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a0071ea0 0x7f15a00703b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.736+0000 7f15a5d58700 1 -- 172.21.15.94:0/933911651 >> 172.21.15.94:0/933911651 conn(0x7f15a00f8550 msgr2=0x7f15a00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.736+0000 7f15a5d58700 1 -- 172.21.15.94:0/933911651 shutdown_connections 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.736+0000 7f15a5d58700 1 -- 172.21.15.94:0/933911651 wait complete. 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.737+0000 7f15a5d58700 1 Processor -- start 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.737+0000 7f15a5d58700 1 -- start start 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.737+0000 7f15a5d58700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a00708f0 0x7f15a0113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.737+0000 7f15a5d58700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15a0071ea0 0x7f15a01144c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.737+0000 7f15a5d58700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0114ac0 con 0x7f15a00708f0 2024-04-06T03:44:45.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.737+0000 7f15a5d58700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0114c30 con 0x7f15a0071ea0 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.737+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a00708f0 0x7f15a0113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.737+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a00708f0 0x7f15a0113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58424/0 (socket says 172.21.15.94:58424) 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.737+0000 7f159f7fe700 1 -- 172.21.15.94:0/2957248022 learned_addr learned my addr 172.21.15.94:0/2957248022 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f159effd700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15a0071ea0 0x7f15a01144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f159f7fe700 1 -- 172.21.15.94:0/2957248022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15a0071ea0 msgr2=0x7f15a01144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f159f7fe700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15a0071ea0 0x7f15a01144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f159f7fe700 1 -- 172.21.15.94:0/2957248022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590005040 con 0x7f15a00708f0 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f159f7fe700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a00708f0 0x7f15a0113f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f159000b3b0 tx=0x7f1590009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f1587fff700 1 -- 172.21.15.94:0/2957248022 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f159000e040 con 0x7f15a00708f0 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f1587fff700 1 -- 172.21.15.94:0/2957248022 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f159000bcc0 con 0x7f15a00708f0 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a0119630 con 0x7f15a00708f0 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f1587fff700 1 -- 172.21.15.94:0/2957248022 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f159001c630 con 0x7f15a00708f0 2024-04-06T03:44:45.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.738+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a0119bf0 con 0x7f15a00708f0 2024-04-06T03:44:45.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.739+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a004c710 con 0x7f15a00708f0 2024-04-06T03:44:45.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.740+0000 7f1587fff700 1 -- 172.21.15.94:0/2957248022 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f15900139f0 con 0x7f15a00708f0 2024-04-06T03:44:45.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.740+0000 7f1587fff700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f158806dca0 0x7f1588070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:45.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.740+0000 7f1587fff700 1 -- 172.21.15.94:0/2957248022 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f159008e1b0 con 0x7f15a00708f0 2024-04-06T03:44:45.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.740+0000 7f159effd700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f158806dca0 0x7f1588070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:45.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.740+0000 7f159effd700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f158806dca0 0x7f1588070150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1594004d60 tx=0x7f159400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:45.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.744+0000 7f1587fff700 1 -- 172.21.15.94:0/2957248022 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f159005b390 con 0x7f15a00708f0 2024-04-06T03:44:45.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.963+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15a00ffa90 con 0x7f158806dca0 2024-04-06T03:44:45.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.964+0000 7f1587fff700 1 -- 172.21.15.94:0/2957248022 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f15a00ffa90 con 0x7f158806dca0 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f158806dca0 msgr2=0x7f1588070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f158806dca0 0x7f1588070150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1594004d60 tx=0x7f159400b040 comp rx=0 tx=0).stop 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a00708f0 msgr2=0x7f15a0113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a00708f0 0x7f15a0113f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f159000b3b0 tx=0x7f1590009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 shutdown_connections 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15a00708f0 0x7f15a0113f80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f158806dca0 0x7f1588070150 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 --2- 172.21.15.94:0/2957248022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15a0071ea0 0x7f15a01144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 >> 172.21.15.94:0/2957248022 conn(0x7f15a00f8550 msgr2=0x7f15a00f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 shutdown_connections 2024-04-06T03:44:45.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:45.966+0000 7f15a5d58700 1 -- 172.21.15.94:0/2957248022 wait complete. 2024-04-06T03:44:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.066+0000 7fbb22879700 1 -- 172.21.15.94:0/2267035714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fcef0 msgr2=0x7fbb1c0fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.066+0000 7fbb22879700 1 --2- 172.21.15.94:0/2267035714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fcef0 0x7fbb1c0fd300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c004940 tx=0x7fbb0c009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.066+0000 7fbb22879700 1 -- 172.21.15.94:0/2267035714 shutdown_connections 2024-04-06T03:44:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.066+0000 7fbb22879700 1 --2- 172.21.15.94:0/2267035714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c0fe060 0x7fbb1c0fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.066+0000 7fbb22879700 1 --2- 172.21.15.94:0/2267035714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fcef0 0x7fbb1c0fd300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.066+0000 7fbb22879700 1 -- 172.21.15.94:0/2267035714 >> 172.21.15.94:0/2267035714 conn(0x7fbb1c0f85b0 msgr2=0x7fbb1c0fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.066+0000 7fbb22879700 1 -- 172.21.15.94:0/2267035714 shutdown_connections 2024-04-06T03:44:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.067+0000 7fbb22879700 1 -- 172.21.15.94:0/2267035714 wait complete. 2024-04-06T03:44:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.067+0000 7fbb22879700 1 Processor -- start 2024-04-06T03:44:46.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.067+0000 7fbb22879700 1 -- start start 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.067+0000 7fbb22879700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c0fcef0 0x7fbb1c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.067+0000 7fbb22879700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fe060 0x7fbb1c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.067+0000 7fbb22879700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c070950 con 0x7fbb1c0fe060 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.067+0000 7fbb22879700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c070ac0 con 0x7fbb1c0fcef0 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb1b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fe060 0x7fbb1c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb1b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fe060 0x7fbb1c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58434/0 (socket says 172.21.15.94:58434) 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb1b7fe700 1 -- 172.21.15.94:0/1226181533 learned_addr learned my addr 172.21.15.94:0/1226181533 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb1bfff700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c0fcef0 0x7fbb1c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb1b7fe700 1 -- 172.21.15.94:0/1226181533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c0fcef0 msgr2=0x7fbb1c071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb1b7fe700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c0fcef0 0x7fbb1c071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb1b7fe700 1 -- 172.21.15.94:0/1226181533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c005040 con 0x7fbb1c0fe060 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb1b7fe700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fe060 0x7fbb1c070310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000b790 tx=0x7fbb10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb18ff9700 1 -- 172.21.15.94:0/1226181533 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb1000f6f0 con 0x7fbb1c0fe060 2024-04-06T03:44:46.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c070da0 con 0x7fbb1c0fe060 2024-04-06T03:44:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb18ff9700 1 -- 172.21.15.94:0/1226181533 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb1000fd20 con 0x7fbb1c0fe060 2024-04-06T03:44:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb18ff9700 1 -- 172.21.15.94:0/1226181533 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb100103a0 con 0x7fbb1c0fe060 2024-04-06T03:44:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.068+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c115ab0 con 0x7fbb1c0fe060 2024-04-06T03:44:46.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.069+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c05f5f0 con 0x7fbb1c0fe060 2024-04-06T03:44:46.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.070+0000 7fbb18ff9700 1 -- 172.21.15.94:0/1226181533 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fbb10011400 con 0x7fbb1c0fe060 2024-04-06T03:44:46.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.070+0000 7fbb18ff9700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbb0406dca0 0x7fbb04070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:46.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.071+0000 7fbb18ff9700 1 -- 172.21.15.94:0/1226181533 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbb1008d2d0 con 0x7fbb1c0fe060 2024-04-06T03:44:46.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.071+0000 7fbb1bfff700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbb0406dca0 0x7fbb04070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:46.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.071+0000 7fbb1bfff700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbb0406dca0 0x7fbb04070150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c004890 tx=0x7fbb0c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:46.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.075+0000 7fbb18ff9700 1 -- 172.21.15.94:0/1226181533 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbb10056980 con 0x7fbb1c0fe060 2024-04-06T03:44:46.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.280+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb1c102380 con 0x7fbb0406dca0 2024-04-06T03:44:46.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.289+0000 7fbb18ff9700 1 -- 172.21.15.94:0/1226181533 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fbb1c102380 con 0x7fbb0406dca0 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (3m) 12s ago 12m 27.6M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (44s) 12s ago 12m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (39s) 36s ago 11m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 12s ago 11m - - 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (6m) 12s ago 6m 14.7M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (6m) 12s ago 6m 153M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (6m) 36s ago 6m 14.6M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (6m) 36s ago 6m 17.0M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:44:46.293 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (2m) 12s ago 13m 500M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (2m) 36s ago 11m 449M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (109s) 12s ago 13m 42.8M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (87s) 36s ago 11m 36.7M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (4m) 12s ago 12m 15.5M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (3m) 36s ago 11m 17.3M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (17s) 12s ago 10m 12.5M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (9m) 12s ago 9m 264M 1557M 17.2.7-819-g6699495c ec300eaa768b d6034d5d9a66 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (9m) 12s ago 9m 226M 1557M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (9m) 36s ago 9m 292M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (8m) 36s ago 8m 257M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (8m) 36s ago 8m 241M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:44:46.294 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (2m) 12s ago 11m 47.5M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:44:46.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.291+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbb0406dca0 msgr2=0x7fbb04070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.291+0000 7fbb22879700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbb0406dca0 0x7fbb04070150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c004890 tx=0x7fbb0c009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:46.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.292+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fe060 msgr2=0x7fbb1c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.292+0000 7fbb22879700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fe060 0x7fbb1c070310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000b790 tx=0x7fbb10009250 comp rx=0 tx=0).stop 2024-04-06T03:44:46.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.292+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 shutdown_connections 2024-04-06T03:44:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.292+0000 7fbb22879700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbb0406dca0 0x7fbb04070150 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.292+0000 7fbb22879700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c0fcef0 0x7fbb1c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.292+0000 7fbb22879700 1 --2- 172.21.15.94:0/1226181533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb1c0fe060 0x7fbb1c070310 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.292+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 >> 172.21.15.94:0/1226181533 conn(0x7fbb1c0f85b0 msgr2=0x7fbb1c0f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.292+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 shutdown_connections 2024-04-06T03:44:46.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.292+0000 7fbb22879700 1 -- 172.21.15.94:0/1226181533 wait complete. 2024-04-06T03:44:46.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.390+0000 7f146a46a700 1 -- 172.21.15.94:0/2432226740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 msgr2=0x7f14640fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.390+0000 7f146a46a700 1 --2- 172.21.15.94:0/2432226740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 0x7f14640fd7b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1454004940 tx=0x7f1454009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:46.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.390+0000 7f146a46a700 1 -- 172.21.15.94:0/2432226740 shutdown_connections 2024-04-06T03:44:46.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.390+0000 7f146a46a700 1 --2- 172.21.15.94:0/2432226740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14640fdcf0 0x7f1464100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.390+0000 7f146a46a700 1 --2- 172.21.15.94:0/2432226740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 0x7f14640fd7b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.390+0000 7f146a46a700 1 -- 172.21.15.94:0/2432226740 >> 172.21.15.94:0/2432226740 conn(0x7f14640f7430 msgr2=0x7f14640f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:46.394 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:46 smithi094 ceph-mon[94053]: pgmap v62: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:46.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.391+0000 7f146a46a700 1 -- 172.21.15.94:0/2432226740 shutdown_connections 2024-04-06T03:44:46.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.391+0000 7f146a46a700 1 -- 172.21.15.94:0/2432226740 wait complete. 2024-04-06T03:44:46.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.391+0000 7f146a46a700 1 Processor -- start 2024-04-06T03:44:46.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.391+0000 7f146a46a700 1 -- start start 2024-04-06T03:44:46.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.391+0000 7f146a46a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 0x7f1464113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:46.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.391+0000 7f146a46a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14640fdcf0 0x7f1464114500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:46.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.391+0000 7f146a46a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1464114b90 con 0x7f1464066f70 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.391+0000 7f146a46a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14641194e0 con 0x7f14640fdcf0 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1463fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 0x7f1464113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1463fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 0x7f1464113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58452/0 (socket says 172.21.15.94:58452) 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1463fff700 1 -- 172.21.15.94:0/2614382607 learned_addr learned my addr 172.21.15.94:0/2614382607 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f14637fe700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14640fdcf0 0x7f1464114500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1463fff700 1 -- 172.21.15.94:0/2614382607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14640fdcf0 msgr2=0x7f1464114500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1463fff700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14640fdcf0 0x7f1464114500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1463fff700 1 -- 172.21.15.94:0/2614382607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454005040 con 0x7f1464066f70 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1463fff700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 0x7f1464113fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1454000c00 tx=0x7f1454009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1460ff9700 1 -- 172.21.15.94:0/2614382607 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145400e040 con 0x7f1464066f70 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1460ff9700 1 -- 172.21.15.94:0/2614382607 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f145400bbf0 con 0x7f1464066f70 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f1460ff9700 1 -- 172.21.15.94:0/2614382607 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145401c630 con 0x7f1464066f70 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14641196e0 con 0x7f1464066f70 2024-04-06T03:44:46.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.392+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1464119c50 con 0x7f1464066f70 2024-04-06T03:44:46.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.394+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146404c710 con 0x7f1464066f70 2024-04-06T03:44:46.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.394+0000 7f1460ff9700 1 -- 172.21.15.94:0/2614382607 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f145400bd60 con 0x7f1464066f70 2024-04-06T03:44:46.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.394+0000 7f1460ff9700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f144c06dca0 0x7f144c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:46.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.394+0000 7f1460ff9700 1 -- 172.21.15.94:0/2614382607 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f145408e6b0 con 0x7f1464066f70 2024-04-06T03:44:46.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.394+0000 7f14637fe700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f144c06dca0 0x7f144c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:46.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.395+0000 7f14637fe700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f144c06dca0 0x7f144c070150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1458009bf0 tx=0x7f14580091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:46.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.399+0000 7f1460ff9700 1 -- 172.21.15.94:0/2614382607 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f145405b930 con 0x7f1464066f70 2024-04-06T03:44:46.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:46 smithi136 ceph-mon[82061]: pgmap v62: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:46.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.676+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f146411a000 con 0x7f1464066f70 2024-04-06T03:44:46.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.676+0000 7f1460ff9700 1 -- 172.21.15.94:0/2614382607 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f145405b4c0 con 0x7f1464066f70 2024-04-06T03:44:46.680 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:44:46.680 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:44:46.680 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:44:46.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:44:46.680 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:44:46.680 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 5, 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 1 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 9, 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 5 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:44:46.681 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:44:46.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f144c06dca0 msgr2=0x7f144c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f144c06dca0 0x7f144c070150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1458009bf0 tx=0x7f14580091a0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 msgr2=0x7f1464113fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 0x7f1464113fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1454000c00 tx=0x7f1454009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:46.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 shutdown_connections 2024-04-06T03:44:46.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1464066f70 0x7f1464113fc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f144c06dca0 0x7f144c070150 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 --2- 172.21.15.94:0/2614382607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14640fdcf0 0x7f1464114500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 >> 172.21.15.94:0/2614382607 conn(0x7f14640f7430 msgr2=0x7f14640f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:46.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 shutdown_connections 2024-04-06T03:44:46.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.678+0000 7f146a46a700 1 -- 172.21.15.94:0/2614382607 wait complete. 2024-04-06T03:44:46.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.779+0000 7f6b581d3700 1 -- 172.21.15.94:0/3911916144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fbcd0 msgr2=0x7f6b500fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.779+0000 7f6b581d3700 1 --2- 172.21.15.94:0/3911916144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fbcd0 0x7f6b500fc0e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b40004940 tx=0x7f6b40009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:46.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.779+0000 7f6b581d3700 1 -- 172.21.15.94:0/3911916144 shutdown_connections 2024-04-06T03:44:46.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.779+0000 7f6b581d3700 1 --2- 172.21.15.94:0/3911916144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b500fcf80 0x7f6b500fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.779+0000 7f6b581d3700 1 --2- 172.21.15.94:0/3911916144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fbcd0 0x7f6b500fc0e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.779+0000 7f6b581d3700 1 -- 172.21.15.94:0/3911916144 >> 172.21.15.94:0/3911916144 conn(0x7f6b500f7450 msgr2=0x7f6b500f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.779+0000 7f6b581d3700 1 -- 172.21.15.94:0/3911916144 shutdown_connections 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.779+0000 7f6b581d3700 1 -- 172.21.15.94:0/3911916144 wait complete. 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b581d3700 1 Processor -- start 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b581d3700 1 -- start start 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b581d3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b500fbcd0 0x7f6b50066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b581d3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fcf80 0x7f6b50112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b581d3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50112c70 con 0x7f6b500fcf80 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b581d3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50112db0 con 0x7f6b500fbcd0 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b5576e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fcf80 0x7f6b50112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b5576e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fcf80 0x7f6b50112730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58470/0 (socket says 172.21.15.94:58470) 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b5576e700 1 -- 172.21.15.94:0/2357582171 learned_addr learned my addr 172.21.15.94:0/2357582171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b55f6f700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b500fbcd0 0x7f6b50066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b5576e700 1 -- 172.21.15.94:0/2357582171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b500fbcd0 msgr2=0x7f6b50066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b5576e700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b500fbcd0 0x7f6b50066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:46.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.780+0000 7f6b5576e700 1 -- 172.21.15.94:0/2357582171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b40005040 con 0x7f6b500fcf80 2024-04-06T03:44:46.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.781+0000 7f6b5576e700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fcf80 0x7f6b50112730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c830 tx=0x7f6b4c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:46.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.781+0000 7f6b467fc700 1 -- 172.21.15.94:0/2357582171 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b4c00a760 con 0x7f6b500fcf80 2024-04-06T03:44:46.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.781+0000 7f6b467fc700 1 -- 172.21.15.94:0/2357582171 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b4c00ad90 con 0x7f6b500fcf80 2024-04-06T03:44:46.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.781+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b50113010 con 0x7f6b500fcf80 2024-04-06T03:44:46.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.781+0000 7f6b467fc700 1 -- 172.21.15.94:0/2357582171 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b4c0113a0 con 0x7f6b500fcf80 2024-04-06T03:44:46.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.781+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b501135a0 con 0x7f6b500fcf80 2024-04-06T03:44:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.782+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5004c710 con 0x7f6b500fcf80 2024-04-06T03:44:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.782+0000 7f6b467fc700 1 -- 172.21.15.94:0/2357582171 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f6b4c012400 con 0x7f6b500fcf80 2024-04-06T03:44:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.783+0000 7f6b467fc700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6b3c06dca0 0x7f6b3c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.783+0000 7f6b467fc700 1 -- 172.21.15.94:0/2357582171 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6b4c08d400 con 0x7f6b500fcf80 2024-04-06T03:44:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.783+0000 7f6b55f6f700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6b3c06dca0 0x7f6b3c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.783+0000 7f6b55f6f700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6b3c06dca0 0x7f6b3c070150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000b3b0 tx=0x7f6b40009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:46.787+0000 7f6b467fc700 1 -- 172.21.15.94:0/2357582171 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6b4c05a680 con 0x7f6b500fcf80 2024-04-06T03:44:47.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.031+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6b501047c0 con 0x7f6b500fcf80 2024-04-06T03:44:47.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.032+0000 7f6b467fc700 1 -- 172.21.15.94:0/2357582171 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f6b4c05a210 con 0x7f6b500fcf80 2024-04-06T03:44:47.035 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:44:47.035 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:44:47.035 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:44:47.035 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:44:47.035 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:44:47.036 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:44:47.037 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-04-06T03:44:47.038 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:44:47.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:44:47.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:44:47.038 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:44:47.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:44:47.038 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:44:47.038 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:44:47.038 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6b3c06dca0 msgr2=0x7f6b3c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6b3c06dca0 0x7f6b3c070150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000b3b0 tx=0x7f6b40009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fcf80 msgr2=0x7f6b50112730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fcf80 0x7f6b50112730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c830 tx=0x7f6b4c004470 comp rx=0 tx=0).stop 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 shutdown_connections 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6b3c06dca0 0x7f6b3c070150 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b500fbcd0 0x7f6b50066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 --2- 172.21.15.94:0/2357582171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b500fcf80 0x7f6b50112730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 >> 172.21.15.94:0/2357582171 conn(0x7f6b500f7450 msgr2=0x7f6b500f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 shutdown_connections 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.034+0000 7f6b581d3700 1 -- 172.21.15.94:0/2357582171 wait complete. 2024-04-06T03:44:47.039 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:44:47.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.134+0000 7f082ae2b700 1 -- 172.21.15.94:0/1306102887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fdfe0 msgr2=0x7f08240fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.134+0000 7f082ae2b700 1 --2- 172.21.15.94:0/1306102887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fdfe0 0x7f08240fe450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0810004940 tx=0x7f0810009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.134+0000 7f082ae2b700 1 -- 172.21.15.94:0/1306102887 shutdown_connections 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.134+0000 7f082ae2b700 1 --2- 172.21.15.94:0/1306102887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fdfe0 0x7f08240fe450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.134+0000 7f082ae2b700 1 --2- 172.21.15.94:0/1306102887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240fce70 0x7f08240fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.134+0000 7f082ae2b700 1 -- 172.21.15.94:0/1306102887 >> 172.21.15.94:0/1306102887 conn(0x7f08240f8550 msgr2=0x7f08240fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.134+0000 7f082ae2b700 1 -- 172.21.15.94:0/1306102887 shutdown_connections 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.135+0000 7f082ae2b700 1 -- 172.21.15.94:0/1306102887 wait complete. 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.135+0000 7f082ae2b700 1 Processor -- start 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.135+0000 7f082ae2b700 1 -- start start 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.135+0000 7f082ae2b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fce70 0x7f0824071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.135+0000 7f082ae2b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240fdfe0 0x7f0824070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.135+0000 7f082ae2b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0824070850 con 0x7f08240fce70 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.135+0000 7f082ae2b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08240709c0 con 0x7f08240fdfe0 2024-04-06T03:44:47.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f0828bc7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fce70 0x7f0824071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f0828bc7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fce70 0x7f0824071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58474/0 (socket says 172.21.15.94:58474) 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f0828bc7700 1 -- 172.21.15.94:0/1994215358 learned_addr learned my addr 172.21.15.94:0/1994215358 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f0823fff700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240fdfe0 0x7f0824070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f0828bc7700 1 -- 172.21.15.94:0/1994215358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240fdfe0 msgr2=0x7f0824070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f0828bc7700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240fdfe0 0x7f0824070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f0828bc7700 1 -- 172.21.15.94:0/1994215358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0810005040 con 0x7f08240fce70 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f0828bc7700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fce70 0x7f0824071c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f081800b790 tx=0x7f0818009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f08217fa700 1 -- 172.21.15.94:0/1994215358 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f081800f6f0 con 0x7f08240fce70 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f08217fa700 1 -- 172.21.15.94:0/1994215358 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f081800fd20 con 0x7f08240fce70 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0824070ca0 con 0x7f08240fce70 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f08217fa700 1 -- 172.21.15.94:0/1994215358 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08180103a0 con 0x7f08240fce70 2024-04-06T03:44:47.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.136+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0824115a20 con 0x7f08240fce70 2024-04-06T03:44:47.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.137+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082404c710 con 0x7f08240fce70 2024-04-06T03:44:47.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.138+0000 7f08217fa700 1 -- 172.21.15.94:0/1994215358 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f0818011400 con 0x7f08240fce70 2024-04-06T03:44:47.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.138+0000 7f08217fa700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f081406dca0 0x7f0814070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:47.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.138+0000 7f0823fff700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f081406dca0 0x7f0814070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:47.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.138+0000 7f08217fa700 1 -- 172.21.15.94:0/1994215358 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f081808d2d0 con 0x7f08240fce70 2024-04-06T03:44:47.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.139+0000 7f0823fff700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f081406dca0 0x7f0814070150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f081000b970 tx=0x7f0810009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.143+0000 7f08217fa700 1 -- 172.21.15.94:0/1994215358 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f081805a4b0 con 0x7f08240fce70 2024-04-06T03:44:47.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.360+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0824102280 con 0x7f081406dca0 2024-04-06T03:44:47.364 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:47 smithi094 ceph-mon[94053]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:47.364 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:47 smithi094 ceph-mon[94053]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:47.364 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:47 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2614382607' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:47.364 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:47 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2357582171' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.362+0000 7f08217fa700 1 -- 172.21.15.94:0/1994215358 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0824102280 con 0x7f081406dca0 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "7/21 daemons upgraded", 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:44:47.365 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:44:47.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f081406dca0 msgr2=0x7f0814070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f081406dca0 0x7f0814070150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f081000b970 tx=0x7f0810009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:47.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fce70 msgr2=0x7f0824071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fce70 0x7f0824071c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f081800b790 tx=0x7f0818009250 comp rx=0 tx=0).stop 2024-04-06T03:44:47.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 shutdown_connections 2024-04-06T03:44:47.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08240fce70 0x7f0824071c60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f081406dca0 0x7f0814070150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 --2- 172.21.15.94:0/1994215358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240fdfe0 0x7f0824070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 >> 172.21.15.94:0/1994215358 conn(0x7f08240f8550 msgr2=0x7f08240f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:47.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 shutdown_connections 2024-04-06T03:44:47.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.365+0000 7f082ae2b700 1 -- 172.21.15.94:0/1994215358 wait complete. 2024-04-06T03:44:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/441810941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 msgr2=0x7fcfd40fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 --2- 172.21.15.94:0/441810941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 0x7fcfd40fe460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8004940 tx=0x7fcfc8009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/441810941 shutdown_connections 2024-04-06T03:44:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 --2- 172.21.15.94:0/441810941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 0x7fcfd40fe460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 --2- 172.21.15.94:0/441810941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfd40fce80 0x7fcfd40fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/441810941 >> 172.21.15.94:0/441810941 conn(0x7fcfd40f8580 msgr2=0x7fcfd40fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/441810941 shutdown_connections 2024-04-06T03:44:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/441810941 wait complete. 2024-04-06T03:44:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 Processor -- start 2024-04-06T03:44:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.471+0000 7fcfd9ac4700 1 -- start start 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd9ac4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfd40fce80 0x7fcfd410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd9ac4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 0x7fcfd410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd9ac4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd410e550 con 0x7fcfd40fdff0 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd9ac4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd410eda0 con 0x7fcfd40fce80 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 0x7fcfd410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 0x7fcfd410df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58482/0 (socket says 172.21.15.94:58482) 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd2ffd700 1 -- 172.21.15.94:0/4010517192 learned_addr learned my addr 172.21.15.94:0/4010517192 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd37fe700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfd40fce80 0x7fcfd410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd2ffd700 1 -- 172.21.15.94:0/4010517192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfd40fce80 msgr2=0x7fcfd410da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd2ffd700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfd40fce80 0x7fcfd410da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd2ffd700 1 -- 172.21.15.94:0/4010517192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8005040 con 0x7fcfd40fdff0 2024-04-06T03:44:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfd2ffd700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 0x7fcfd410df50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009150 tx=0x7fcfc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfbbfff700 1 -- 172.21.15.94:0/4010517192 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc800e040 con 0x7fcfd40fdff0 2024-04-06T03:44:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfbbfff700 1 -- 172.21.15.94:0/4010517192 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfc800bcc0 con 0x7fcfd40fdff0 2024-04-06T03:44:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.472+0000 7fcfbbfff700 1 -- 172.21.15.94:0/4010517192 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc801c630 con 0x7fcfd40fdff0 2024-04-06T03:44:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.473+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd410f020 con 0x7fcfd40fdff0 2024-04-06T03:44:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.473+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd410f510 con 0x7fcfd40fdff0 2024-04-06T03:44:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.474+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd405f5f0 con 0x7fcfd40fdff0 2024-04-06T03:44:47.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.474+0000 7fcfbbfff700 1 -- 172.21.15.94:0/4010517192 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fcfc8013420 con 0x7fcfd40fdff0 2024-04-06T03:44:47.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.475+0000 7fcfbbfff700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fcfbc06dca0 0x7fcfbc070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:44:47.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.475+0000 7fcfbbfff700 1 -- 172.21.15.94:0/4010517192 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcfc808e180 con 0x7fcfd40fdff0 2024-04-06T03:44:47.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.475+0000 7fcfd37fe700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fcfbc06dca0 0x7fcfbc070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:44:47.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.475+0000 7fcfd37fe700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fcfbc06dca0 0x7fcfbc070150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4004990 tx=0x7fcfc4009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:44:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.479+0000 7fcfbbfff700 1 -- 172.21.15.94:0/4010517192 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcfc805b360 con 0x7fcfd40fdff0 2024-04-06T03:44:47.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:47 smithi136 ceph-mon[82061]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:47.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:47 smithi136 ceph-mon[82061]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:47.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:47 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2614382607' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:44:47.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:47 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2357582171' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:44:47.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.759+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcfd404bed0 con 0x7fcfd40fdff0 2024-04-06T03:44:47.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.759+0000 7fcfbbfff700 1 -- 172.21.15.94:0/4010517192 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+157 (secure 0 0 0) 0x7fcfc805aef0 con 0x7fcfd40fdff0 2024-04-06T03:44:47.762 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:44:47.762 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:44:47.763 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:44:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fcfbc06dca0 msgr2=0x7fcfbc070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fcfbc06dca0 0x7fcfbc070150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4004990 tx=0x7fcfc4009ee0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 msgr2=0x7fcfd410df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:44:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 0x7fcfd410df50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009150 tx=0x7fcfc8009e30 comp rx=0 tx=0).stop 2024-04-06T03:44:47.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 shutdown_connections 2024-04-06T03:44:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fcfbc06dca0 0x7fcfbc070150 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfd40fce80 0x7fcfd410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 --2- 172.21.15.94:0/4010517192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfd40fdff0 0x7fcfd410df50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:44:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 >> 172.21.15.94:0/4010517192 conn(0x7fcfd40f8580 msgr2=0x7fcfd40f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:44:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 shutdown_connections 2024-04-06T03:44:47.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:44:47.761+0000 7fcfd9ac4700 1 -- 172.21.15.94:0/4010517192 wait complete. 2024-04-06T03:44:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:48 smithi094 ceph-mon[94053]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:48 smithi094 ceph-mon[94053]: pgmap v63: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:48 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/4010517192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:44:48.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:48 smithi136 ceph-mon[82061]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:48.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:48 smithi136 ceph-mon[82061]: pgmap v63: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:48.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:48 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/4010517192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:44:49.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:49 smithi094 ceph-mon[94053]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:49.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:49 smithi136 ceph-mon[82061]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:44:50.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:50 smithi094 ceph-mon[94053]: pgmap v64: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:50.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:50 smithi136 ceph-mon[82061]: pgmap v64: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:51.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:51.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:44:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:51.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:51.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:44:51.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:52.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:52 smithi136 ceph-mon[82061]: pgmap v65: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:52.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:52 smithi094 ceph-mon[94053]: pgmap v65: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:54.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:54 smithi136 ceph-mon[82061]: pgmap v66: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:54 smithi094 ceph-mon[94053]: pgmap v66: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:55.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:55 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-06T03:44:55.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:55 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-06T03:44:56.529 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:56 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-06T03:44:56.529 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:56 smithi094 ceph-mon[94053]: Upgrade: osd.1 is safe to restart 2024-04-06T03:44:56.529 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:56 smithi094 ceph-mon[94053]: pgmap v67: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:56.529 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:56 smithi094 ceph-mon[94053]: Upgrade: Updating osd.1 2024-04-06T03:44:56.529 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:56.529 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-06T03:44:56.530 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:56.530 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:56 smithi094 ceph-mon[94053]: Deploying daemon osd.1 on smithi094 2024-04-06T03:44:56.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:56 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-06T03:44:56.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:56 smithi136 ceph-mon[82061]: Upgrade: osd.1 is safe to restart 2024-04-06T03:44:56.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:56 smithi136 ceph-mon[82061]: pgmap v67: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:56.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:56 smithi136 ceph-mon[82061]: Upgrade: Updating osd.1 2024-04-06T03:44:56.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:44:56.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-06T03:44:56.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:44:56.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:56 smithi136 ceph-mon[82061]: Deploying daemon osd.1 on smithi094 2024-04-06T03:44:57.481 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:44:57 smithi094 ceph-mon[94053]: pgmap v68: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:57.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:44:57 smithi136 ceph-mon[82061]: pgmap v68: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:44:59.418 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:44:59 smithi094 systemd[1]: Stopping Ceph osd.1 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:44:59.419 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:44:59 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[55408]: 2024-04-06T03:44:59.347+0000 7f8731095700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:44:59.419 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:44:59 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[55408]: 2024-04-06T03:44:59.347+0000 7f8731095700 -1 osd.1 53 *** Got signal Terminated *** 2024-04-06T03:44:59.419 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:44:59 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[55408]: 2024-04-06T03:44:59.347+0000 7f8731095700 -1 osd.1 53 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:45:00.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:00 smithi136 ceph-mon[82061]: pgmap v69: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:45:00.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:00 smithi136 ceph-mon[82061]: osd.1 marked itself down and dead 2024-04-06T03:45:00.652 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:00 smithi094 ceph-mon[94053]: pgmap v69: 65 pgs: 65 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:45:00.652 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:00 smithi094 ceph-mon[94053]: osd.1 marked itself down and dead 2024-04-06T03:45:00.918 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:00 smithi094 podman[109049]: 2024-04-06 03:45:00.649857181 +0000 UTC m=+1.404725033 container died d6034d5d9a6606209f5c07e1ead45b1782c86dc7430528f978ae984c69442389 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=quincy-6699495, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:45:01.192 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:01 smithi094 podman[109049]: 2024-04-06 03:45:01.031842925 +0000 UTC m=+1.786710782 container cleanup d6034d5d9a6606209f5c07e1ead45b1782c86dc7430528f978ae984c69442389 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=quincy-6699495) 2024-04-06T03:45:01.192 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:01 smithi094 bash[109049]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1 2024-04-06T03:45:01.468 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:01 smithi094 podman[109068]: 2024-04-06 03:45:01.250341305 +0000 UTC m=+0.604162551 container remove d6034d5d9a6606209f5c07e1ead45b1782c86dc7430528f978ae984c69442389 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=quincy-6699495, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:45:01.469 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:01 smithi094 ceph-mon[94053]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:45:01.469 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:01 smithi094 ceph-mon[94053]: osdmap e54: 6 total, 5 up, 6 in 2024-04-06T03:45:01.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:01 smithi136 ceph-mon[82061]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:45:01.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:01 smithi136 ceph-mon[82061]: osdmap e54: 6 total, 5 up, 6 in 2024-04-06T03:45:02.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:02 smithi094 ceph-mon[94053]: pgmap v71: 65 pgs: 12 stale+active+clean, 53 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:45:02.421 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:02 smithi094 ceph-mon[94053]: osdmap e55: 6 total, 5 up, 6 in 2024-04-06T03:45:02.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:02 smithi136 ceph-mon[82061]: pgmap v71: 65 pgs: 12 stale+active+clean, 53 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:45:02.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:02 smithi136 ceph-mon[82061]: osdmap e55: 6 total, 5 up, 6 in 2024-04-06T03:45:02.843 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:02 smithi094 podman[109281]: 2024-04-06 03:45:02.652141968 +0000 UTC m=+0.020832380 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:45:02.843 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:02 smithi094 podman[109281]: 2024-04-06 03:45:02.757965415 +0000 UTC m=+0.126655788 container create b9bb1dc01f54ad285c47f25363b00b499809015f2dab621ade26cb7fa47ac997 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-06T03:45:03.168 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:03 smithi094 podman[109281]: 2024-04-06 03:45:03.002405104 +0000 UTC m=+0.371095483 container init b9bb1dc01f54ad285c47f25363b00b499809015f2dab621ade26cb7fa47ac997 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-9f19235, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-06T03:45:03.168 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:03 smithi094 podman[109281]: 2024-04-06 03:45:03.005593373 +0000 UTC m=+0.374283757 container start b9bb1dc01f54ad285c47f25363b00b499809015f2dab621ade26cb7fa47ac997 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-9f19235, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-06T03:45:03.168 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:03 smithi094 podman[109281]: 2024-04-06 03:45:03.060409251 +0000 UTC m=+0.429099624 container attach b9bb1dc01f54ad285c47f25363b00b499809015f2dab621ade26cb7fa47ac997 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-06T03:45:03.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:03 smithi136 ceph-mon[82061]: pgmap v73: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:45:03.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:03 smithi136 ceph-mon[82061]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-06T03:45:03.625 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:03 smithi094 ceph-mon[94053]: pgmap v73: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:45:03.625 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:03 smithi094 ceph-mon[94053]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-06T03:45:03.625 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:03 smithi094 podman[109281]: 2024-04-06 03:45:03.309195925 +0000 UTC m=+0.677886308 container died b9bb1dc01f54ad285c47f25363b00b499809015f2dab621ade26cb7fa47ac997 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-9f19235) 2024-04-06T03:45:04.359 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:03 smithi094 podman[109322]: 2024-04-06 03:45:03.922439165 +0000 UTC m=+0.607726173 container remove b9bb1dc01f54ad285c47f25363b00b499809015f2dab621ade26cb7fa47ac997 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, RELEASE=squid-9f19235, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, io.buildah.version=1.33.5) 2024-04-06T03:45:04.359 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:03 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.1.service: Deactivated successfully. 2024-04-06T03:45:04.359 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:03 smithi094 systemd[1]: Stopped Ceph osd.1 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:45:04.359 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:03 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.1.service: Consumed 1min 9.684s CPU time. 2024-04-06T03:45:04.359 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:04 smithi094 systemd[1]: Starting Ceph osd.1 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:45:04.669 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:04 smithi094 podman[109428]: 2024-04-06 03:45:04.258709209 +0000 UTC m=+0.017770214 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:45:04.669 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:04 smithi094 podman[109428]: 2024-04-06 03:45:04.3564764 +0000 UTC m=+0.115537386 container create 4f8360e77acfdced8693bfb6a1dc24e6e8358e4c86a2e4b76527ffd1b1d60ce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403) 2024-04-06T03:45:04.669 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:04 smithi094 podman[109428]: 2024-04-06 03:45:04.601066802 +0000 UTC m=+0.360127797 container init 4f8360e77acfdced8693bfb6a1dc24e6e8358e4c86a2e4b76527ffd1b1d60ce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:45:04.669 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:04 smithi094 podman[109428]: 2024-04-06 03:45:04.604102411 +0000 UTC m=+0.363163400 container start 4f8360e77acfdced8693bfb6a1dc24e6e8358e4c86a2e4b76527ffd1b1d60ce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, RELEASE=squid-9f19235, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-06T03:45:05.049 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:04 smithi094 podman[109428]: 2024-04-06 03:45:04.684185905 +0000 UTC m=+0.443246886 container attach 4f8360e77acfdced8693bfb6a1dc24e6e8358e4c86a2e4b76527ffd1b1d60ce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:45:05.365 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate[109443]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate[109443]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 bash[109428]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 bash[109428]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 bash[109428]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 bash[109428]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 bash[109428]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 bash[109428]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 bash[109428]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate[109443]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate[109443]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate[109443]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate[109443]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate[109443]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-06T03:45:05.366 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 podman[109428]: 2024-04-06 03:45:05.071447561 +0000 UTC m=+0.830508545 container died 4f8360e77acfdced8693bfb6a1dc24e6e8358e4c86a2e4b76527ffd1b1d60ce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-9f19235, org.label-schema.license=GPLv2, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:45:05.918 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 podman[109581]: 2024-04-06 03:45:05.65469448 +0000 UTC m=+0.577990886 container remove 4f8360e77acfdced8693bfb6a1dc24e6e8358e4c86a2e4b76527ffd1b1d60ce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-activate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-06T03:45:05.918 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 podman[109641]: 2024-04-06 03:45:05.85555452 +0000 UTC m=+0.104752508 container create 108ce348aea2d29a8084bba6747871db98a4e77470ac441a6599b15b3d234511 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-9f19235, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD) 2024-04-06T03:45:05.918 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:05 smithi094 podman[109641]: 2024-04-06 03:45:05.769236698 +0000 UTC m=+0.018434689 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:45:06.241 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:06 smithi094 podman[109641]: 2024-04-06 03:45:06.066233579 +0000 UTC m=+0.315431568 container init 108ce348aea2d29a8084bba6747871db98a4e77470ac441a6599b15b3d234511 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, RELEASE=squid-9f19235, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-06T03:45:06.242 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:06 smithi094 podman[109641]: 2024-04-06 03:45:06.069274268 +0000 UTC m=+0.318472262 container start 108ce348aea2d29a8084bba6747871db98a4e77470ac441a6599b15b3d234511 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, RELEASE=squid-9f19235) 2024-04-06T03:45:06.242 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:06 smithi094 ceph-osd[109660]: -- 172.21.15.94:0/3992754815 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c436aad680 con 0x55c436a76c00 2024-04-06T03:45:06.242 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:06 smithi094 bash[109641]: 108ce348aea2d29a8084bba6747871db98a4e77470ac441a6599b15b3d234511 2024-04-06T03:45:06.242 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:06 smithi094 systemd[1]: Started Ceph osd.1 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:45:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:06 smithi136 ceph-mon[82061]: pgmap v74: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:45:06.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:06.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:45:06.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:06 smithi094 ceph-mon[94053]: pgmap v74: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-06T03:45:06.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:06.613 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:45:06.864 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:06 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[109656]: 2024-04-06T03:45:06.610+0000 7f96ee52f7c0 -1 Falling back to public interface 2024-04-06T03:45:07.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:07 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:07.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:07 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:07.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:07 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:45:07.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:07 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:07.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:07 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:07.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:07 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:45:08.538 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:08 smithi094 ceph-mon[94053]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:08.538 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:08 smithi094 ceph-mon[94053]: Health check failed: Degraded data redundancy: 32/219 objects degraded (14.612%), 13 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:08.538 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:08 smithi094 ceph-mon[94053]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-06T03:45:08.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:08 smithi136 ceph-mon[82061]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:08.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:08 smithi136 ceph-mon[82061]: Health check failed: Degraded data redundancy: 32/219 objects degraded (14.612%), 13 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:08.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:08 smithi136 ceph-mon[82061]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-06T03:45:10.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:10 smithi094 ceph-mon[94053]: pgmap v76: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:10.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:10 smithi136 ceph-mon[82061]: pgmap v76: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:12.168 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:11 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[109656]: 2024-04-06T03:45:11.873+0000 7f96ee52f7c0 -1 osd.1 53 log_to_monitors true 2024-04-06T03:45:12.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:12.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:12.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:11 smithi094 ceph-mon[94053]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:12.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:11 smithi094 ceph-mon[94053]: from='osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-06T03:45:12.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:11 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:12.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:11 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:12.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:11 smithi136 ceph-mon[82061]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:12.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:11 smithi136 ceph-mon[82061]: from='osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-06T03:45:12.905 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:45:12 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[109656]: 2024-04-06T03:45:12.616+0000 7f96db97f700 -1 osd.1 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:45:13.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:12 smithi094 ceph-mon[94053]: from='osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-06T03:45:13.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:12 smithi094 ceph-mon[94053]: osdmap e56: 6 total, 5 up, 6 in 2024-04-06T03:45:13.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:12 smithi094 ceph-mon[94053]: from='osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:45:13.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:13.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:13.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:12 smithi094 ceph-mon[94053]: from='osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734]' entity='osd.1' 2024-04-06T03:45:13.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:12 smithi136 ceph-mon[82061]: from='osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-06T03:45:13.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:12 smithi136 ceph-mon[82061]: osdmap e56: 6 total, 5 up, 6 in 2024-04-06T03:45:13.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:12 smithi136 ceph-mon[82061]: from='osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:45:13.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:12 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:13.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:12 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:13.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:12 smithi136 ceph-mon[82061]: from='osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734]' entity='osd.1' 2024-04-06T03:45:14.092 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:13 smithi094 ceph-mon[94053]: pgmap v79: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:14.092 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:13 smithi094 ceph-mon[94053]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:45:14.092 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:13 smithi094 ceph-mon[94053]: osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734] boot 2024-04-06T03:45:14.092 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:13 smithi094 ceph-mon[94053]: osdmap e57: 6 total, 6 up, 6 in 2024-04-06T03:45:14.092 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:13 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:45:14.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:13 smithi136 ceph-mon[82061]: pgmap v79: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:14.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:13 smithi136 ceph-mon[82061]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:45:14.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:13 smithi136 ceph-mon[82061]: osd.1 [v2:172.21.15.94:6810/3698645734,v1:172.21.15.94:6811/3698645734] boot 2024-04-06T03:45:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:13 smithi136 ceph-mon[82061]: osdmap e57: 6 total, 6 up, 6 in 2024-04-06T03:45:14.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:13 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-06T03:45:15.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:15 smithi094 ceph-mon[94053]: osdmap e58: 6 total, 6 up, 6 in 2024-04-06T03:45:15.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:15 smithi094 ceph-mon[94053]: pgmap v82: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:16.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:15 smithi136 ceph-mon[82061]: osdmap e58: 6 total, 6 up, 6 in 2024-04-06T03:45:16.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:15 smithi136 ceph-mon[82061]: pgmap v82: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 32/219 objects degraded (14.612%) 2024-04-06T03:45:17.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: pgmap v83: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%) 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: pgmap v84: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%) 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-06T03:45:17.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-06T03:45:17.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:17 smithi136 ceph-mon[82061]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-04-06T03:45:17.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: pgmap v83: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%) 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: pgmap v84: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 16/219 objects degraded (7.306%) 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-06T03:45:17.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:17 smithi094 ceph-mon[94053]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-04-06T03:45:17.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.877+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3543698308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 msgr2=0x7fbe940fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:17.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.877+0000 7fbe9c7cb700 1 --2- 172.21.15.94:0/3543698308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 0x7fbe940fc0e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009ce0 tx=0x7fbe84004d10 comp rx=0 tx=0).stop 2024-04-06T03:45:17.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.877+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3543698308 shutdown_connections 2024-04-06T03:45:17.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.877+0000 7fbe9c7cb700 1 --2- 172.21.15.94:0/3543698308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940fcf80 0x7fbe940fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.877+0000 7fbe9c7cb700 1 --2- 172.21.15.94:0/3543698308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 0x7fbe940fc0e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.877+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3543698308 >> 172.21.15.94:0/3543698308 conn(0x7fbe940f7430 msgr2=0x7fbe940f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.877+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3543698308 shutdown_connections 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.877+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3543698308 wait complete. 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe9c7cb700 1 Processor -- start 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe9c7cb700 1 -- start start 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe9c7cb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 0x7fbe94110830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe9c7cb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940fcf80 0x7fbe94112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe9c7cb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940663b0 con 0x7fbe940fbcd0 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe9c7cb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940664f0 con 0x7fbe940fcf80 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe9a567700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 0x7fbe94110830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe9a567700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 0x7fbe94110830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60336/0 (socket says 172.21.15.94:60336) 2024-04-06T03:45:17.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe9a567700 1 -- 172.21.15.94:0/3149661771 learned_addr learned my addr 172.21.15.94:0/3149661771 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.878+0000 7fbe99d66700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940fcf80 0x7fbe94112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.879+0000 7fbe9a567700 1 -- 172.21.15.94:0/3149661771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940fcf80 msgr2=0x7fbe94112d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.879+0000 7fbe9a567700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940fcf80 0x7fbe94112d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.879+0000 7fbe9a567700 1 -- 172.21.15.94:0/3149661771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe84009940 con 0x7fbe940fbcd0 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.879+0000 7fbe9a567700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 0x7fbe94110830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009380 tx=0x7fbe84004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.879+0000 7fbe92ffd700 1 -- 172.21.15.94:0/3149661771 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe8400e040 con 0x7fbe940fbcd0 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.879+0000 7fbe92ffd700 1 -- 172.21.15.94:0/3149661771 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe840128e0 con 0x7fbe940fbcd0 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.879+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe94113320 con 0x7fbe940fbcd0 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.879+0000 7fbe92ffd700 1 -- 172.21.15.94:0/3149661771 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe8401ca10 con 0x7fbe940fbcd0 2024-04-06T03:45:17.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.879+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941138e0 con 0x7fbe940fbcd0 2024-04-06T03:45:17.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.880+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9405f5f0 con 0x7fbe940fbcd0 2024-04-06T03:45:17.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.880+0000 7fbe92ffd700 1 -- 172.21.15.94:0/3149661771 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fbe84012a50 con 0x7fbe940fbcd0 2024-04-06T03:45:17.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.881+0000 7fbe92ffd700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbe7c06dca0 0x7fbe7c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:17.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.881+0000 7fbe92ffd700 1 -- 172.21.15.94:0/3149661771 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbe8408efe0 con 0x7fbe940fbcd0 2024-04-06T03:45:17.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.881+0000 7fbe99d66700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbe7c06dca0 0x7fbe7c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:17.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.881+0000 7fbe99d66700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbe7c06dca0 0x7fbe7c070150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbe88004990 tx=0x7fbe88009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:17.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:17.886+0000 7fbe92ffd700 1 -- 172.21.15.94:0/3149661771 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe84058650 con 0x7fbe940fbcd0 2024-04-06T03:45:18.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.098+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe94101290 con 0x7fbe7c06dca0 2024-04-06T03:45:18.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.099+0000 7fbe92ffd700 1 -- 172.21.15.94:0/3149661771 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbe94101290 con 0x7fbe7c06dca0 2024-04-06T03:45:18.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbe7c06dca0 msgr2=0x7fbe7c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbe7c06dca0 0x7fbe7c070150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbe88004990 tx=0x7fbe88009ee0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 msgr2=0x7fbe94110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 0x7fbe94110830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009380 tx=0x7fbe84004d10 comp rx=0 tx=0).stop 2024-04-06T03:45:18.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 shutdown_connections 2024-04-06T03:45:18.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe940fbcd0 0x7fbe94110830 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbe7c06dca0 0x7fbe7c070150 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 --2- 172.21.15.94:0/3149661771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940fcf80 0x7fbe94112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 >> 172.21.15.94:0/3149661771 conn(0x7fbe940f7430 msgr2=0x7fbe940f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:18.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 shutdown_connections 2024-04-06T03:45:18.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.102+0000 7fbe9c7cb700 1 -- 172.21.15.94:0/3149661771 wait complete. 2024-04-06T03:45:18.115 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 -- 172.21.15.94:0/1680413641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 msgr2=0x7fe0500ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 --2- 172.21.15.94:0/1680413641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 0x7fe0500ff9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe040004940 tx=0x7fe040009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 -- 172.21.15.94:0/1680413641 shutdown_connections 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 --2- 172.21.15.94:0/1680413641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0500fff00 0x7fe050102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 --2- 172.21.15.94:0/1680413641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 0x7fe0500ff9c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 -- 172.21.15.94:0/1680413641 >> 172.21.15.94:0/1680413641 conn(0x7fe0500f7430 msgr2=0x7fe0500f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 -- 172.21.15.94:0/1680413641 shutdown_connections 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 -- 172.21.15.94:0/1680413641 wait complete. 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 Processor -- start 2024-04-06T03:45:18.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 -- start start 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 0x7fe05010dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.204+0000 7fe0558d2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0500fff00 0x7fe05010e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe0558d2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05010e6a0 con 0x7fe0500662e0 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe0558d2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05010edf0 con 0x7fe0500fff00 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe04effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 0x7fe05010dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe04effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 0x7fe05010dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60346/0 (socket says 172.21.15.94:60346) 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe04effd700 1 -- 172.21.15.94:0/204229923 learned_addr learned my addr 172.21.15.94:0/204229923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe0467fc700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0500fff00 0x7fe05010e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe04effd700 1 -- 172.21.15.94:0/204229923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0500fff00 msgr2=0x7fe05010e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe04effd700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0500fff00 0x7fe05010e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe04effd700 1 -- 172.21.15.94:0/204229923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040005040 con 0x7fe0500662e0 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe04effd700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 0x7fe05010dad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe04000b3b0 tx=0x7fe040009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe047fff700 1 -- 172.21.15.94:0/204229923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe04000e040 con 0x7fe0500662e0 2024-04-06T03:45:18.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe047fff700 1 -- 172.21.15.94:0/204229923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe04000bcc0 con 0x7fe0500662e0 2024-04-06T03:45:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05010f070 con 0x7fe0500662e0 2024-04-06T03:45:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe047fff700 1 -- 172.21.15.94:0/204229923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe04001c630 con 0x7fe0500662e0 2024-04-06T03:45:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.205+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05010f630 con 0x7fe0500662e0 2024-04-06T03:45:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.206+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05005f5f0 con 0x7fe0500662e0 2024-04-06T03:45:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.207+0000 7fe047fff700 1 -- 172.21.15.94:0/204229923 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fe0400139f0 con 0x7fe0500662e0 2024-04-06T03:45:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.207+0000 7fe047fff700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe03c06dca0 0x7fe03c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.207+0000 7fe0467fc700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe03c06dca0 0x7fe03c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.207+0000 7fe047fff700 1 -- 172.21.15.94:0/204229923 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe04008e160 con 0x7fe0500662e0 2024-04-06T03:45:18.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.208+0000 7fe0467fc700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe03c06dca0 0x7fe03c070150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe038006fd0 tx=0x7fe038008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:18.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.212+0000 7fe047fff700 1 -- 172.21.15.94:0/204229923 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe04005b360 con 0x7fe0500662e0 2024-04-06T03:45:18.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.431+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe05010f960 con 0x7fe03c06dca0 2024-04-06T03:45:18.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.432+0000 7fe047fff700 1 -- 172.21.15.94:0/204229923 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe05010f960 con 0x7fe03c06dca0 2024-04-06T03:45:18.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe03c06dca0 msgr2=0x7fe03c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe03c06dca0 0x7fe03c070150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe038006fd0 tx=0x7fe038008040 comp rx=0 tx=0).stop 2024-04-06T03:45:18.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 msgr2=0x7fe05010dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 0x7fe05010dad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe04000b3b0 tx=0x7fe040009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:18.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 shutdown_connections 2024-04-06T03:45:18.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0500662e0 0x7fe05010dad0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe03c06dca0 0x7fe03c070150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 --2- 172.21.15.94:0/204229923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0500fff00 0x7fe05010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 >> 172.21.15.94:0/204229923 conn(0x7fe0500f7430 msgr2=0x7fe0500f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:18.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 shutdown_connections 2024-04-06T03:45:18.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.434+0000 7fe0558d2700 1 -- 172.21.15.94:0/204229923 wait complete. 2024-04-06T03:45:18.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.535+0000 7f383f324700 1 -- 172.21.15.94:0/2854802649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 msgr2=0x7f3838070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.535+0000 7f383f324700 1 --2- 172.21.15.94:0/2854802649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 0x7f3838070350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3834004940 tx=0x7f3834009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:18.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.535+0000 7f383f324700 1 -- 172.21.15.94:0/2854802649 shutdown_connections 2024-04-06T03:45:18.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.535+0000 7f383f324700 1 --2- 172.21.15.94:0/2854802649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3838070890 0x7f3838070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.535+0000 7f383f324700 1 --2- 172.21.15.94:0/2854802649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 0x7f3838070350 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.535+0000 7f383f324700 1 -- 172.21.15.94:0/2854802649 >> 172.21.15.94:0/2854802649 conn(0x7f38380f8550 msgr2=0x7f38380fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:18.539 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:18 smithi094 ceph-mon[94053]: Health check update: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:18.540 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:18 smithi094 ceph-mon[94053]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.536+0000 7f383f324700 1 -- 172.21.15.94:0/2854802649 shutdown_connections 2024-04-06T03:45:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.536+0000 7f383f324700 1 -- 172.21.15.94:0/2854802649 wait complete. 2024-04-06T03:45:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.536+0000 7f383f324700 1 Processor -- start 2024-04-06T03:45:18.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.536+0000 7f383f324700 1 -- start start 2024-04-06T03:45:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.536+0000 7f383f324700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3838070890 0x7f383810fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.536+0000 7f383f324700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 0x7f3838110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.536+0000 7f383f324700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838110690 con 0x7f3838071e40 2024-04-06T03:45:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.536+0000 7f383f324700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838110800 con 0x7f3838070890 2024-04-06T03:45:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f383c8bf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 0x7f3838110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f383c8bf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 0x7f3838110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60358/0 (socket says 172.21.15.94:60358) 2024-04-06T03:45:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f383c8bf700 1 -- 172.21.15.94:0/3162840908 learned_addr learned my addr 172.21.15.94:0/3162840908 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:18.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f383d0c0700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3838070890 0x7f383810fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f383c8bf700 1 -- 172.21.15.94:0/3162840908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3838070890 msgr2=0x7f383810fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f383c8bf700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3838070890 0x7f383810fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f383c8bf700 1 -- 172.21.15.94:0/3162840908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3834005040 con 0x7f3838071e40 2024-04-06T03:45:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f383c8bf700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 0x7f3838110090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3828002a50 tx=0x7f38280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f382dffb700 1 -- 172.21.15.94:0/3162840908 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3828007cc0 con 0x7f3838071e40 2024-04-06T03:45:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f382dffb700 1 -- 172.21.15.94:0/3162840908 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f382800f040 con 0x7f3838071e40 2024-04-06T03:45:18.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.537+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3838115260 con 0x7f3838071e40 2024-04-06T03:45:18.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.538+0000 7f382dffb700 1 -- 172.21.15.94:0/3162840908 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f382800da30 con 0x7f3838071e40 2024-04-06T03:45:18.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.538+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3838115820 con 0x7f3838071e40 2024-04-06T03:45:18.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.539+0000 7f382dffb700 1 -- 172.21.15.94:0/3162840908 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f382800db90 con 0x7f3838071e40 2024-04-06T03:45:18.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.539+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383804c710 con 0x7f3838071e40 2024-04-06T03:45:18.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.539+0000 7f382dffb700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f382406dca0 0x7f3824070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:18.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.539+0000 7f382dffb700 1 -- 172.21.15.94:0/3162840908 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f382808d180 con 0x7f3838071e40 2024-04-06T03:45:18.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.539+0000 7f383d0c0700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f382406dca0 0x7f3824070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:18.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.540+0000 7f383d0c0700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f382406dca0 0x7f3824070150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3834004890 tx=0x7f3834009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:18.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.544+0000 7f382dffb700 1 -- 172.21.15.94:0/3162840908 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f382805a360 con 0x7f3838071e40 2024-04-06T03:45:18.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:18 smithi136 ceph-mon[82061]: Health check update: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:18.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:18 smithi136 ceph-mon[82061]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:18.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.748+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3838103ef0 con 0x7f382406dca0 2024-04-06T03:45:18.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.758+0000 7f382dffb700 1 -- 172.21.15.94:0/3162840908 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f3838103ef0 con 0x7f382406dca0 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (4m) 7s ago 12m 27.6M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (77s) 7s ago 12m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (71s) 68s ago 11m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 7s ago 12m - - 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (7m) 7s ago 7m 14.8M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (7m) 7s ago 7m 153M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (7m) 68s ago 7m 14.6M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (7m) 68s ago 7m 17.0M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (3m) 7s ago 13m 501M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (2m) 68s ago 11m 449M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (2m) 7s ago 14m 44.9M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:45:18.762 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (119s) 68s ago 11m 36.7M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:45:18.763 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (4m) 7s ago 12m 15.2M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:45:18.763 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (3m) 68s ago 11m 17.3M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:45:18.763 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (50s) 7s ago 10m 102M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:45:18.763 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (12s) 7s ago 10m 12.1M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:45:18.763 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (10m) 7s ago 10m 225M 1557M 17.2.7-819-g6699495c ec300eaa768b 50775006e096 2024-04-06T03:45:18.763 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (9m) 68s ago 9m 292M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:45:18.763 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (9m) 68s ago 9m 257M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:45:18.763 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (8m) 68s ago 8m 241M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:45:18.763 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (2m) 7s ago 12m 54.7M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:45:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f382406dca0 msgr2=0x7f3824070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f382406dca0 0x7f3824070150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3834004890 tx=0x7f3834009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 msgr2=0x7f3838110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 0x7f3838110090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3828002a50 tx=0x7f38280046b0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 shutdown_connections 2024-04-06T03:45:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f382406dca0 0x7f3824070150 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3838070890 0x7f383810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 --2- 172.21.15.94:0/3162840908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3838071e40 0x7f3838110090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 >> 172.21.15.94:0/3162840908 conn(0x7f38380f8550 msgr2=0x7f38380f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 shutdown_connections 2024-04-06T03:45:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.761+0000 7f383f324700 1 -- 172.21.15.94:0/3162840908 wait complete. 2024-04-06T03:45:18.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.861+0000 7fc05ae17700 1 -- 172.21.15.94:0/2205327668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 msgr2=0x7fc0540fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.861+0000 7fc05ae17700 1 --2- 172.21.15.94:0/2205327668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 0x7fc0540fd3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc044009ce0 tx=0x7fc044004d10 comp rx=0 tx=0).stop 2024-04-06T03:45:18.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.861+0000 7fc05ae17700 1 -- 172.21.15.94:0/2205327668 shutdown_connections 2024-04-06T03:45:18.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.861+0000 7fc05ae17700 1 --2- 172.21.15.94:0/2205327668 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 0x7fc0540fd3e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.861+0000 7fc05ae17700 1 --2- 172.21.15.94:0/2205327668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0540fbe00 0x7fc0540fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.861+0000 7fc05ae17700 1 -- 172.21.15.94:0/2205327668 >> 172.21.15.94:0/2205327668 conn(0x7fc0540f74e0 msgr2=0x7fc0540f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.861+0000 7fc05ae17700 1 -- 172.21.15.94:0/2205327668 shutdown_connections 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.861+0000 7fc05ae17700 1 -- 172.21.15.94:0/2205327668 wait complete. 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc05ae17700 1 Processor -- start 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc05ae17700 1 -- start start 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc05ae17700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0540fbe00 0x7fc054110740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc05ae17700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 0x7fc054112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc05ae17700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc054066be0 con 0x7fc0540fcf70 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc05ae17700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc054066d20 con 0x7fc0540fbe00 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc053fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 0x7fc054112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc053fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 0x7fc054112c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60378/0 (socket says 172.21.15.94:60378) 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc053fff700 1 -- 172.21.15.94:0/1317589941 learned_addr learned my addr 172.21.15.94:0/1317589941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.862+0000 7fc058bb3700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0540fbe00 0x7fc054110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.863+0000 7fc053fff700 1 -- 172.21.15.94:0/1317589941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0540fbe00 msgr2=0x7fc054110740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.863+0000 7fc053fff700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0540fbe00 0x7fc054110740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.863+0000 7fc053fff700 1 -- 172.21.15.94:0/1317589941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc044009940 con 0x7fc0540fcf70 2024-04-06T03:45:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.863+0000 7fc053fff700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 0x7fc054112c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc04400bfd0 tx=0x7fc044004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.863+0000 7fc0517fa700 1 -- 172.21.15.94:0/1317589941 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc04400e040 con 0x7fc0540fcf70 2024-04-06T03:45:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.863+0000 7fc0517fa700 1 -- 172.21.15.94:0/1317589941 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc04401cd00 con 0x7fc0540fcf70 2024-04-06T03:45:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.863+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0541131d0 con 0x7fc0540fcf70 2024-04-06T03:45:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.863+0000 7fc0517fa700 1 -- 172.21.15.94:0/1317589941 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc044012c40 con 0x7fc0540fcf70 2024-04-06T03:45:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.863+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc054113790 con 0x7fc0540fcf70 2024-04-06T03:45:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.864+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05405f5f0 con 0x7fc0540fcf70 2024-04-06T03:45:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.864+0000 7fc0517fa700 1 -- 172.21.15.94:0/1317589941 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fc0440124a0 con 0x7fc0540fcf70 2024-04-06T03:45:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.865+0000 7fc0517fa700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc04006dca0 0x7fc040070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.865+0000 7fc0517fa700 1 -- 172.21.15.94:0/1317589941 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc04408f6e0 con 0x7fc0540fcf70 2024-04-06T03:45:18.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.865+0000 7fc058bb3700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc04006dca0 0x7fc040070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.865+0000 7fc058bb3700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc04006dca0 0x7fc040070150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc054066860 tx=0x7fc04c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:18.870+0000 7fc0517fa700 1 -- 172.21.15.94:0/1317589941 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc044058d90 con 0x7fc0540fcf70 2024-04-06T03:45:19.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.145+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc05404bed0 con 0x7fc0540fcf70 2024-04-06T03:45:19.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.146+0000 7fc0517fa700 1 -- 172.21.15.94:0/1317589941 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fc04405c450 con 0x7fc0540fcf70 2024-04-06T03:45:19.149 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:45:19.149 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:45:19.149 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:45:19.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4, 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 8, 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 6 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:45:19.150 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:45:19.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc04006dca0 msgr2=0x7fc040070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc04006dca0 0x7fc040070150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc054066860 tx=0x7fc04c00b040 comp rx=0 tx=0).stop 2024-04-06T03:45:19.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 msgr2=0x7fc054112c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 0x7fc054112c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc04400bfd0 tx=0x7fc044004d10 comp rx=0 tx=0).stop 2024-04-06T03:45:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 shutdown_connections 2024-04-06T03:45:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc04006dca0 0x7fc040070150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0540fbe00 0x7fc054110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 --2- 172.21.15.94:0/1317589941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0540fcf70 0x7fc054112c90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 >> 172.21.15.94:0/1317589941 conn(0x7fc0540f74e0 msgr2=0x7fc0540f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 shutdown_connections 2024-04-06T03:45:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.148+0000 7fc05ae17700 1 -- 172.21.15.94:0/1317589941 wait complete. 2024-04-06T03:45:19.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.249+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2337158585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 msgr2=0x7f0b580fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.249+0000 7f0b5e1fd700 1 --2- 172.21.15.94:0/2337158585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 0x7f0b580fd3f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b40004940 tx=0x7f0b40009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:19.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.249+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2337158585 shutdown_connections 2024-04-06T03:45:19.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.249+0000 7f0b5e1fd700 1 --2- 172.21.15.94:0/2337158585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 0x7f0b580fd3f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.249+0000 7f0b5e1fd700 1 --2- 172.21.15.94:0/2337158585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b580fbcd0 0x7f0b580fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.249+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2337158585 >> 172.21.15.94:0/2337158585 conn(0x7f0b580f7430 msgr2=0x7f0b580f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:19.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.249+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2337158585 shutdown_connections 2024-04-06T03:45:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.249+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2337158585 wait complete. 2024-04-06T03:45:19.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.249+0000 7f0b5e1fd700 1 Processor -- start 2024-04-06T03:45:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b5e1fd700 1 -- start start 2024-04-06T03:45:19.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b5e1fd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b580fbcd0 0x7f0b58110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b5e1fd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 0x7f0b58112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b5e1fd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58066400 con 0x7f0b580fcf80 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b5e1fd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58066540 con 0x7f0b580fbcd0 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b4effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 0x7f0b58112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b4effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 0x7f0b58112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60400/0 (socket says 172.21.15.94:60400) 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b4effd700 1 -- 172.21.15.94:0/2966638621 learned_addr learned my addr 172.21.15.94:0/2966638621 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b577fe700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b580fbcd0 0x7f0b58110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b4effd700 1 -- 172.21.15.94:0/2966638621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b580fbcd0 msgr2=0x7f0b58110710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b4effd700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b580fbcd0 0x7f0b58110710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b4effd700 1 -- 172.21.15.94:0/2966638621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b40005040 con 0x7f0b580fcf80 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b577fe700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b580fbcd0 0x7f0b58110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b4effd700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 0x7f0b58112c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009380 tx=0x7f0b40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.250+0000 7f0b54ff9700 1 -- 172.21.15.94:0/2966638621 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b4000e040 con 0x7f0b580fcf80 2024-04-06T03:45:19.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.251+0000 7f0b54ff9700 1 -- 172.21.15.94:0/2966638621 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b4000bcc0 con 0x7f0b580fcf80 2024-04-06T03:45:19.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.251+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b58113260 con 0x7f0b580fcf80 2024-04-06T03:45:19.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.251+0000 7f0b54ff9700 1 -- 172.21.15.94:0/2966638621 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b4001c5f0 con 0x7f0b580fcf80 2024-04-06T03:45:19.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.251+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b58113820 con 0x7f0b580fcf80 2024-04-06T03:45:19.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.252+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5805f5f0 con 0x7f0b580fcf80 2024-04-06T03:45:19.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.252+0000 7f0b54ff9700 1 -- 172.21.15.94:0/2966638621 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f0b40013420 con 0x7f0b580fcf80 2024-04-06T03:45:19.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.252+0000 7f0b54ff9700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0b4406dca0 0x7f0b44070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:19.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.252+0000 7f0b54ff9700 1 -- 172.21.15.94:0/2966638621 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0b4008dfe0 con 0x7f0b580fcf80 2024-04-06T03:45:19.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.252+0000 7f0b577fe700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0b4406dca0 0x7f0b44070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:19.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.253+0000 7f0b577fe700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0b4406dca0 0x7f0b44070150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b48006fd0 tx=0x7f0b48008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:19.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.257+0000 7f0b54ff9700 1 -- 172.21.15.94:0/2966638621 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0b40057740 con 0x7f0b580fcf80 2024-04-06T03:45:19.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:19 smithi094 ceph-mon[94053]: pgmap v85: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:19.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:19 smithi094 ceph-mon[94053]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:19.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:19 smithi094 ceph-mon[94053]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:19.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:19 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1317589941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:19.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.495+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0b5804bed0 con 0x7f0b580fcf80 2024-04-06T03:45:19.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.495+0000 7f0b54ff9700 1 -- 172.21.15.94:0/2966638621 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f0b4005ae00 con 0x7f0b580fcf80 2024-04-06T03:45:19.498 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:45:19.498 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:45:19.498 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:45:19.499 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:45:19.500 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:45:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:45:19.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0b4406dca0 msgr2=0x7f0b44070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0b4406dca0 0x7f0b44070150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b48006fd0 tx=0x7f0b48008040 comp rx=0 tx=0).stop 2024-04-06T03:45:19.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 msgr2=0x7f0b58112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 0x7f0b58112c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009380 tx=0x7f0b40009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 shutdown_connections 2024-04-06T03:45:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0b4406dca0 0x7f0b44070150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b580fbcd0 0x7f0b58110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 --2- 172.21.15.94:0/2966638621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b580fcf80 0x7f0b58112c60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 >> 172.21.15.94:0/2966638621 conn(0x7f0b580f7430 msgr2=0x7f0b580f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 shutdown_connections 2024-04-06T03:45:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.497+0000 7f0b5e1fd700 1 -- 172.21.15.94:0/2966638621 wait complete. 2024-04-06T03:45:19.503 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:45:19.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:19 smithi136 ceph-mon[82061]: pgmap v85: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:19.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:19 smithi136 ceph-mon[82061]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:19.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:19 smithi136 ceph-mon[82061]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:19.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:19 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1317589941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:19.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.598+0000 7fb082b41700 1 -- 172.21.15.94:0/932357755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c0708f0 msgr2=0x7fb07c070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.598+0000 7fb082b41700 1 --2- 172.21.15.94:0/932357755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c0708f0 0x7fb07c070d60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb068004940 tx=0x7fb068009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.598+0000 7fb082b41700 1 -- 172.21.15.94:0/932357755 shutdown_connections 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.598+0000 7fb082b41700 1 --2- 172.21.15.94:0/932357755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c0708f0 0x7fb07c070d60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.598+0000 7fb082b41700 1 --2- 172.21.15.94:0/932357755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb07c071ea0 0x7fb07c0703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.598+0000 7fb082b41700 1 -- 172.21.15.94:0/932357755 >> 172.21.15.94:0/932357755 conn(0x7fb07c0f8580 msgr2=0x7fb07c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.598+0000 7fb082b41700 1 -- 172.21.15.94:0/932357755 shutdown_connections 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.598+0000 7fb082b41700 1 -- 172.21.15.94:0/932357755 wait complete. 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.599+0000 7fb082b41700 1 Processor -- start 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.599+0000 7fb082b41700 1 -- start start 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.599+0000 7fb082b41700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb07c0708f0 0x7fb07c113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.599+0000 7fb082b41700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c071ea0 0x7fb07c114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.599+0000 7fb082b41700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c114a80 con 0x7fb07c071ea0 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.599+0000 7fb082b41700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c114bf0 con 0x7fb07c0708f0 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.599+0000 7fb07bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c071ea0 0x7fb07c114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.599+0000 7fb07bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c071ea0 0x7fb07c114480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60418/0 (socket says 172.21.15.94:60418) 2024-04-06T03:45:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.599+0000 7fb07bfff700 1 -- 172.21.15.94:0/2768833018 learned_addr learned my addr 172.21.15.94:0/2768833018 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:19.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb0808dd700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb07c0708f0 0x7fb07c113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:19.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb07bfff700 1 -- 172.21.15.94:0/2768833018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb07c0708f0 msgr2=0x7fb07c113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb07bfff700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb07c0708f0 0x7fb07c113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb07bfff700 1 -- 172.21.15.94:0/2768833018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb068005040 con 0x7fb07c071ea0 2024-04-06T03:45:19.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb07bfff700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c071ea0 0x7fb07c114480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb06800b3b0 tx=0x7fb068009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:19.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb0797fa700 1 -- 172.21.15.94:0/2768833018 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06800e040 con 0x7fb07c071ea0 2024-04-06T03:45:19.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb0797fa700 1 -- 172.21.15.94:0/2768833018 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb06800bcc0 con 0x7fb07c071ea0 2024-04-06T03:45:19.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c1195f0 con 0x7fb07c071ea0 2024-04-06T03:45:19.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb0797fa700 1 -- 172.21.15.94:0/2768833018 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06801c630 con 0x7fb07c071ea0 2024-04-06T03:45:19.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.600+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c119bb0 con 0x7fb07c071ea0 2024-04-06T03:45:19.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.601+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c04c710 con 0x7fb07c071ea0 2024-04-06T03:45:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.601+0000 7fb0797fa700 1 -- 172.21.15.94:0/2768833018 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fb068013420 con 0x7fb07c071ea0 2024-04-06T03:45:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.602+0000 7fb0797fa700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb06c06dca0 0x7fb06c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.602+0000 7fb0797fa700 1 -- 172.21.15.94:0/2768833018 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb06808e0c0 con 0x7fb07c071ea0 2024-04-06T03:45:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.602+0000 7fb0808dd700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb06c06dca0 0x7fb06c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.602+0000 7fb0808dd700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb06c06dca0 0x7fb06c070150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb070009fd0 tx=0x7fb070009f00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:19.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.606+0000 7fb0797fa700 1 -- 172.21.15.94:0/2768833018 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb06805b2a0 con 0x7fb07c071ea0 2024-04-06T03:45:19.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.819+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb07c0ff9f0 con 0x7fb06c06dca0 2024-04-06T03:45:19.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.821+0000 7fb0797fa700 1 -- 172.21.15.94:0/2768833018 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb07c0ff9f0 con 0x7fb06c06dca0 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "8/21 daemons upgraded", 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:45:19.824 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:45:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb06c06dca0 msgr2=0x7fb06c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb06c06dca0 0x7fb06c070150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb070009fd0 tx=0x7fb070009f00 comp rx=0 tx=0).stop 2024-04-06T03:45:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c071ea0 msgr2=0x7fb07c114480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c071ea0 0x7fb07c114480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb06800b3b0 tx=0x7fb068009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 shutdown_connections 2024-04-06T03:45:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb06c06dca0 0x7fb06c070150 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb07c0708f0 0x7fb07c113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 --2- 172.21.15.94:0/2768833018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c071ea0 0x7fb07c114480 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 >> 172.21.15.94:0/2768833018 conn(0x7fb07c0f8580 msgr2=0x7fb07c0f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 shutdown_connections 2024-04-06T03:45:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.823+0000 7fb082b41700 1 -- 172.21.15.94:0/2768833018 wait complete. 2024-04-06T03:45:19.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.922+0000 7ff3e9cde700 1 -- 172.21.15.94:0/4229134776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e4066f70 msgr2=0x7ff3e40ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.922+0000 7ff3e9cde700 1 --2- 172.21.15.94:0/4229134776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e4066f70 0x7ff3e40ff9c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4004940 tx=0x7ff3d4009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:19.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.923+0000 7ff3e9cde700 1 -- 172.21.15.94:0/4229134776 shutdown_connections 2024-04-06T03:45:19.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.923+0000 7ff3e9cde700 1 --2- 172.21.15.94:0/4229134776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3e40fff00 0x7ff3e4102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.923+0000 7ff3e9cde700 1 --2- 172.21.15.94:0/4229134776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e4066f70 0x7ff3e40ff9c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.923+0000 7ff3e9cde700 1 -- 172.21.15.94:0/4229134776 >> 172.21.15.94:0/4229134776 conn(0x7ff3e40f7430 msgr2=0x7ff3e40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:19.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.923+0000 7ff3e9cde700 1 -- 172.21.15.94:0/4229134776 shutdown_connections 2024-04-06T03:45:19.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.923+0000 7ff3e9cde700 1 -- 172.21.15.94:0/4229134776 wait complete. 2024-04-06T03:45:19.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.923+0000 7ff3e9cde700 1 Processor -- start 2024-04-06T03:45:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.923+0000 7ff3e9cde700 1 -- start start 2024-04-06T03:45:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e9cde700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3e4066f70 0x7ff3e410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e9cde700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40fff00 0x7ff3e410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e9cde700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e410e4a0 con 0x7ff3e40fff00 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e9cde700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e410e610 con 0x7ff3e4066f70 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40fff00 0x7ff3e410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40fff00 0x7ff3e410dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60450/0 (socket says 172.21.15.94:60450) 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e2ffd700 1 -- 172.21.15.94:0/2058225238 learned_addr learned my addr 172.21.15.94:0/2058225238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e37fe700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3e4066f70 0x7ff3e410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e2ffd700 1 -- 172.21.15.94:0/2058225238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3e4066f70 msgr2=0x7ff3e410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e2ffd700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3e4066f70 0x7ff3e410d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e2ffd700 1 -- 172.21.15.94:0/2058225238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4005040 con 0x7ff3e40fff00 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e2ffd700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40fff00 0x7ff3e410dea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c830 tx=0x7ff3d8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3cbfff700 1 -- 172.21.15.94:0/2058225238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d800a760 con 0x7ff3e40fff00 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3cbfff700 1 -- 172.21.15.94:0/2058225238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3d800ad90 con 0x7ff3e40fff00 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e4113070 con 0x7ff3e40fff00 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3cbfff700 1 -- 172.21.15.94:0/2058225238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3d80113a0 con 0x7ff3e40fff00 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.924+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e4113630 con 0x7ff3e40fff00 2024-04-06T03:45:19.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.925+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e404c710 con 0x7ff3e40fff00 2024-04-06T03:45:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.926+0000 7ff3cbfff700 1 -- 172.21.15.94:0/2058225238 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7ff3d800a8c0 con 0x7ff3e40fff00 2024-04-06T03:45:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.926+0000 7ff3cbfff700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff3cc06dca0 0x7ff3cc070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.926+0000 7ff3cbfff700 1 -- 172.21.15.94:0/2058225238 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff3d808d4f0 con 0x7ff3e40fff00 2024-04-06T03:45:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.926+0000 7ff3e37fe700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff3cc06dca0 0x7ff3cc070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:19.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.927+0000 7ff3e37fe700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff3cc06dca0 0x7ff3cc070150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4004890 tx=0x7ff3d4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:19.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:19.931+0000 7ff3cbfff700 1 -- 172.21.15.94:0/2058225238 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff3d805a6d0 con 0x7ff3e40fff00 2024-04-06T03:45:20.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.206+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff3e4113af0 con 0x7ff3e40fff00 2024-04-06T03:45:20.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.206+0000 7ff3cbfff700 1 -- 172.21.15.94:0/2058225238 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+157 (secure 0 0 0) 0x7ff3d805a260 con 0x7ff3e40fff00 2024-04-06T03:45:20.209 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:45:20.209 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:45:20.210 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.208+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff3cc06dca0 msgr2=0x7ff3cc070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.208+0000 7ff3e9cde700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff3cc06dca0 0x7ff3cc070150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4004890 tx=0x7ff3d4009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.208+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40fff00 msgr2=0x7ff3e410dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.208+0000 7ff3e9cde700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40fff00 0x7ff3e410dea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c830 tx=0x7ff3d8004470 comp rx=0 tx=0).stop 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.208+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 shutdown_connections 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.208+0000 7ff3e9cde700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff3cc06dca0 0x7ff3cc070150 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.208+0000 7ff3e9cde700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3e4066f70 0x7ff3e410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.208+0000 7ff3e9cde700 1 --2- 172.21.15.94:0/2058225238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e40fff00 0x7ff3e410dea0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.208+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 >> 172.21.15.94:0/2058225238 conn(0x7ff3e40f7430 msgr2=0x7ff3e40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.209+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 shutdown_connections 2024-04-06T03:45:20.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:20.209+0000 7ff3e9cde700 1 -- 172.21.15.94:0/2058225238 wait complete. 2024-04-06T03:45:20.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:20 smithi136 ceph-mon[82061]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded) 2024-04-06T03:45:20.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:20 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2966638621' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:45:20.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:20 smithi136 ceph-mon[82061]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:20.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:20 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2058225238' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:45:20.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:20.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:45:20.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:20 smithi094 ceph-mon[94053]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/219 objects degraded (7.306%), 7 pgs degraded) 2024-04-06T03:45:20.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:20 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2966638621' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:45:20.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:20 smithi094 ceph-mon[94053]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:20.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:20 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2058225238' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:45:20.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:20.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:45:21.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:21 smithi136 ceph-mon[82061]: pgmap v86: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:21 smithi094 ceph-mon[94053]: pgmap v86: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:23.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:23 smithi136 ceph-mon[82061]: pgmap v87: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:23.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:23 smithi094 ceph-mon[94053]: pgmap v87: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:25.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:25 smithi136 ceph-mon[82061]: pgmap v88: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:25.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:25 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:25.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:25 smithi094 ceph-mon[94053]: pgmap v88: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:25.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:25 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:27.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:27 smithi136 ceph-mon[82061]: pgmap v89: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:27.657 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:27 smithi094 ceph-mon[94053]: pgmap v89: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:29.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:29 smithi136 ceph-mon[82061]: pgmap v90: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:29.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:29 smithi094 ceph-mon[94053]: pgmap v90: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:31.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:31 smithi136 ceph-mon[82061]: pgmap v91: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:31.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:31 smithi094 ceph-mon[94053]: pgmap v91: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:32.398 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:32 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-06T03:45:32.398 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:32 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-06T03:45:32.399 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:32 smithi094 ceph-mon[94053]: Upgrade: osd.2 is safe to restart 2024-04-06T03:45:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:32 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-06T03:45:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:32 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-06T03:45:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:32 smithi136 ceph-mon[82061]: Upgrade: osd.2 is safe to restart 2024-04-06T03:45:34.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:34 smithi094 ceph-mon[94053]: pgmap v92: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:34.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:34 smithi094 ceph-mon[94053]: Upgrade: Updating osd.2 2024-04-06T03:45:34.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:34.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-06T03:45:34.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:45:34.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:34 smithi094 ceph-mon[94053]: Deploying daemon osd.2 on smithi094 2024-04-06T03:45:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:34 smithi136 ceph-mon[82061]: pgmap v92: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:34 smithi136 ceph-mon[82061]: Upgrade: Updating osd.2 2024-04-06T03:45:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:34.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-06T03:45:34.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:45:34.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:34 smithi136 ceph-mon[82061]: Deploying daemon osd.2 on smithi094 2024-04-06T03:45:36.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:36 smithi094 ceph-mon[94053]: pgmap v93: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:36.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:45:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:36 smithi136 ceph-mon[82061]: pgmap v93: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:45:36.727 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:36 smithi094 systemd[1]: Stopping Ceph osd.2 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:45:37.168 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:36 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[62756]: 2024-04-06T03:45:36.724+0000 7f267a89a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:45:37.168 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:36 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[62756]: 2024-04-06T03:45:36.724+0000 7f267a89a700 -1 osd.2 58 *** Got signal Terminated *** 2024-04-06T03:45:37.169 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:36 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[62756]: 2024-04-06T03:45:36.724+0000 7f267a89a700 -1 osd.2 58 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:45:37.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:37 smithi136 ceph-mon[82061]: osd.2 marked itself down and dead 2024-04-06T03:45:37.665 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:37 smithi094 ceph-mon[94053]: osd.2 marked itself down and dead 2024-04-06T03:45:37.918 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:37 smithi094 podman[114938]: 2024-04-06 03:45:37.663126171 +0000 UTC m=+1.049172536 container died 50775006e09609d3108de9a9ddfece17d788977243569d97f5152cbf9ef3ba86 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=quincy-6699495, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:45:38.214 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:38 smithi094 podman[114938]: 2024-04-06 03:45:38.047675282 +0000 UTC m=+1.433721646 container cleanup 50775006e09609d3108de9a9ddfece17d788977243569d97f5152cbf9ef3ba86 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-6699495, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-06T03:45:38.214 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:38 smithi094 bash[114938]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2 2024-04-06T03:45:38.504 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:38 smithi094 ceph-mon[94053]: pgmap v94: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:38.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:38 smithi094 ceph-mon[94053]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:45:38.505 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:38 smithi094 ceph-mon[94053]: osdmap e59: 6 total, 5 up, 6 in 2024-04-06T03:45:38.505 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:38 smithi094 podman[114956]: 2024-04-06 03:45:38.307888943 +0000 UTC m=+0.648395880 container remove 50775006e09609d3108de9a9ddfece17d788977243569d97f5152cbf9ef3ba86 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=quincy-6699495, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-06T03:45:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:38 smithi136 ceph-mon[82061]: pgmap v94: 65 pgs: 65 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:38 smithi136 ceph-mon[82061]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:45:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:38 smithi136 ceph-mon[82061]: osdmap e59: 6 total, 5 up, 6 in 2024-04-06T03:45:39.559 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:39 smithi094 ceph-mon[94053]: osdmap e60: 6 total, 5 up, 6 in 2024-04-06T03:45:39.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:39 smithi136 ceph-mon[82061]: osdmap e60: 6 total, 5 up, 6 in 2024-04-06T03:45:39.842 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:39 smithi094 podman[115139]: 2024-04-06 03:45:39.626291472 +0000 UTC m=+0.648640035 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:45:39.842 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:39 smithi094 podman[115139]: 2024-04-06 03:45:39.724277572 +0000 UTC m=+0.746626134 container create 90d6df6192cd0fd7055fa7af6bc86bfcb8874a71d0183cd6498a8cb82fb37180 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-06T03:45:40.168 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:39 smithi094 podman[115139]: 2024-04-06 03:45:39.960444927 +0000 UTC m=+0.982793489 container init 90d6df6192cd0fd7055fa7af6bc86bfcb8874a71d0183cd6498a8cb82fb37180 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-06T03:45:40.168 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:39 smithi094 podman[115139]: 2024-04-06 03:45:39.963815506 +0000 UTC m=+0.986164069 container start 90d6df6192cd0fd7055fa7af6bc86bfcb8874a71d0183cd6498a8cb82fb37180 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:45:40.541 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:40 smithi094 ceph-mon[94053]: pgmap v97: 65 pgs: 9 stale+active+clean, 56 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:40.541 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:40 smithi094 podman[115139]: 2024-04-06 03:45:40.243810165 +0000 UTC m=+1.266158735 container attach 90d6df6192cd0fd7055fa7af6bc86bfcb8874a71d0183cd6498a8cb82fb37180 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:45:40.542 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:40 smithi094 podman[115139]: 2024-04-06 03:45:40.249811714 +0000 UTC m=+1.272160281 container died 90d6df6192cd0fd7055fa7af6bc86bfcb8874a71d0183cd6498a8cb82fb37180 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, ceph=True, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-06T03:45:40.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:40 smithi136 ceph-mon[82061]: pgmap v97: 65 pgs: 9 stale+active+clean, 56 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:41.168 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:40 smithi094 podman[115196]: 2024-04-06 03:45:40.813895562 +0000 UTC m=+0.558978523 container remove 90d6df6192cd0fd7055fa7af6bc86bfcb8874a71d0183cd6498a8cb82fb37180 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-9f19235, org.label-schema.license=GPLv2, ceph=True) 2024-04-06T03:45:41.168 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:40 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.2.service: Deactivated successfully. 2024-04-06T03:45:41.168 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:40 smithi094 systemd[1]: Stopped Ceph osd.2 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:45:41.168 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:40 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.2.service: Consumed 57.905s CPU time. 2024-04-06T03:45:41.168 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 systemd[1]: Starting Ceph osd.2 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:45:41.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:41 smithi094 ceph-mon[94053]: pgmap v98: 65 pgs: 3 active+undersized+degraded, 6 stale+active+clean, 5 active+undersized, 51 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 7/219 objects degraded (3.196%) 2024-04-06T03:45:41.512 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:41 smithi094 ceph-mon[94053]: Health check failed: Degraded data redundancy: 7/219 objects degraded (3.196%), 3 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:41.513 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 podman[115301]: 2024-04-06 03:45:41.15626799 +0000 UTC m=+0.018167170 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:45:41.513 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 podman[115301]: 2024-04-06 03:45:41.273574509 +0000 UTC m=+0.135473688 container create 4218b1dd937e37040bbc559cae0a044b7d336f11875bb9bb88b8a8d4cb4f3bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-06T03:45:41.513 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 podman[115301]: 2024-04-06 03:45:41.510507925 +0000 UTC m=+0.372407111 container init 4218b1dd937e37040bbc559cae0a044b7d336f11875bb9bb88b8a8d4cb4f3bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:45:41.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:41 smithi136 ceph-mon[82061]: pgmap v98: 65 pgs: 3 active+undersized+degraded, 6 stale+active+clean, 5 active+undersized, 51 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 7/219 objects degraded (3.196%) 2024-04-06T03:45:41.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:41 smithi136 ceph-mon[82061]: Health check failed: Degraded data redundancy: 7/219 objects degraded (3.196%), 3 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:41.918 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 podman[115301]: 2024-04-06 03:45:41.513617275 +0000 UTC m=+0.375516462 container start 4218b1dd937e37040bbc559cae0a044b7d336f11875bb9bb88b8a8d4cb4f3bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:45:41.918 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 podman[115301]: 2024-04-06 03:45:41.618104328 +0000 UTC m=+0.480003524 container attach 4218b1dd937e37040bbc559cae0a044b7d336f11875bb9bb88b8a8d4cb4f3bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:45:42.207 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate[115316]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-06T03:45:42.208 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate[115316]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-06T03:45:42.208 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 bash[115301]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-06T03:45:42.208 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 bash[115301]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-06T03:45:42.208 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 bash[115301]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-06T03:45:42.208 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 bash[115301]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-06T03:45:42.208 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 bash[115301]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-06T03:45:42.208 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 bash[115301]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-06T03:45:42.209 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 bash[115301]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-06T03:45:42.209 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate[115316]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-06T03:45:42.209 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate[115316]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-06T03:45:42.209 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate[115316]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-06T03:45:42.209 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate[115316]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-06T03:45:42.209 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate[115316]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-06T03:45:42.209 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:41 smithi094 podman[115301]: 2024-04-06 03:45:41.975613048 +0000 UTC m=+0.837512230 container died 4218b1dd937e37040bbc559cae0a044b7d336f11875bb9bb88b8a8d4cb4f3bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-06T03:45:42.918 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:42 smithi094 podman[115453]: 2024-04-06 03:45:42.65569142 +0000 UTC m=+0.675422882 container remove 4218b1dd937e37040bbc559cae0a044b7d336f11875bb9bb88b8a8d4cb4f3bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-activate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:45:42.918 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:42 smithi094 podman[115518]: 2024-04-06 03:45:42.773113761 +0000 UTC m=+0.020003233 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:45:42.918 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:42 smithi094 podman[115518]: 2024-04-06 03:45:42.890021233 +0000 UTC m=+0.136910683 container create eb8d21d9c0dc91216688cfdbce37f2c697a26d034c1ee39d123dc49eb62296f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-06T03:45:43.418 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:43 smithi094 podman[115518]: 2024-04-06 03:45:43.084415518 +0000 UTC m=+0.331304970 container init eb8d21d9c0dc91216688cfdbce37f2c697a26d034c1ee39d123dc49eb62296f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2, RELEASE=squid-9f19235, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=) 2024-04-06T03:45:43.418 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:43 smithi094 podman[115518]: 2024-04-06 03:45:43.087448789 +0000 UTC m=+0.334338246 container start eb8d21d9c0dc91216688cfdbce37f2c697a26d034c1ee39d123dc49eb62296f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, RELEASE=squid-9f19235, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-06T03:45:43.418 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:43 smithi094 ceph-osd[115537]: -- 172.21.15.94:0/46298448 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5568014db680 con 0x5568014a4c00 2024-04-06T03:45:43.418 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:43 smithi094 bash[115518]: eb8d21d9c0dc91216688cfdbce37f2c697a26d034c1ee39d123dc49eb62296f8 2024-04-06T03:45:43.419 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:43 smithi094 systemd[1]: Started Ceph osd.2 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:45:43.918 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:43 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[115533]: 2024-04-06T03:45:43.624+0000 7f4ae13317c0 -1 Falling back to public interface 2024-04-06T03:45:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:44 smithi094 ceph-mon[94053]: pgmap v99: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:44.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:45:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:44 smithi136 ceph-mon[82061]: pgmap v99: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:45:46.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:46 smithi094 ceph-mon[94053]: pgmap v100: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:46.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:46 smithi136 ceph-mon[82061]: pgmap v100: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:47.527 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:47 smithi094 ceph-mon[94053]: Health check update: Degraded data redundancy: 29/219 objects degraded (13.242%), 11 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:47.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:47 smithi136 ceph-mon[82061]: Health check update: Degraded data redundancy: 29/219 objects degraded (13.242%), 11 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:48.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:48 smithi136 ceph-mon[82061]: pgmap v101: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:48.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:48.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:48.661 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:48 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[115533]: 2024-04-06T03:45:48.323+0000 7f4ae13317c0 -1 osd.2 58 log_to_monitors true 2024-04-06T03:45:48.661 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:48 smithi094 ceph-mon[94053]: pgmap v101: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:48.661 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:48.661 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:49.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:49 smithi136 ceph-mon[82061]: from='osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-06T03:45:49.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:49 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:49.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:49 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:49.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:49 smithi094 ceph-mon[94053]: from='osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-06T03:45:49.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:49 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:49.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:49 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:50.140 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:45:49 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[115533]: 2024-04-06T03:45:49.943+0000 7f4ace781700 -1 osd.2 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:45:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 -- 172.21.15.94:0/766172937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fe430 msgr2=0x7fc9d00fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 --2- 172.21.15.94:0/766172937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fe430 0x7fc9d00fe840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0004940 tx=0x7fc9c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 -- 172.21.15.94:0/766172937 shutdown_connections 2024-04-06T03:45:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 --2- 172.21.15.94:0/766172937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d00fed80 0x7fc9d00faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 --2- 172.21.15.94:0/766172937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fe430 0x7fc9d00fe840 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 -- 172.21.15.94:0/766172937 >> 172.21.15.94:0/766172937 conn(0x7fc9d00f6bd0 msgr2=0x7fc9d00f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 -- 172.21.15.94:0/766172937 shutdown_connections 2024-04-06T03:45:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 -- 172.21.15.94:0/766172937 wait complete. 2024-04-06T03:45:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 Processor -- start 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.318+0000 7fc9d6978700 1 -- start start 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9d6978700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d00fe430 0x7fc9d010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9d6978700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fed80 0x7fc9d010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9d6978700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d010e450 con 0x7fc9d00fed80 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9d6978700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d010e5c0 con 0x7fc9d00fe430 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9cf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fed80 0x7fc9d010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9cf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fed80 0x7fc9d010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47858/0 (socket says 172.21.15.94:47858) 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9cf7fe700 1 -- 172.21.15.94:0/2878756354 learned_addr learned my addr 172.21.15.94:0/2878756354 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9cffff700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d00fe430 0x7fc9d010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9cf7fe700 1 -- 172.21.15.94:0/2878756354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d00fe430 msgr2=0x7fc9d010d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:50.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9cf7fe700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d00fe430 0x7fc9d010d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9cf7fe700 1 -- 172.21.15.94:0/2878756354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0005040 con 0x7fc9d00fed80 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9cffff700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d00fe430 0x7fc9d010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9cf7fe700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fed80 0x7fc9d010de50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c830 tx=0x7fc9c4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9ccff9700 1 -- 172.21.15.94:0/2878756354 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9c400a760 con 0x7fc9d00fed80 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.319+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d0113020 con 0x7fc9d00fed80 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.320+0000 7fc9ccff9700 1 -- 172.21.15.94:0/2878756354 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9c400ad90 con 0x7fc9d00fed80 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.320+0000 7fc9ccff9700 1 -- 172.21.15.94:0/2878756354 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9c40113a0 con 0x7fc9d00fed80 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.320+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d01135b0 con 0x7fc9d00fed80 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.320+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d004c710 con 0x7fc9d00fed80 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.321+0000 7fc9ccff9700 1 -- 172.21.15.94:0/2878756354 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fc9c4012400 con 0x7fc9d00fed80 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.321+0000 7fc9ccff9700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc9b806dca0 0x7fc9b8070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:50.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.321+0000 7fc9ccff9700 1 -- 172.21.15.94:0/2878756354 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc9c408d420 con 0x7fc9d00fed80 2024-04-06T03:45:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.321+0000 7fc9cffff700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc9b806dca0 0x7fc9b8070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.322+0000 7fc9cffff700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc9b806dca0 0x7fc9b8070150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000b970 tx=0x7fc9c0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:50.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.326+0000 7fc9ccff9700 1 -- 172.21.15.94:0/2878756354 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc9c405a6a0 con 0x7fc9d00fed80 2024-04-06T03:45:50.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:50 smithi094 ceph-mon[94053]: pgmap v102: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:50.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:50 smithi094 ceph-mon[94053]: from='osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-06T03:45:50.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:50 smithi094 ceph-mon[94053]: osdmap e61: 6 total, 5 up, 6 in 2024-04-06T03:45:50.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:50 smithi094 ceph-mon[94053]: from='osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:45:50.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:50 smithi094 ceph-mon[94053]: from='osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192]' entity='osd.2' 2024-04-06T03:45:50.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:50 smithi136 ceph-mon[82061]: pgmap v102: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:50.500 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:50 smithi136 ceph-mon[82061]: from='osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-06T03:45:50.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:50 smithi136 ceph-mon[82061]: osdmap e61: 6 total, 5 up, 6 in 2024-04-06T03:45:50.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:50 smithi136 ceph-mon[82061]: from='osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-06T03:45:50.501 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:50 smithi136 ceph-mon[82061]: from='osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192]' entity='osd.2' 2024-04-06T03:45:50.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.550+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9d01024d0 con 0x7fc9b806dca0 2024-04-06T03:45:50.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.551+0000 7fc9ccff9700 1 -- 172.21.15.94:0/2878756354 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc9d01024d0 con 0x7fc9b806dca0 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc9b806dca0 msgr2=0x7fc9b8070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc9b806dca0 0x7fc9b8070150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000b970 tx=0x7fc9c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fed80 msgr2=0x7fc9d010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fed80 0x7fc9d010de50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c830 tx=0x7fc9c4004470 comp rx=0 tx=0).stop 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 shutdown_connections 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc9b806dca0 0x7fc9b8070150 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d00fe430 0x7fc9d010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 --2- 172.21.15.94:0/2878756354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d00fed80 0x7fc9d010de50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 >> 172.21.15.94:0/2878756354 conn(0x7fc9d00f6bd0 msgr2=0x7fc9d00f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 shutdown_connections 2024-04-06T03:45:50.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.554+0000 7fc9d6978700 1 -- 172.21.15.94:0/2878756354 wait complete. 2024-04-06T03:45:50.567 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:45:50.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.657+0000 7fc28a5f0700 1 -- 172.21.15.94:0/2176125799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fbcd0 msgr2=0x7fc2840fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:50.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.657+0000 7fc28a5f0700 1 --2- 172.21.15.94:0/2176125799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fbcd0 0x7fc2840fc0e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc274004940 tx=0x7fc274009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:50.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.657+0000 7fc28a5f0700 1 -- 172.21.15.94:0/2176125799 shutdown_connections 2024-04-06T03:45:50.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.657+0000 7fc28a5f0700 1 --2- 172.21.15.94:0/2176125799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2840fcf80 0x7fc2840fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.657+0000 7fc28a5f0700 1 --2- 172.21.15.94:0/2176125799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fbcd0 0x7fc2840fc0e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.657+0000 7fc28a5f0700 1 -- 172.21.15.94:0/2176125799 >> 172.21.15.94:0/2176125799 conn(0x7fc2840f7430 msgr2=0x7fc2840f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:50.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.657+0000 7fc28a5f0700 1 -- 172.21.15.94:0/2176125799 shutdown_connections 2024-04-06T03:45:50.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.657+0000 7fc28a5f0700 1 -- 172.21.15.94:0/2176125799 wait complete. 2024-04-06T03:45:50.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.657+0000 7fc28a5f0700 1 Processor -- start 2024-04-06T03:45:50.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc28a5f0700 1 -- start start 2024-04-06T03:45:50.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc28a5f0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2840fbcd0 0x7fc28410c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:50.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc28a5f0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fcf80 0x7fc28410ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:50.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc28a5f0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28410d4e0 con 0x7fc2840fcf80 2024-04-06T03:45:50.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc28a5f0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28410dcd0 con 0x7fc2840fbcd0 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc2837fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fcf80 0x7fc28410ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc2837fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fcf80 0x7fc28410ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47888/0 (socket says 172.21.15.94:47888) 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc2837fe700 1 -- 172.21.15.94:0/1140112384 learned_addr learned my addr 172.21.15.94:0/1140112384 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc283fff700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2840fbcd0 0x7fc28410c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc2837fe700 1 -- 172.21.15.94:0/1140112384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2840fbcd0 msgr2=0x7fc28410c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc2837fe700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2840fbcd0 0x7fc28410c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc2837fe700 1 -- 172.21.15.94:0/1140112384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274005040 con 0x7fc2840fcf80 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc283fff700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2840fbcd0 0x7fc28410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc2837fe700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fcf80 0x7fc28410ce50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc27800b790 tx=0x7fc278009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc280ff9700 1 -- 172.21.15.94:0/1140112384 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc27800f770 con 0x7fc2840fcf80 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc280ff9700 1 -- 172.21.15.94:0/1140112384 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc27800fda0 con 0x7fc2840fcf80 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28410dfb0 con 0x7fc2840fcf80 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.658+0000 7fc280ff9700 1 -- 172.21.15.94:0/1140112384 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc27800d310 con 0x7fc2840fcf80 2024-04-06T03:45:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.659+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28410e570 con 0x7fc2840fcf80 2024-04-06T03:45:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.660+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28405f5f0 con 0x7fc2840fcf80 2024-04-06T03:45:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.660+0000 7fc280ff9700 1 -- 172.21.15.94:0/1140112384 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fc27800d4d0 con 0x7fc2840fcf80 2024-04-06T03:45:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.660+0000 7fc280ff9700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc26c06dca0 0x7fc26c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.661+0000 7fc283fff700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc26c06dca0 0x7fc26c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.661+0000 7fc280ff9700 1 -- 172.21.15.94:0/1140112384 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc27808d510 con 0x7fc2840fcf80 2024-04-06T03:45:50.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.661+0000 7fc283fff700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc26c06dca0 0x7fc26c070150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc274000c00 tx=0x7fc274009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:50.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.665+0000 7fc280ff9700 1 -- 172.21.15.94:0/1140112384 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc27805a760 con 0x7fc2840fcf80 2024-04-06T03:45:50.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.884+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc28410e860 con 0x7fc26c06dca0 2024-04-06T03:45:50.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.885+0000 7fc280ff9700 1 -- 172.21.15.94:0/1140112384 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc28410e860 con 0x7fc26c06dca0 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc26c06dca0 msgr2=0x7fc26c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc26c06dca0 0x7fc26c070150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc274000c00 tx=0x7fc274009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fcf80 msgr2=0x7fc28410ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fcf80 0x7fc28410ce50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc27800b790 tx=0x7fc278009250 comp rx=0 tx=0).stop 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 shutdown_connections 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc26c06dca0 0x7fc26c070150 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2840fbcd0 0x7fc28410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 --2- 172.21.15.94:0/1140112384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2840fcf80 0x7fc28410ce50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 >> 172.21.15.94:0/1140112384 conn(0x7fc2840f7430 msgr2=0x7fc2840f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 shutdown_connections 2024-04-06T03:45:50.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:50.888+0000 7fc28a5f0700 1 -- 172.21.15.94:0/1140112384 wait complete. 2024-04-06T03:45:51.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 -- 172.21.15.94:0/3222853862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 msgr2=0x7fc9880ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 --2- 172.21.15.94:0/3222853862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 0x7fc9880ffab0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc980004940 tx=0x7fc980009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:51.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 -- 172.21.15.94:0/3222853862 shutdown_connections 2024-04-06T03:45:51.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 --2- 172.21.15.94:0/3222853862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9880ffff0 0x7fc9881023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 --2- 172.21.15.94:0/3222853862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 0x7fc9880ffab0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 -- 172.21.15.94:0/3222853862 >> 172.21.15.94:0/3222853862 conn(0x7fc9880f74e0 msgr2=0x7fc9880f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:51.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 -- 172.21.15.94:0/3222853862 shutdown_connections 2024-04-06T03:45:51.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 -- 172.21.15.94:0/3222853862 wait complete. 2024-04-06T03:45:51.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 Processor -- start 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 -- start start 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 0x7fc98810dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9880ffff0 0x7fc98810e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98810e6b0 con 0x7fc988066b10 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.015+0000 7fc98d317700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98810ee50 con 0x7fc9880ffff0 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc986ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 0x7fc98810dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc986ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 0x7fc98810dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47906/0 (socket says 172.21.15.94:47906) 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc986ffd700 1 -- 172.21.15.94:0/391140815 learned_addr learned my addr 172.21.15.94:0/391140815 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc9867fc700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9880ffff0 0x7fc98810e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc986ffd700 1 -- 172.21.15.94:0/391140815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9880ffff0 msgr2=0x7fc98810e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc986ffd700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9880ffff0 0x7fc98810e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc986ffd700 1 -- 172.21.15.94:0/391140815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980005040 con 0x7fc988066b10 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc986ffd700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 0x7fc98810dae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc98000b970 tx=0x7fc980009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc97f7fe700 1 -- 172.21.15.94:0/391140815 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc98000e040 con 0x7fc988066b10 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98810f0d0 con 0x7fc988066b10 2024-04-06T03:45:51.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc97f7fe700 1 -- 172.21.15.94:0/391140815 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc98000bbf0 con 0x7fc988066b10 2024-04-06T03:45:51.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc97f7fe700 1 -- 172.21.15.94:0/391140815 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc98001c3f0 con 0x7fc988066b10 2024-04-06T03:45:51.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.016+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98810f690 con 0x7fc988066b10 2024-04-06T03:45:51.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.017+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98805f5f0 con 0x7fc988066b10 2024-04-06T03:45:51.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.018+0000 7fc97f7fe700 1 -- 172.21.15.94:0/391140815 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fc98000bd60 con 0x7fc988066b10 2024-04-06T03:45:51.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.018+0000 7fc97f7fe700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc97406dcb0 0x7fc974070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:51.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.018+0000 7fc97f7fe700 1 -- 172.21.15.94:0/391140815 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc98008e7b0 con 0x7fc988066b10 2024-04-06T03:45:51.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.018+0000 7fc9867fc700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc97406dcb0 0x7fc974070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:51.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.018+0000 7fc9867fc700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc97406dcb0 0x7fc974070160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc970007580 tx=0x7fc970009780 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:51.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.023+0000 7fc97f7fe700 1 -- 172.21.15.94:0/391140815 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc98005b990 con 0x7fc988066b10 2024-04-06T03:45:51.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.235+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc98810f970 con 0x7fc97406dcb0 2024-04-06T03:45:51.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.245+0000 7fc97f7fe700 1 -- 172.21.15.94:0/391140815 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc98810f970 con 0x7fc97406dcb0 2024-04-06T03:45:51.248 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:45:51.248 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (4m) 3s ago 13m 27.6M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (109s) 3s ago 13m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (104s) 101s ago 12m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 3s ago 12m - - 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (7m) 3s ago 7m 15.0M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (7m) 3s ago 7m 153M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (7m) 101s ago 7m 14.6M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (7m) 101s ago 7m 17.0M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (4m) 3s ago 14m 502M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (3m) 101s ago 12m 449M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (2m) 3s ago 14m 46.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (2m) 101s ago 12m 36.7M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (5m) 3s ago 13m 15.3M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (4m) 101s ago 12m 17.3M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (82s) 3s ago 11m 103M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:45:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (45s) 3s ago 11m 107M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:45:51.250 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (8s) 3s ago 10m 12.0M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:45:51.250 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (10m) 101s ago 10m 292M 2581M 17.2.7-819-g6699495c ec300eaa768b 49aad532cde6 2024-04-06T03:45:51.250 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (9m) 101s ago 9m 257M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:45:51.250 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (9m) 101s ago 9m 241M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:45:51.250 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (3m) 3s ago 12m 57.0M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc97406dcb0 msgr2=0x7fc974070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc97406dcb0 0x7fc974070160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc970007580 tx=0x7fc970009780 comp rx=0 tx=0).stop 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 msgr2=0x7fc98810dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 0x7fc98810dae0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc98000b970 tx=0x7fc980009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 shutdown_connections 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc988066b10 0x7fc98810dae0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc97406dcb0 0x7fc974070160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 --2- 172.21.15.94:0/391140815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9880ffff0 0x7fc98810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 >> 172.21.15.94:0/391140815 conn(0x7fc9880f74e0 msgr2=0x7fc9880f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 shutdown_connections 2024-04-06T03:45:51.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.248+0000 7fc98d317700 1 -- 172.21.15.94:0/391140815 wait complete. 2024-04-06T03:45:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.351+0000 7feafc747700 1 -- 172.21.15.94:0/3633661412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fe400 msgr2=0x7feaf40fe810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.351+0000 7feafc747700 1 --2- 172.21.15.94:0/3633661412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fe400 0x7feaf40fe810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7feae4004940 tx=0x7feae4009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.351+0000 7feafc747700 1 -- 172.21.15.94:0/3633661412 shutdown_connections 2024-04-06T03:45:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.351+0000 7feafc747700 1 --2- 172.21.15.94:0/3633661412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf40fed50 0x7feaf40fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.351+0000 7feafc747700 1 --2- 172.21.15.94:0/3633661412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fe400 0x7feaf40fe810 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.351+0000 7feafc747700 1 -- 172.21.15.94:0/3633661412 >> 172.21.15.94:0/3633661412 conn(0x7feaf40f6ba0 msgr2=0x7feaf40f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.351+0000 7feafc747700 1 -- 172.21.15.94:0/3633661412 shutdown_connections 2024-04-06T03:45:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.351+0000 7feafc747700 1 -- 172.21.15.94:0/3633661412 wait complete. 2024-04-06T03:45:51.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feafc747700 1 Processor -- start 2024-04-06T03:45:51.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feafc747700 1 -- start start 2024-04-06T03:45:51.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feafc747700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf40fe400 0x7feaf410d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:51.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feafc747700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fed50 0x7feaf410df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:51.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feafc747700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf410e530 con 0x7feaf40fed50 2024-04-06T03:45:51.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feafc747700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf410ed80 con 0x7feaf40fe400 2024-04-06T03:45:51.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feaf9ce2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fed50 0x7feaf410df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feaf9ce2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fed50 0x7feaf410df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47924/0 (socket says 172.21.15.94:47924) 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feaf9ce2700 1 -- 172.21.15.94:0/1348275153 learned_addr learned my addr 172.21.15.94:0/1348275153 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feafa4e3700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf40fe400 0x7feaf410d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feaf9ce2700 1 -- 172.21.15.94:0/1348275153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf40fe400 msgr2=0x7feaf410d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feaf9ce2700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf40fe400 0x7feaf410d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feaf9ce2700 1 -- 172.21.15.94:0/1348275153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae4005040 con 0x7feaf40fed50 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feafa4e3700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf40fe400 0x7feaf410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feaf9ce2700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fed50 0x7feaf410df30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feaf000c8c0 tx=0x7feaf00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.352+0000 7feaeaffd700 1 -- 172.21.15.94:0/1348275153 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaf000a980 con 0x7feaf40fed50 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.353+0000 7feaeaffd700 1 -- 172.21.15.94:0/1348275153 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaf000ce60 con 0x7feaf40fed50 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.353+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf410f060 con 0x7feaf40fed50 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.353+0000 7feaeaffd700 1 -- 172.21.15.94:0/1348275153 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaf00115e0 con 0x7feaf40fed50 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.353+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf410f5a0 con 0x7feaf40fed50 2024-04-06T03:45:51.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.353+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf405f5f0 con 0x7feaf40fed50 2024-04-06T03:45:51.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.354+0000 7feaeaffd700 1 -- 172.21.15.94:0/1348275153 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7feaf0012400 con 0x7feaf40fed50 2024-04-06T03:45:51.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.354+0000 7feaeaffd700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feae006dca0 0x7feae0070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:51.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.355+0000 7feaeaffd700 1 -- 172.21.15.94:0/1348275153 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7feaf008d5a0 con 0x7feaf40fed50 2024-04-06T03:45:51.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.355+0000 7feafa4e3700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feae006dca0 0x7feae0070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:51.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.355+0000 7feafa4e3700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feae006dca0 0x7feae0070150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feae4000c00 tx=0x7feae4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:51.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.359+0000 7feaeaffd700 1 -- 172.21.15.94:0/1348275153 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feaf005a780 con 0x7feaf40fed50 2024-04-06T03:45:51.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:51.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:45:51.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:51 smithi094 ceph-mon[94053]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:45:51.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:51 smithi094 ceph-mon[94053]: osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192] boot 2024-04-06T03:45:51.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:51 smithi094 ceph-mon[94053]: osdmap e62: 6 total, 6 up, 6 in 2024-04-06T03:45:51.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:45:51.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:51.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:45:51.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:51 smithi136 ceph-mon[82061]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:45:51.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:51 smithi136 ceph-mon[82061]: osd.2 [v2:172.21.15.94:6818/1970793192,v1:172.21.15.94:6819/1970793192] boot 2024-04-06T03:45:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:51 smithi136 ceph-mon[82061]: osdmap e62: 6 total, 6 up, 6 in 2024-04-06T03:45:51.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-06T03:45:51.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.655+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feaf410f1f0 con 0x7feaf40fed50 2024-04-06T03:45:51.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.656+0000 7feaeaffd700 1 -- 172.21.15.94:0/1348275153 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7feaf410f1f0 con 0x7feaf40fed50 2024-04-06T03:45:51.721 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:45:51.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:45:51.721 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:45:51.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:45:51.721 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:45:51.721 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:45:51.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:45:51.721 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 3, 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 3 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 7, 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 7 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:45:51.722 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:45:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feae006dca0 msgr2=0x7feae0070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feae006dca0 0x7feae0070150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feae4000c00 tx=0x7feae4009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fed50 msgr2=0x7feaf410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fed50 0x7feaf410df30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feaf000c8c0 tx=0x7feaf00044c0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 shutdown_connections 2024-04-06T03:45:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feae006dca0 0x7feae0070150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feaf40fe400 0x7feaf410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 --2- 172.21.15.94:0/1348275153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf40fed50 0x7feaf410df30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 >> 172.21.15.94:0/1348275153 conn(0x7feaf40f6ba0 msgr2=0x7feaf40f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 shutdown_connections 2024-04-06T03:45:51.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.658+0000 7feafc747700 1 -- 172.21.15.94:0/1348275153 wait complete. 2024-04-06T03:45:51.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.764+0000 7fc15a690700 1 -- 172.21.15.94:0/2002713687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 msgr2=0x7fc1540fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.764+0000 7fc15a690700 1 --2- 172.21.15.94:0/2002713687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 0x7fc1540fc210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc144004940 tx=0x7fc144009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.764+0000 7fc15a690700 1 -- 172.21.15.94:0/2002713687 shutdown_connections 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.764+0000 7fc15a690700 1 --2- 172.21.15.94:0/2002713687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1540fcf70 0x7fc1540fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.764+0000 7fc15a690700 1 --2- 172.21.15.94:0/2002713687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 0x7fc1540fc210 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.764+0000 7fc15a690700 1 -- 172.21.15.94:0/2002713687 >> 172.21.15.94:0/2002713687 conn(0x7fc1540f74e0 msgr2=0x7fc1540f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.764+0000 7fc15a690700 1 -- 172.21.15.94:0/2002713687 shutdown_connections 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.765+0000 7fc15a690700 1 -- 172.21.15.94:0/2002713687 wait complete. 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.765+0000 7fc15a690700 1 Processor -- start 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.765+0000 7fc15a690700 1 -- start start 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.765+0000 7fc15a690700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 0x7fc15410dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc15a690700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1540fcf70 0x7fc15410e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc15a690700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15410e610 con 0x7fc1540fbe00 2024-04-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc15a690700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15410edc0 con 0x7fc1540fcf70 2024-04-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc153fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 0x7fc15410dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc153fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 0x7fc15410dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47936/0 (socket says 172.21.15.94:47936) 2024-04-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc153fff700 1 -- 172.21.15.94:0/2997797191 learned_addr learned my addr 172.21.15.94:0/2997797191 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc1537fe700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1540fcf70 0x7fc15410e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc153fff700 1 -- 172.21.15.94:0/2997797191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1540fcf70 msgr2=0x7fc15410e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc153fff700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1540fcf70 0x7fc15410e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc153fff700 1 -- 172.21.15.94:0/2997797191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc144005040 con 0x7fc1540fbe00 2024-04-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc153fff700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 0x7fc15410dad0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc144009150 tx=0x7fc144009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc150ff9700 1 -- 172.21.15.94:0/2997797191 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc14400e040 con 0x7fc1540fbe00 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15410f040 con 0x7fc1540fbe00 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc150ff9700 1 -- 172.21.15.94:0/2997797191 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc14400bbf0 con 0x7fc1540fbe00 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.766+0000 7fc150ff9700 1 -- 172.21.15.94:0/2997797191 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc14401c630 con 0x7fc1540fbe00 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.767+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15410f580 con 0x7fc1540fbe00 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.768+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc154101b00 con 0x7fc1540fbe00 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.768+0000 7fc150ff9700 1 -- 172.21.15.94:0/2997797191 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fc14400bd60 con 0x7fc1540fbe00 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.769+0000 7fc150ff9700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc13c06dcb0 0x7fc13c070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.769+0000 7fc150ff9700 1 -- 172.21.15.94:0/2997797191 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc14408e7d0 con 0x7fc1540fbe00 2024-04-06T03:45:51.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.769+0000 7fc1537fe700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc13c06dcb0 0x7fc13c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:51.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.769+0000 7fc1537fe700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc13c06dcb0 0x7fc13c070160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc148004990 tx=0x7fc148009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:51.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:51.774+0000 7fc150ff9700 1 -- 172.21.15.94:0/2997797191 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc14405ba50 con 0x7fc1540fbe00 2024-04-06T03:45:52.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.096+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc1541048f0 con 0x7fc1540fbe00 2024-04-06T03:45:52.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.097+0000 7fc150ff9700 1 -- 172.21.15.94:0/2997797191 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fc14405b5e0 con 0x7fc1540fbe00 2024-04-06T03:45:52.100 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:45:52.100 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:45:52.101 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:45:52.102 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:45:52.103 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:45:52.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.100+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc13c06dcb0 msgr2=0x7fc13c070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.100+0000 7fc15a690700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc13c06dcb0 0x7fc13c070160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc148004990 tx=0x7fc148009ee0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.100+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 msgr2=0x7fc15410dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.100+0000 7fc15a690700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 0x7fc15410dad0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc144009150 tx=0x7fc144009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:52.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.100+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 shutdown_connections 2024-04-06T03:45:52.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.100+0000 7fc15a690700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1540fbe00 0x7fc15410dad0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.100+0000 7fc15a690700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc13c06dcb0 0x7fc13c070160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.100+0000 7fc15a690700 1 --2- 172.21.15.94:0/2997797191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1540fcf70 0x7fc15410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.100+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 >> 172.21.15.94:0/2997797191 conn(0x7fc1540f74e0 msgr2=0x7fc154066430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:52.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.101+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 shutdown_connections 2024-04-06T03:45:52.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.101+0000 7fc15a690700 1 -- 172.21.15.94:0/2997797191 wait complete. 2024-04-06T03:45:52.105 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:45:52.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.212+0000 7f8a86ad8700 1 -- 172.21.15.94:0/4141868252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fcfa0 msgr2=0x7f8a800fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.212+0000 7f8a86ad8700 1 --2- 172.21.15.94:0/4141868252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fcfa0 0x7f8a800fd410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a68004940 tx=0x7f8a68009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.212+0000 7f8a86ad8700 1 -- 172.21.15.94:0/4141868252 shutdown_connections 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.212+0000 7f8a86ad8700 1 --2- 172.21.15.94:0/4141868252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fcfa0 0x7f8a800fd410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.212+0000 7f8a86ad8700 1 --2- 172.21.15.94:0/4141868252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a800fbcf0 0x7f8a800fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.212+0000 7f8a86ad8700 1 -- 172.21.15.94:0/4141868252 >> 172.21.15.94:0/4141868252 conn(0x7f8a800f7450 msgr2=0x7f8a800f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.212+0000 7f8a86ad8700 1 -- 172.21.15.94:0/4141868252 shutdown_connections 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.212+0000 7f8a86ad8700 1 -- 172.21.15.94:0/4141868252 wait complete. 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.212+0000 7f8a86ad8700 1 Processor -- start 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a86ad8700 1 -- start start 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a86ad8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fbcf0 0x7f8a8010dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a86ad8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a800fcfa0 0x7f8a8010dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a86ad8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8010e680 con 0x7f8a800fbcf0 2024-04-06T03:45:52.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a86ad8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8010edb0 con 0x7f8a800fcfa0 2024-04-06T03:45:52.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a84874700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fbcf0 0x7f8a8010dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:52.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a84874700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fbcf0 0x7f8a8010dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47950/0 (socket says 172.21.15.94:47950) 2024-04-06T03:45:52.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a84874700 1 -- 172.21.15.94:0/2141033686 learned_addr learned my addr 172.21.15.94:0/2141033686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:52.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a7ffff700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a800fcfa0 0x7f8a8010dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:52.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a84874700 1 -- 172.21.15.94:0/2141033686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a800fcfa0 msgr2=0x7f8a8010dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a84874700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a800fcfa0 0x7f8a8010dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.213+0000 7f8a84874700 1 -- 172.21.15.94:0/2141033686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a68005040 con 0x7f8a800fbcf0 2024-04-06T03:45:52.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.214+0000 7f8a84874700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fbcf0 0x7f8a8010dab0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800bbb0 tx=0x7f8a78009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:52.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.214+0000 7f8a7d7fa700 1 -- 172.21.15.94:0/2141033686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a780092e0 con 0x7f8a800fbcf0 2024-04-06T03:45:52.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.214+0000 7f8a7d7fa700 1 -- 172.21.15.94:0/2141033686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a7800cbd0 con 0x7f8a800fbcf0 2024-04-06T03:45:52.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.214+0000 7f8a7d7fa700 1 -- 172.21.15.94:0/2141033686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a78007400 con 0x7f8a800fbcf0 2024-04-06T03:45:52.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.214+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8010f090 con 0x7f8a800fbcf0 2024-04-06T03:45:52.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.214+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8010f650 con 0x7f8a800fbcf0 2024-04-06T03:45:52.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.215+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a80101b30 con 0x7f8a800fbcf0 2024-04-06T03:45:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.215+0000 7f8a7d7fa700 1 -- 172.21.15.94:0/2141033686 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f8a78012070 con 0x7f8a800fbcf0 2024-04-06T03:45:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.216+0000 7f8a7d7fa700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8a6c06dcb0 0x7f8a6c070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.216+0000 7f8a7d7fa700 1 -- 172.21.15.94:0/2141033686 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8a7808d0a0 con 0x7f8a800fbcf0 2024-04-06T03:45:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.216+0000 7f8a7ffff700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8a6c06dcb0 0x7f8a6c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:52.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.217+0000 7f8a7ffff700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8a6c06dcb0 0x7f8a6c070160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a68004f90 tx=0x7f8a68009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:52.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.221+0000 7f8a7d7fa700 1 -- 172.21.15.94:0/2141033686 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8a7805a280 con 0x7f8a800fbcf0 2024-04-06T03:45:52.372 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:52 smithi094 ceph-mon[94053]: pgmap v104: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:52.373 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:52 smithi094 ceph-mon[94053]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:52.373 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:52 smithi094 ceph-mon[94053]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:52.373 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:52 smithi094 ceph-mon[94053]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:52.373 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:52 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1348275153' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:52.373 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:52 smithi094 ceph-mon[94053]: osdmap e63: 6 total, 6 up, 6 in 2024-04-06T03:45:52.373 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:52 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2997797191' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:45:52.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.448+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a8010f8c0 con 0x7f8a6c06dcb0 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.449+0000 7f8a7d7fa700 1 -- 172.21.15.94:0/2141033686 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8a8010f8c0 con 0x7f8a6c06dcb0 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:45:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "9/21 daemons upgraded", 2024-04-06T03:45:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-04-06T03:45:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:45:52.453 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.451+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8a6c06dcb0 msgr2=0x7f8a6c070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.451+0000 7f8a86ad8700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8a6c06dcb0 0x7f8a6c070160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a68004f90 tx=0x7f8a68009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.451+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fbcf0 msgr2=0x7f8a8010dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.451+0000 7f8a86ad8700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fbcf0 0x7f8a8010dab0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800bbb0 tx=0x7f8a78009840 comp rx=0 tx=0).stop 2024-04-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.452+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 shutdown_connections 2024-04-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.452+0000 7f8a86ad8700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a800fbcf0 0x7f8a8010dab0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.452+0000 7f8a86ad8700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8a6c06dcb0 0x7f8a6c070160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.452+0000 7f8a86ad8700 1 --2- 172.21.15.94:0/2141033686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a800fcfa0 0x7f8a8010dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.452+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 >> 172.21.15.94:0/2141033686 conn(0x7f8a800f7450 msgr2=0x7f8a800663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.452+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 shutdown_connections 2024-04-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.452+0000 7f8a86ad8700 1 -- 172.21.15.94:0/2141033686 wait complete. 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.559+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3898195292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740662e0 msgr2=0x7f4d740ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.559+0000 7f4d7bc41700 1 --2- 172.21.15.94:0/3898195292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740662e0 0x7f4d740ff9c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4d70004940 tx=0x7f4d70009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.559+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3898195292 shutdown_connections 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.559+0000 7f4d7bc41700 1 --2- 172.21.15.94:0/3898195292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d740fff00 0x7f4d74102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.559+0000 7f4d7bc41700 1 --2- 172.21.15.94:0/3898195292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740662e0 0x7f4d740ff9c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.559+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3898195292 >> 172.21.15.94:0/3898195292 conn(0x7f4d740f7430 msgr2=0x7f4d740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.559+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3898195292 shutdown_connections 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.559+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3898195292 wait complete. 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d7bc41700 1 Processor -- start 2024-04-06T03:45:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d7bc41700 1 -- start start 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d7bc41700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d740662e0 0x7f4d7410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d7bc41700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740fff00 0x7f4d7410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d7bc41700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7410e570 con 0x7f4d740fff00 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d7bc41700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7410edc0 con 0x7f4d740662e0 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d791dc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740fff00 0x7f4d7410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d791dc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740fff00 0x7f4d7410df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47962/0 (socket says 172.21.15.94:47962) 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d791dc700 1 -- 172.21.15.94:0/3180198592 learned_addr learned my addr 172.21.15.94:0/3180198592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d799dd700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d740662e0 0x7f4d7410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d791dc700 1 -- 172.21.15.94:0/3180198592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d740662e0 msgr2=0x7f4d7410da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d791dc700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d740662e0 0x7f4d7410da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d791dc700 1 -- 172.21.15.94:0/3180198592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d70005040 con 0x7f4d740fff00 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.560+0000 7f4d791dc700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740fff00 0x7f4d7410df70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d64002a50 tx=0x7f4d640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.561+0000 7f4d6a7fc700 1 -- 172.21.15.94:0/3180198592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d64007cc0 con 0x7f4d740fff00 2024-04-06T03:45:52.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.561+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7410f0a0 con 0x7f4d740fff00 2024-04-06T03:45:52.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.561+0000 7f4d6a7fc700 1 -- 172.21.15.94:0/3180198592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d6400f040 con 0x7f4d740fff00 2024-04-06T03:45:52.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.561+0000 7f4d6a7fc700 1 -- 172.21.15.94:0/3180198592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d6400da30 con 0x7f4d740fff00 2024-04-06T03:45:52.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.561+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7410f5e0 con 0x7f4d740fff00 2024-04-06T03:45:52.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.562+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7405f5f0 con 0x7f4d740fff00 2024-04-06T03:45:52.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.562+0000 7f4d6a7fc700 1 -- 172.21.15.94:0/3180198592 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f4d6400db90 con 0x7f4d740fff00 2024-04-06T03:45:52.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.563+0000 7f4d6a7fc700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d6006dca0 0x7f4d60070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:45:52.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.563+0000 7f4d799dd700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d6006dca0 0x7f4d60070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:45:52.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.563+0000 7f4d6a7fc700 1 -- 172.21.15.94:0/3180198592 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4d6408d180 con 0x7f4d740fff00 2024-04-06T03:45:52.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.563+0000 7f4d799dd700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d6006dca0 0x7f4d60070150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d70004890 tx=0x7f4d70009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:45:52.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.568+0000 7f4d6a7fc700 1 -- 172.21.15.94:0/3180198592 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4d6405a360 con 0x7f4d740fff00 2024-04-06T03:45:52.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:52 smithi136 ceph-mon[82061]: pgmap v104: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29/219 objects degraded (13.242%) 2024-04-06T03:45:52.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:52 smithi136 ceph-mon[82061]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:52.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:52 smithi136 ceph-mon[82061]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:52.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:52 smithi136 ceph-mon[82061]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:52.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:52 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1348275153' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:52.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:52 smithi136 ceph-mon[82061]: osdmap e63: 6 total, 6 up, 6 in 2024-04-06T03:45:52.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:52 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2997797191' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:45:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.854+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4d7404bed0 con 0x7f4d740fff00 2024-04-06T03:45:52.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.854+0000 7f4d6a7fc700 1 -- 172.21.15.94:0/3180198592 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+941 (secure 0 0 0) 0x7f4d64059ef0 con 0x7f4d740fff00 2024-04-06T03:45:52.857 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s); Degraded data redundancy: 29/219 objects degraded (13.242%), 11 pgs degraded 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 29/219 objects degraded (13.242%), 11 pgs degraded 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-04-06T03:45:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.856+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d6006dca0 msgr2=0x7f4d60070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.856+0000 7f4d7bc41700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d6006dca0 0x7f4d60070150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d70004890 tx=0x7f4d70009e30 comp rx=0 tx=0).stop 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.856+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740fff00 msgr2=0x7f4d7410df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.856+0000 7f4d7bc41700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740fff00 0x7f4d7410df70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d64002a50 tx=0x7f4d640046b0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.856+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 shutdown_connections 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.856+0000 7f4d7bc41700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d6006dca0 0x7f4d60070150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.857+0000 7f4d7bc41700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d740662e0 0x7f4d7410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.857+0000 7f4d7bc41700 1 --2- 172.21.15.94:0/3180198592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d740fff00 0x7f4d7410df70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.857+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 >> 172.21.15.94:0/3180198592 conn(0x7f4d740f7430 msgr2=0x7f4d740f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.857+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 shutdown_connections 2024-04-06T03:45:52.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:45:52.857+0000 7f4d7bc41700 1 -- 172.21.15.94:0/3180198592 wait complete. 2024-04-06T03:45:53.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:53 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3180198592' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:45:53.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:53 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3180198592' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:45:54.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: pgmap v107: 65 pgs: 3 activating, 10 peering, 6 active+undersized+degraded, 8 active+undersized, 38 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19/219 objects degraded (8.676%) 2024-04-06T03:45:54.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:54.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: Health check update: Degraded data redundancy: 19/219 objects degraded (8.676%), 6 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:54.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: pgmap v108: 65 pgs: 3 activating, 10 peering, 6 active+undersized+degraded, 8 active+undersized, 38 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19/219 objects degraded (8.676%) 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: pgmap v109: 65 pgs: 3 activating, 10 peering, 6 active+undersized+degraded, 8 active+undersized, 38 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19/219 objects degraded (8.676%) 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-06T03:45:54.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:54 smithi136 ceph-mon[82061]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-06T03:45:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: pgmap v107: 65 pgs: 3 activating, 10 peering, 6 active+undersized+degraded, 8 active+undersized, 38 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19/219 objects degraded (8.676%) 2024-04-06T03:45:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:45:54.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: Health check update: Degraded data redundancy: 19/219 objects degraded (8.676%), 6 pgs degraded (PG_DEGRADED) 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: pgmap v108: 65 pgs: 3 activating, 10 peering, 6 active+undersized+degraded, 8 active+undersized, 38 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19/219 objects degraded (8.676%) 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: pgmap v109: 65 pgs: 3 activating, 10 peering, 6 active+undersized+degraded, 8 active+undersized, 38 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19/219 objects degraded (8.676%) 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-06T03:45:54.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:54 smithi094 ceph-mon[94053]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-06T03:45:56.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:55 smithi136 ceph-mon[82061]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/219 objects degraded (8.676%), 6 pgs degraded) 2024-04-06T03:45:56.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:55 smithi094 ceph-mon[94053]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/219 objects degraded (8.676%), 6 pgs degraded) 2024-04-06T03:45:57.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:56 smithi136 ceph-mon[82061]: pgmap v110: 65 pgs: 3 activating, 10 peering, 52 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:57.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:56 smithi094 ceph-mon[94053]: pgmap v110: 65 pgs: 3 activating, 10 peering, 52 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:58.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:45:58 smithi094 ceph-mon[94053]: pgmap v111: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:45:59.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:45:58 smithi136 ceph-mon[82061]: pgmap v111: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:01.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:00 smithi136 ceph-mon[82061]: pgmap v112: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:00 smithi094 ceph-mon[94053]: pgmap v112: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:02.970 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:02 smithi094 ceph-mon[94053]: pgmap v113: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:03.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:02 smithi136 ceph-mon[82061]: pgmap v113: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:04.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:04 smithi094 ceph-mon[94053]: pgmap v114: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:05.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:04 smithi136 ceph-mon[82061]: pgmap v114: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:46:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:06.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:06.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:46:06.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:07.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:07 smithi136 ceph-mon[82061]: pgmap v115: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:07.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:07 smithi094 ceph-mon[94053]: pgmap v115: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:09.522 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:09 smithi136 ceph-mon[82061]: pgmap v116: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:09.522 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:09 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-06T03:46:09.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:09 smithi094 ceph-mon[94053]: pgmap v116: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:09.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:09 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-06T03:46:10.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:10 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-06T03:46:10.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:10 smithi136 ceph-mon[82061]: Upgrade: osd.3 is safe to restart 2024-04-06T03:46:10.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:10 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-06T03:46:10.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:10 smithi094 ceph-mon[94053]: Upgrade: osd.3 is safe to restart 2024-04-06T03:46:11.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:11 smithi136 ceph-mon[82061]: pgmap v117: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:11.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:11 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:11.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:11 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-06T03:46:11.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:11 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:46:11.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:11 smithi094 ceph-mon[94053]: pgmap v117: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:11.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:11.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-06T03:46:11.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:46:12.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:12 smithi136 ceph-mon[82061]: Upgrade: Updating osd.3 2024-04-06T03:46:12.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:12 smithi136 ceph-mon[82061]: Deploying daemon osd.3 on smithi136 2024-04-06T03:46:12.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:12 smithi094 ceph-mon[94053]: Upgrade: Updating osd.3 2024-04-06T03:46:12.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:12 smithi094 ceph-mon[94053]: Deploying daemon osd.3 on smithi136 2024-04-06T03:46:13.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:13 smithi136 ceph-mon[82061]: pgmap v118: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:13.599 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:13 smithi136 systemd[1]: Stopping Ceph osd.3 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:46:13.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:13 smithi094 ceph-mon[94053]: pgmap v118: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:14.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:13 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[41886]: 2024-04-06T03:46:13.697+0000 7f624e1f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:46:14.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:13 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[41886]: 2024-04-06T03:46:13.697+0000 7f624e1f0700 -1 osd.3 63 *** Got signal Terminated *** 2024-04-06T03:46:14.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:13 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[41886]: 2024-04-06T03:46:13.697+0000 7f624e1f0700 -1 osd.3 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:46:14.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:14 smithi136 ceph-mon[82061]: pgmap v119: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:14.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:14 smithi136 ceph-mon[82061]: osd.3 marked itself down and dead 2024-04-06T03:46:14.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:14 smithi094 ceph-mon[94053]: pgmap v119: 65 pgs: 65 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:14.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:14 smithi094 ceph-mon[94053]: osd.3 marked itself down and dead 2024-04-06T03:46:14.935 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:14 smithi136 podman[91503]: 2024-04-06 03:46:14.654859207 +0000 UTC m=+1.051395983 container died 49aad532cde6147072bd30d6c646f308c87198d750d9d93de20150bf6ea3bff1 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=quincy-6699495, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:46:15.272 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:15 smithi136 podman[91503]: 2024-04-06 03:46:15.04037455 +0000 UTC m=+1.436911312 container cleanup 49aad532cde6147072bd30d6c646f308c87198d750d9d93de20150bf6ea3bff1 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, RELEASE=quincy-6699495, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux ) 2024-04-06T03:46:15.272 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:15 smithi136 bash[91503]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3 2024-04-06T03:46:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:15 smithi136 ceph-mon[82061]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:46:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:15 smithi136 ceph-mon[82061]: osdmap e64: 6 total, 5 up, 6 in 2024-04-06T03:46:15.599 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:15 smithi136 podman[91531]: 2024-04-06 03:46:15.308341846 +0000 UTC m=+0.648866621 container remove 49aad532cde6147072bd30d6c646f308c87198d750d9d93de20150bf6ea3bff1 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=quincy-6699495, org.label-schema.build-date=20240403) 2024-04-06T03:46:15.646 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:15 smithi094 ceph-mon[94053]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:46:15.647 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:15 smithi094 ceph-mon[94053]: osdmap e64: 6 total, 5 up, 6 in 2024-04-06T03:46:15.954 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:15 smithi136 podman[91622]: 2024-04-06 03:46:15.650739677 +0000 UTC m=+0.112163971 container create d62e1c32a9668d6545c13a38e6580b942f059187505edcf81947946408941c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=squid-9f19235, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-06T03:46:15.955 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:15 smithi136 podman[91622]: 2024-04-06 03:46:15.556365589 +0000 UTC m=+0.017789891 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:46:15.955 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:15 smithi136 podman[91622]: 2024-04-06 03:46:15.861574547 +0000 UTC m=+0.322998842 container init d62e1c32a9668d6545c13a38e6580b942f059187505edcf81947946408941c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS) 2024-04-06T03:46:15.955 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:15 smithi136 podman[91622]: 2024-04-06 03:46:15.864590891 +0000 UTC m=+0.326015189 container start d62e1c32a9668d6545c13a38e6580b942f059187505edcf81947946408941c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, ceph=True) 2024-04-06T03:46:15.955 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:15 smithi136 podman[91622]: 2024-04-06 03:46:15.952996173 +0000 UTC m=+0.414420469 container attach d62e1c32a9668d6545c13a38e6580b942f059187505edcf81947946408941c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-06T03:46:16.276 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:16 smithi136 podman[91622]: 2024-04-06 03:46:16.153829479 +0000 UTC m=+0.615253775 container died d62e1c32a9668d6545c13a38e6580b942f059187505edcf81947946408941c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS) 2024-04-06T03:46:16.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:16 smithi136 ceph-mon[82061]: osdmap e65: 6 total, 5 up, 6 in 2024-04-06T03:46:16.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:16 smithi136 ceph-mon[82061]: pgmap v122: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:16 smithi094 ceph-mon[94053]: osdmap e65: 6 total, 5 up, 6 in 2024-04-06T03:46:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:16 smithi094 ceph-mon[94053]: pgmap v122: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:17.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:16 smithi136 podman[91648]: 2024-04-06 03:46:16.723221818 +0000 UTC m=+0.564066482 container remove d62e1c32a9668d6545c13a38e6580b942f059187505edcf81947946408941c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:46:17.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:16 smithi136 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.3.service: Deactivated successfully. 2024-04-06T03:46:17.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:16 smithi136 systemd[1]: Stopped Ceph osd.3 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:46:17.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:16 smithi136 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.3.service: Consumed 1min 15.694s CPU time. 2024-04-06T03:46:17.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:16 smithi136 systemd[1]: Starting Ceph osd.3 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:46:17.487 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:17 smithi136 ceph-mon[82061]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-06T03:46:17.487 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 podman[91751]: 2024-04-06 03:46:17.066433201 +0000 UTC m=+0.017784250 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:46:17.487 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 podman[91751]: 2024-04-06 03:46:17.190999137 +0000 UTC m=+0.142350177 container create b37d7468047619adc462ad65ce42147173378e23ff0a690e2d0ba7b8c8a49112 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-9f19235, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-06T03:46:17.487 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 podman[91751]: 2024-04-06 03:46:17.409957777 +0000 UTC m=+0.361308818 container init b37d7468047619adc462ad65ce42147173378e23ff0a690e2d0ba7b8c8a49112 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240403, RELEASE=squid-9f19235) 2024-04-06T03:46:17.487 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 podman[91751]: 2024-04-06 03:46:17.41328703 +0000 UTC m=+0.364638071 container start b37d7468047619adc462ad65ce42147173378e23ff0a690e2d0ba7b8c8a49112 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-9f19235) 2024-04-06T03:46:17.488 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 podman[91751]: 2024-04-06 03:46:17.484800662 +0000 UTC m=+0.436151727 container attach b37d7468047619adc462ad65ce42147173378e23ff0a690e2d0ba7b8c8a49112 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, RELEASE=squid-9f19235, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:46:17.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:17 smithi094 ceph-mon[94053]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-06T03:46:18.165 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate[91765]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate[91765]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 bash[91751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 bash[91751]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 bash[91751]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 bash[91751]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 bash[91751]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 bash[91751]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 bash[91751]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate[91765]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate[91765]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate[91765]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate[91765]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-06T03:46:18.166 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate[91765]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-06T03:46:18.167 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:17 smithi136 podman[91751]: 2024-04-06 03:46:17.87579326 +0000 UTC m=+0.827144292 container died b37d7468047619adc462ad65ce42147173378e23ff0a690e2d0ba7b8c8a49112 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, ceph=True) 2024-04-06T03:46:18.473 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:18 smithi136 ceph-mon[82061]: pgmap v123: 65 pgs: 12 active+undersized+degraded, 7 peering, 1 stale+active+clean, 15 active+undersized, 30 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:46:18.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:18 smithi094 ceph-mon[94053]: pgmap v123: 65 pgs: 12 active+undersized+degraded, 7 peering, 1 stale+active+clean, 15 active+undersized, 30 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:46:18.787 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:18 smithi136 podman[91884]: 2024-04-06 03:46:18.471727948 +0000 UTC m=+0.590714151 container remove b37d7468047619adc462ad65ce42147173378e23ff0a690e2d0ba7b8c8a49112 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-06T03:46:18.787 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:18 smithi136 podman[91949]: 2024-04-06 03:46:18.587667171 +0000 UTC m=+0.018077803 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:46:18.787 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:18 smithi136 podman[91949]: 2024-04-06 03:46:18.705881148 +0000 UTC m=+0.136291769 container create 0bdf47d78dc3cbb4550a4b621db935a574c2aa5ca9ddebc735d5e442c8dd4851 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=) 2024-04-06T03:46:19.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:18 smithi136 podman[91949]: 2024-04-06 03:46:18.874970506 +0000 UTC m=+0.305381129 container init 0bdf47d78dc3cbb4550a4b621db935a574c2aa5ca9ddebc735d5e442c8dd4851 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-06T03:46:19.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:18 smithi136 podman[91949]: 2024-04-06 03:46:18.878534058 +0000 UTC m=+0.308944681 container start 0bdf47d78dc3cbb4550a4b621db935a574c2aa5ca9ddebc735d5e442c8dd4851 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3, ceph=True, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-9f19235) 2024-04-06T03:46:19.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:18 smithi136 ceph-osd[91968]: -- 172.21.15.136:0/1183380650 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631580df680 con 0x5631580a8c00 2024-04-06T03:46:19.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:18 smithi136 bash[91949]: 0bdf47d78dc3cbb4550a4b621db935a574c2aa5ca9ddebc735d5e442c8dd4851 2024-04-06T03:46:19.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:18 smithi136 systemd[1]: Started Ceph osd.3 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:46:19.599 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:19 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:46:19.426+0000 7fa789b587c0 -1 Falling back to public interface 2024-04-06T03:46:19.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:19 smithi136 ceph-mon[82061]: Health check failed: Degraded data redundancy: 36/219 objects degraded (16.438%), 12 pgs degraded (PG_DEGRADED) 2024-04-06T03:46:19.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:19 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:19.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:19 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:19.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:19 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:46:19.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:19 smithi094 ceph-mon[94053]: Health check failed: Degraded data redundancy: 36/219 objects degraded (16.438%), 12 pgs degraded (PG_DEGRADED) 2024-04-06T03:46:19.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:19 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:19.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:19 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:19.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:19 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:46:21.553 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:21 smithi136 ceph-mon[82061]: pgmap v124: 65 pgs: 14 active+undersized+degraded, 7 peering, 17 active+undersized, 27 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-04-06T03:46:21.553 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:21.553 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:46:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:21 smithi094 ceph-mon[94053]: pgmap v124: 65 pgs: 14 active+undersized+degraded, 7 peering, 17 active+undersized, 27 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39/219 objects degraded (17.808%) 2024-04-06T03:46:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.973+0000 7f331ce91700 1 -- 172.21.15.94:0/3564126039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fdfe0 msgr2=0x7f33180fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.973+0000 7f331ce91700 1 --2- 172.21.15.94:0/3564126039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fdfe0 0x7f33180fe450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f330c004940 tx=0x7f330c009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.973+0000 7f331ce91700 1 -- 172.21.15.94:0/3564126039 shutdown_connections 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.973+0000 7f331ce91700 1 --2- 172.21.15.94:0/3564126039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fdfe0 0x7f33180fe450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.973+0000 7f331ce91700 1 --2- 172.21.15.94:0/3564126039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33180fce70 0x7f33180fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.973+0000 7f331ce91700 1 -- 172.21.15.94:0/3564126039 >> 172.21.15.94:0/3564126039 conn(0x7f33180f8550 msgr2=0x7f33180fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.973+0000 7f331ce91700 1 -- 172.21.15.94:0/3564126039 shutdown_connections 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.973+0000 7f331ce91700 1 -- 172.21.15.94:0/3564126039 wait complete. 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.973+0000 7f331ce91700 1 Processor -- start 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331ce91700 1 -- start start 2024-04-06T03:46:22.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331ce91700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fce70 0x7f331810d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331ce91700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33180fdfe0 0x7f331810ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331ce91700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331810e3d0 con 0x7f33180fce70 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331ce91700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331810e540 con 0x7f33180fdfe0 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fce70 0x7f331810d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fce70 0x7f331810d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46222/0 (socket says 172.21.15.94:46222) 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331659c700 1 -- 172.21.15.94:0/3764679080 learned_addr learned my addr 172.21.15.94:0/3764679080 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f3315d9b700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33180fdfe0 0x7f331810ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331659c700 1 -- 172.21.15.94:0/3764679080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33180fdfe0 msgr2=0x7f331810ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331659c700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33180fdfe0 0x7f331810ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331659c700 1 -- 172.21.15.94:0/3764679080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c005040 con 0x7f33180fce70 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.974+0000 7f331659c700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fce70 0x7f331810d890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f330800eb40 tx=0x7f330800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:22.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.975+0000 7f3306ffd700 1 -- 172.21.15.94:0/3764679080 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3308007cc0 con 0x7f33180fce70 2024-04-06T03:46:22.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.975+0000 7f3306ffd700 1 -- 172.21.15.94:0/3764679080 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f330800f040 con 0x7f33180fce70 2024-04-06T03:46:22.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.975+0000 7f3306ffd700 1 -- 172.21.15.94:0/3764679080 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f330800ca10 con 0x7f33180fce70 2024-04-06T03:46:22.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.975+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3318112f40 con 0x7f33180fce70 2024-04-06T03:46:22.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.975+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33181133a0 con 0x7f33180fce70 2024-04-06T03:46:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.976+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331804c710 con 0x7f33180fce70 2024-04-06T03:46:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.976+0000 7f3306ffd700 1 -- 172.21.15.94:0/3764679080 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f330800cc80 con 0x7f33180fce70 2024-04-06T03:46:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.977+0000 7f3306ffd700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f330006dca0 0x7f3300070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.977+0000 7f3315d9b700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f330006dca0 0x7f3300070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.977+0000 7f3306ffd700 1 -- 172.21.15.94:0/3764679080 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f330808d070 con 0x7f33180fce70 2024-04-06T03:46:22.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.977+0000 7f3315d9b700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f330006dca0 0x7f3300070150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f330c009b30 tx=0x7f330c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:22.981+0000 7f3306ffd700 1 -- 172.21.15.94:0/3764679080 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f330805a280 con 0x7f33180fce70 2024-04-06T03:46:23.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.201+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33181022b0 con 0x7f330006dca0 2024-04-06T03:46:23.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:22 smithi094 ceph-mon[94053]: pgmap v125: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-04-06T03:46:23.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:22 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:23.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:22 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:23.204 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:22 smithi094 ceph-mon[94053]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-06T03:46:23.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.202+0000 7f3306ffd700 1 -- 172.21.15.94:0/3764679080 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f33181022b0 con 0x7f330006dca0 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f330006dca0 msgr2=0x7f3300070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f330006dca0 0x7f3300070150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f330c009b30 tx=0x7f330c009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fce70 msgr2=0x7f331810d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fce70 0x7f331810d890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f330800eb40 tx=0x7f330800b040 comp rx=0 tx=0).stop 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 shutdown_connections 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33180fce70 0x7f331810d890 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f330006dca0 0x7f3300070150 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 --2- 172.21.15.94:0/3764679080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33180fdfe0 0x7f331810ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 >> 172.21.15.94:0/3764679080 conn(0x7f33180f8550 msgr2=0x7f33180f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 shutdown_connections 2024-04-06T03:46:23.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.205+0000 7f331ce91700 1 -- 172.21.15.94:0/3764679080 wait complete. 2024-04-06T03:46:23.218 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:46:23.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.309+0000 7f8d93ddd700 1 -- 172.21.15.94:0/4214974623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fcf80 msgr2=0x7f8d8c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.309+0000 7f8d93ddd700 1 --2- 172.21.15.94:0/4214974623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fcf80 0x7f8d8c0fd3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8d88004940 tx=0x7f8d88009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.309+0000 7f8d93ddd700 1 -- 172.21.15.94:0/4214974623 shutdown_connections 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.309+0000 7f8d93ddd700 1 --2- 172.21.15.94:0/4214974623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fcf80 0x7f8d8c0fd3f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.309+0000 7f8d93ddd700 1 --2- 172.21.15.94:0/4214974623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c0fbcd0 0x7f8d8c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.309+0000 7f8d93ddd700 1 -- 172.21.15.94:0/4214974623 >> 172.21.15.94:0/4214974623 conn(0x7f8d8c0f7430 msgr2=0x7f8d8c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.309+0000 7f8d93ddd700 1 -- 172.21.15.94:0/4214974623 shutdown_connections 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.309+0000 7f8d93ddd700 1 -- 172.21.15.94:0/4214974623 wait complete. 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d93ddd700 1 Processor -- start 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d93ddd700 1 -- start start 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d93ddd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fbcd0 0x7f8d8c10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d93ddd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c0fcf80 0x7f8d8c10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d93ddd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c10e520 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d93ddd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c10ed20 con 0x7f8d8c0fcf80 2024-04-06T03:46:23.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d91b79700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fbcd0 0x7f8d8c10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d91b79700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fbcd0 0x7f8d8c10d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46236/0 (socket says 172.21.15.94:46236) 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d91b79700 1 -- 172.21.15.94:0/2122873158 learned_addr learned my addr 172.21.15.94:0/2122873158 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d91378700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c0fcf80 0x7f8d8c10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d91b79700 1 -- 172.21.15.94:0/2122873158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c0fcf80 msgr2=0x7f8d8c10df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d91b79700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c0fcf80 0x7f8d8c10df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d91b79700 1 -- 172.21.15.94:0/2122873158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d88005040 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d91b79700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fbcd0 0x7f8d8c10d9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c002a80 tx=0x7f8d7c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d827fc700 1 -- 172.21.15.94:0/2122873158 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7c007cc0 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d827fc700 1 -- 172.21.15.94:0/2122873158 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d7c00f040 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.310+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c10f000 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.311+0000 7f8d827fc700 1 -- 172.21.15.94:0/2122873158 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7c00dac0 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.311+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c10f540 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.311+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c101b10 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.312+0000 7f8d827fc700 1 -- 172.21.15.94:0/2122873158 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f8d7c00dc20 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.312+0000 7f8d827fc700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8d7806dca0 0x7f8d78070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:23.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.313+0000 7f8d827fc700 1 -- 172.21.15.94:0/2122873158 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8d7c08d0c0 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.313+0000 7f8d91378700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8d7806dca0 0x7f8d78070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:23.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.313+0000 7f8d91378700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8d7806dca0 0x7f8d78070150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800b970 tx=0x7f8d88009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:23.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.317+0000 7f8d827fc700 1 -- 172.21.15.94:0/2122873158 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8d7c05a220 con 0x7f8d8c0fbcd0 2024-04-06T03:46:23.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:22 smithi136 ceph-mon[82061]: pgmap v125: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-04-06T03:46:23.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:22 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:23.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:22 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:23.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:22 smithi136 ceph-mon[82061]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-06T03:46:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.533+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d8c10f760 con 0x7f8d7806dca0 2024-04-06T03:46:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.534+0000 7f8d827fc700 1 -- 172.21.15.94:0/2122873158 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f8d8c10f760 con 0x7f8d7806dca0 2024-04-06T03:46:23.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8d7806dca0 msgr2=0x7f8d78070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8d7806dca0 0x7f8d78070150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800b970 tx=0x7f8d88009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:23.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fbcd0 msgr2=0x7f8d8c10d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fbcd0 0x7f8d8c10d9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c002a80 tx=0x7f8d7c0046b0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 shutdown_connections 2024-04-06T03:46:23.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d8c0fbcd0 0x7f8d8c10d9e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8d7806dca0 0x7f8d78070150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 --2- 172.21.15.94:0/2122873158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c0fcf80 0x7f8d8c10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 >> 172.21.15.94:0/2122873158 conn(0x7f8d8c0f7430 msgr2=0x7f8d8c065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:23.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 shutdown_connections 2024-04-06T03:46:23.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.536+0000 7f8d93ddd700 1 -- 172.21.15.94:0/2122873158 wait complete. 2024-04-06T03:46:23.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.637+0000 7f1fe1328700 1 -- 172.21.15.94:0/1207289882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 msgr2=0x7f1fdc0703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.637+0000 7f1fe1328700 1 --2- 172.21.15.94:0/1207289882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 0x7f1fdc0703b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc004940 tx=0x7f1fcc009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:23.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.637+0000 7f1fe1328700 1 -- 172.21.15.94:0/1207289882 shutdown_connections 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.637+0000 7f1fe1328700 1 --2- 172.21.15.94:0/1207289882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fdc0708f0 0x7f1fdc070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.637+0000 7f1fe1328700 1 --2- 172.21.15.94:0/1207289882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 0x7f1fdc0703b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.637+0000 7f1fe1328700 1 -- 172.21.15.94:0/1207289882 >> 172.21.15.94:0/1207289882 conn(0x7f1fdc0f8550 msgr2=0x7f1fdc0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.637+0000 7f1fe1328700 1 -- 172.21.15.94:0/1207289882 shutdown_connections 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.637+0000 7f1fe1328700 1 -- 172.21.15.94:0/1207289882 wait complete. 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fe1328700 1 Processor -- start 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fe1328700 1 -- start start 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fe1328700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fdc0708f0 0x7f1fdc113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fe1328700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 0x7f1fdc1144c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fe1328700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc114ac0 con 0x7f1fdc071ea0 2024-04-06T03:46:23.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fe1328700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc114c30 con 0x7f1fdc0708f0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fda7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 0x7f1fdc1144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fda7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 0x7f1fdc1144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46258/0 (socket says 172.21.15.94:46258) 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fda7fc700 1 -- 172.21.15.94:0/3683708462 learned_addr learned my addr 172.21.15.94:0/3683708462 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.638+0000 7f1fdaffd700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fdc0708f0 0x7f1fdc113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.639+0000 7f1fda7fc700 1 -- 172.21.15.94:0/3683708462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fdc0708f0 msgr2=0x7f1fdc113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.639+0000 7f1fda7fc700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fdc0708f0 0x7f1fdc113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.639+0000 7f1fda7fc700 1 -- 172.21.15.94:0/3683708462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc005040 con 0x7f1fdc071ea0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.639+0000 7f1fda7fc700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 0x7f1fdc1144c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0002a50 tx=0x7f1fd00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.639+0000 7f1fc37fe700 1 -- 172.21.15.94:0/3683708462 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd0007cc0 con 0x7f1fdc071ea0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.639+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fdc119690 con 0x7f1fdc071ea0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.639+0000 7f1fc37fe700 1 -- 172.21.15.94:0/3683708462 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fd000f040 con 0x7f1fdc071ea0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.639+0000 7f1fc37fe700 1 -- 172.21.15.94:0/3683708462 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd000da30 con 0x7f1fdc071ea0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.639+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc119c50 con 0x7f1fdc071ea0 2024-04-06T03:46:23.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.640+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fdc04c710 con 0x7f1fdc071ea0 2024-04-06T03:46:23.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.640+0000 7f1fc37fe700 1 -- 172.21.15.94:0/3683708462 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f1fd000db90 con 0x7f1fdc071ea0 2024-04-06T03:46:23.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.641+0000 7f1fc37fe700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1fc406dca0 0x7f1fc4070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:23.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.641+0000 7f1fc37fe700 1 -- 172.21.15.94:0/3683708462 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1fd008d2c0 con 0x7f1fdc071ea0 2024-04-06T03:46:23.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.641+0000 7f1fdaffd700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1fc406dca0 0x7f1fc4070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:23.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.641+0000 7f1fdaffd700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1fc406dca0 0x7f1fc4070150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc004890 tx=0x7f1fcc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:23.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.645+0000 7f1fc37fe700 1 -- 172.21.15.94:0/3683708462 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1fd005a3c0 con 0x7f1fdc071ea0 2024-04-06T03:46:23.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.850+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fdc0ffa60 con 0x7f1fc406dca0 2024-04-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.862+0000 7f1fc37fe700 1 -- 172.21.15.94:0/3683708462 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f1fdc0ffa60 con 0x7f1fc406dca0 2024-04-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (5m) 36s ago 13m 27.6M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (2m) 36s ago 13m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (2m) 1s ago 12m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 36s ago 13m - - 2024-04-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (8m) 36s ago 8m 15.0M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:46:23.865 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (8m) 36s ago 8m 153M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (8m) 1s ago 8m 15.1M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (8m) 1s ago 8m 17.5M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (4m) 36s ago 15m 502M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (3m) 1s ago 12m 451M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (3m) 36s ago 15m 46.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (3m) 1s ago 12m 41.6M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (5m) 36s ago 13m 15.3M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (4m) 1s ago 12m 16.7M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (115s) 36s ago 12m 103M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (78s) 36s ago 11m 107M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (41s) 36s ago 11m 12.0M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:46:23.866 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (5s) 1s ago 10m 12.0M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:46:23.867 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (10m) 1s ago 10m 257M 2581M 17.2.7-819-g6699495c ec300eaa768b 2f6660744da9 2024-04-06T03:46:23.867 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (9m) 1s ago 9m 241M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:46:23.867 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (3m) 36s ago 13m 57.0M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1fc406dca0 msgr2=0x7f1fc4070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1fc406dca0 0x7f1fc4070150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc004890 tx=0x7f1fcc009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 msgr2=0x7f1fdc1144c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 0x7f1fdc1144c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0002a50 tx=0x7f1fd00046b0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 shutdown_connections 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1fc406dca0 0x7f1fc4070150 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fdc0708f0 0x7f1fdc113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 --2- 172.21.15.94:0/3683708462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fdc071ea0 0x7f1fdc1144c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 >> 172.21.15.94:0/3683708462 conn(0x7f1fdc0f8550 msgr2=0x7f1fdc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 shutdown_connections 2024-04-06T03:46:23.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.864+0000 7f1fe1328700 1 -- 172.21.15.94:0/3683708462 wait complete. 2024-04-06T03:46:23.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.964+0000 7f6969572700 1 -- 172.21.15.94:0/3000717434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 msgr2=0x7f69640fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.964+0000 7f6969572700 1 --2- 172.21.15.94:0/3000717434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 0x7f69640fd310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f695c009ce0 tx=0x7f695c004d10 comp rx=0 tx=0).stop 2024-04-06T03:46:23.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.964+0000 7f6969572700 1 -- 172.21.15.94:0/3000717434 shutdown_connections 2024-04-06T03:46:23.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.964+0000 7f6969572700 1 --2- 172.21.15.94:0/3000717434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69640fe0d0 0x7f69640fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.964+0000 7f6969572700 1 --2- 172.21.15.94:0/3000717434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 0x7f69640fd310 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.964+0000 7f6969572700 1 -- 172.21.15.94:0/3000717434 >> 172.21.15.94:0/3000717434 conn(0x7f69640f8560 msgr2=0x7f69640fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:23.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.964+0000 7f6969572700 1 -- 172.21.15.94:0/3000717434 shutdown_connections 2024-04-06T03:46:23.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.964+0000 7f6969572700 1 -- 172.21.15.94:0/3000717434 wait complete. 2024-04-06T03:46:23.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6969572700 1 Processor -- start 2024-04-06T03:46:23.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6969572700 1 -- start start 2024-04-06T03:46:23.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6969572700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 0x7f696410da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:23.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6969572700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69640fe0d0 0x7f696410dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:23.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6969572700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696410e5c0 con 0x7f69640fcee0 2024-04-06T03:46:23.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6969572700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696410ee10 con 0x7f69640fe0d0 2024-04-06T03:46:23.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6962ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 0x7f696410da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:23.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6962ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 0x7f696410da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46282/0 (socket says 172.21.15.94:46282) 2024-04-06T03:46:23.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6962ffd700 1 -- 172.21.15.94:0/935538956 learned_addr learned my addr 172.21.15.94:0/935538956 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:23.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f69627fc700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69640fe0d0 0x7f696410dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:23.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6962ffd700 1 -- 172.21.15.94:0/935538956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69640fe0d0 msgr2=0x7f696410dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:23.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6962ffd700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69640fe0d0 0x7f696410dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:23.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6962ffd700 1 -- 172.21.15.94:0/935538956 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c009940 con 0x7f69640fcee0 2024-04-06T03:46:23.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f6962ffd700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 0x7f696410da80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f695c0098e0 tx=0x7f695c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:23.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.965+0000 7f694b7fe700 1 -- 172.21.15.94:0/935538956 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695c00e040 con 0x7f69640fcee0 2024-04-06T03:46:23.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.966+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696410f090 con 0x7f69640fcee0 2024-04-06T03:46:23.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.966+0000 7f694b7fe700 1 -- 172.21.15.94:0/935538956 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f695c0049f0 con 0x7f69640fcee0 2024-04-06T03:46:23.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.966+0000 7f694b7fe700 1 -- 172.21.15.94:0/935538956 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695c012c00 con 0x7f69640fcee0 2024-04-06T03:46:23.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.966+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696410f5d0 con 0x7f69640fcee0 2024-04-06T03:46:23.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.966+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696405f5f0 con 0x7f69640fcee0 2024-04-06T03:46:23.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.967+0000 7f694b7fe700 1 -- 172.21.15.94:0/935538956 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f695c012410 con 0x7f69640fcee0 2024-04-06T03:46:23.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.967+0000 7f694b7fe700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f694c06dca0 0x7f694c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:23.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.968+0000 7f694b7fe700 1 -- 172.21.15.94:0/935538956 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f695c08f770 con 0x7f69640fcee0 2024-04-06T03:46:23.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.968+0000 7f69627fc700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f694c06dca0 0x7f694c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:23.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.968+0000 7f69627fc700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f694c06dca0 0x7f694c070150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6958009b80 tx=0x7f695800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:23.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:23.972+0000 7f694b7fe700 1 -- 172.21.15.94:0/935538956 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f695c05c8d0 con 0x7f69640fcee0 2024-04-06T03:46:24.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.249+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f696410f830 con 0x7f69640fcee0 2024-04-06T03:46:24.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.249+0000 7f694b7fe700 1 -- 172.21.15.94:0/935538956 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f695c05c460 con 0x7f69640fcee0 2024-04-06T03:46:24.252 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:46:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:46:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:46:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 2, 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 3 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 6, 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 7 2024-04-06T03:46:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:46:24.254 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:46:24.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.251+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f694c06dca0 msgr2=0x7f694c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.251+0000 7f6969572700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f694c06dca0 0x7f694c070150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6958009b80 tx=0x7f695800b040 comp rx=0 tx=0).stop 2024-04-06T03:46:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.251+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 msgr2=0x7f696410da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.251+0000 7f6969572700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 0x7f696410da80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f695c0098e0 tx=0x7f695c004d10 comp rx=0 tx=0).stop 2024-04-06T03:46:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.251+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 shutdown_connections 2024-04-06T03:46:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.251+0000 7f6969572700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69640fcee0 0x7f696410da80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.251+0000 7f6969572700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f694c06dca0 0x7f694c070150 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.251+0000 7f6969572700 1 --2- 172.21.15.94:0/935538956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69640fe0d0 0x7f696410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.251+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 >> 172.21.15.94:0/935538956 conn(0x7f69640f8560 msgr2=0x7f69640f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:24.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.252+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 shutdown_connections 2024-04-06T03:46:24.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.252+0000 7f6969572700 1 -- 172.21.15.94:0/935538956 wait complete. 2024-04-06T03:46:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:24 smithi136 ceph-mon[82061]: from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:24.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 -- 172.21.15.94:0/249054636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1900708f0 msgr2=0x7fa190070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 --2- 172.21.15.94:0/249054636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1900708f0 0x7fa190070d60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa180004940 tx=0x7fa180009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:24.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 -- 172.21.15.94:0/249054636 shutdown_connections 2024-04-06T03:46:24.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 --2- 172.21.15.94:0/249054636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1900708f0 0x7fa190070d60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 --2- 172.21.15.94:0/249054636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa190071ea0 0x7fa1900703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 -- 172.21.15.94:0/249054636 >> 172.21.15.94:0/249054636 conn(0x7fa1900f8550 msgr2=0x7fa1900fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:24.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 -- 172.21.15.94:0/249054636 shutdown_connections 2024-04-06T03:46:24.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 -- 172.21.15.94:0/249054636 wait complete. 2024-04-06T03:46:24.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 Processor -- start 2024-04-06T03:46:24.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 -- start start 2024-04-06T03:46:24.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1900708f0 0x7fa190113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:24.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa190071ea0 0x7fa1901143e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:24.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1901149e0 con 0x7fa190071ea0 2024-04-06T03:46:24.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa1976ca700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa190114b50 con 0x7fa1900708f0 2024-04-06T03:46:24.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.354+0000 7fa194c65700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa190071ea0 0x7fa1901143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:24.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa194c65700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa190071ea0 0x7fa1901143e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46300/0 (socket says 172.21.15.94:46300) 2024-04-06T03:46:24.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa194c65700 1 -- 172.21.15.94:0/1933749349 learned_addr learned my addr 172.21.15.94:0/1933749349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:24.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa195466700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1900708f0 0x7fa190113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:24.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa194c65700 1 -- 172.21.15.94:0/1933749349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1900708f0 msgr2=0x7fa190113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa194c65700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1900708f0 0x7fa190113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa194c65700 1 -- 172.21.15.94:0/1933749349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180005040 con 0x7fa190071ea0 2024-04-06T03:46:24.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa194c65700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa190071ea0 0x7fa1901143e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa180013870 tx=0x7fa180009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:24.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa185ffb700 1 -- 172.21.15.94:0/1933749349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa180021070 con 0x7fa190071ea0 2024-04-06T03:46:24.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa185ffb700 1 -- 172.21.15.94:0/1933749349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa180009430 con 0x7fa190071ea0 2024-04-06T03:46:24.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa185ffb700 1 -- 172.21.15.94:0/1933749349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa18000e070 con 0x7fa190071ea0 2024-04-06T03:46:24.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa190119550 con 0x7fa190071ea0 2024-04-06T03:46:24.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.355+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa190119a90 con 0x7fa190071ea0 2024-04-06T03:46:24.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.356+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19004c710 con 0x7fa190071ea0 2024-04-06T03:46:24.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.357+0000 7fa185ffb700 1 -- 172.21.15.94:0/1933749349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fa18001c750 con 0x7fa190071ea0 2024-04-06T03:46:24.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.357+0000 7fa185ffb700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fa17c06dc40 0x7fa17c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:24.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.357+0000 7fa185ffb700 1 -- 172.21.15.94:0/1933749349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa18008e680 con 0x7fa190071ea0 2024-04-06T03:46:24.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.357+0000 7fa195466700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fa17c06dc40 0x7fa17c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:24.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.358+0000 7fa195466700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fa17c06dc40 0x7fa17c0700f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009b80 tx=0x7fa18c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:24.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.362+0000 7fa185ffb700 1 -- 172.21.15.94:0/1933749349 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa18005b860 con 0x7fa190071ea0 2024-04-06T03:46:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:24 smithi094 ceph-mon[94053]: from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:24.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.600+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa19010bed0 con 0x7fa190071ea0 2024-04-06T03:46:24.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.600+0000 7fa185ffb700 1 -- 172.21.15.94:0/1933749349 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fa18005b3f0 con 0x7fa190071ea0 2024-04-06T03:46:24.603 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:46:24.603 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:46:24.603 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:46:24.603 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:46:24.603 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:46:24.603 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:46:24.604 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:46:24.605 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-04-06T03:46:24.606 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:46:24.606 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:46:24.606 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:46:24.606 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:46:24.606 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:46:24.606 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:46:24.606 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:46:24.606 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:46:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.602+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fa17c06dc40 msgr2=0x7fa17c0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.602+0000 7fa1976ca700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fa17c06dc40 0x7fa17c0700f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009b80 tx=0x7fa18c00b040 comp rx=0 tx=0).stop 2024-04-06T03:46:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.602+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa190071ea0 msgr2=0x7fa1901143e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.602+0000 7fa1976ca700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa190071ea0 0x7fa1901143e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa180013870 tx=0x7fa180009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.602+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 shutdown_connections 2024-04-06T03:46:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.602+0000 7fa1976ca700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fa17c06dc40 0x7fa17c0700f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.602+0000 7fa1976ca700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1900708f0 0x7fa190113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.602+0000 7fa1976ca700 1 --2- 172.21.15.94:0/1933749349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa190071ea0 0x7fa1901143e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.602+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 >> 172.21.15.94:0/1933749349 conn(0x7fa1900f8550 msgr2=0x7fa1900f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.603+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 shutdown_connections 2024-04-06T03:46:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.603+0000 7fa1976ca700 1 -- 172.21.15.94:0/1933749349 wait complete. 2024-04-06T03:46:24.608 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.704+0000 7f76eae79700 1 -- 172.21.15.94:0/1824190546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fe060 msgr2=0x7f76e40fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.704+0000 7f76eae79700 1 --2- 172.21.15.94:0/1824190546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fe060 0x7f76e40fe4d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76d0004940 tx=0x7f76d0009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 -- 172.21.15.94:0/1824190546 shutdown_connections 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 --2- 172.21.15.94:0/1824190546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fe060 0x7f76e40fe4d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 --2- 172.21.15.94:0/1824190546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e40fcef0 0x7f76e40fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 -- 172.21.15.94:0/1824190546 >> 172.21.15.94:0/1824190546 conn(0x7f76e40f85b0 msgr2=0x7f76e40fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 -- 172.21.15.94:0/1824190546 shutdown_connections 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 -- 172.21.15.94:0/1824190546 wait complete. 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 Processor -- start 2024-04-06T03:46:24.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 -- start start 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fcef0 0x7f76e4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e40fe060 0x7f76e4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40708c0 con 0x7f76e40fcef0 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76eae79700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4070a30 con 0x7f76e40fe060 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76e8c15700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fcef0 0x7f76e4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.705+0000 7f76e8c15700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fcef0 0x7f76e4071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46318/0 (socket says 172.21.15.94:46318) 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76e8c15700 1 -- 172.21.15.94:0/3745754661 learned_addr learned my addr 172.21.15.94:0/3745754661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76e3fff700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e40fe060 0x7f76e4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76e8c15700 1 -- 172.21.15.94:0/3745754661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e40fe060 msgr2=0x7f76e4070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76e8c15700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e40fe060 0x7f76e4070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76e8c15700 1 -- 172.21.15.94:0/3745754661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0005040 con 0x7f76e40fcef0 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76e8c15700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fcef0 0x7f76e4071e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76d8002a50 tx=0x7f76d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76e17fa700 1 -- 172.21.15.94:0/3745754661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d8007cc0 con 0x7f76e40fcef0 2024-04-06T03:46:24.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76e17fa700 1 -- 172.21.15.94:0/3745754661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76d800f040 con 0x7f76e40fcef0 2024-04-06T03:46:24.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e4070d10 con 0x7f76e40fcef0 2024-04-06T03:46:24.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76e17fa700 1 -- 172.21.15.94:0/3745754661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d800da30 con 0x7f76e40fcef0 2024-04-06T03:46:24.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.706+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e4115a70 con 0x7f76e40fcef0 2024-04-06T03:46:24.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.707+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e405f5f0 con 0x7f76e40fcef0 2024-04-06T03:46:24.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.708+0000 7f76e17fa700 1 -- 172.21.15.94:0/3745754661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f76d800db90 con 0x7f76e40fcef0 2024-04-06T03:46:24.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.708+0000 7f76e17fa700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f76d406dcb0 0x7f76d4070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:24.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.708+0000 7f76e17fa700 1 -- 172.21.15.94:0/3745754661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f76d808d260 con 0x7f76e40fcef0 2024-04-06T03:46:24.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.708+0000 7f76e3fff700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f76d406dcb0 0x7f76d4070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:24.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.708+0000 7f76e3fff700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f76d406dcb0 0x7f76d4070160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009fd0 tx=0x7f76d0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:24.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.713+0000 7f76e17fa700 1 -- 172.21.15.94:0/3745754661 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f76d8056890 con 0x7f76e40fcef0 2024-04-06T03:46:24.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.929+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76e41023c0 con 0x7f76d406dcb0 2024-04-06T03:46:24.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.930+0000 7f76e17fa700 1 -- 172.21.15.94:0/3745754661 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f76e41023c0 con 0x7f76d406dcb0 2024-04-06T03:46:24.933 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:46:24.933 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:46:24.933 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:46:24.933 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:46:24.933 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:46:24.933 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:46:24.933 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:46:24.934 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:46:24.934 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:46:24.934 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "10/21 daemons upgraded", 2024-04-06T03:46:24.934 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-04-06T03:46:24.934 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:46:24.934 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:46:24.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f76d406dcb0 msgr2=0x7f76d4070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f76d406dcb0 0x7f76d4070160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009fd0 tx=0x7f76d0009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fcef0 msgr2=0x7f76e4071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fcef0 0x7f76e4071e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76d8002a50 tx=0x7f76d80046b0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 shutdown_connections 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76e40fcef0 0x7f76e4071e40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f76d406dcb0 0x7f76d4070160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 --2- 172.21.15.94:0/3745754661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76e40fe060 0x7f76e4070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 >> 172.21.15.94:0/3745754661 conn(0x7f76e40f85b0 msgr2=0x7f76e40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 shutdown_connections 2024-04-06T03:46:24.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:24.933+0000 7f76eae79700 1 -- 172.21.15.94:0/3745754661 wait complete. 2024-04-06T03:46:24.972 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:24 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:46:24.640+0000 7fa789b587c0 -1 osd.3 63 log_to_monitors true 2024-04-06T03:46:25.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.033+0000 7f8b32528700 1 -- 172.21.15.94:0/2893980839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c066f10 msgr2=0x7f8b2c0ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:25.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.033+0000 7f8b32528700 1 --2- 172.21.15.94:0/2893980839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c066f10 0x7f8b2c0ffa50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8b20004940 tx=0x7f8b20009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:25.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.033+0000 7f8b32528700 1 -- 172.21.15.94:0/2893980839 shutdown_connections 2024-04-06T03:46:25.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.033+0000 7f8b32528700 1 --2- 172.21.15.94:0/2893980839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b2c0fff90 0x7f8b2c102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:25.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.034+0000 7f8b32528700 1 --2- 172.21.15.94:0/2893980839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c066f10 0x7f8b2c0ffa50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:25.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.034+0000 7f8b32528700 1 -- 172.21.15.94:0/2893980839 >> 172.21.15.94:0/2893980839 conn(0x7f8b2c0f7480 msgr2=0x7f8b2c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:25.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.034+0000 7f8b32528700 1 -- 172.21.15.94:0/2893980839 shutdown_connections 2024-04-06T03:46:25.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.034+0000 7f8b32528700 1 -- 172.21.15.94:0/2893980839 wait complete. 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.034+0000 7f8b32528700 1 Processor -- start 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.034+0000 7f8b32528700 1 -- start start 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b32528700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b2c066f10 0x7f8b2c10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b32528700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c0fff90 0x7f8b2c10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b32528700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c10e560 con 0x7f8b2c0fff90 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b32528700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c112eb0 con 0x7f8b2c066f10 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b2b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c0fff90 0x7f8b2c10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b2b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c0fff90 0x7f8b2c10ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46336/0 (socket says 172.21.15.94:46336) 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b2b7fe700 1 -- 172.21.15.94:0/415550699 learned_addr learned my addr 172.21.15.94:0/415550699 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b2bfff700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b2c066f10 0x7f8b2c10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b2b7fe700 1 -- 172.21.15.94:0/415550699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b2c066f10 msgr2=0x7f8b2c10d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b2b7fe700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b2c066f10 0x7f8b2c10d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b2b7fe700 1 -- 172.21.15.94:0/415550699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b20005040 con 0x7f8b2c0fff90 2024-04-06T03:46:25.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b2b7fe700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c0fff90 0x7f8b2c10ded0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00b790 tx=0x7f8b1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:25.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b28ff9700 1 -- 172.21.15.94:0/415550699 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1c00f6f0 con 0x7f8b2c0fff90 2024-04-06T03:46:25.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c113110 con 0x7f8b2c0fff90 2024-04-06T03:46:25.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b28ff9700 1 -- 172.21.15.94:0/415550699 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b1c00fd20 con 0x7f8b2c0fff90 2024-04-06T03:46:25.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b28ff9700 1 -- 172.21.15.94:0/415550699 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1c0103a0 con 0x7f8b2c0fff90 2024-04-06T03:46:25.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.035+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2c113680 con 0x7f8b2c0fff90 2024-04-06T03:46:25.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.036+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2c04c710 con 0x7f8b2c0fff90 2024-04-06T03:46:25.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.037+0000 7f8b28ff9700 1 -- 172.21.15.94:0/415550699 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f8b1c011400 con 0x7f8b2c0fff90 2024-04-06T03:46:25.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.037+0000 7f8b28ff9700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8b1406dca0 0x7f8b14070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:25.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.037+0000 7f8b28ff9700 1 -- 172.21.15.94:0/415550699 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8b1c08d2f0 con 0x7f8b2c0fff90 2024-04-06T03:46:25.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.037+0000 7f8b2bfff700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8b1406dca0 0x7f8b14070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:25.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.038+0000 7f8b2bfff700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8b1406dca0 0x7f8b14070150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000b970 tx=0x7f8b20009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:25.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.042+0000 7f8b28ff9700 1 -- 172.21.15.94:0/415550699 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8b1c05a4d0 con 0x7f8b2c0fff90 2024-04-06T03:46:25.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.320+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8b2c105960 con 0x7f8b2c0fff90 2024-04-06T03:46:25.323 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:25 smithi094 ceph-mon[94053]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:25.323 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:25 smithi094 ceph-mon[94053]: pgmap v126: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-04-06T03:46:25.323 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:25 smithi094 ceph-mon[94053]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:25.323 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:25 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/935538956' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:25.323 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:25 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1933749349' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:46:25.323 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:25 smithi094 ceph-mon[94053]: from='osd.3 [v2:172.21.15.136:6800/2267317275,v1:172.21.15.136:6801/2267317275]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-06T03:46:25.323 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:25 smithi094 ceph-mon[94053]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.320+0000 7f8b28ff9700 1 -- 172.21.15.94:0/415550699 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1362 (secure 0 0 0) 0x7f8b1c05a060 con 0x7f8b2c0fff90 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 osds down; Degraded data redundancy: 48/219 objects degraded (21.918%), 17 pgs degraded 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout: osd.3 (root=default,host=smithi136) is down 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 48/219 objects degraded (21.918%), 17 pgs degraded 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-04-06T03:46:25.324 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-04-06T03:46:25.325 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-04-06T03:46:25.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8b1406dca0 msgr2=0x7f8b14070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:25.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8b1406dca0 0x7f8b14070150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000b970 tx=0x7f8b20009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:25.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c0fff90 msgr2=0x7f8b2c10ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:25.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c0fff90 0x7f8b2c10ded0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00b790 tx=0x7f8b1c009250 comp rx=0 tx=0).stop 2024-04-06T03:46:25.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 shutdown_connections 2024-04-06T03:46:25.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8b1406dca0 0x7f8b14070150 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:25.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b2c066f10 0x7f8b2c10d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:25.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 --2- 172.21.15.94:0/415550699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b2c0fff90 0x7f8b2c10ded0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:25.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 >> 172.21.15.94:0/415550699 conn(0x7f8b2c0f7480 msgr2=0x7f8b2c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:25.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 shutdown_connections 2024-04-06T03:46:25.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:25.323+0000 7f8b32528700 1 -- 172.21.15.94:0/415550699 wait complete. 2024-04-06T03:46:25.349 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:46:25 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:46:25.141+0000 7fa77afb0700 -1 osd.3 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:46:25.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:25 smithi136 ceph-mon[82061]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:25.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:25 smithi136 ceph-mon[82061]: pgmap v126: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 194 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-04-06T03:46:25.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:25 smithi136 ceph-mon[82061]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:25.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:25 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/935538956' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:25.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:25 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1933749349' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:46:25.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:25 smithi136 ceph-mon[82061]: from='osd.3 [v2:172.21.15.136:6800/2267317275,v1:172.21.15.136:6801/2267317275]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-06T03:46:25.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:25 smithi136 ceph-mon[82061]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-06T03:46:26.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:26 smithi094 ceph-mon[94053]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:26.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:26 smithi094 ceph-mon[94053]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-06T03:46:26.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:26 smithi094 ceph-mon[94053]: osdmap e66: 6 total, 5 up, 6 in 2024-04-06T03:46:26.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:26 smithi094 ceph-mon[94053]: from='osd.3 [v2:172.21.15.136:6800/2267317275,v1:172.21.15.136:6801/2267317275]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:46:26.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:26 smithi094 ceph-mon[94053]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:46:26.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:26 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/415550699' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:46:26.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:26 smithi136 ceph-mon[82061]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:26.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:26 smithi136 ceph-mon[82061]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-06T03:46:26.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:26 smithi136 ceph-mon[82061]: osdmap e66: 6 total, 5 up, 6 in 2024-04-06T03:46:26.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:26 smithi136 ceph-mon[82061]: from='osd.3 [v2:172.21.15.136:6800/2267317275,v1:172.21.15.136:6801/2267317275]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:46:26.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:26 smithi136 ceph-mon[82061]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:46:26.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:26 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/415550699' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:46:27.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: pgmap v128: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-04-06T03:46:27.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:46:27.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: osd.3 [v2:172.21.15.136:6800/2267317275,v1:172.21.15.136:6801/2267317275] boot 2024-04-06T03:46:27.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: osdmap e67: 6 total, 6 up, 6 in 2024-04-06T03:46:27.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:46:27.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: Health check update: Degraded data redundancy: 48/219 objects degraded (21.918%), 17 pgs degraded (PG_DEGRADED) 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:27.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:27 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-06T03:46:27.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: pgmap v128: 65 pgs: 17 active+undersized+degraded, 21 active+undersized, 27 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 48/219 objects degraded (21.918%) 2024-04-06T03:46:27.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:46:27.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: osd.3 [v2:172.21.15.136:6800/2267317275,v1:172.21.15.136:6801/2267317275] boot 2024-04-06T03:46:27.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: osdmap e67: 6 total, 6 up, 6 in 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: Health check update: Degraded data redundancy: 48/219 objects degraded (21.918%), 17 pgs degraded (PG_DEGRADED) 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:27.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:27 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-06T03:46:28.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:28 smithi094 ceph-mon[94053]: pgmap v130: 65 pgs: 8 peering, 15 active+undersized+degraded, 15 active+undersized, 27 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 42/219 objects degraded (19.178%) 2024-04-06T03:46:28.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:28 smithi094 ceph-mon[94053]: pgmap v131: 65 pgs: 8 peering, 15 active+undersized+degraded, 15 active+undersized, 27 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 42/219 objects degraded (19.178%) 2024-04-06T03:46:28.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:28 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-06T03:46:28.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:28 smithi094 ceph-mon[94053]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-06T03:46:28.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:28 smithi094 ceph-mon[94053]: osdmap e68: 6 total, 6 up, 6 in 2024-04-06T03:46:28.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:28 smithi136 ceph-mon[82061]: pgmap v130: 65 pgs: 8 peering, 15 active+undersized+degraded, 15 active+undersized, 27 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 42/219 objects degraded (19.178%) 2024-04-06T03:46:28.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:28 smithi136 ceph-mon[82061]: pgmap v131: 65 pgs: 8 peering, 15 active+undersized+degraded, 15 active+undersized, 27 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 42/219 objects degraded (19.178%) 2024-04-06T03:46:28.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:28 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-06T03:46:28.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:28 smithi136 ceph-mon[82061]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-06T03:46:28.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:28 smithi136 ceph-mon[82061]: osdmap e68: 6 total, 6 up, 6 in 2024-04-06T03:46:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:30 smithi094 ceph-mon[94053]: pgmap v133: 65 pgs: 15 peering, 11 active+undersized+degraded, 12 active+undersized, 27 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:46:30.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:30 smithi136 ceph-mon[82061]: pgmap v133: 65 pgs: 15 peering, 11 active+undersized+degraded, 12 active+undersized, 27 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:46:31.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:31 smithi136 ceph-mon[82061]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/219 objects degraded (13.699%), 11 pgs degraded) 2024-04-06T03:46:31.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:31 smithi094 ceph-mon[94053]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/219 objects degraded (13.699%), 11 pgs degraded) 2024-04-06T03:46:32.569 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:32 smithi094 ceph-mon[94053]: pgmap v134: 65 pgs: 15 peering, 50 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:32.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:32 smithi136 ceph-mon[82061]: pgmap v134: 65 pgs: 15 peering, 50 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:34 smithi136 ceph-mon[82061]: pgmap v135: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:34.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:34 smithi094 ceph-mon[94053]: pgmap v135: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:36 smithi136 ceph-mon[82061]: pgmap v136: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:46:36.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:36 smithi094 ceph-mon[94053]: pgmap v136: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:36.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:36.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:46:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:38 smithi136 ceph-mon[82061]: pgmap v137: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:38.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:38 smithi094 ceph-mon[94053]: pgmap v137: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:40.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:40 smithi136 ceph-mon[82061]: pgmap v138: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:40.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:40 smithi094 ceph-mon[94053]: pgmap v138: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:41.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:41 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:41.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:41 smithi136 ceph-mon[82061]: pgmap v139: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:41.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:41 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:41.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:41 smithi094 ceph-mon[94053]: pgmap v139: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:42.545 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:42 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-06T03:46:42.545 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:42 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-06T03:46:42.545 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:42 smithi136 ceph-mon[82061]: Upgrade: osd.4 is safe to restart 2024-04-06T03:46:42.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:42 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-06T03:46:42.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:42 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-06T03:46:42.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:42 smithi094 ceph-mon[94053]: Upgrade: osd.4 is safe to restart 2024-04-06T03:46:43.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:43 smithi136 ceph-mon[82061]: pgmap v140: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:43.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:43 smithi094 ceph-mon[94053]: pgmap v140: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:44 smithi136 ceph-mon[82061]: Upgrade: Updating osd.4 2024-04-06T03:46:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-06T03:46:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:46:44.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:44 smithi136 ceph-mon[82061]: Deploying daemon osd.4 on smithi136 2024-04-06T03:46:44.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:44 smithi094 ceph-mon[94053]: Upgrade: Updating osd.4 2024-04-06T03:46:44.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:44.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-06T03:46:44.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:46:44.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:44 smithi094 ceph-mon[94053]: Deploying daemon osd.4 on smithi136 2024-04-06T03:46:45.592 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:45 smithi136 ceph-mon[82061]: pgmap v141: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:45.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:45 smithi094 ceph-mon[94053]: pgmap v141: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:46.774 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:46 smithi136 systemd[1]: Stopping Ceph osd.4 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:46:47.099 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:46 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[48296]: 2024-04-06T03:46:46.771+0000 7f1f9c0ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:46:47.099 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:46 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[48296]: 2024-04-06T03:46:46.771+0000 7f1f9c0ed700 -1 osd.4 68 *** Got signal Terminated *** 2024-04-06T03:46:47.099 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:46 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[48296]: 2024-04-06T03:46:46.771+0000 7f1f9c0ed700 -1 osd.4 68 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:46:48.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:47 smithi136 ceph-mon[82061]: pgmap v142: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:48.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:47 smithi136 ceph-mon[82061]: osd.4 marked itself down and dead 2024-04-06T03:46:48.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:47 smithi094 ceph-mon[94053]: pgmap v142: 65 pgs: 65 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:48.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:47 smithi094 ceph-mon[94053]: osd.4 marked itself down and dead 2024-04-06T03:46:48.574 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:48 smithi136 podman[96638]: 2024-04-06 03:46:48.308854938 +0000 UTC m=+1.637024516 container died 2f6660744da9431e4bb09fb21f57403bf18e6e0ca74dff065fe92b6ead8eb673 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=quincy-6699495, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-06T03:46:48.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:48 smithi136 ceph-mon[82061]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:46:48.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:48 smithi136 ceph-mon[82061]: osdmap e69: 6 total, 5 up, 6 in 2024-04-06T03:46:48.849 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:48 smithi136 podman[96638]: 2024-04-06 03:46:48.655163591 +0000 UTC m=+1.983333187 container cleanup 2f6660744da9431e4bb09fb21f57403bf18e6e0ca74dff065fe92b6ead8eb673 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-6699495, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-06T03:46:48.849 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:48 smithi136 bash[96638]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4 2024-04-06T03:46:49.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:48 smithi094 ceph-mon[94053]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:46:49.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:48 smithi094 ceph-mon[94053]: osdmap e69: 6 total, 5 up, 6 in 2024-04-06T03:46:49.249 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:48 smithi136 podman[96668]: 2024-04-06 03:46:48.939669727 +0000 UTC m=+0.630859013 container remove 2f6660744da9431e4bb09fb21f57403bf18e6e0ca74dff065fe92b6ead8eb673 (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, RELEASE=quincy-6699495, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-06T03:46:49.586 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:49 smithi136 podman[96758]: 2024-04-06 03:46:49.150335271 +0000 UTC m=+0.017647119 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:46:49.586 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:49 smithi136 podman[96758]: 2024-04-06 03:46:49.248713971 +0000 UTC m=+0.116025810 container create f551544a5f342362dc4c9a9ee2a4f959ba4b09585be9b55d4e4499ea1d1d3f7c (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-06T03:46:49.586 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:49 smithi136 podman[96758]: 2024-04-06 03:46:49.476055678 +0000 UTC m=+0.343367516 container init f551544a5f342362dc4c9a9ee2a4f959ba4b09585be9b55d4e4499ea1d1d3f7c (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, GIT_BRANCH=HEAD) 2024-04-06T03:46:49.586 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:49 smithi136 podman[96758]: 2024-04-06 03:46:49.479171292 +0000 UTC m=+0.346483130 container start f551544a5f342362dc4c9a9ee2a4f959ba4b09585be9b55d4e4499ea1d1d3f7c (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-06T03:46:49.586 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:49 smithi136 podman[96758]: 2024-04-06 03:46:49.584376158 +0000 UTC m=+0.451688007 container attach f551544a5f342362dc4c9a9ee2a4f959ba4b09585be9b55d4e4499ea1d1d3f7c (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:46:49.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:49 smithi136 ceph-mon[82061]: pgmap v144: 65 pgs: 6 stale+active+clean, 59 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:49.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:49 smithi136 ceph-mon[82061]: osdmap e70: 6 total, 5 up, 6 in 2024-04-06T03:46:49.849 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:49 smithi136 podman[96758]: 2024-04-06 03:46:49.773017415 +0000 UTC m=+0.640329245 container died f551544a5f342362dc4c9a9ee2a4f959ba4b09585be9b55d4e4499ea1d1d3f7c (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-06T03:46:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:49 smithi094 ceph-mon[94053]: pgmap v144: 65 pgs: 6 stale+active+clean, 59 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:46:50.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:49 smithi094 ceph-mon[94053]: osdmap e70: 6 total, 5 up, 6 in 2024-04-06T03:46:50.599 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:50 smithi136 podman[96785]: 2024-04-06 03:46:50.337968714 +0000 UTC m=+0.559954180 container remove f551544a5f342362dc4c9a9ee2a4f959ba4b09585be9b55d4e4499ea1d1d3f7c (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9f19235) 2024-04-06T03:46:50.599 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:50 smithi136 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.4.service: Deactivated successfully. 2024-04-06T03:46:50.599 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:50 smithi136 systemd[1]: Stopped Ceph osd.4 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:46:50.599 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:50 smithi136 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.4.service: Consumed 1min 3.906s CPU time. 2024-04-06T03:46:50.599 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:50 smithi136 systemd[1]: Starting Ceph osd.4 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:46:51.026 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:50 smithi136 podman[96891]: 2024-04-06 03:46:50.763745796 +0000 UTC m=+0.115219653 container create 328a3f229dc3c9a7201cd942db66e67d8b91de437c821dd7685a44e59d72f8ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:46:51.026 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:50 smithi136 podman[96891]: 2024-04-06 03:46:50.666426111 +0000 UTC m=+0.017899976 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:46:51.026 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 podman[96891]: 2024-04-06 03:46:51.024649527 +0000 UTC m=+0.376123381 container init 328a3f229dc3c9a7201cd942db66e67d8b91de437c821dd7685a44e59d72f8ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:46:51.349 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 podman[96891]: 2024-04-06 03:46:51.028369312 +0000 UTC m=+0.379843180 container start 328a3f229dc3c9a7201cd942db66e67d8b91de437c821dd7685a44e59d72f8ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate, ceph=True, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-9f19235) 2024-04-06T03:46:51.349 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 podman[96891]: 2024-04-06 03:46:51.116152248 +0000 UTC m=+0.467626120 container attach 328a3f229dc3c9a7201cd942db66e67d8b91de437c821dd7685a44e59d72f8ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-06T03:46:51.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:51.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:51 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:46:51.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:51.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:51 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:46:51.838 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate[96906]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate[96906]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 bash[96891]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 bash[96891]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 bash[96891]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 bash[96891]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 bash[96891]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 bash[96891]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 bash[96891]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate[96906]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate[96906]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate[96906]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate[96906]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate[96906]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-06T03:46:51.839 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:51 smithi136 podman[96891]: 2024-04-06 03:46:51.490703415 +0000 UTC m=+0.842177270 container died 328a3f229dc3c9a7201cd942db66e67d8b91de437c821dd7685a44e59d72f8ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-06T03:46:52.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:52 smithi136 ceph-mon[82061]: pgmap v146: 65 pgs: 8 active+undersized+degraded, 2 stale+active+clean, 13 active+undersized, 42 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26/219 objects degraded (11.872%) 2024-04-06T03:46:52.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:52 smithi136 ceph-mon[82061]: Health check failed: Degraded data redundancy: 26/219 objects degraded (11.872%), 8 pgs degraded (PG_DEGRADED) 2024-04-06T03:46:52.349 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:52 smithi136 podman[97031]: 2024-04-06 03:46:52.095114012 +0000 UTC m=+0.599417776 container remove 328a3f229dc3c9a7201cd942db66e67d8b91de437c821dd7685a44e59d72f8ad (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-activate, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-06T03:46:52.349 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:52 smithi136 podman[97099]: 2024-04-06 03:46:52.21380157 +0000 UTC m=+0.017931558 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:46:52.349 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:52 smithi136 podman[97099]: 2024-04-06 03:46:52.329213061 +0000 UTC m=+0.133343068 container create c31367e1c2d53d57c59534dfa2bcf245ed64c52613f8cbe7b906e624fe08fef0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-06T03:46:52.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:52 smithi094 ceph-mon[94053]: pgmap v146: 65 pgs: 8 active+undersized+degraded, 2 stale+active+clean, 13 active+undersized, 42 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26/219 objects degraded (11.872%) 2024-04-06T03:46:52.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:52 smithi094 ceph-mon[94053]: Health check failed: Degraded data redundancy: 26/219 objects degraded (11.872%), 8 pgs degraded (PG_DEGRADED) 2024-04-06T03:46:52.849 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:52 smithi136 podman[97099]: 2024-04-06 03:46:52.523107717 +0000 UTC m=+0.327237686 container init c31367e1c2d53d57c59534dfa2bcf245ed64c52613f8cbe7b906e624fe08fef0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-9f19235) 2024-04-06T03:46:52.849 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:52 smithi136 podman[97099]: 2024-04-06 03:46:52.52620885 +0000 UTC m=+0.330338815 container start c31367e1c2d53d57c59534dfa2bcf245ed64c52613f8cbe7b906e624fe08fef0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-06T03:46:52.849 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:52 smithi136 ceph-osd[97118]: -- 172.21.15.136:0/1115883613 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e37b88b680 con 0x55e37b855000 2024-04-06T03:46:52.849 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:52 smithi136 bash[97099]: c31367e1c2d53d57c59534dfa2bcf245ed64c52613f8cbe7b906e624fe08fef0 2024-04-06T03:46:52.849 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:52 smithi136 systemd[1]: Started Ceph osd.4 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:46:53.349 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:53 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:46:53.077+0000 7f3d37dc57c0 -1 Falling back to public interface 2024-04-06T03:46:53.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:53 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:53.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:53 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:53.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:53 smithi136 ceph-mon[82061]: pgmap v147: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:46:53.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:53 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:46:54.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:53 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:54.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:53 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:54.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:53 smithi094 ceph-mon[94053]: pgmap v147: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:46:54.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:53 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.438+0000 7f05d4031700 1 -- 172.21.15.94:0/2988665856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fce40 msgr2=0x7f05cc0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.438+0000 7f05d4031700 1 --2- 172.21.15.94:0/2988665856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fce40 0x7f05cc0fd250 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f05c0004940 tx=0x7f05c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.438+0000 7f05d4031700 1 -- 172.21.15.94:0/2988665856 shutdown_connections 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.438+0000 7f05d4031700 1 --2- 172.21.15.94:0/2988665856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05cc0fe040 0x7f05cc0fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.438+0000 7f05d4031700 1 --2- 172.21.15.94:0/2988665856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fce40 0x7f05cc0fd250 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.438+0000 7f05d4031700 1 -- 172.21.15.94:0/2988665856 >> 172.21.15.94:0/2988665856 conn(0x7f05cc0f8550 msgr2=0x7f05cc0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.438+0000 7f05d4031700 1 -- 172.21.15.94:0/2988665856 shutdown_connections 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.438+0000 7f05d4031700 1 -- 172.21.15.94:0/2988665856 wait complete. 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d4031700 1 Processor -- start 2024-04-06T03:46:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d4031700 1 -- start start 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d4031700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05cc0fce40 0x7f05cc10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d4031700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fe040 0x7f05cc10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d4031700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc10e4f0 con 0x7f05cc0fe040 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d4031700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc10ed40 con 0x7f05cc0fce40 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d15cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fe040 0x7f05cc10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d15cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fe040 0x7f05cc10def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51516/0 (socket says 172.21.15.94:51516) 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d15cc700 1 -- 172.21.15.94:0/3794469445 learned_addr learned my addr 172.21.15.94:0/3794469445 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.439+0000 7f05d1dcd700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05cc0fce40 0x7f05cc10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.440+0000 7f05d15cc700 1 -- 172.21.15.94:0/3794469445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05cc0fce40 msgr2=0x7f05cc10d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.440+0000 7f05d15cc700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05cc0fce40 0x7f05cc10d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.440+0000 7f05d15cc700 1 -- 172.21.15.94:0/3794469445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c0005040 con 0x7f05cc0fe040 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.440+0000 7f05d15cc700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fe040 0x7f05cc10def0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f05c800eb40 tx=0x7f05c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.440+0000 7f05be7fc700 1 -- 172.21.15.94:0/3794469445 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c8007cc0 con 0x7f05cc0fe040 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.440+0000 7f05be7fc700 1 -- 172.21.15.94:0/3794469445 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05c800f040 con 0x7f05cc0fe040 2024-04-06T03:46:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.440+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc10efc0 con 0x7f05cc0fe040 2024-04-06T03:46:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.440+0000 7f05be7fc700 1 -- 172.21.15.94:0/3794469445 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c800ca10 con 0x7f05cc0fe040 2024-04-06T03:46:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.440+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc10f500 con 0x7f05cc0fe040 2024-04-06T03:46:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.441+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc05f5f0 con 0x7f05cc0fe040 2024-04-06T03:46:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.442+0000 7f05be7fc700 1 -- 172.21.15.94:0/3794469445 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f05c800cc80 con 0x7f05cc0fe040 2024-04-06T03:46:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.442+0000 7f05be7fc700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f05b806dca0 0x7f05b8070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.442+0000 7f05be7fc700 1 -- 172.21.15.94:0/3794469445 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f05c808d090 con 0x7f05cc0fe040 2024-04-06T03:46:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.442+0000 7f05d1dcd700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f05b806dca0 0x7f05b8070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.442+0000 7f05d1dcd700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f05b806dca0 0x7f05b8070150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f05c0004890 tx=0x7f05c0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.447+0000 7f05be7fc700 1 -- 172.21.15.94:0/3794469445 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f05c805a2a0 con 0x7f05cc0fe040 2024-04-06T03:46:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.661+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f05cc10f720 con 0x7f05b806dca0 2024-04-06T03:46:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.662+0000 7f05be7fc700 1 -- 172.21.15.94:0/3794469445 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f05cc10f720 con 0x7f05b806dca0 2024-04-06T03:46:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f05b806dca0 msgr2=0x7f05b8070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f05b806dca0 0x7f05b8070150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f05c0004890 tx=0x7f05c0009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fe040 msgr2=0x7f05cc10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fe040 0x7f05cc10def0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f05c800eb40 tx=0x7f05c800b040 comp rx=0 tx=0).stop 2024-04-06T03:46:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 shutdown_connections 2024-04-06T03:46:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f05b806dca0 0x7f05b8070150 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05cc0fce40 0x7f05cc10d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 --2- 172.21.15.94:0/3794469445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05cc0fe040 0x7f05cc10def0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 >> 172.21.15.94:0/3794469445 conn(0x7f05cc0f8550 msgr2=0x7f05cc0f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:55.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 shutdown_connections 2024-04-06T03:46:55.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.664+0000 7f05d4031700 1 -- 172.21.15.94:0/3794469445 wait complete. 2024-04-06T03:46:55.677 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:46:55.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.768+0000 7f9784ecd700 1 -- 172.21.15.94:0/1841780250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97801002a0 msgr2=0x7f9780100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:55.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.768+0000 7f9784ecd700 1 --2- 172.21.15.94:0/1841780250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97801002a0 0x7f9780100710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9774004940 tx=0x7f9774009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.768+0000 7f9784ecd700 1 -- 172.21.15.94:0/1841780250 shutdown_connections 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.768+0000 7f9784ecd700 1 --2- 172.21.15.94:0/1841780250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97801002a0 0x7f9780100710 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.768+0000 7f9784ecd700 1 --2- 172.21.15.94:0/1841780250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97800feff0 0x7f97800ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.768+0000 7f9784ecd700 1 -- 172.21.15.94:0/1841780250 >> 172.21.15.94:0/1841780250 conn(0x7f97800fa950 msgr2=0x7f97800fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.768+0000 7f9784ecd700 1 -- 172.21.15.94:0/1841780250 shutdown_connections 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.768+0000 7f9784ecd700 1 -- 172.21.15.94:0/1841780250 wait complete. 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f9784ecd700 1 Processor -- start 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f9784ecd700 1 -- start start 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f9784ecd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97800feff0 0x7f978010fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f9784ecd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97801002a0 0x7f9780110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f9784ecd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9780110740 con 0x7f97801002a0 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f9784ecd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9780110f40 con 0x7f97800feff0 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f977dd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97801002a0 0x7f9780110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:55.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f977dd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97801002a0 0x7f9780110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51538/0 (socket says 172.21.15.94:51538) 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f977dd9b700 1 -- 172.21.15.94:0/1364133887 learned_addr learned my addr 172.21.15.94:0/1364133887 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f977e59c700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97800feff0 0x7f978010fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f977dd9b700 1 -- 172.21.15.94:0/1364133887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97800feff0 msgr2=0x7f978010fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f977dd9b700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97800feff0 0x7f978010fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f977dd9b700 1 -- 172.21.15.94:0/1364133887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780048f0 con 0x7f97801002a0 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.769+0000 7f977dd9b700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97801002a0 0x7f9780110140 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f977400b0d0 tx=0x7f9774009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.770+0000 7f976effd700 1 -- 172.21.15.94:0/1364133887 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f977400e030 con 0x7f97801002a0 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.770+0000 7f976effd700 1 -- 172.21.15.94:0/1364133887 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f977401c400 con 0x7f97801002a0 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.770+0000 7f976effd700 1 -- 172.21.15.94:0/1364133887 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f977400b630 con 0x7f97801002a0 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.770+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9774005040 con 0x7f97801002a0 2024-04-06T03:46:55.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.770+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9780111570 con 0x7f97801002a0 2024-04-06T03:46:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.771+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978005f5f0 con 0x7f97801002a0 2024-04-06T03:46:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.771+0000 7f976effd700 1 -- 172.21.15.94:0/1364133887 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f977401c570 con 0x7f97801002a0 2024-04-06T03:46:55.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.772+0000 7f976effd700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f976806dca0 0x7f9768070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.772+0000 7f976effd700 1 -- 172.21.15.94:0/1364133887 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f977408ed50 con 0x7f97801002a0 2024-04-06T03:46:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.772+0000 7f977e59c700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f976806dca0 0x7f9768070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:55.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.772+0000 7f977e59c700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f976806dca0 0x7f9768070150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9778007d60 tx=0x7f977800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:55.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.776+0000 7f976effd700 1 -- 172.21.15.94:0/1364133887 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f977405bf30 con 0x7f97801002a0 2024-04-06T03:46:55.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.988+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97801118a0 con 0x7f976806dca0 2024-04-06T03:46:55.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.990+0000 7f976effd700 1 -- 172.21.15.94:0/1364133887 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f97801118a0 con 0x7f976806dca0 2024-04-06T03:46:55.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f976806dca0 msgr2=0x7f9768070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:55.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f976806dca0 0x7f9768070150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9778007d60 tx=0x7f977800b040 comp rx=0 tx=0).stop 2024-04-06T03:46:55.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97801002a0 msgr2=0x7f9780110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:55.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97801002a0 0x7f9780110140 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f977400b0d0 tx=0x7f9774009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:55.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 shutdown_connections 2024-04-06T03:46:55.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f976806dca0 0x7f9768070150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97800feff0 0x7f978010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 --2- 172.21.15.94:0/1364133887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97801002a0 0x7f9780110140 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:55.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 >> 172.21.15.94:0/1364133887 conn(0x7f97800fa950 msgr2=0x7f97800fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:55.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 shutdown_connections 2024-04-06T03:46:55.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:55.992+0000 7f9784ecd700 1 -- 172.21.15.94:0/1364133887 wait complete. 2024-04-06T03:46:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.092+0000 7f19e84df700 1 -- 172.21.15.94:0/3463831237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 msgr2=0x7f19e00fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.092+0000 7f19e84df700 1 --2- 172.21.15.94:0/3463831237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 0x7f19e00fe450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f19d0004940 tx=0x7f19d0009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.093+0000 7f19e84df700 1 -- 172.21.15.94:0/3463831237 shutdown_connections 2024-04-06T03:46:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.093+0000 7f19e84df700 1 --2- 172.21.15.94:0/3463831237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 0x7f19e00fe450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.093+0000 7f19e84df700 1 --2- 172.21.15.94:0/3463831237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e00fce70 0x7f19e00fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.093+0000 7f19e84df700 1 -- 172.21.15.94:0/3463831237 >> 172.21.15.94:0/3463831237 conn(0x7f19e00f8550 msgr2=0x7f19e00fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:56.096 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:55 smithi094 ceph-mon[94053]: pgmap v148: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:46:56.096 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:55 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:56.096 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:55 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.093+0000 7f19e84df700 1 -- 172.21.15.94:0/3463831237 shutdown_connections 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.093+0000 7f19e84df700 1 -- 172.21.15.94:0/3463831237 wait complete. 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.093+0000 7f19e84df700 1 Processor -- start 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.093+0000 7f19e84df700 1 -- start start 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e84df700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e00fce70 0x7f19e0071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e84df700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 0x7f19e0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e84df700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0072260 con 0x7f19e00fdfe0 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e84df700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0070850 con 0x7f19e00fce70 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e5a7a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 0x7f19e0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e5a7a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 0x7f19e0070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51552/0 (socket says 172.21.15.94:51552) 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e5a7a700 1 -- 172.21.15.94:0/1135000234 learned_addr learned my addr 172.21.15.94:0/1135000234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e627b700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e00fce70 0x7f19e0071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e5a7a700 1 -- 172.21.15.94:0/1135000234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e00fce70 msgr2=0x7f19e0071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e5a7a700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e00fce70 0x7f19e0071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e5a7a700 1 -- 172.21.15.94:0/1135000234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d0005040 con 0x7f19e00fdfe0 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e5a7a700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 0x7f19e0070310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19d000b3b0 tx=0x7f19d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19d6ffd700 1 -- 172.21.15.94:0/1135000234 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d000e040 con 0x7f19e00fdfe0 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19d6ffd700 1 -- 172.21.15.94:0/1135000234 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19d000bbf0 con 0x7f19e00fdfe0 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e0070aa0 con 0x7f19e00fdfe0 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.094+0000 7f19d6ffd700 1 -- 172.21.15.94:0/1135000234 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d001c630 con 0x7f19e00fdfe0 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.095+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e0071060 con 0x7f19e00fdfe0 2024-04-06T03:46:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.095+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e004c710 con 0x7f19e00fdfe0 2024-04-06T03:46:56.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:55 smithi136 ceph-mon[82061]: pgmap v148: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:46:56.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:55 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:56.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:55 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:56.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.096+0000 7f19d6ffd700 1 -- 172.21.15.94:0/1135000234 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f19d000bd60 con 0x7f19e00fdfe0 2024-04-06T03:46:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.097+0000 7f19d6ffd700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f19cc06dca0 0x7f19cc070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.097+0000 7f19e627b700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f19cc06dca0 0x7f19cc070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.097+0000 7f19d6ffd700 1 -- 172.21.15.94:0/1135000234 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f19d008dfb0 con 0x7f19e00fdfe0 2024-04-06T03:46:56.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.097+0000 7f19e627b700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f19cc06dca0 0x7f19cc070150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19dc004f40 tx=0x7f19dc00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:56.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.101+0000 7f19d6ffd700 1 -- 172.21.15.94:0/1135000234 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f19d005b230 con 0x7f19e00fdfe0 2024-04-06T03:46:56.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.309+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f19e01023a0 con 0x7f19cc06dca0 2024-04-06T03:46:56.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.319+0000 7f19d6ffd700 1 -- 172.21.15.94:0/1135000234 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f19e01023a0 con 0x7f19cc06dca0 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (6m) 68s ago 14m 27.6M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (2m) 68s ago 14m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (2m) 0s ago 13m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 68s ago 14m - - 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (8m) 68s ago 8m 15.0M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (9m) 68s ago 9m 153M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (9m) 0s ago 9m 15.2M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (8m) 0s ago 8m 17.6M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:46:56.322 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (5m) 68s ago 15m 502M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (4m) 0s ago 13m 451M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (3m) 68s ago 15m 46.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (3m) 0s ago 13m 42.7M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (6m) 68s ago 14m 15.3M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (5m) 0s ago 13m 16.8M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (2m) 68s ago 12m 103M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (110s) 68s ago 12m 107M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (73s) 68s ago 11m 12.0M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (37s) 0s ago 11m 103M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (4s) 0s ago 10m 14.3M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 c31367e1c2d5 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (10m) 0s ago 10m 241M 2581M 17.2.7-819-g6699495c ec300eaa768b 8653ba3da3ac 2024-04-06T03:46:56.323 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (4m) 68s ago 13m 57.0M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:46:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.321+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f19cc06dca0 msgr2=0x7f19cc070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.321+0000 7f19e84df700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f19cc06dca0 0x7f19cc070150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19dc004f40 tx=0x7f19dc00b040 comp rx=0 tx=0).stop 2024-04-06T03:46:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.321+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 msgr2=0x7f19e0070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.321+0000 7f19e84df700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 0x7f19e0070310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19d000b3b0 tx=0x7f19d0009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.321+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 shutdown_connections 2024-04-06T03:46:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.321+0000 7f19e84df700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f19cc06dca0 0x7f19cc070150 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.321+0000 7f19e84df700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19e00fce70 0x7f19e0071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.321+0000 7f19e84df700 1 --2- 172.21.15.94:0/1135000234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19e00fdfe0 0x7f19e0070310 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.321+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 >> 172.21.15.94:0/1135000234 conn(0x7f19e00f8550 msgr2=0x7f19e00f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:56.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.322+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 shutdown_connections 2024-04-06T03:46:56.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.322+0000 7f19e84df700 1 -- 172.21.15.94:0/1135000234 wait complete. 2024-04-06T03:46:56.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.421+0000 7f93bdef2700 1 -- 172.21.15.94:0/603235325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fd380 msgr2=0x7f93b80fd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.421+0000 7f93bdef2700 1 --2- 172.21.15.94:0/603235325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fd380 0x7f93b80fd790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009ce0 tx=0x7f93b0004d10 comp rx=0 tx=0).stop 2024-04-06T03:46:56.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.421+0000 7f93bdef2700 1 -- 172.21.15.94:0/603235325 shutdown_connections 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.421+0000 7f93bdef2700 1 --2- 172.21.15.94:0/603235325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b80fdcd0 0x7f93b8104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.421+0000 7f93bdef2700 1 --2- 172.21.15.94:0/603235325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fd380 0x7f93b80fd790 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.421+0000 7f93bdef2700 1 -- 172.21.15.94:0/603235325 >> 172.21.15.94:0/603235325 conn(0x7f93b80f6c10 msgr2=0x7f93b80f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.421+0000 7f93bdef2700 1 -- 172.21.15.94:0/603235325 shutdown_connections 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.421+0000 7f93bdef2700 1 -- 172.21.15.94:0/603235325 wait complete. 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.421+0000 7f93bdef2700 1 Processor -- start 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93bdef2700 1 -- start start 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93bdef2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b80fd380 0x7f93b810fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93bdef2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fdcd0 0x7f93b8110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93bdef2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8110990 con 0x7f93b80fdcd0 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93bdef2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8115250 con 0x7f93b80fd380 2024-04-06T03:46:56.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93b6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fdcd0 0x7f93b8110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93b6ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fdcd0 0x7f93b8110280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51566/0 (socket says 172.21.15.94:51566) 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93b6ffd700 1 -- 172.21.15.94:0/3853427568 learned_addr learned my addr 172.21.15.94:0/3853427568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93b77fe700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b80fd380 0x7f93b810fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93b6ffd700 1 -- 172.21.15.94:0/3853427568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b80fd380 msgr2=0x7f93b810fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93b6ffd700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b80fd380 0x7f93b810fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.422+0000 7f93b6ffd700 1 -- 172.21.15.94:0/3853427568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b0009940 con 0x7f93b80fdcd0 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.423+0000 7f93b6ffd700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fdcd0 0x7f93b8110280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00b790 tx=0x7f93ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.423+0000 7f939ffff700 1 -- 172.21.15.94:0/3853427568 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93ac00f770 con 0x7f93b80fdcd0 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.423+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b81154b0 con 0x7f93b80fdcd0 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.423+0000 7f939ffff700 1 -- 172.21.15.94:0/3853427568 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93ac00fda0 con 0x7f93b80fdcd0 2024-04-06T03:46:56.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.423+0000 7f939ffff700 1 -- 172.21.15.94:0/3853427568 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93ac00d550 con 0x7f93b80fdcd0 2024-04-06T03:46:56.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.423+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b8115a40 con 0x7f93b80fdcd0 2024-04-06T03:46:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.424+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b81083c0 con 0x7f93b80fdcd0 2024-04-06T03:46:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.424+0000 7f939ffff700 1 -- 172.21.15.94:0/3853427568 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f93ac010420 con 0x7f93b80fdcd0 2024-04-06T03:46:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.425+0000 7f939ffff700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f93a006dca0 0x7f93a0070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.425+0000 7f939ffff700 1 -- 172.21.15.94:0/3853427568 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f93ac08d500 con 0x7f93b80fdcd0 2024-04-06T03:46:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.425+0000 7f93b77fe700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f93a006dca0 0x7f93a0070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.425+0000 7f93b77fe700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f93a006dca0 0x7f93a0070150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f93b0015040 tx=0x7f93b0004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:56.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.429+0000 7f939ffff700 1 -- 172.21.15.94:0/3853427568 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f93ac056bb0 con 0x7f93b80fdcd0 2024-04-06T03:46:56.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.710+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f93b804c710 con 0x7f93b80fdcd0 2024-04-06T03:46:56.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.711+0000 7f939ffff700 1 -- 172.21.15.94:0/3853427568 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f93ac05a270 con 0x7f93b80fdcd0 2024-04-06T03:46:56.713 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 1, 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 4 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:46:56.714 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:46:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 5, 2024-04-06T03:46:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 8 2024-04-06T03:46:56.715 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:46:56.715 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:46:56.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f93a006dca0 msgr2=0x7f93a0070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f93a006dca0 0x7f93a0070150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f93b0015040 tx=0x7f93b0004d10 comp rx=0 tx=0).stop 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fdcd0 msgr2=0x7f93b8110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fdcd0 0x7f93b8110280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00b790 tx=0x7f93ac0091a0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 shutdown_connections 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f93a006dca0 0x7f93a0070150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b80fd380 0x7f93b810fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 --2- 172.21.15.94:0/3853427568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80fdcd0 0x7f93b8110280 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 >> 172.21.15.94:0/3853427568 conn(0x7f93b80f6c10 msgr2=0x7f93b80f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 shutdown_connections 2024-04-06T03:46:56.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.713+0000 7f93bdef2700 1 -- 172.21.15.94:0/3853427568 wait complete. 2024-04-06T03:46:56.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.817+0000 7fd61fd49700 1 -- 172.21.15.94:0/988428601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fcee0 msgr2=0x7fd6180fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.817+0000 7fd61fd49700 1 --2- 172.21.15.94:0/988428601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fcee0 0x7fd6180fd310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd614004940 tx=0x7fd614009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.817+0000 7fd61fd49700 1 -- 172.21.15.94:0/988428601 shutdown_connections 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.817+0000 7fd61fd49700 1 --2- 172.21.15.94:0/988428601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6180fe0d0 0x7fd6180fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.817+0000 7fd61fd49700 1 --2- 172.21.15.94:0/988428601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fcee0 0x7fd6180fd310 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.817+0000 7fd61fd49700 1 -- 172.21.15.94:0/988428601 >> 172.21.15.94:0/988428601 conn(0x7fd6180f8560 msgr2=0x7fd6180fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.817+0000 7fd61fd49700 1 -- 172.21.15.94:0/988428601 shutdown_connections 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.817+0000 7fd61fd49700 1 -- 172.21.15.94:0/988428601 wait complete. 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61fd49700 1 Processor -- start 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61fd49700 1 -- start start 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61fd49700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6180fcee0 0x7fd61810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61fd49700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fe0d0 0x7fd61810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61fd49700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61810e5a0 con 0x7fd6180fe0d0 2024-04-06T03:46:56.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61fd49700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61810edf0 con 0x7fd6180fcee0 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61d2e4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fe0d0 0x7fd61810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61d2e4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fe0d0 0x7fd61810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51588/0 (socket says 172.21.15.94:51588) 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61d2e4700 1 -- 172.21.15.94:0/1508653992 learned_addr learned my addr 172.21.15.94:0/1508653992 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61dae5700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6180fcee0 0x7fd61810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61d2e4700 1 -- 172.21.15.94:0/1508653992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6180fcee0 msgr2=0x7fd61810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61d2e4700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6180fcee0 0x7fd61810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.818+0000 7fd61d2e4700 1 -- 172.21.15.94:0/1508653992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614005040 con 0x7fd6180fe0d0 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.819+0000 7fd61d2e4700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fe0d0 0x7fd61810dfa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd608002a50 tx=0x7fd6080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.819+0000 7fd60e7fc700 1 -- 172.21.15.94:0/1508653992 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd608007cc0 con 0x7fd6180fe0d0 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.819+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61810f0d0 con 0x7fd6180fe0d0 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.819+0000 7fd60e7fc700 1 -- 172.21.15.94:0/1508653992 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd60800f040 con 0x7fd6180fe0d0 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.819+0000 7fd60e7fc700 1 -- 172.21.15.94:0/1508653992 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60800da30 con 0x7fd6180fe0d0 2024-04-06T03:46:56.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.819+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61810f610 con 0x7fd6180fe0d0 2024-04-06T03:46:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.820+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61805f5f0 con 0x7fd6180fe0d0 2024-04-06T03:46:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.820+0000 7fd60e7fc700 1 -- 172.21.15.94:0/1508653992 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fd60800db90 con 0x7fd6180fe0d0 2024-04-06T03:46:56.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.821+0000 7fd60e7fc700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd60406dca0 0x7fd604070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.821+0000 7fd60e7fc700 1 -- 172.21.15.94:0/1508653992 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd60808d260 con 0x7fd6180fe0d0 2024-04-06T03:46:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.821+0000 7fd61dae5700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd60406dca0 0x7fd604070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:56.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.821+0000 7fd61dae5700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd60406dca0 0x7fd604070150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd61400b970 tx=0x7fd614009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:56.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:56.825+0000 7fd60e7fc700 1 -- 172.21.15.94:0/1508653992 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd60805a3c0 con 0x7fd6180fe0d0 2024-04-06T03:46:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.066+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd61810f8d0 con 0x7fd6180fe0d0 2024-04-06T03:46:57.069 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:56 smithi094 ceph-mon[94053]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:57.069 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:56 smithi094 ceph-mon[94053]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:57.069 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:57.069 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:57.069 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:56 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3853427568' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.066+0000 7fd60e7fc700 1 -- 172.21.15.94:0/1508653992 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fd608059f50 con 0x7fd6180fe0d0 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:46:57.070 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:46:57.071 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:46:57.072 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:46:57.073 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:46:57.073 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:46:57.073 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:46:57.073 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:46:57.073 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.069+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd60406dca0 msgr2=0x7fd604070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.069+0000 7fd61fd49700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd60406dca0 0x7fd604070150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd61400b970 tx=0x7fd614009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.069+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fe0d0 msgr2=0x7fd61810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.069+0000 7fd61fd49700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fe0d0 0x7fd61810dfa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd608002a50 tx=0x7fd6080046b0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.069+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 shutdown_connections 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.069+0000 7fd61fd49700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd60406dca0 0x7fd604070150 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.069+0000 7fd61fd49700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6180fcee0 0x7fd61810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.069+0000 7fd61fd49700 1 --2- 172.21.15.94:0/1508653992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6180fe0d0 0x7fd61810dfa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.069+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 >> 172.21.15.94:0/1508653992 conn(0x7fd6180f8560 msgr2=0x7fd6180f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:57.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.070+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 shutdown_connections 2024-04-06T03:46:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.070+0000 7fd61fd49700 1 -- 172.21.15.94:0/1508653992 wait complete. 2024-04-06T03:46:57.075 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:46:57.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:56 smithi136 ceph-mon[82061]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:57.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:56 smithi136 ceph-mon[82061]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:57.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:57.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:46:57.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:56 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3853427568' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:46:57.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 -- 172.21.15.94:0/2911204564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fcf80 msgr2=0x7f033c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 --2- 172.21.15.94:0/2911204564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fcf80 0x7f033c0fd3f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0328004940 tx=0x7f0328009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:57.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 -- 172.21.15.94:0/2911204564 shutdown_connections 2024-04-06T03:46:57.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 --2- 172.21.15.94:0/2911204564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fcf80 0x7f033c0fd3f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 --2- 172.21.15.94:0/2911204564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f033c0fbcd0 0x7f033c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 -- 172.21.15.94:0/2911204564 >> 172.21.15.94:0/2911204564 conn(0x7f033c0f7430 msgr2=0x7f033c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 -- 172.21.15.94:0/2911204564 shutdown_connections 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 -- 172.21.15.94:0/2911204564 wait complete. 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 Processor -- start 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.172+0000 7f0342bed700 1 -- start start 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0342bed700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fbcd0 0x7f033c1106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0342bed700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f033c0fcf80 0x7f033c112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0342bed700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c066350 con 0x7f033c0fbcd0 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0342bed700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c066490 con 0x7f033c0fcf80 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0340989700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fbcd0 0x7f033c1106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0340989700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fbcd0 0x7f033c1106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51598/0 (socket says 172.21.15.94:51598) 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0340989700 1 -- 172.21.15.94:0/2722223239 learned_addr learned my addr 172.21.15.94:0/2722223239 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:57.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f033bfff700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f033c0fcf80 0x7f033c112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:57.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0340989700 1 -- 172.21.15.94:0/2722223239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f033c0fcf80 msgr2=0x7f033c112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0340989700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f033c0fcf80 0x7f033c112bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0340989700 1 -- 172.21.15.94:0/2722223239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328005040 con 0x7f033c0fbcd0 2024-04-06T03:46:57.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0340989700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fbcd0 0x7f033c1106a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f033000eb40 tx=0x7f033000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:57.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f03397fa700 1 -- 172.21.15.94:0/2722223239 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0330007cc0 con 0x7f033c0fbcd0 2024-04-06T03:46:57.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c113130 con 0x7f033c0fbcd0 2024-04-06T03:46:57.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f03397fa700 1 -- 172.21.15.94:0/2722223239 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f033000f040 con 0x7f033c0fbcd0 2024-04-06T03:46:57.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.173+0000 7f03397fa700 1 -- 172.21.15.94:0/2722223239 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f033000ca10 con 0x7f033c0fbcd0 2024-04-06T03:46:57.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.174+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c113670 con 0x7f033c0fbcd0 2024-04-06T03:46:57.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.174+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c05f5f0 con 0x7f033c0fbcd0 2024-04-06T03:46:57.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.175+0000 7f03397fa700 1 -- 172.21.15.94:0/2722223239 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f033000cc80 con 0x7f033c0fbcd0 2024-04-06T03:46:57.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.175+0000 7f03397fa700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f032c06dca0 0x7f032c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:57.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.176+0000 7f033bfff700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f032c06dca0 0x7f032c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:57.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.176+0000 7f03397fa700 1 -- 172.21.15.94:0/2722223239 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f033008d090 con 0x7f033c0fbcd0 2024-04-06T03:46:57.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.176+0000 7f033bfff700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f032c06dca0 0x7f032c070150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0328009b30 tx=0x7f0328009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:57.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.180+0000 7f03397fa700 1 -- 172.21.15.94:0/2722223239 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0330056770 con 0x7f033c0fbcd0 2024-04-06T03:46:57.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.391+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f033c1012a0 con 0x7f032c06dca0 2024-04-06T03:46:57.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.392+0000 7f03397fa700 1 -- 172.21.15.94:0/2722223239 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f033c1012a0 con 0x7f032c06dca0 2024-04-06T03:46:57.395 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:46:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:46:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:46:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:46:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:46:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:46:57.395 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:46:57.396 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:46:57.396 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:46:57.396 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "11/21 daemons upgraded", 2024-04-06T03:46:57.396 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-04-06T03:46:57.396 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:46:57.396 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:46:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.394+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f032c06dca0 msgr2=0x7f032c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.394+0000 7f0342bed700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f032c06dca0 0x7f032c070150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0328009b30 tx=0x7f0328009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.394+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fbcd0 msgr2=0x7f033c1106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.394+0000 7f0342bed700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fbcd0 0x7f033c1106a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f033000eb40 tx=0x7f033000b040 comp rx=0 tx=0).stop 2024-04-06T03:46:57.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.395+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 shutdown_connections 2024-04-06T03:46:57.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.395+0000 7f0342bed700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f033c0fbcd0 0x7f033c1106a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.395+0000 7f0342bed700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f032c06dca0 0x7f032c070150 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.395+0000 7f0342bed700 1 --2- 172.21.15.94:0/2722223239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f033c0fcf80 0x7f033c112bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.395+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 >> 172.21.15.94:0/2722223239 conn(0x7f033c0f7430 msgr2=0x7f033c0f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:57.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.395+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 shutdown_connections 2024-04-06T03:46:57.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.395+0000 7f0342bed700 1 -- 172.21.15.94:0/2722223239 wait complete. 2024-04-06T03:46:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.495+0000 7f3d1e038700 1 -- 172.21.15.94:0/2614134766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 msgr2=0x7f3d180fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.495+0000 7f3d1e038700 1 --2- 172.21.15.94:0/2614134766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 0x7f3d180fd420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004940 tx=0x7f3d00009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.495+0000 7f3d1e038700 1 -- 172.21.15.94:0/2614134766 shutdown_connections 2024-04-06T03:46:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.495+0000 7f3d1e038700 1 --2- 172.21.15.94:0/2614134766 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 0x7f3d180fd420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.495+0000 7f3d1e038700 1 --2- 172.21.15.94:0/2614134766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d180fbd00 0x7f3d180fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.495+0000 7f3d1e038700 1 -- 172.21.15.94:0/2614134766 >> 172.21.15.94:0/2614134766 conn(0x7f3d180f7480 msgr2=0x7f3d180f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.495+0000 7f3d1e038700 1 -- 172.21.15.94:0/2614134766 shutdown_connections 2024-04-06T03:46:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.495+0000 7f3d1e038700 1 -- 172.21.15.94:0/2614134766 wait complete. 2024-04-06T03:46:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.495+0000 7f3d1e038700 1 Processor -- start 2024-04-06T03:46:57.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d1e038700 1 -- start start 2024-04-06T03:46:57.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d1e038700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d180fbd00 0x7f3d18066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:57.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d1e038700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 0x7f3d18112710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:57.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d1e038700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d18112c50 con 0x7f3d180fcfb0 2024-04-06T03:46:57.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d1e038700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d18112d90 con 0x7f3d180fbd00 2024-04-06T03:46:57.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d0effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 0x7f3d18112710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:57.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d0effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 0x7f3d18112710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59978/0 (socket says 172.21.15.94:59978) 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d0effd700 1 -- 172.21.15.94:0/3157022553 learned_addr learned my addr 172.21.15.94:0/3157022553 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d177fe700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d180fbd00 0x7f3d18066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d0effd700 1 -- 172.21.15.94:0/3157022553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d180fbd00 msgr2=0x7f3d18066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d0effd700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d180fbd00 0x7f3d18066d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d0effd700 1 -- 172.21.15.94:0/3157022553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d00005040 con 0x7f3d180fcfb0 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.496+0000 7f3d0effd700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 0x7f3d18112710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009150 tx=0x7f3d00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.497+0000 7f3d14ff9700 1 -- 172.21.15.94:0/3157022553 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0000e040 con 0x7f3d180fcfb0 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.497+0000 7f3d14ff9700 1 -- 172.21.15.94:0/3157022553 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d00004290 con 0x7f3d180fcfb0 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.497+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d18112fc0 con 0x7f3d180fcfb0 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.497+0000 7f3d14ff9700 1 -- 172.21.15.94:0/3157022553 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0001c630 con 0x7f3d180fcfb0 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.497+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d181134f0 con 0x7f3d180fcfb0 2024-04-06T03:46:57.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.497+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1804c710 con 0x7f3d180fcfb0 2024-04-06T03:46:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.498+0000 7f3d14ff9700 1 -- 172.21.15.94:0/3157022553 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f3d0000bbf0 con 0x7f3d180fcfb0 2024-04-06T03:46:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.498+0000 7f3d14ff9700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d0406dca0 0x7f3d04070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:46:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.499+0000 7f3d14ff9700 1 -- 172.21.15.94:0/3157022553 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f3d0008e120 con 0x7f3d180fcfb0 2024-04-06T03:46:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.499+0000 7f3d177fe700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d0406dca0 0x7f3d04070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:46:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.499+0000 7f3d177fe700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d0406dca0 0x7f3d04070150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d08004f40 tx=0x7f3d0800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:46:57.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.503+0000 7f3d14ff9700 1 -- 172.21.15.94:0/3157022553 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3d0005b300 con 0x7f3d180fcfb0 2024-04-06T03:46:57.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.781+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3d181047f0 con 0x7f3d180fcfb0 2024-04-06T03:46:57.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.782+0000 7f3d14ff9700 1 -- 172.21.15.94:0/3157022553 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1196 (secure 0 0 0) 0x7f3d0005ae90 con 0x7f3d180fcfb0 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 osds down; Degraded data redundancy: 36/219 objects degraded (16.438%), 14 pgs degraded 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: osd.4 (root=default,host=smithi136) is down 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 36/219 objects degraded (16.438%), 14 pgs degraded 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-04-06T03:46:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-04-06T03:46:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-04-06T03:46:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-04-06T03:46:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-04-06T03:46:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-04-06T03:46:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-04-06T03:46:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-04-06T03:46:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-04-06T03:46:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-04-06T03:46:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-04-06T03:46:57.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d0406dca0 msgr2=0x7f3d04070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d0406dca0 0x7f3d04070150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d08004f40 tx=0x7f3d0800b040 comp rx=0 tx=0).stop 2024-04-06T03:46:57.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 msgr2=0x7f3d18112710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:46:57.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 0x7f3d18112710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009150 tx=0x7f3d00009e30 comp rx=0 tx=0).stop 2024-04-06T03:46:57.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 shutdown_connections 2024-04-06T03:46:57.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d0406dca0 0x7f3d04070150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d180fbd00 0x7f3d18066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 --2- 172.21.15.94:0/3157022553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d180fcfb0 0x7f3d18112710 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:46:57.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 >> 172.21.15.94:0/3157022553 conn(0x7f3d180f7480 msgr2=0x7f3d180f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:46:57.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 shutdown_connections 2024-04-06T03:46:57.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:46:57.784+0000 7f3d1e038700 1 -- 172.21.15.94:0/3157022553 wait complete. 2024-04-06T03:46:58.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:57 smithi136 ceph-mon[82061]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:58.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:57 smithi136 ceph-mon[82061]: pgmap v149: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:46:58.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:57 smithi136 ceph-mon[82061]: Health check update: Degraded data redundancy: 36/219 objects degraded (16.438%), 14 pgs degraded (PG_DEGRADED) 2024-04-06T03:46:58.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:57 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1508653992' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:46:58.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:57 smithi094 ceph-mon[94053]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:58.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:57 smithi094 ceph-mon[94053]: pgmap v149: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:46:58.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:57 smithi094 ceph-mon[94053]: Health check update: Degraded data redundancy: 36/219 objects degraded (16.438%), 14 pgs degraded (PG_DEGRADED) 2024-04-06T03:46:58.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:57 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1508653992' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:46:58.530 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:58 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:46:58.233+0000 7f3d37dc57c0 -1 osd.4 68 log_to_monitors true 2024-04-06T03:46:58.786 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:58 smithi136 ceph-mon[82061]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:58.787 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:58 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3157022553' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:46:58.787 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:58 smithi136 ceph-mon[82061]: from='osd.4 [v2:172.21.15.136:6808/3530002524,v1:172.21.15.136:6809/3530002524]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-06T03:46:58.787 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:58 smithi136 ceph-mon[82061]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-06T03:46:59.099 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:46:58 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:46:58.798+0000 7f3d2921d700 -1 osd.4 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:46:59.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:58 smithi094 ceph-mon[94053]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:46:59.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:58 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3157022553' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:46:59.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:58 smithi094 ceph-mon[94053]: from='osd.4 [v2:172.21.15.136:6808/3530002524,v1:172.21.15.136:6809/3530002524]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-06T03:46:59.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:58 smithi094 ceph-mon[94053]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-06T03:47:00.041 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:59 smithi136 ceph-mon[82061]: pgmap v150: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:47:00.041 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:59 smithi136 ceph-mon[82061]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-06T03:47:00.041 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:59 smithi136 ceph-mon[82061]: osdmap e71: 6 total, 5 up, 6 in 2024-04-06T03:47:00.041 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:59 smithi136 ceph-mon[82061]: from='osd.4 [v2:172.21.15.136:6808/3530002524,v1:172.21.15.136:6809/3530002524]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:47:00.041 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:46:59 smithi136 ceph-mon[82061]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:47:00.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:59 smithi094 ceph-mon[94053]: pgmap v150: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:47:00.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:59 smithi094 ceph-mon[94053]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-06T03:47:00.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:59 smithi094 ceph-mon[94053]: osdmap e71: 6 total, 5 up, 6 in 2024-04-06T03:47:00.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:59 smithi094 ceph-mon[94053]: from='osd.4 [v2:172.21.15.136:6808/3530002524,v1:172.21.15.136:6809/3530002524]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:47:00.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:46:59 smithi094 ceph-mon[94053]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:47:01.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:47:01.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: osd.4 [v2:172.21.15.136:6808/3530002524,v1:172.21.15.136:6809/3530002524] boot 2024-04-06T03:47:01.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: osdmap e72: 6 total, 6 up, 6 in 2024-04-06T03:47:01.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: pgmap v153: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: pgmap v154: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-06T03:47:01.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:00 smithi136 ceph-mon[82061]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-06T03:47:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:47:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: osd.4 [v2:172.21.15.136:6808/3530002524,v1:172.21.15.136:6809/3530002524] boot 2024-04-06T03:47:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: osdmap e72: 6 total, 6 up, 6 in 2024-04-06T03:47:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-06T03:47:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:47:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:47:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: pgmap v153: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:47:01.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: pgmap v154: 65 pgs: 14 active+undersized+degraded, 20 active+undersized, 31 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 36/219 objects degraded (16.438%) 2024-04-06T03:47:01.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:01.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:47:01.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:01.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:01.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:01.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-06T03:47:01.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-06T03:47:01.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:00 smithi094 ceph-mon[94053]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-06T03:47:01.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:01 smithi136 ceph-mon[82061]: osdmap e73: 6 total, 6 up, 6 in 2024-04-06T03:47:02.069 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:01 smithi094 ceph-mon[94053]: osdmap e73: 6 total, 6 up, 6 in 2024-04-06T03:47:03.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:02 smithi136 ceph-mon[82061]: pgmap v156: 65 pgs: 16 peering, 2 active+undersized+degraded, 5 active+undersized, 42 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3/219 objects degraded (1.370%) 2024-04-06T03:47:03.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:02 smithi094 ceph-mon[94053]: pgmap v156: 65 pgs: 16 peering, 2 active+undersized+degraded, 5 active+undersized, 42 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3/219 objects degraded (1.370%) 2024-04-06T03:47:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:03 smithi136 ceph-mon[82061]: Health check update: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded (PG_DEGRADED) 2024-04-06T03:47:04.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:03 smithi094 ceph-mon[94053]: Health check update: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded (PG_DEGRADED) 2024-04-06T03:47:05.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:04 smithi136 ceph-mon[82061]: pgmap v157: 65 pgs: 16 peering, 1 active+undersized, 48 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:05.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:04 smithi094 ceph-mon[94053]: pgmap v157: 65 pgs: 16 peering, 1 active+undersized, 48 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:06.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:05 smithi136 ceph-mon[82061]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded) 2024-04-06T03:47:06.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:05 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:06.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:05 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:47:06.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:05 smithi094 ceph-mon[94053]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/219 objects degraded (1.370%), 2 pgs degraded) 2024-04-06T03:47:06.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:05 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:06.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:05 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:47:07.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:06 smithi136 ceph-mon[82061]: pgmap v158: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:07.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:06 smithi094 ceph-mon[94053]: pgmap v158: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:09.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:09 smithi094 ceph-mon[94053]: pgmap v159: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:09.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:09 smithi136 ceph-mon[82061]: pgmap v159: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:11.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:11 smithi094 ceph-mon[94053]: pgmap v160: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:11.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:11 smithi136 ceph-mon[82061]: pgmap v160: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:13.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:13 smithi094 ceph-mon[94053]: pgmap v161: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:13.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:13 smithi136 ceph-mon[82061]: pgmap v161: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:15.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:15 smithi094 ceph-mon[94053]: pgmap v162: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:15.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:15 smithi136 ceph-mon[82061]: pgmap v162: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:16.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-06T03:47:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:16 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-06T03:47:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:16 smithi094 ceph-mon[94053]: Upgrade: osd.5 is safe to restart 2024-04-06T03:47:16.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:16.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-06T03:47:16.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:16 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-06T03:47:16.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:16 smithi136 ceph-mon[82061]: Upgrade: osd.5 is safe to restart 2024-04-06T03:47:16.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:17.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:17 smithi094 ceph-mon[94053]: pgmap v163: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:17.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:17.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-06T03:47:17.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:47:17.474 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:17 smithi136 ceph-mon[82061]: pgmap v163: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:17.474 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:17.475 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-06T03:47:17.475 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:47:18.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:18 smithi136 ceph-mon[82061]: Upgrade: Updating osd.5 2024-04-06T03:47:18.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:18 smithi136 ceph-mon[82061]: Deploying daemon osd.5 on smithi136 2024-04-06T03:47:18.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:18 smithi094 ceph-mon[94053]: Upgrade: Updating osd.5 2024-04-06T03:47:18.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:18 smithi094 ceph-mon[94053]: Deploying daemon osd.5 on smithi136 2024-04-06T03:47:19.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:19 smithi094 ceph-mon[94053]: pgmap v164: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:19.499 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:19 smithi136 ceph-mon[82061]: pgmap v164: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:20.119 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:19 smithi136 systemd[1]: Stopping Ceph osd.5 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:47:20.119 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:20 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[54793]: 2024-04-06T03:47:20.093+0000 7fcf2901f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:47:20.119 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:20 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[54793]: 2024-04-06T03:47:20.093+0000 7fcf2901f700 -1 osd.5 73 *** Got signal Terminated *** 2024-04-06T03:47:20.119 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:20 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[54793]: 2024-04-06T03:47:20.093+0000 7fcf2901f700 -1 osd.5 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:47:20.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:20 smithi094 ceph-mon[94053]: osd.5 marked itself down and dead 2024-04-06T03:47:20.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:20 smithi136 ceph-mon[82061]: osd.5 marked itself down and dead 2024-04-06T03:47:20.994 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:20 smithi136 podman[101657]: 2024-04-06 03:47:20.693813279 +0000 UTC m=+0.688738664 container died 8653ba3da3ac5535bc4cf8dd896b2beda71100ee95772e4132460a607f1688aa (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5, ceph=True, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-6699495) 2024-04-06T03:47:20.995 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:20 smithi136 podman[101657]: 2024-04-06 03:47:20.993283154 +0000 UTC m=+0.988208536 container cleanup 8653ba3da3ac5535bc4cf8dd896b2beda71100ee95772e4132460a607f1688aa (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, GIT_CLEAN=True, RELEASE=quincy-6699495, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-06T03:47:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:21 smithi136 ceph-mon[82061]: pgmap v165: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:21 smithi136 ceph-mon[82061]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:47:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:21 smithi136 ceph-mon[82061]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-06T03:47:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:21 smithi136 ceph-mon[82061]: osdmap e74: 6 total, 5 up, 6 in 2024-04-06T03:47:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:21.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:47:21.350 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:20 smithi136 bash[101657]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5 2024-04-06T03:47:21.350 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:21 smithi136 podman[101684]: 2024-04-06 03:47:21.22824611 +0000 UTC m=+0.533275031 container remove 8653ba3da3ac5535bc4cf8dd896b2beda71100ee95772e4132460a607f1688aa (image=quay.ceph.io/ceph-ci/ceph@sha256:d049538dc20db76390a0232d25def63f9e4809620d469762c761112812aa14b8, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=quincy-6699495, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240403, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:47:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:21 smithi094 ceph-mon[94053]: pgmap v165: 65 pgs: 65 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-06T03:47:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:21 smithi094 ceph-mon[94053]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-06T03:47:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:21 smithi094 ceph-mon[94053]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-06T03:47:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:21 smithi094 ceph-mon[94053]: osdmap e74: 6 total, 5 up, 6 in 2024-04-06T03:47:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:21.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:47:21.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:21 smithi136 podman[101773]: 2024-04-06 03:47:21.553951981 +0000 UTC m=+0.112225547 container create f0eb8137df1a27734e8b4fc91fba2120192b6cf3b8e76c48b7e31dd80232fc95 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-9f19235, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:47:21.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:21 smithi136 podman[101773]: 2024-04-06 03:47:21.459173952 +0000 UTC m=+0.017447544 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:47:21.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:21 smithi136 podman[101773]: 2024-04-06 03:47:21.790114293 +0000 UTC m=+0.348387860 container init f0eb8137df1a27734e8b4fc91fba2120192b6cf3b8e76c48b7e31dd80232fc95 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:47:21.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:21 smithi136 podman[101773]: 2024-04-06 03:47:21.793252797 +0000 UTC m=+0.351526370 container start f0eb8137df1a27734e8b4fc91fba2120192b6cf3b8e76c48b7e31dd80232fc95 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-06T03:47:22.178 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:21 smithi136 podman[101773]: 2024-04-06 03:47:21.869316437 +0000 UTC m=+0.427590011 container attach f0eb8137df1a27734e8b4fc91fba2120192b6cf3b8e76c48b7e31dd80232fc95 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403) 2024-04-06T03:47:22.178 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:22 smithi136 podman[101773]: 2024-04-06 03:47:22.080433408 +0000 UTC m=+0.638706993 container died f0eb8137df1a27734e8b4fc91fba2120192b6cf3b8e76c48b7e31dd80232fc95 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0) 2024-04-06T03:47:22.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:22 smithi136 ceph-mon[82061]: osdmap e75: 6 total, 5 up, 6 in 2024-04-06T03:47:22.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:22 smithi094 ceph-mon[94053]: osdmap e75: 6 total, 5 up, 6 in 2024-04-06T03:47:23.099 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:22 smithi136 podman[101799]: 2024-04-06 03:47:22.693053444 +0000 UTC m=+0.607278767 container remove f0eb8137df1a27734e8b4fc91fba2120192b6cf3b8e76c48b7e31dd80232fc95 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:47:23.099 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:22 smithi136 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.5.service: Deactivated successfully. 2024-04-06T03:47:23.099 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:22 smithi136 systemd[1]: Stopped Ceph osd.5 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:47:23.099 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:22 smithi136 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.5.service: Consumed 1min 5.034s CPU time. 2024-04-06T03:47:23.099 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:22 smithi136 systemd[1]: Starting Ceph osd.5 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:47:23.565 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:23 smithi136 ceph-mon[82061]: pgmap v168: 65 pgs: 2 active+undersized+degraded, 3 peering, 10 stale+active+clean, 1 active+undersized, 49 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2/219 objects degraded (0.913%) 2024-04-06T03:47:23.565 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:23 smithi136 ceph-mon[82061]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-06T03:47:23.565 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:23 smithi136 ceph-mon[82061]: Health check failed: Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded (PG_DEGRADED) 2024-04-06T03:47:23.565 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:23 smithi136 podman[101902]: 2024-04-06 03:47:23.02515963 +0000 UTC m=+0.018030352 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:47:23.565 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:23 smithi136 podman[101902]: 2024-04-06 03:47:23.143655861 +0000 UTC m=+0.136526622 container create a5fc8539dd18390b5ddd12f031f32872629b7e565e2e95d7f7ccf5848d9bd132 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate, RELEASE=squid-9f19235, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux ) 2024-04-06T03:47:23.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:23 smithi094 ceph-mon[94053]: pgmap v168: 65 pgs: 2 active+undersized+degraded, 3 peering, 10 stale+active+clean, 1 active+undersized, 49 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2/219 objects degraded (0.913%) 2024-04-06T03:47:23.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:23 smithi094 ceph-mon[94053]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-06T03:47:23.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:23 smithi094 ceph-mon[94053]: Health check failed: Degraded data redundancy: 2/219 objects degraded (0.913%), 2 pgs degraded (PG_DEGRADED) 2024-04-06T03:47:23.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:23 smithi136 podman[101902]: 2024-04-06 03:47:23.563446268 +0000 UTC m=+0.556316992 container init a5fc8539dd18390b5ddd12f031f32872629b7e565e2e95d7f7ccf5848d9bd132 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-9f19235, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:47:23.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:23 smithi136 podman[101902]: 2024-04-06 03:47:23.566953251 +0000 UTC m=+0.559823978 container start a5fc8539dd18390b5ddd12f031f32872629b7e565e2e95d7f7ccf5848d9bd132 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9f19235, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:47:23.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:23 smithi136 podman[101902]: 2024-04-06 03:47:23.64621776 +0000 UTC m=+0.639088483 container attach a5fc8539dd18390b5ddd12f031f32872629b7e565e2e95d7f7ccf5848d9bd132 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, RELEASE=squid-9f19235) 2024-04-06T03:47:24.285 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate[101917]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-06T03:47:24.285 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate[101917]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-06T03:47:24.285 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 bash[101902]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-06T03:47:24.285 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 bash[101902]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-06T03:47:24.285 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 bash[101902]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-06T03:47:24.285 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 bash[101902]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-06T03:47:24.285 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 bash[101902]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-06T03:47:24.285 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 bash[101902]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-06T03:47:24.286 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 bash[101902]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-06T03:47:24.286 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate[101917]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-06T03:47:24.286 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate[101917]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-06T03:47:24.286 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate[101917]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-06T03:47:24.286 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate[101917]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-06T03:47:24.286 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate[101917]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-06T03:47:24.286 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 podman[101902]: 2024-04-06 03:47:24.031608663 +0000 UTC m=+1.024479374 container died a5fc8539dd18390b5ddd12f031f32872629b7e565e2e95d7f7ccf5848d9bd132 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403) 2024-04-06T03:47:24.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 podman[102036]: 2024-04-06 03:47:24.549748004 +0000 UTC m=+0.512644031 container remove a5fc8539dd18390b5ddd12f031f32872629b7e565e2e95d7f7ccf5848d9bd132 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-06T03:47:24.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 podman[102097]: 2024-04-06 03:47:24.664729211 +0000 UTC m=+0.017814462 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:47:24.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:24 smithi136 podman[102097]: 2024-04-06 03:47:24.767104591 +0000 UTC m=+0.120189874 container create 16386e8f8583d884ca32bdbeccc2c5dcf72e51a2b42696e719da0d39095b151e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-9f19235) 2024-04-06T03:47:25.182 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:25 smithi136 podman[102097]: 2024-04-06 03:47:25.002581694 +0000 UTC m=+0.355666932 container init 16386e8f8583d884ca32bdbeccc2c5dcf72e51a2b42696e719da0d39095b151e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-9f19235, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-06T03:47:25.182 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:25 smithi136 podman[102097]: 2024-04-06 03:47:25.005578474 +0000 UTC m=+0.358663715 container start 16386e8f8583d884ca32bdbeccc2c5dcf72e51a2b42696e719da0d39095b151e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:47:25.182 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:25 smithi136 ceph-osd[102116]: -- 172.21.15.136:0/356095262 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624f150d680 con 0x5624f14d7000 2024-04-06T03:47:25.182 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:25 smithi136 bash[102097]: 16386e8f8583d884ca32bdbeccc2c5dcf72e51a2b42696e719da0d39095b151e 2024-04-06T03:47:25.182 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:25 smithi136 systemd[1]: Started Ceph osd.5 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:47:25.557 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:25 smithi136 ceph-mon[82061]: pgmap v169: 65 pgs: 5 active+undersized+degraded, 3 peering, 6 stale+active+clean, 5 active+undersized, 46 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 8/219 objects degraded (3.653%) 2024-04-06T03:47:25.557 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:25 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:25.557 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:25 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:25.557 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:25 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:47:25.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:25 smithi094 ceph-mon[94053]: pgmap v169: 65 pgs: 5 active+undersized+degraded, 3 peering, 6 stale+active+clean, 5 active+undersized, 46 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 8/219 objects degraded (3.653%) 2024-04-06T03:47:25.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:25 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:25.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:25 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:25.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:25 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:47:25.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:25 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:47:25.554+0000 7fc0b351c7c0 -1 Falling back to public interface 2024-04-06T03:47:26.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:26 smithi136 ceph-mon[82061]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-06T03:47:26.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:26 smithi094 ceph-mon[94053]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-06T03:47:27.554 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:27 smithi136 ceph-mon[82061]: pgmap v170: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:27.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:27 smithi094 ceph-mon[94053]: pgmap v170: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.899+0000 7f16435b7700 1 -- 172.21.15.94:0/334697455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fe0d0 msgr2=0x7f163c0fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.899+0000 7f16435b7700 1 --2- 172.21.15.94:0/334697455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fe0d0 0x7f163c0fe540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f162c004940 tx=0x7f162c009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.899+0000 7f16435b7700 1 -- 172.21.15.94:0/334697455 shutdown_connections 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.899+0000 7f16435b7700 1 --2- 172.21.15.94:0/334697455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fe0d0 0x7f163c0fe540 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.899+0000 7f16435b7700 1 --2- 172.21.15.94:0/334697455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f163c0fcee0 0x7f163c0fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.899+0000 7f16435b7700 1 -- 172.21.15.94:0/334697455 >> 172.21.15.94:0/334697455 conn(0x7f163c0f8560 msgr2=0x7f163c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.899+0000 7f16435b7700 1 -- 172.21.15.94:0/334697455 shutdown_connections 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.899+0000 7f16435b7700 1 -- 172.21.15.94:0/334697455 wait complete. 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f16435b7700 1 Processor -- start 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f16435b7700 1 -- start start 2024-04-06T03:47:27.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f16435b7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fcee0 0x7f163c10dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f16435b7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f163c0fe0d0 0x7f163c10e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f16435b7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c10e600 con 0x7f163c0fcee0 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f16435b7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c10ede0 con 0x7f163c0fe0d0 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f1641353700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fcee0 0x7f163c10dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f1641353700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fcee0 0x7f163c10dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43780/0 (socket says 172.21.15.94:43780) 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f1641353700 1 -- 172.21.15.94:0/3746624799 learned_addr learned my addr 172.21.15.94:0/3746624799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.900+0000 7f1640b52700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f163c0fe0d0 0x7f163c10e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.901+0000 7f1641353700 1 -- 172.21.15.94:0/3746624799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f163c0fe0d0 msgr2=0x7f163c10e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.901+0000 7f1641353700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f163c0fe0d0 0x7f163c10e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.901+0000 7f1641353700 1 -- 172.21.15.94:0/3746624799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c005040 con 0x7f163c0fcee0 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.901+0000 7f1641353700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fcee0 0x7f163c10dac0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1638002a80 tx=0x7f1638004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.901+0000 7f1631ffb700 1 -- 172.21.15.94:0/3746624799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1638007cc0 con 0x7f163c0fcee0 2024-04-06T03:47:27.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.901+0000 7f1631ffb700 1 -- 172.21.15.94:0/3746624799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f163800f040 con 0x7f163c0fcee0 2024-04-06T03:47:27.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.901+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c10f0c0 con 0x7f163c0fcee0 2024-04-06T03:47:27.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.901+0000 7f1631ffb700 1 -- 172.21.15.94:0/3746624799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163800da30 con 0x7f163c0fcee0 2024-04-06T03:47:27.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.901+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c10f600 con 0x7f163c0fcee0 2024-04-06T03:47:27.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.902+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c05f5f0 con 0x7f163c0fcee0 2024-04-06T03:47:27.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.903+0000 7f1631ffb700 1 -- 172.21.15.94:0/3746624799 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f163800db90 con 0x7f163c0fcee0 2024-04-06T03:47:27.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.903+0000 7f1631ffb700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f162806dca0 0x7f1628070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:27.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.903+0000 7f1631ffb700 1 -- 172.21.15.94:0/3746624799 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f163808d260 con 0x7f163c0fcee0 2024-04-06T03:47:27.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.903+0000 7f1640b52700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f162806dca0 0x7f1628070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:27.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.904+0000 7f1640b52700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f162806dca0 0x7f1628070150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f162c00b970 tx=0x7f162c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:27.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:27.908+0000 7f1631ffb700 1 -- 172.21.15.94:0/3746624799 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f163805a3c0 con 0x7f163c0fcee0 2024-04-06T03:47:28.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.127+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f163c10f990 con 0x7f162806dca0 2024-04-06T03:47:28.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.128+0000 7f1631ffb700 1 -- 172.21.15.94:0/3746624799 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f163c10f990 con 0x7f162806dca0 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f162806dca0 msgr2=0x7f1628070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f162806dca0 0x7f1628070150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f162c00b970 tx=0x7f162c009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fcee0 msgr2=0x7f163c10dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fcee0 0x7f163c10dac0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1638002a80 tx=0x7f1638004600 comp rx=0 tx=0).stop 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 shutdown_connections 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f163c0fcee0 0x7f163c10dac0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f162806dca0 0x7f1628070150 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 --2- 172.21.15.94:0/3746624799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f163c0fe0d0 0x7f163c10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 >> 172.21.15.94:0/3746624799 conn(0x7f163c0f8560 msgr2=0x7f163c0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 shutdown_connections 2024-04-06T03:47:28.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.130+0000 7f16435b7700 1 -- 172.21.15.94:0/3746624799 wait complete. 2024-04-06T03:47:28.143 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:47:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.232+0000 7f5cc369a700 1 -- 172.21.15.94:0/3103638072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 msgr2=0x7f5cbc0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.232+0000 7f5cc369a700 1 --2- 172.21.15.94:0/3103638072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 0x7f5cbc0fd280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8004940 tx=0x7f5cb8009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.232+0000 7f5cc369a700 1 -- 172.21.15.94:0/3103638072 shutdown_connections 2024-04-06T03:47:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.232+0000 7f5cc369a700 1 --2- 172.21.15.94:0/3103638072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cbc0fdfe0 0x7f5cbc0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.232+0000 7f5cc369a700 1 --2- 172.21.15.94:0/3103638072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 0x7f5cbc0fd280 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.232+0000 7f5cc369a700 1 -- 172.21.15.94:0/3103638072 >> 172.21.15.94:0/3103638072 conn(0x7f5cbc0f8550 msgr2=0x7f5cbc0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc369a700 1 -- 172.21.15.94:0/3103638072 shutdown_connections 2024-04-06T03:47:28.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc369a700 1 -- 172.21.15.94:0/3103638072 wait complete. 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc369a700 1 Processor -- start 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc369a700 1 -- start start 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc369a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 0x7f5cbc071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc369a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cbc0fdfe0 0x7f5cbc070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc369a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc072260 con 0x7f5cbc0fce70 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc369a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc070850 con 0x7f5cbc0fdfe0 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc1436700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 0x7f5cbc071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc1436700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 0x7f5cbc071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43800/0 (socket says 172.21.15.94:43800) 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.233+0000 7f5cc1436700 1 -- 172.21.15.94:0/2798316938 learned_addr learned my addr 172.21.15.94:0/2798316938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.234+0000 7f5cc0c35700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cbc0fdfe0 0x7f5cbc070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:28.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.234+0000 7f5cc1436700 1 -- 172.21.15.94:0/2798316938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cbc0fdfe0 msgr2=0x7f5cbc070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.234+0000 7f5cc1436700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cbc0fdfe0 0x7f5cbc070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.234+0000 7f5cc1436700 1 -- 172.21.15.94:0/2798316938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb8005040 con 0x7f5cbc0fce70 2024-04-06T03:47:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.234+0000 7f5cc1436700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 0x7f5cbc071c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b3b0 tx=0x7f5cb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.234+0000 7f5cb1ffb700 1 -- 172.21.15.94:0/2798316938 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb800e040 con 0x7f5cbc0fce70 2024-04-06T03:47:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.234+0000 7f5cb1ffb700 1 -- 172.21.15.94:0/2798316938 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cb800bbf0 con 0x7f5cbc0fce70 2024-04-06T03:47:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.234+0000 7f5cb1ffb700 1 -- 172.21.15.94:0/2798316938 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb801c630 con 0x7f5cbc0fce70 2024-04-06T03:47:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.235+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc070aa0 con 0x7f5cbc0fce70 2024-04-06T03:47:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.235+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc071060 con 0x7f5cbc0fce70 2024-04-06T03:47:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.236+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc04c710 con 0x7f5cbc0fce70 2024-04-06T03:47:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.236+0000 7f5cb1ffb700 1 -- 172.21.15.94:0/2798316938 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f5cb800bd60 con 0x7f5cbc0fce70 2024-04-06T03:47:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.237+0000 7f5cb1ffb700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5ca806dca0 0x7f5ca8070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.237+0000 7f5cb1ffb700 1 -- 172.21.15.94:0/2798316938 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5cb808dfd0 con 0x7f5cbc0fce70 2024-04-06T03:47:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.237+0000 7f5cc0c35700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5ca806dca0 0x7f5ca8070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.237+0000 7f5cc0c35700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5ca806dca0 0x7f5ca8070150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5cac004f40 tx=0x7f5cac00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:28.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.241+0000 7f5cb1ffb700 1 -- 172.21.15.94:0/2798316938 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5cb805b250 con 0x7f5cbc0fce70 2024-04-06T03:47:28.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.453+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cbc102340 con 0x7f5ca806dca0 2024-04-06T03:47:28.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.454+0000 7f5cb1ffb700 1 -- 172.21.15.94:0/2798316938 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5cbc102340 con 0x7f5ca806dca0 2024-04-06T03:47:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5ca806dca0 msgr2=0x7f5ca8070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5ca806dca0 0x7f5ca8070150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5cac004f40 tx=0x7f5cac00b040 comp rx=0 tx=0).stop 2024-04-06T03:47:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 msgr2=0x7f5cbc071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 0x7f5cbc071c60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b3b0 tx=0x7f5cb8009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 shutdown_connections 2024-04-06T03:47:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cbc0fce70 0x7f5cbc071c60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5ca806dca0 0x7f5ca8070150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 --2- 172.21.15.94:0/2798316938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cbc0fdfe0 0x7f5cbc070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 >> 172.21.15.94:0/2798316938 conn(0x7f5cbc0f8550 msgr2=0x7f5cbc0f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:28.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 shutdown_connections 2024-04-06T03:47:28.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.456+0000 7f5cc369a700 1 -- 172.21.15.94:0/2798316938 wait complete. 2024-04-06T03:47:28.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 -- 172.21.15.94:0/1085836633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340fff80 msgr2=0x7ffa34102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 --2- 172.21.15.94:0/1085836633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340fff80 0x7ffa34102360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffa28004940 tx=0x7ffa28009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:28.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 -- 172.21.15.94:0/1085836633 shutdown_connections 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 --2- 172.21.15.94:0/1085836633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340fff80 0x7ffa34102360 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 --2- 172.21.15.94:0/1085836633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa34066eb0 0x7ffa340ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 -- 172.21.15.94:0/1085836633 >> 172.21.15.94:0/1085836633 conn(0x7ffa340f7430 msgr2=0x7ffa340f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 -- 172.21.15.94:0/1085836633 shutdown_connections 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 -- 172.21.15.94:0/1085836633 wait complete. 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 Processor -- start 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.556+0000 7ffa3a027700 1 -- start start 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa3a027700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa34066eb0 0x7ffa3410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa3a027700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340fff80 0x7ffa3410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa3a027700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3410e490 con 0x7ffa340fff80 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa3a027700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3410e600 con 0x7ffa34066eb0 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340fff80 0x7ffa3410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340fff80 0x7ffa3410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43820/0 (socket says 172.21.15.94:43820) 2024-04-06T03:47:28.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa32ffd700 1 -- 172.21.15.94:0/1374170053 learned_addr learned my addr 172.21.15.94:0/1374170053 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:47:28.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa337fe700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa34066eb0 0x7ffa3410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:28.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa337fe700 1 -- 172.21.15.94:0/1374170053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340fff80 msgr2=0x7ffa3410de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa337fe700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340fff80 0x7ffa3410de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa337fe700 1 -- 172.21.15.94:0/1374170053 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa28005040 con 0x7ffa34066eb0 2024-04-06T03:47:28.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa337fe700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa34066eb0 0x7ffa3410d950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffa24002840 tx=0x7ffa24004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:28.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa1bfff700 1 -- 172.21.15.94:0/1374170053 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa24007cc0 con 0x7ffa34066eb0 2024-04-06T03:47:28.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.558+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa34113060 con 0x7ffa34066eb0 2024-04-06T03:47:28.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.557+0000 7ffa1bfff700 1 -- 172.21.15.94:0/1374170053 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa2400f040 con 0x7ffa34066eb0 2024-04-06T03:47:28.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.558+0000 7ffa1bfff700 1 -- 172.21.15.94:0/1374170053 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa24013920 con 0x7ffa34066eb0 2024-04-06T03:47:28.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.558+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa34113620 con 0x7ffa34066eb0 2024-04-06T03:47:28.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.559+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3404c710 con 0x7ffa34066eb0 2024-04-06T03:47:28.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.559+0000 7ffa1bfff700 1 -- 172.21.15.94:0/1374170053 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7ffa24013b90 con 0x7ffa34066eb0 2024-04-06T03:47:28.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.560+0000 7ffa1bfff700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ffa1c06dca0 0x7ffa1c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:28.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.560+0000 7ffa32ffd700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ffa1c06dca0 0x7ffa1c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:28.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.560+0000 7ffa1bfff700 1 -- 172.21.15.94:0/1374170053 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ffa2408d250 con 0x7ffa34066eb0 2024-04-06T03:47:28.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.560+0000 7ffa32ffd700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ffa1c06dca0 0x7ffa1c070150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa28004f90 tx=0x7ffa28009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:28.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.565+0000 7ffa1bfff700 1 -- 172.21.15.94:0/1374170053 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffa2405a310 con 0x7ffa34066eb0 2024-04-06T03:47:28.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.769+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa340f8f70 con 0x7ffa1c06dca0 2024-04-06T03:47:28.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.780+0000 7ffa1bfff700 1 -- 172.21.15.94:0/1374170053 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ffa340f8f70 con 0x7ffa1c06dca0 2024-04-06T03:47:28.782 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:47:28.782 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (6m) 100s ago 14m 27.6M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:47:28.782 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (3m) 100s ago 15m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:47:28.782 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (3m) 0s ago 14m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:47:28.782 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 unknown 100s ago 14m - - 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (9m) 100s ago 9m 15.0M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (9m) 100s ago 9m 153M - 17.2.7-819-g6699495c ec300eaa768b 3e99d04556bb 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (9m) 0s ago 9m 15.4M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (9m) 0s ago 9m 17.8M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (5m) 100s ago 16m 502M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (4m) 0s ago 13m 451M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (4m) 100s ago 16m 46.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (4m) 0s ago 13m 43.8M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (6m) 100s ago 14m 15.3M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (5m) 0s ago 14m 17.2M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (3m) 100s ago 13m 103M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (2m) 100s ago 12m 107M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (105s) 100s ago 12m 12.0M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:47:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (70s) 0s ago 11m 105M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:47:28.784 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (36s) 0s ago 11m 102M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 c31367e1c2d5 2024-04-06T03:47:28.784 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (4s) 0s ago 10m 11.7M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 16386e8f8583 2024-04-06T03:47:28.784 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (4m) 100s ago 14m 57.0M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:47:28.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ffa1c06dca0 msgr2=0x7ffa1c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ffa1c06dca0 0x7ffa1c070150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa28004f90 tx=0x7ffa28009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa34066eb0 msgr2=0x7ffa3410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa34066eb0 0x7ffa3410d950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffa24002840 tx=0x7ffa24004570 comp rx=0 tx=0).stop 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 shutdown_connections 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ffa1c06dca0 0x7ffa1c070150 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa34066eb0 0x7ffa3410d950 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 --2- 172.21.15.94:0/1374170053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa340fff80 0x7ffa3410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 >> 172.21.15.94:0/1374170053 conn(0x7ffa340f7430 msgr2=0x7ffa340f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 shutdown_connections 2024-04-06T03:47:28.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.782+0000 7ffa3a027700 1 -- 172.21.15.94:0/1374170053 wait complete. 2024-04-06T03:47:28.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.882+0000 7f60fc2ed700 1 -- 172.21.15.94:0/1902471314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fbcd0 msgr2=0x7f60f40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.882+0000 7f60fc2ed700 1 --2- 172.21.15.94:0/1902471314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fbcd0 0x7f60f40fc0e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009ce0 tx=0x7f60ec004d10 comp rx=0 tx=0).stop 2024-04-06T03:47:28.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.882+0000 7f60fc2ed700 1 -- 172.21.15.94:0/1902471314 shutdown_connections 2024-04-06T03:47:28.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.882+0000 7f60fc2ed700 1 --2- 172.21.15.94:0/1902471314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f40fcf80 0x7f60f40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.882+0000 7f60fc2ed700 1 --2- 172.21.15.94:0/1902471314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fbcd0 0x7f60f40fc0e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.882+0000 7f60fc2ed700 1 -- 172.21.15.94:0/1902471314 >> 172.21.15.94:0/1902471314 conn(0x7f60f40f7430 msgr2=0x7f60f40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:28.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.882+0000 7f60fc2ed700 1 -- 172.21.15.94:0/1902471314 shutdown_connections 2024-04-06T03:47:28.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.882+0000 7f60fc2ed700 1 -- 172.21.15.94:0/1902471314 wait complete. 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60fc2ed700 1 Processor -- start 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60fc2ed700 1 -- start start 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60fc2ed700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f40fbcd0 0x7f60f4066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60fc2ed700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fcf80 0x7f60f41128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60fc2ed700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f4112e10 con 0x7f60f40fcf80 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60fc2ed700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f4112f50 con 0x7f60f40fbcd0 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60f9888700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fcf80 0x7f60f41128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60f9888700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fcf80 0x7f60f41128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43832/0 (socket says 172.21.15.94:43832) 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60f9888700 1 -- 172.21.15.94:0/3539367228 learned_addr learned my addr 172.21.15.94:0/3539367228 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60fa089700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f40fbcd0 0x7f60f4066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60f9888700 1 -- 172.21.15.94:0/3539367228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f40fbcd0 msgr2=0x7f60f4066d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:28.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60f9888700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f40fbcd0 0x7f60f4066d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:28.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60f9888700 1 -- 172.21.15.94:0/3539367228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec009940 con 0x7f60f40fcf80 2024-04-06T03:47:28.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.883+0000 7f60f9888700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fcf80 0x7f60f41128d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60e4002a50 tx=0x7f60e40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:28.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.884+0000 7f60eaffd700 1 -- 172.21.15.94:0/3539367228 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e4007cc0 con 0x7f60f40fcf80 2024-04-06T03:47:28.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.884+0000 7f60eaffd700 1 -- 172.21.15.94:0/3539367228 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60e400f040 con 0x7f60f40fcf80 2024-04-06T03:47:28.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.884+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f4113230 con 0x7f60f40fcf80 2024-04-06T03:47:28.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.884+0000 7f60eaffd700 1 -- 172.21.15.94:0/3539367228 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e400da30 con 0x7f60f40fcf80 2024-04-06T03:47:28.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.884+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f41137f0 con 0x7f60f40fcf80 2024-04-06T03:47:28.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.885+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f404c710 con 0x7f60f40fcf80 2024-04-06T03:47:28.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.885+0000 7f60eaffd700 1 -- 172.21.15.94:0/3539367228 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f60e400dc70 con 0x7f60f40fcf80 2024-04-06T03:47:28.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.886+0000 7f60eaffd700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f60e006dca0 0x7f60e0070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:28.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.886+0000 7f60fa089700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f60e006dca0 0x7f60e0070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:28.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.886+0000 7f60eaffd700 1 -- 172.21.15.94:0/3539367228 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f60e408d8f0 con 0x7f60f40fcf80 2024-04-06T03:47:28.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.886+0000 7f60fa089700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f60e006dca0 0x7f60e0070150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60ec015040 tx=0x7f60ec004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:28.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:28.891+0000 7f60eaffd700 1 -- 172.21.15.94:0/3539367228 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f60e405aad0 con 0x7f60f40fcf80 2024-04-06T03:47:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.163+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f60f41047c0 con 0x7f60f40fcf80 2024-04-06T03:47:29.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:29 smithi094 ceph-mon[94053]: pgmap v171: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:29.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:29 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:29.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:29 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:29.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:29 smithi094 ceph-mon[94053]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:47:29.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.164+0000 7f60eaffd700 1 -- 172.21.15.94:0/3539367228 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+732 (secure 0 0 0) 0x7f60e405a660 con 0x7f60f40fcf80 2024-04-06T03:47:29.167 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:47:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:47:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:47:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:47:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:47:29.167 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 5 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 4, 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 9 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:47:29.168 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:47:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f60e006dca0 msgr2=0x7f60e0070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f60e006dca0 0x7f60e0070150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60ec015040 tx=0x7f60ec004d10 comp rx=0 tx=0).stop 2024-04-06T03:47:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fcf80 msgr2=0x7f60f41128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fcf80 0x7f60f41128d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60e4002a50 tx=0x7f60e40046b0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 shutdown_connections 2024-04-06T03:47:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f60e006dca0 0x7f60e0070150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60f40fbcd0 0x7f60f4066d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 --2- 172.21.15.94:0/3539367228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60f40fcf80 0x7f60f41128d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 >> 172.21.15.94:0/3539367228 conn(0x7f60f40f7430 msgr2=0x7f60f40f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 shutdown_connections 2024-04-06T03:47:29.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.166+0000 7f60fc2ed700 1 -- 172.21.15.94:0/3539367228 wait complete. 2024-04-06T03:47:29.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 -- 172.21.15.94:0/2335005516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fce50 msgr2=0x7fe02c0fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 --2- 172.21.15.94:0/2335005516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fce50 0x7fe02c0fd260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe01c004940 tx=0x7fe01c009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:29.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 -- 172.21.15.94:0/2335005516 shutdown_connections 2024-04-06T03:47:29.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 --2- 172.21.15.94:0/2335005516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe02c0fe050 0x7fe02c0fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 --2- 172.21.15.94:0/2335005516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fce50 0x7fe02c0fd260 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 -- 172.21.15.94:0/2335005516 >> 172.21.15.94:0/2335005516 conn(0x7fe02c0f8580 msgr2=0x7fe02c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 -- 172.21.15.94:0/2335005516 shutdown_connections 2024-04-06T03:47:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 -- 172.21.15.94:0/2335005516 wait complete. 2024-04-06T03:47:29.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 Processor -- start 2024-04-06T03:47:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.270+0000 7fe031ea0700 1 -- start start 2024-04-06T03:47:29.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe031ea0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe02c0fce50 0x7fe02c071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe031ea0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fe050 0x7fe02c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe031ea0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c072260 con 0x7fe02c0fe050 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe031ea0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c070850 con 0x7fe02c0fce50 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe02affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fe050 0x7fe02c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe02affd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fe050 0x7fe02c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43850/0 (socket says 172.21.15.94:43850) 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe02b7fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe02c0fce50 0x7fe02c071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe02affd700 1 -- 172.21.15.94:0/3673760155 learned_addr learned my addr 172.21.15.94:0/3673760155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe02affd700 1 -- 172.21.15.94:0/3673760155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe02c0fce50 msgr2=0x7fe02c071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe02affd700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe02c0fce50 0x7fe02c071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe02affd700 1 -- 172.21.15.94:0/3673760155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c005040 con 0x7fe02c0fe050 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe02affd700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fe050 0x7fe02c070310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe01400eb40 tx=0x7fe01400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe023fff700 1 -- 172.21.15.94:0/3673760155 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe014007cc0 con 0x7fe02c0fe050 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe023fff700 1 -- 172.21.15.94:0/3673760155 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe01400f040 con 0x7fe02c0fe050 2024-04-06T03:47:29.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c070aa0 con 0x7fe02c0fe050 2024-04-06T03:47:29.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe023fff700 1 -- 172.21.15.94:0/3673760155 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe01400ca10 con 0x7fe02c0fe050 2024-04-06T03:47:29.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.271+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c070fe0 con 0x7fe02c0fe050 2024-04-06T03:47:29.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.272+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02c04c710 con 0x7fe02c0fe050 2024-04-06T03:47:29.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.273+0000 7fe023fff700 1 -- 172.21.15.94:0/3673760155 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fe01400cc80 con 0x7fe02c0fe050 2024-04-06T03:47:29.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.273+0000 7fe023fff700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe01806dca0 0x7fe018070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:29.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.273+0000 7fe023fff700 1 -- 172.21.15.94:0/3673760155 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe01408d090 con 0x7fe02c0fe050 2024-04-06T03:47:29.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.273+0000 7fe02b7fe700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe01806dca0 0x7fe018070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:29.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.274+0000 7fe02b7fe700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe01806dca0 0x7fe018070150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe01c004890 tx=0x7fe01c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:29.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.278+0000 7fe023fff700 1 -- 172.21.15.94:0/3673760155 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe01405a2a0 con 0x7fe02c0fe050 2024-04-06T03:47:29.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:29 smithi136 ceph-mon[82061]: pgmap v171: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:29.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:29 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:29.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:29 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:29.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:29 smithi136 ceph-mon[82061]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:47:29.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.519+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe02c1058e0 con 0x7fe02c0fe050 2024-04-06T03:47:29.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.519+0000 7fe023fff700 1 -- 172.21.15.94:0/3673760155 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fe014059e30 con 0x7fe02c0fe050 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout:e12 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 11 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:47:29.522 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:38:04.308946+0000 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:47:29.523 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14460} 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:14460} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.94:6826/1913624529,v1:172.21.15.94:6827/1913624529] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:47:29.524 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:47:29.525 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:47:29.525 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:14472} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:47:29.525 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:47:29.525 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:47:29.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe01806dca0 msgr2=0x7fe018070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe01806dca0 0x7fe018070150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe01c004890 tx=0x7fe01c009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fe050 msgr2=0x7fe02c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fe050 0x7fe02c070310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe01400eb40 tx=0x7fe01400b040 comp rx=0 tx=0).stop 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 shutdown_connections 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe01806dca0 0x7fe018070150 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe02c0fce50 0x7fe02c071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 --2- 172.21.15.94:0/3673760155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe02c0fe050 0x7fe02c070310 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 >> 172.21.15.94:0/3673760155 conn(0x7fe02c0f8580 msgr2=0x7fe02c0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 shutdown_connections 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.521+0000 7fe031ea0700 1 -- 172.21.15.94:0/3673760155 wait complete. 2024-04-06T03:47:29.526 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:47:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 -- 172.21.15.94:0/2969037373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 msgr2=0x7f11540fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 --2- 172.21.15.94:0/2969037373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 0x7f11540fc0e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1148004940 tx=0x7f1148009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 -- 172.21.15.94:0/2969037373 shutdown_connections 2024-04-06T03:47:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 --2- 172.21.15.94:0/2969037373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11540fcf80 0x7f11540fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 --2- 172.21.15.94:0/2969037373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 0x7f11540fc0e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 -- 172.21.15.94:0/2969037373 >> 172.21.15.94:0/2969037373 conn(0x7f11540f7430 msgr2=0x7f11540f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:29.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 -- 172.21.15.94:0/2969037373 shutdown_connections 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 -- 172.21.15.94:0/2969037373 wait complete. 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 Processor -- start 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.623+0000 7f115925f700 1 -- start start 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f115925f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 0x7f1154110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f115925f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11540fcf80 0x7f1154112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f115925f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1154066350 con 0x7f11540fbcd0 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f115925f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1154066490 con 0x7f11540fcf80 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f1152d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 0x7f1154110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f1152d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 0x7f1154110690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43868/0 (socket says 172.21.15.94:43868) 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f1152d9d700 1 -- 172.21.15.94:0/1219650629 learned_addr learned my addr 172.21.15.94:0/1219650629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f115259c700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11540fcf80 0x7f1154112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:29.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f1152d9d700 1 -- 172.21.15.94:0/1219650629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11540fcf80 msgr2=0x7f1154112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f1152d9d700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11540fcf80 0x7f1154112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f1152d9d700 1 -- 172.21.15.94:0/1219650629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1148005040 con 0x7f11540fbcd0 2024-04-06T03:47:29.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f1152d9d700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 0x7f1154110690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1148013870 tx=0x7f1148009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:29.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f113b7fe700 1 -- 172.21.15.94:0/1219650629 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1148021070 con 0x7f11540fbcd0 2024-04-06T03:47:29.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f113b7fe700 1 -- 172.21.15.94:0/1219650629 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1148009430 con 0x7f11540fbcd0 2024-04-06T03:47:29.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1154113120 con 0x7f11540fbcd0 2024-04-06T03:47:29.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f113b7fe700 1 -- 172.21.15.94:0/1219650629 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114800e070 con 0x7f11540fbcd0 2024-04-06T03:47:29.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.624+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1154113660 con 0x7f11540fbcd0 2024-04-06T03:47:29.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.625+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115405f5f0 con 0x7f11540fbcd0 2024-04-06T03:47:29.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.626+0000 7f113b7fe700 1 -- 172.21.15.94:0/1219650629 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f114801c750 con 0x7f11540fbcd0 2024-04-06T03:47:29.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.626+0000 7f113b7fe700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f113c06dc30 0x7f113c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:29.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.626+0000 7f113b7fe700 1 -- 172.21.15.94:0/1219650629 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f114808e680 con 0x7f11540fbcd0 2024-04-06T03:47:29.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.626+0000 7f115259c700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f113c06dc30 0x7f113c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:29.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.627+0000 7f115259c700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f113c06dc30 0x7f113c0700e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1144009b80 tx=0x7f114400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:29.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.631+0000 7f113b7fe700 1 -- 172.21.15.94:0/1219650629 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1148057d30 con 0x7f11540fbcd0 2024-04-06T03:47:29.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.843+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11541012f0 con 0x7f113c06dc30 2024-04-06T03:47:29.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.844+0000 7f113b7fe700 1 -- 172.21.15.94:0/1219650629 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f11541012f0 con 0x7f113c06dc30 2024-04-06T03:47:29.846 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "osd", 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "12/21 daemons upgraded", 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading osd daemons", 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:47:29.847 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.846+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f113c06dc30 msgr2=0x7f113c0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.846+0000 7f115925f700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f113c06dc30 0x7f113c0700e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1144009b80 tx=0x7f114400b040 comp rx=0 tx=0).stop 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.846+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 msgr2=0x7f1154110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.846+0000 7f115925f700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 0x7f1154110690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1148013870 tx=0x7f1148009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.846+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 shutdown_connections 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.846+0000 7f115925f700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11540fbcd0 0x7f1154110690 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.846+0000 7f115925f700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f113c06dc30 0x7f113c0700e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.846+0000 7f115925f700 1 --2- 172.21.15.94:0/1219650629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11540fcf80 0x7f1154112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.846+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 >> 172.21.15.94:0/1219650629 conn(0x7f11540f7430 msgr2=0x7f11540f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:29.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.847+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 shutdown_connections 2024-04-06T03:47:29.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.847+0000 7f115925f700 1 -- 172.21.15.94:0/1219650629 wait complete. 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.945+0000 7fe8464c0700 1 -- 172.21.15.94:0/1650840659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fdfe0 msgr2=0x7fe8400fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.945+0000 7fe8464c0700 1 --2- 172.21.15.94:0/1650840659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fdfe0 0x7fe8400fe450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe834004940 tx=0x7fe834009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.945+0000 7fe8464c0700 1 -- 172.21.15.94:0/1650840659 shutdown_connections 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.945+0000 7fe8464c0700 1 --2- 172.21.15.94:0/1650840659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fdfe0 0x7fe8400fe450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.945+0000 7fe8464c0700 1 --2- 172.21.15.94:0/1650840659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8400fce70 0x7fe8400fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.945+0000 7fe8464c0700 1 -- 172.21.15.94:0/1650840659 >> 172.21.15.94:0/1650840659 conn(0x7fe8400f8550 msgr2=0x7fe8400fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.945+0000 7fe8464c0700 1 -- 172.21.15.94:0/1650840659 shutdown_connections 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.945+0000 7fe8464c0700 1 -- 172.21.15.94:0/1650840659 wait complete. 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe8464c0700 1 Processor -- start 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe8464c0700 1 -- start start 2024-04-06T03:47:29.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe8464c0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fce70 0x7fe84010d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe8464c0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8400fdfe0 0x7fe84010ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe8464c0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84010e3d0 con 0x7fe8400fce70 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe8464c0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84010e540 con 0x7fe8400fdfe0 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe83ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fce70 0x7fe84010d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe83ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fce70 0x7fe84010d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43888/0 (socket says 172.21.15.94:43888) 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe83ffff700 1 -- 172.21.15.94:0/3800058254 learned_addr learned my addr 172.21.15.94:0/3800058254 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.946+0000 7fe83f7fe700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8400fdfe0 0x7fe84010ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.947+0000 7fe83ffff700 1 -- 172.21.15.94:0/3800058254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8400fdfe0 msgr2=0x7fe84010ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.947+0000 7fe83ffff700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8400fdfe0 0x7fe84010ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.947+0000 7fe83ffff700 1 -- 172.21.15.94:0/3800058254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834005040 con 0x7fe8400fce70 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.947+0000 7fe83ffff700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fce70 0x7fe84010d890 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe830002a80 tx=0x7fe8300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.947+0000 7fe83cff9700 1 -- 172.21.15.94:0/3800058254 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe830007cc0 con 0x7fe8400fce70 2024-04-06T03:47:29.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.947+0000 7fe83cff9700 1 -- 172.21.15.94:0/3800058254 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe83000f040 con 0x7fe8400fce70 2024-04-06T03:47:29.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.947+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe840112fa0 con 0x7fe8400fce70 2024-04-06T03:47:29.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.947+0000 7fe83cff9700 1 -- 172.21.15.94:0/3800058254 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83000dac0 con 0x7fe8400fce70 2024-04-06T03:47:29.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.947+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8401134d0 con 0x7fe8400fce70 2024-04-06T03:47:29.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.948+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84004c710 con 0x7fe8400fce70 2024-04-06T03:47:29.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.949+0000 7fe83cff9700 1 -- 172.21.15.94:0/3800058254 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fe83000dc20 con 0x7fe8400fce70 2024-04-06T03:47:29.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.949+0000 7fe83cff9700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe82806dca0 0x7fe828070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:47:29.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.949+0000 7fe83cff9700 1 -- 172.21.15.94:0/3800058254 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe83008d0d0 con 0x7fe8400fce70 2024-04-06T03:47:29.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.949+0000 7fe83f7fe700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe82806dca0 0x7fe828070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:47:29.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.949+0000 7fe83f7fe700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe82806dca0 0x7fe828070150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe83400b3b0 tx=0x7fe834009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:47:29.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:29.953+0000 7fe83cff9700 1 -- 172.21.15.94:0/3800058254 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe83005a230 con 0x7fe8400fce70 2024-04-06T03:47:30.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.230+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe8401136c0 con 0x7fe8400fce70 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.230+0000 7fe83cff9700 1 -- 172.21.15.94:0/3800058254 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1325 (secure 0 0 0) 0x7fe830059dc0 con 0x7fe8400fce70 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in unknown state 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: osd.5 (root=default,host=smithi136) is down 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-04-06T03:47:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-04-06T03:47:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe82806dca0 msgr2=0x7fe828070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe82806dca0 0x7fe828070150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe83400b3b0 tx=0x7fe834009e30 comp rx=0 tx=0).stop 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fce70 msgr2=0x7fe84010d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fce70 0x7fe84010d890 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe830002a80 tx=0x7fe8300046b0 comp rx=0 tx=0).stop 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 shutdown_connections 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8400fce70 0x7fe84010d890 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe82806dca0 0x7fe828070150 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 --2- 172.21.15.94:0/3800058254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8400fdfe0 0x7fe84010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 >> 172.21.15.94:0/3800058254 conn(0x7fe8400f8550 msgr2=0x7fe8400f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 shutdown_connections 2024-04-06T03:47:30.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:47:30.232+0000 7fe8464c0700 1 -- 172.21.15.94:0/3800058254 wait complete. 2024-04-06T03:47:30.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:30 smithi136 ceph-mon[82061]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:47:30.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:30 smithi136 ceph-mon[82061]: from='client.44261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:47:30.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:30 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3539367228' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:30.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:30 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:30.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:30 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:30.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:30 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3673760155' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:47:30.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:30 smithi094 ceph-mon[94053]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:47:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:30 smithi094 ceph-mon[94053]: from='client.44261 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:47:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:30 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3539367228' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:30 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:30 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:30.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:30 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3673760155' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:47:30.808 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:30 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:47:30.485+0000 7fc0b351c7c0 -1 osd.5 73 log_to_monitors true 2024-04-06T03:47:31.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:31 smithi136 ceph-mon[82061]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:47:31.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:31 smithi136 ceph-mon[82061]: pgmap v172: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:31.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:31 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3800058254' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:47:31.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:31 smithi136 ceph-mon[82061]: from='osd.5 [v2:172.21.15.136:6816/853857034,v1:172.21.15.136:6817/853857034]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-06T03:47:31.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:31 smithi136 ceph-mon[82061]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-06T03:47:31.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:31 smithi094 ceph-mon[94053]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:47:31.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:31 smithi094 ceph-mon[94053]: pgmap v172: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:31.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:31 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3800058254' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:47:31.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:31 smithi094 ceph-mon[94053]: from='osd.5 [v2:172.21.15.136:6816/853857034,v1:172.21.15.136:6817/853857034]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-06T03:47:31.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:31 smithi094 ceph-mon[94053]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-06T03:47:31.996 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:47:31 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:47:31.897+0000 7fc0a096c700 -1 osd.5 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-06T03:47:32.323 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:32 smithi136 ceph-mon[82061]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-06T03:47:32.323 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:32 smithi136 ceph-mon[82061]: from='osd.5 [v2:172.21.15.136:6816/853857034,v1:172.21.15.136:6817/853857034]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:47:32.323 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:32 smithi136 ceph-mon[82061]: osdmap e76: 6 total, 5 up, 6 in 2024-04-06T03:47:32.323 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:32 smithi136 ceph-mon[82061]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:47:32.324 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:32 smithi136 ceph-mon[82061]: Health check update: Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded (PG_DEGRADED) 2024-04-06T03:47:32.324 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:32 smithi136 ceph-mon[82061]: from='osd.5 ' entity='osd.5' 2024-04-06T03:47:32.568 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:32 smithi094 ceph-mon[94053]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-06T03:47:32.568 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:32 smithi094 ceph-mon[94053]: from='osd.5 [v2:172.21.15.136:6816/853857034,v1:172.21.15.136:6817/853857034]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:47:32.568 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:32 smithi094 ceph-mon[94053]: osdmap e76: 6 total, 5 up, 6 in 2024-04-06T03:47:32.569 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:32 smithi094 ceph-mon[94053]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-06T03:47:32.569 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:32 smithi094 ceph-mon[94053]: Health check update: Degraded data redundancy: 30/219 objects degraded (13.699%), 12 pgs degraded (PG_DEGRADED) 2024-04-06T03:47:32.569 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:32 smithi094 ceph-mon[94053]: from='osd.5 ' entity='osd.5' 2024-04-06T03:47:33.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: pgmap v174: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 838 MiB used, 536 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:33.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:33.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-06T03:47:33.600 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-06T03:47:33.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-06T03:47:33.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-06T03:47:33.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-06T03:47:33.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-06T03:47:33.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:47:33.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-06T03:47:33.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-06T03:47:33.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: osd.5 [v2:172.21.15.136:6816/853857034,v1:172.21.15.136:6817/853857034] boot 2024-04-06T03:47:33.601 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: osdmap e77: 6 total, 6 up, 6 in 2024-04-06T03:47:33.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:47:33.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-06T03:47:33.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-06T03:47:33.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-06T03:47:33.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-06T03:47:33.602 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:33 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-06T03:47:33.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: pgmap v174: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 838 MiB used, 536 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:33.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:33.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:33.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:47:33.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:47:33.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:33.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:47:33.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-06T03:47:33.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: osd.5 [v2:172.21.15.136:6816/853857034,v1:172.21.15.136:6817/853857034] boot 2024-04-06T03:47:33.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: osdmap e77: 6 total, 6 up, 6 in 2024-04-06T03:47:33.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-06T03:47:33.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-06T03:47:33.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-06T03:47:33.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-06T03:47:33.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-06T03:47:33.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:33 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-06T03:47:34.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:34 smithi094 ceph-mon[94053]: pgmap v175: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 838 MiB used, 536 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:34.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:34 smithi094 ceph-mon[94053]: pgmap v176: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 838 MiB used, 536 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:34.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:34 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all osd 2024-04-06T03:47:34.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:34 smithi094 ceph-mon[94053]: Upgrade: Setting require_osd_release to 19 squid 2024-04-06T03:47:34.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:34 smithi094 ceph-mon[94053]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-06T03:47:34.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-06T03:47:34.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:34 smithi094 ceph-mon[94053]: osdmap e78: 6 total, 6 up, 6 in 2024-04-06T03:47:34.501 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:34 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi094.qogyzw"]}]: dispatch 2024-04-06T03:47:34.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:34 smithi136 ceph-mon[82061]: pgmap v175: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 838 MiB used, 536 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:34 smithi136 ceph-mon[82061]: pgmap v176: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 194 MiB data, 838 MiB used, 536 GiB / 536 GiB avail; 30/219 objects degraded (13.699%) 2024-04-06T03:47:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:34 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all osd 2024-04-06T03:47:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:34 smithi136 ceph-mon[82061]: Upgrade: Setting require_osd_release to 19 squid 2024-04-06T03:47:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:34 smithi136 ceph-mon[82061]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-06T03:47:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-06T03:47:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:34 smithi136 ceph-mon[82061]: osdmap e78: 6 total, 6 up, 6 in 2024-04-06T03:47:34.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:34 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi094.qogyzw"]}]: dispatch 2024-04-06T03:47:35.535 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:35 smithi094 ceph-mon[94053]: Upgrade: It appears safe to stop mds.cephfs.smithi094.qogyzw 2024-04-06T03:47:35.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:35 smithi136 ceph-mon[82061]: Upgrade: It appears safe to stop mds.cephfs.smithi094.qogyzw 2024-04-06T03:47:36.493 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:36 smithi094 ceph-mon[94053]: pgmap v179: 65 pgs: 3 peering, 9 active+undersized+degraded, 14 active+undersized, 39 active+clean; 194 MiB data, 838 MiB used, 536 GiB / 536 GiB avail; 24/219 objects degraded (10.959%) 2024-04-06T03:47:36.493 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:36.493 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:47:36.493 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:36.493 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.qogyzw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:47:36.493 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:47:36.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:36 smithi136 ceph-mon[82061]: pgmap v179: 65 pgs: 3 peering, 9 active+undersized+degraded, 14 active+undersized, 39 active+clean; 194 MiB data, 838 MiB used, 536 GiB / 536 GiB avail; 24/219 objects degraded (10.959%) 2024-04-06T03:47:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:47:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.qogyzw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:47:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:47:37.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:37 smithi094 ceph-mon[94053]: Upgrade: Updating mds.cephfs.smithi094.qogyzw 2024-04-06T03:47:37.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:37 smithi094 ceph-mon[94053]: Deploying daemon mds.cephfs.smithi094.qogyzw on smithi094 2024-04-06T03:47:37.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:37 smithi094 ceph-mon[94053]: Health check update: Degraded data redundancy: 24/219 objects degraded (10.959%), 9 pgs degraded (PG_DEGRADED) 2024-04-06T03:47:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:37 smithi136 ceph-mon[82061]: Upgrade: Updating mds.cephfs.smithi094.qogyzw 2024-04-06T03:47:37.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:37 smithi136 ceph-mon[82061]: Deploying daemon mds.cephfs.smithi094.qogyzw on smithi094 2024-04-06T03:47:37.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:37 smithi136 ceph-mon[82061]: Health check update: Degraded data redundancy: 24/219 objects degraded (10.959%), 9 pgs degraded (PG_DEGRADED) 2024-04-06T03:47:37.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:37 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[94049]: 2024-04-06T03:47:37.708+0000 7f71b5c08700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:47:38.098 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:47:37 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:47:37.730+0000 7fa766f88700 -1 bdev(0x563158eaa000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-04-06T03:47:38.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:47:37 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:47:37.730+0000 7fa765f86700 -1 bdev(0x563158eaa000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 6 2024-04-06T03:47:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:38 smithi136 ceph-mon[82061]: pgmap v180: 65 pgs: 3 peering, 1 active+undersized+degraded, 1 active+undersized, 60 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1/219 objects degraded (0.457%) 2024-04-06T03:47:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:38 smithi136 ceph-mon[82061]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-06T03:47:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:38 smithi136 ceph-mon[82061]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:47:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:38 smithi136 ceph-mon[82061]: osdmap e79: 6 total, 6 up, 6 in 2024-04-06T03:47:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:38 smithi136 ceph-mon[82061]: Standby daemon mds.cephfs.smithi094.kvxcqo assigned to filesystem cephfs as rank 0 2024-04-06T03:47:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:38 smithi136 ceph-mon[82061]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:47:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:38 smithi136 ceph-mon[82061]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-06T03:47:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:38 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi094.kvxcqo=up:replay} 2 up:standby 2024-04-06T03:47:38.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:38 smithi094 ceph-mon[94053]: pgmap v180: 65 pgs: 3 peering, 1 active+undersized+degraded, 1 active+undersized, 60 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 1/219 objects degraded (0.457%) 2024-04-06T03:47:38.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:38 smithi094 ceph-mon[94053]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-06T03:47:38.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:38 smithi094 ceph-mon[94053]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:47:38.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:38 smithi094 ceph-mon[94053]: osdmap e79: 6 total, 6 up, 6 in 2024-04-06T03:47:38.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:38 smithi094 ceph-mon[94053]: Standby daemon mds.cephfs.smithi094.kvxcqo assigned to filesystem cephfs as rank 0 2024-04-06T03:47:38.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:38 smithi094 ceph-mon[94053]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:47:38.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:38 smithi094 ceph-mon[94053]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-06T03:47:38.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:38 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi094.kvxcqo=up:replay} 2 up:standby 2024-04-06T03:47:39.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:39 smithi136 ceph-mon[82061]: pgmap v182: 65 pgs: 3 peering, 62 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:47:39.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:39 smithi136 ceph-mon[82061]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/219 objects degraded (0.457%), 1 pg degraded) 2024-04-06T03:47:39.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:39 smithi094 ceph-mon[94053]: pgmap v182: 65 pgs: 3 peering, 62 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:47:39.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:39 smithi094 ceph-mon[94053]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/219 objects degraded (0.457%), 1 pg degraded) 2024-04-06T03:47:42.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:41 smithi136 ceph-mon[82061]: pgmap v183: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s rd, 2 op/s 2024-04-06T03:47:42.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:41 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:47:42.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:41 smithi094 ceph-mon[94053]: pgmap v183: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 6.8 MiB/s rd, 2 op/s 2024-04-06T03:47:42.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:41 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:47:44.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:43 smithi136 ceph-mon[82061]: pgmap v184: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-06T03:47:44.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:43 smithi094 ceph-mon[94053]: pgmap v184: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-06T03:47:45.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:45 smithi094 ceph-mon[94053]: pgmap v185: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-06T03:47:46.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:45 smithi136 ceph-mon[82061]: pgmap v185: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-06T03:47:47.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:46 smithi136 ceph-mon[82061]: reconnect by client.24295 v1:172.21.15.94:0/2208962971 after 0 2024-04-06T03:47:47.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:46 smithi136 ceph-mon[82061]: reconnect by client.14524 v1:172.21.15.136:0/1309930737 after 0 2024-04-06T03:47:47.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:46 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:reconnect 2024-04-06T03:47:47.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:46 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi094.kvxcqo=up:reconnect} 2 up:standby 2024-04-06T03:47:47.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:46 smithi094 ceph-mon[94053]: reconnect by client.24295 v1:172.21.15.94:0/2208962971 after 0 2024-04-06T03:47:47.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:46 smithi094 ceph-mon[94053]: reconnect by client.14524 v1:172.21.15.136:0/1309930737 after 0 2024-04-06T03:47:47.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:46 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:reconnect 2024-04-06T03:47:47.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:46 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi094.kvxcqo=up:reconnect} 2 up:standby 2024-04-06T03:47:48.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:47 smithi136 ceph-mon[82061]: pgmap v186: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-06T03:47:48.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:47 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:rejoin 2024-04-06T03:47:48.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:47 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi094.kvxcqo=up:rejoin} 2 up:standby 2024-04-06T03:47:48.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:47 smithi136 ceph-mon[82061]: daemon mds.cephfs.smithi094.kvxcqo is now active in filesystem cephfs as rank 0 2024-04-06T03:47:48.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:47 smithi094 ceph-mon[94053]: pgmap v186: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-06T03:47:48.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:47 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:rejoin 2024-04-06T03:47:48.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:47 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi094.kvxcqo=up:rejoin} 2 up:standby 2024-04-06T03:47:48.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:47 smithi094 ceph-mon[94053]: daemon mds.cephfs.smithi094.kvxcqo is now active in filesystem cephfs as rank 0 2024-04-06T03:47:49.093 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:48 smithi094 ceph-mon[94053]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-06T03:47:49.094 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:48 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:active 2024-04-06T03:47:49.094 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:48 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi094.kvxcqo=up:active} 2 up:standby 2024-04-06T03:47:49.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:48 smithi136 ceph-mon[82061]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-06T03:47:49.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:48 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] up:active 2024-04-06T03:47:49.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:48 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi094.kvxcqo=up:active} 2 up:standby 2024-04-06T03:47:50.086 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:49 smithi094 ceph-mon[94053]: pgmap v187: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-06T03:47:50.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:49 smithi136 ceph-mon[82061]: pgmap v187: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-06T03:47:51.098 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:50 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] up:boot 2024-04-06T03:47:51.098 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:50 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi094.kvxcqo=up:active} 3 up:standby 2024-04-06T03:47:51.098 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:47:51.098 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:51.098 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:51.098 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:47:51.098 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:51.099 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:47:51.099 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:51.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:50 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] up:boot 2024-04-06T03:47:51.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:50 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi094.kvxcqo=up:active} 3 up:standby 2024-04-06T03:47:51.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:47:51.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:51.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:51.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:47:51.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:51.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:47:51.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:52.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:51 smithi136 ceph-mon[82061]: pgmap v188: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-06T03:47:52.126 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:51 smithi094 ceph-mon[94053]: pgmap v188: 65 pgs: 65 active+clean; 194 MiB data, 842 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-06T03:47:54.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:53 smithi136 ceph-mon[82061]: pgmap v189: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-06T03:47:54.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:53 smithi094 ceph-mon[94053]: pgmap v189: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-06T03:47:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:55 smithi094 ceph-mon[94053]: pgmap v190: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-06T03:47:56.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:55 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:56.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:55 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:56.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:55 smithi136 ceph-mon[82061]: pgmap v190: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-04-06T03:47:56.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:55 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:56.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:55 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:57.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:57.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:56 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:57.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:57.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:56 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:47:58.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:57 smithi094 ceph-mon[94053]: pgmap v191: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 3.9 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-06T03:47:58.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:57 smithi136 ceph-mon[82061]: pgmap v191: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 3.9 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-06T03:48:00.300 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: pgmap v192: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-06T03:48:00.300 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:00.300 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:00.300 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:00.301 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:48:00.301 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:00.301 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:00.301 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:00.301 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:00.301 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:00.301 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:00.301 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:47:59 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi094.kvxcqo"]}]: dispatch 2024-04-06T03:48:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.338+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1299282403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 msgr2=0x7f8dd00fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.338+0000 7f8dd6d24700 1 --2- 172.21.15.94:0/1299282403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 0x7f8dd00fc110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0004940 tx=0x7f8dc0009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.338+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1299282403 shutdown_connections 2024-04-06T03:48:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.338+0000 7f8dd6d24700 1 --2- 172.21.15.94:0/1299282403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd00fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.338+0000 7f8dd6d24700 1 --2- 172.21.15.94:0/1299282403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 0x7f8dd00fc110 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.338+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1299282403 >> 172.21.15.94:0/1299282403 conn(0x7f8dd00f7480 msgr2=0x7f8dd00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:00.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.338+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1299282403 shutdown_connections 2024-04-06T03:48:00.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.338+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1299282403 wait complete. 2024-04-06T03:48:00.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dd6d24700 1 Processor -- start 2024-04-06T03:48:00.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dd6d24700 1 -- start start 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dd6d24700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 0x7f8dd010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dd6d24700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dd6d24700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd010e520 con 0x7f8dd00fbd00 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dd6d24700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd0112ec0 con 0x7f8dd00fcfb0 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dd4ac0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 0x7f8dd010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dd4ac0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 0x7f8dd010d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57072/0 (socket says 172.21.15.94:57072) 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dd4ac0700 1 -- 172.21.15.94:0/1971997336 learned_addr learned my addr 172.21.15.94:0/1971997336 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.339+0000 7f8dcffff700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.340+0000 7f8dd4ac0700 1 -- 172.21.15.94:0/1971997336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd00fcfb0 msgr2=0x7f8dd010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.340+0000 7f8dd4ac0700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd010de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.340+0000 7f8dd4ac0700 1 -- 172.21.15.94:0/1971997336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc0005040 con 0x7f8dd00fbd00 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.340+0000 7f8dd4ac0700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 0x7f8dd010d950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b3b0 tx=0x7f8dc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.340+0000 7f8dcd7fa700 1 -- 172.21.15.94:0/1971997336 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dc000e070 con 0x7f8dd00fbd00 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.340+0000 7f8dcd7fa700 1 -- 172.21.15.94:0/1971997336 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dc000bbf0 con 0x7f8dd00fbd00 2024-04-06T03:48:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.340+0000 7f8dcd7fa700 1 -- 172.21.15.94:0/1971997336 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dc001c400 con 0x7f8dd00fbd00 2024-04-06T03:48:00.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.340+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd01130c0 con 0x7f8dd00fbd00 2024-04-06T03:48:00.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.340+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd0113570 con 0x7f8dd00fbd00 2024-04-06T03:48:00.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.341+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd0101b40 con 0x7f8dd00fbd00 2024-04-06T03:48:00.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.342+0000 7f8dcd7fa700 1 -- 172.21.15.94:0/1971997336 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f8dc0013420 con 0x7f8dd00fbd00 2024-04-06T03:48:00.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.342+0000 7f8dcd7fa700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8dbc06dce0 0x7f8dbc070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:00.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.342+0000 7f8dcd7fa700 1 -- 172.21.15.94:0/1971997336 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8dc008eec0 con 0x7f8dd00fbd00 2024-04-06T03:48:00.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.342+0000 7f8dcffff700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8dbc06dce0 0x7f8dbc070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:00.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.343+0000 7f8dcffff700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8dbc06dce0 0x7f8dbc070190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006fd0 tx=0x7f8db8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:00.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: pgmap v192: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-06T03:48:00.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:00.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:00.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:00.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:48:00.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:00.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:00.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:00.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:00.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:00.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:00.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:47:59 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi094.kvxcqo"]}]: dispatch 2024-04-06T03:48:00.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.347+0000 7f8dcd7fa700 1 -- 172.21.15.94:0/1971997336 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8dc005c0f0 con 0x7f8dd00fbd00 2024-04-06T03:48:00.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.565+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dd002b310 con 0x7f8dbc06dce0 2024-04-06T03:48:00.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.566+0000 7f8dcd7fa700 1 -- 172.21.15.94:0/1971997336 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8dd002b310 con 0x7f8dbc06dce0 2024-04-06T03:48:00.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8dbc06dce0 msgr2=0x7f8dbc070190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:00.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8dbc06dce0 0x7f8dbc070190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006fd0 tx=0x7f8db8008040 comp rx=0 tx=0).stop 2024-04-06T03:48:00.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 msgr2=0x7f8dd010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:00.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 0x7f8dd010d950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b3b0 tx=0x7f8dc0009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:00.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 shutdown_connections 2024-04-06T03:48:00.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8dd00fbd00 0x7f8dd010d950 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8dbc06dce0 0x7f8dbc070190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 --2- 172.21.15.94:0/1971997336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 >> 172.21.15.94:0/1971997336 conn(0x7f8dd00f7480 msgr2=0x7f8dd00667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:00.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 shutdown_connections 2024-04-06T03:48:00.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.569+0000 7f8dd6d24700 1 -- 172.21.15.94:0/1971997336 wait complete. 2024-04-06T03:48:00.581 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:48:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 -- 172.21.15.94:0/670745072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 msgr2=0x7f05800fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 --2- 172.21.15.94:0/670745072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 0x7f05800fe4b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0570004940 tx=0x7f0570009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 -- 172.21.15.94:0/670745072 shutdown_connections 2024-04-06T03:48:00.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 --2- 172.21.15.94:0/670745072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 0x7f05800fe4b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 --2- 172.21.15.94:0/670745072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05800fce40 0x7f05800fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 -- 172.21.15.94:0/670745072 >> 172.21.15.94:0/670745072 conn(0x7f05800f8570 msgr2=0x7f05800fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 -- 172.21.15.94:0/670745072 shutdown_connections 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 -- 172.21.15.94:0/670745072 wait complete. 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 Processor -- start 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.675+0000 7f0587868700 1 -- start start 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0587868700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05800fce40 0x7f058010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0587868700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 0x7f058010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0587868700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058010e4f0 con 0x7f05800fe040 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0587868700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058010ed40 con 0x7f05800fce40 2024-04-06T03:48:00.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0584e03700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 0x7f058010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0584e03700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 0x7f058010def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57082/0 (socket says 172.21.15.94:57082) 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0584e03700 1 -- 172.21.15.94:0/1563540156 learned_addr learned my addr 172.21.15.94:0/1563540156 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0585604700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05800fce40 0x7f058010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0584e03700 1 -- 172.21.15.94:0/1563540156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05800fce40 msgr2=0x7f058010d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0584e03700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05800fce40 0x7f058010d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0584e03700 1 -- 172.21.15.94:0/1563540156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0570005040 con 0x7f05800fe040 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0584e03700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 0x7f058010def0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0570009150 tx=0x7f0570009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.676+0000 7f0575ffb700 1 -- 172.21.15.94:0/1563540156 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f057000e040 con 0x7f05800fe040 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.677+0000 7f0575ffb700 1 -- 172.21.15.94:0/1563540156 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0570004290 con 0x7f05800fe040 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.677+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058010efc0 con 0x7f05800fe040 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.677+0000 7f0575ffb700 1 -- 172.21.15.94:0/1563540156 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f057001c5d0 con 0x7f05800fe040 2024-04-06T03:48:00.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.677+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058010f440 con 0x7f05800fe040 2024-04-06T03:48:00.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.678+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058005f5f0 con 0x7f05800fe040 2024-04-06T03:48:00.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.678+0000 7f0575ffb700 1 -- 172.21.15.94:0/1563540156 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f0570013420 con 0x7f05800fe040 2024-04-06T03:48:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.679+0000 7f0575ffb700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056c06dca0 0x7f056c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.679+0000 7f0585604700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056c06dca0 0x7f056c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.679+0000 7f0575ffb700 1 -- 172.21.15.94:0/1563540156 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f057008dfb0 con 0x7f05800fe040 2024-04-06T03:48:00.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.679+0000 7f0585604700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056c06dca0 0x7f056c070150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f05800fdbb0 tx=0x7f057c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:00.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.686+0000 7f0575ffb700 1 -- 172.21.15.94:0/1563540156 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f057005b1f0 con 0x7f05800fe040 2024-04-06T03:48:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.903+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f058010f830 con 0x7f056c06dca0 2024-04-06T03:48:00.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.904+0000 7f0575ffb700 1 -- 172.21.15.94:0/1563540156 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f058010f830 con 0x7f056c06dca0 2024-04-06T03:48:00.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056c06dca0 msgr2=0x7f056c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:00.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056c06dca0 0x7f056c070150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f05800fdbb0 tx=0x7f057c00b040 comp rx=0 tx=0).stop 2024-04-06T03:48:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 msgr2=0x7f058010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 0x7f058010def0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0570009150 tx=0x7f0570009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 shutdown_connections 2024-04-06T03:48:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056c06dca0 0x7f056c070150 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05800fce40 0x7f058010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 --2- 172.21.15.94:0/1563540156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05800fe040 0x7f058010def0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 >> 172.21.15.94:0/1563540156 conn(0x7f05800f8570 msgr2=0x7f05800f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 shutdown_connections 2024-04-06T03:48:00.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:00.906+0000 7f0587868700 1 -- 172.21.15.94:0/1563540156 wait complete. 2024-04-06T03:48:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.009+0000 7f6eca855700 1 -- 172.21.15.94:0/3332858482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40fff00 msgr2=0x7f6ec4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.009+0000 7f6eca855700 1 --2- 172.21.15.94:0/3332858482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40fff00 0x7f6ec4102320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4004940 tx=0x7f6eb4009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.009+0000 7f6eca855700 1 -- 172.21.15.94:0/3332858482 shutdown_connections 2024-04-06T03:48:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.009+0000 7f6eca855700 1 --2- 172.21.15.94:0/3332858482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40fff00 0x7f6ec4102320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.009+0000 7f6eca855700 1 --2- 172.21.15.94:0/3332858482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec40662e0 0x7f6ec40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.009+0000 7f6eca855700 1 -- 172.21.15.94:0/3332858482 >> 172.21.15.94:0/3332858482 conn(0x7f6ec40f7430 msgr2=0x7f6ec40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.009+0000 7f6eca855700 1 -- 172.21.15.94:0/3332858482 shutdown_connections 2024-04-06T03:48:01.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.009+0000 7f6eca855700 1 -- 172.21.15.94:0/3332858482 wait complete. 2024-04-06T03:48:01.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6eca855700 1 Processor -- start 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6eca855700 1 -- start start 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6eca855700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40662e0 0x7f6ec410da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6eca855700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec40fff00 0x7f6ec410dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6eca855700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec410e5a0 con 0x7f6ec40662e0 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6eca855700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec410edf0 con 0x7f6ec40fff00 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6ec3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40662e0 0x7f6ec410da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6ec3fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40662e0 0x7f6ec410da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57102/0 (socket says 172.21.15.94:57102) 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6ec3fff700 1 -- 172.21.15.94:0/1740322492 learned_addr learned my addr 172.21.15.94:0/1740322492 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.010+0000 7f6ec37fe700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec40fff00 0x7f6ec410dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.011+0000 7f6ec3fff700 1 -- 172.21.15.94:0/1740322492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec40fff00 msgr2=0x7f6ec410dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.011+0000 7f6ec3fff700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec40fff00 0x7f6ec410dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.011+0000 7f6ec3fff700 1 -- 172.21.15.94:0/1740322492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb4005040 con 0x7f6ec40662e0 2024-04-06T03:48:01.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.011+0000 7f6ec3fff700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40662e0 0x7f6ec410da60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800c800 tx=0x7f6eb8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:01.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.011+0000 7f6ec0ff9700 1 -- 172.21.15.94:0/1740322492 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb80106f0 con 0x7f6ec40662e0 2024-04-06T03:48:01.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.011+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec410f0d0 con 0x7f6ec40662e0 2024-04-06T03:48:01.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.011+0000 7f6ec0ff9700 1 -- 172.21.15.94:0/1740322492 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eb8010d20 con 0x7f6ec40662e0 2024-04-06T03:48:01.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.011+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec410f610 con 0x7f6ec40662e0 2024-04-06T03:48:01.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.011+0000 7f6ec0ff9700 1 -- 172.21.15.94:0/1740322492 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb80113a0 con 0x7f6ec40662e0 2024-04-06T03:48:01.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.012+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec405f5f0 con 0x7f6ec40662e0 2024-04-06T03:48:01.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.012+0000 7f6ec0ff9700 1 -- 172.21.15.94:0/1740322492 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f6eb8012400 con 0x7f6ec40662e0 2024-04-06T03:48:01.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.013+0000 7f6ec0ff9700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6eac06dca0 0x7f6eac070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:01.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.013+0000 7f6ec0ff9700 1 -- 172.21.15.94:0/1740322492 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6eb808d460 con 0x7f6ec40662e0 2024-04-06T03:48:01.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.013+0000 7f6ec37fe700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6eac06dca0 0x7f6eac070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:01.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.013+0000 7f6ec37fe700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6eac06dca0 0x7f6eac070150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400b970 tx=0x7f6eb4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:01.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.017+0000 7f6ec0ff9700 1 -- 172.21.15.94:0/1740322492 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6eb805a5f0 con 0x7f6ec40662e0 2024-04-06T03:48:01.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:00 smithi094 ceph-mon[94053]: Detected new or changed devices on smithi094 2024-04-06T03:48:01.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:00 smithi094 ceph-mon[94053]: pgmap v193: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 5.3 KiB/s wr, 6 op/s 2024-04-06T03:48:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:00 smithi094 ceph-mon[94053]: pgmap v194: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 347 B/s rd, 231 B/s wr, 0 op/s 2024-04-06T03:48:01.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:00 smithi094 ceph-mon[94053]: Upgrade: It appears safe to stop mds.cephfs.smithi094.kvxcqo 2024-04-06T03:48:01.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.226+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ec410fa00 con 0x7f6eac06dca0 2024-04-06T03:48:01.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.236+0000 7f6ec0ff9700 1 -- 172.21.15.94:0/1740322492 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6ec410fa00 con 0x7f6eac06dca0 2024-04-06T03:48:01.238 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (7m) 6s ago 15m 26.7M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (3m) 6s ago 15m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (3m) 33s ago 14m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 error 6s ago 15m - - 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (10m) 6s ago 10m 84.2M - 17.2.7-819-g6699495c ec300eaa768b 56624146b934 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (15s) 6s ago 10m 14.7M - 19.0.0-2027-g9f192350 a5511c7deba9 c189927827c8 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (10m) 33s ago 10m 15.4M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (10m) 33s ago 10m 17.8M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (6m) 6s ago 16m 506M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (5m) 33s ago 14m 451M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (5m) 6s ago 16m 49.9M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (4m) 33s ago 14m 43.8M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (7m) 6s ago 15m 15.4M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:48:01.239 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (6m) 33s ago 14m 17.2M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:48:01.240 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (3m) 6s ago 13m 110M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:48:01.240 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (2m) 6s ago 13m 120M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:48:01.240 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (2m) 6s ago 12m 105M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:48:01.240 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (102s) 33s ago 12m 105M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:48:01.240 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (69s) 33s ago 11m 102M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 c31367e1c2d5 2024-04-06T03:48:01.240 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (36s) 33s ago 11m 11.7M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 16386e8f8583 2024-04-06T03:48:01.240 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (5m) 6s ago 14m 61.2M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6eac06dca0 msgr2=0x7f6eac070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6eac06dca0 0x7f6eac070150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400b970 tx=0x7f6eb4009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40662e0 msgr2=0x7f6ec410da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40662e0 0x7f6ec410da60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800c800 tx=0x7f6eb8004470 comp rx=0 tx=0).stop 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 shutdown_connections 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ec40662e0 0x7f6ec410da60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6eac06dca0 0x7f6eac070150 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 --2- 172.21.15.94:0/1740322492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec40fff00 0x7f6ec410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 >> 172.21.15.94:0/1740322492 conn(0x7f6ec40f7430 msgr2=0x7f6ec40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 shutdown_connections 2024-04-06T03:48:01.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.238+0000 7f6eca855700 1 -- 172.21.15.94:0/1740322492 wait complete. 2024-04-06T03:48:01.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.341+0000 7f70b5d77700 1 -- 172.21.15.94:0/3422189077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fbce0 msgr2=0x7f70b00fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.341+0000 7f70b5d77700 1 --2- 172.21.15.94:0/3422189077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fbce0 0x7f70b00fc0f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f70a0004940 tx=0x7f70a0009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:01.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.341+0000 7f70b5d77700 1 -- 172.21.15.94:0/3422189077 shutdown_connections 2024-04-06T03:48:01.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.341+0000 7f70b5d77700 1 --2- 172.21.15.94:0/3422189077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70b00fcf90 0x7f70b00fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.341+0000 7f70b5d77700 1 --2- 172.21.15.94:0/3422189077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fbce0 0x7f70b00fc0f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.341+0000 7f70b5d77700 1 -- 172.21.15.94:0/3422189077 >> 172.21.15.94:0/3422189077 conn(0x7f70b00f7480 msgr2=0x7f70b00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:01.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.341+0000 7f70b5d77700 1 -- 172.21.15.94:0/3422189077 shutdown_connections 2024-04-06T03:48:01.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.341+0000 7f70b5d77700 1 -- 172.21.15.94:0/3422189077 wait complete. 2024-04-06T03:48:01.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.342+0000 7f70b5d77700 1 Processor -- start 2024-04-06T03:48:01.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.342+0000 7f70b5d77700 1 -- start start 2024-04-06T03:48:01.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.342+0000 7f70b5d77700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70b00fbce0 0x7f70b00ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:01.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.342+0000 7f70b5d77700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fcf90 0x7f70b00fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:01.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.342+0000 7f70b5d77700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0100220 con 0x7f70b00fcf90 2024-04-06T03:48:01.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.342+0000 7f70b5d77700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00fe810 con 0x7f70b00fbce0 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70aeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fcf90 0x7f70b00fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70aeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fcf90 0x7f70b00fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57118/0 (socket says 172.21.15.94:57118) 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70aeffd700 1 -- 172.21.15.94:0/4020287833 learned_addr learned my addr 172.21.15.94:0/4020287833 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70af7fe700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70b00fbce0 0x7f70b00ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70aeffd700 1 -- 172.21.15.94:0/4020287833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70b00fbce0 msgr2=0x7f70b00ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70aeffd700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70b00fbce0 0x7f70b00ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70aeffd700 1 -- 172.21.15.94:0/4020287833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a0005040 con 0x7f70b00fcf90 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70aeffd700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fcf90 0x7f70b00fe2d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c830 tx=0x7f70a4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f7097fff700 1 -- 172.21.15.94:0/4020287833 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a40106f0 con 0x7f70b00fcf90 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f7097fff700 1 -- 172.21.15.94:0/4020287833 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70a4010d20 con 0x7f70b00fcf90 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b00feac0 con 0x7f70b00fcf90 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f7097fff700 1 -- 172.21.15.94:0/4020287833 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a40113a0 con 0x7f70b00fcf90 2024-04-06T03:48:01.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.343+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b00ff080 con 0x7f70b00fcf90 2024-04-06T03:48:01.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.344+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b0101b20 con 0x7f70b00fcf90 2024-04-06T03:48:01.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.345+0000 7f7097fff700 1 -- 172.21.15.94:0/4020287833 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f70a4010850 con 0x7f70b00fcf90 2024-04-06T03:48:01.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.345+0000 7f7097fff700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f709806dca0 0x7f7098070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:01.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.345+0000 7f7097fff700 1 -- 172.21.15.94:0/4020287833 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f70a408d4c0 con 0x7f70b00fcf90 2024-04-06T03:48:01.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.345+0000 7f70af7fe700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f709806dca0 0x7f7098070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:01.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.346+0000 7f70af7fe700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f709806dca0 0x7f7098070150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70a000b970 tx=0x7f70a0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:00 smithi136 ceph-mon[82061]: Detected new or changed devices on smithi094 2024-04-06T03:48:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:00 smithi136 ceph-mon[82061]: pgmap v193: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 5.3 KiB/s wr, 6 op/s 2024-04-06T03:48:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:00 smithi136 ceph-mon[82061]: pgmap v194: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 347 B/s rd, 231 B/s wr, 0 op/s 2024-04-06T03:48:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:00 smithi136 ceph-mon[82061]: Upgrade: It appears safe to stop mds.cephfs.smithi094.kvxcqo 2024-04-06T03:48:01.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.350+0000 7f7097fff700 1 -- 172.21.15.94:0/4020287833 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f70a405a650 con 0x7f70b00fcf90 2024-04-06T03:48:01.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.644+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f70b004c710 con 0x7f70b00fcf90 2024-04-06T03:48:01.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.645+0000 7f7097fff700 1 -- 172.21.15.94:0/4020287833 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f70a405a1e0 con 0x7f70b00fcf90 2024-04-06T03:48:01.647 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:48:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:48:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:48:01.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 6 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 3, 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 1 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 3, 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 11 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:48:01.648 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:48:01.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f709806dca0 msgr2=0x7f7098070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f709806dca0 0x7f7098070150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70a000b970 tx=0x7f70a0009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:01.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fcf90 msgr2=0x7f70b00fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fcf90 0x7f70b00fe2d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c830 tx=0x7f70a4004470 comp rx=0 tx=0).stop 2024-04-06T03:48:01.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 shutdown_connections 2024-04-06T03:48:01.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f709806dca0 0x7f7098070150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70b00fbce0 0x7f70b00ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 --2- 172.21.15.94:0/4020287833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70b00fcf90 0x7f70b00fe2d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 >> 172.21.15.94:0/4020287833 conn(0x7f70b00f7480 msgr2=0x7f70b00f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:01.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 shutdown_connections 2024-04-06T03:48:01.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.648+0000 7f70b5d77700 1 -- 172.21.15.94:0/4020287833 wait complete. 2024-04-06T03:48:01.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.754+0000 7f0581140700 1 -- 172.21.15.94:0/1213009861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 msgr2=0x7f057c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.754+0000 7f0581140700 1 --2- 172.21.15.94:0/1213009861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 0x7f057c0fc0e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f056c004940 tx=0x7f056c009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.754+0000 7f0581140700 1 -- 172.21.15.94:0/1213009861 shutdown_connections 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.754+0000 7f0581140700 1 --2- 172.21.15.94:0/1213009861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0fcf80 0x7f057c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.754+0000 7f0581140700 1 --2- 172.21.15.94:0/1213009861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 0x7f057c0fc0e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.754+0000 7f0581140700 1 -- 172.21.15.94:0/1213009861 >> 172.21.15.94:0/1213009861 conn(0x7f057c0f7430 msgr2=0x7f057c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.754+0000 7f0581140700 1 -- 172.21.15.94:0/1213009861 shutdown_connections 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.754+0000 7f0581140700 1 -- 172.21.15.94:0/1213009861 wait complete. 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f0581140700 1 Processor -- start 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f0581140700 1 -- start start 2024-04-06T03:48:01.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f0581140700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 0x7f057c10c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f0581140700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0fcf80 0x7f057c10ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f0581140700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c10d4e0 con 0x7f057c0fbcd0 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f0581140700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c10dcd0 con 0x7f057c0fcf80 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f057ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 0x7f057c10c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f057ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 0x7f057c10c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57130/0 (socket says 172.21.15.94:57130) 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f057ad9d700 1 -- 172.21.15.94:0/2246717950 learned_addr learned my addr 172.21.15.94:0/2246717950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f057a59c700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0fcf80 0x7f057c10ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f057ad9d700 1 -- 172.21.15.94:0/2246717950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0fcf80 msgr2=0x7f057c10ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f057ad9d700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0fcf80 0x7f057c10ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.755+0000 7f057ad9d700 1 -- 172.21.15.94:0/2246717950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c005040 con 0x7f057c0fbcd0 2024-04-06T03:48:01.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.756+0000 7f057ad9d700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 0x7f057c10c910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f056c00b3b0 tx=0x7f056c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.756+0000 7f05637fe700 1 -- 172.21.15.94:0/2246717950 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f056c00e040 con 0x7f057c0fbcd0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.756+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c10df50 con 0x7f057c0fbcd0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.756+0000 7f05637fe700 1 -- 172.21.15.94:0/2246717950 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f056c00bbf0 con 0x7f057c0fbcd0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.756+0000 7f05637fe700 1 -- 172.21.15.94:0/2246717950 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f056c01c630 con 0x7f057c0fbcd0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.756+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c10e510 con 0x7f057c0fbcd0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.757+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c05f5f0 con 0x7f057c0fbcd0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.758+0000 7f05637fe700 1 -- 172.21.15.94:0/2246717950 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f056c00bd60 con 0x7f057c0fbcd0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.758+0000 7f05637fe700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056406dca0 0x7f0564070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.758+0000 7f05637fe700 1 -- 172.21.15.94:0/2246717950 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f056c08e090 con 0x7f057c0fbcd0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.758+0000 7f057a59c700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056406dca0 0x7f0564070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.758+0000 7f057a59c700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056406dca0 0x7f0564070150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0570009bf0 tx=0x7f05700091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:01.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:01.763+0000 7f05637fe700 1 -- 172.21.15.94:0/2246717950 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f056c05b200 con 0x7f057c0fbcd0 2024-04-06T03:48:02.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.020+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f057c04bed0 con 0x7f057c0fbcd0 2024-04-06T03:48:02.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.020+0000 7f05637fe700 1 -- 172.21.15.94:0/2246717950 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1890 (secure 0 0 0) 0x7f056c05ad90 con 0x7f057c0fbcd0 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:e18 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 17 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:47:47.799730+0000 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:48:02.023 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 79 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=14472} 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:48:02.024 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 14472 members: 14472 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{0:14472} state up:active seq 150 join_fscid=1 addr [v2:172.21.15.94:6828/3794043365,v1:172.21.15.94:6829/3794043365] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:24255} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:48:02.025 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:48:02.026 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{-1:34400} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:48:02.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056406dca0 msgr2=0x7f0564070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056406dca0 0x7f0564070150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0570009bf0 tx=0x7f05700091a0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 msgr2=0x7f057c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 0x7f057c10c910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f056c00b3b0 tx=0x7f056c009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 shutdown_connections 2024-04-06T03:48:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f057c0fbcd0 0x7f057c10c910 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f056406dca0 0x7f0564070150 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 --2- 172.21.15.94:0/2246717950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0fcf80 0x7f057c10ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 >> 172.21.15.94:0/2246717950 conn(0x7f057c0f7430 msgr2=0x7f057c0f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 shutdown_connections 2024-04-06T03:48:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.022+0000 7f0581140700 1 -- 172.21.15.94:0/2246717950 wait complete. 2024-04-06T03:48:02.027 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-04-06T03:48:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.125+0000 7f168227f700 1 -- 172.21.15.94:0/369000268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 msgr2=0x7f167c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.125+0000 7f168227f700 1 --2- 172.21.15.94:0/369000268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 0x7f167c0fd700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f166c004940 tx=0x7f166c009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.125+0000 7f168227f700 1 -- 172.21.15.94:0/369000268 shutdown_connections 2024-04-06T03:48:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.125+0000 7f168227f700 1 --2- 172.21.15.94:0/369000268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f167c0fdc40 0x7f167c0fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.125+0000 7f168227f700 1 --2- 172.21.15.94:0/369000268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 0x7f167c0fd700 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.125+0000 7f168227f700 1 -- 172.21.15.94:0/369000268 >> 172.21.15.94:0/369000268 conn(0x7f167c0f6bd0 msgr2=0x7f167c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.125+0000 7f168227f700 1 -- 172.21.15.94:0/369000268 shutdown_connections 2024-04-06T03:48:02.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.125+0000 7f168227f700 1 -- 172.21.15.94:0/369000268 wait complete. 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.126+0000 7f168227f700 1 Processor -- start 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.126+0000 7f168227f700 1 -- start start 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.126+0000 7f168227f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 0x7f167c0fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.126+0000 7f168227f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f167c0fdc40 0x7f167c112770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.126+0000 7f168227f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c112cb0 con 0x7f167c0fd2f0 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.126+0000 7f168227f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c112df0 con 0x7f167c0fdc40 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.126+0000 7f167bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 0x7f167c0fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.126+0000 7f167bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 0x7f167c0fb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57140/0 (socket says 172.21.15.94:57140) 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.126+0000 7f167bfff700 1 -- 172.21.15.94:0/2368580736 learned_addr learned my addr 172.21.15.94:0/2368580736 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f167b7fe700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f167c0fdc40 0x7f167c112770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:02.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f167bfff700 1 -- 172.21.15.94:0/2368580736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f167c0fdc40 msgr2=0x7f167c112770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f167bfff700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f167c0fdc40 0x7f167c112770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f167bfff700 1 -- 172.21.15.94:0/2368580736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c005040 con 0x7f167c0fd2f0 2024-04-06T03:48:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f167bfff700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 0x7f167c0fb9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b3b0 tx=0x7f166c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f1678ff9700 1 -- 172.21.15.94:0/2368580736 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166c00e040 con 0x7f167c0fd2f0 2024-04-06T03:48:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c113020 con 0x7f167c0fd2f0 2024-04-06T03:48:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f1678ff9700 1 -- 172.21.15.94:0/2368580736 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f166c00bbf0 con 0x7f167c0fd2f0 2024-04-06T03:48:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f1678ff9700 1 -- 172.21.15.94:0/2368580736 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166c01c630 con 0x7f167c0fd2f0 2024-04-06T03:48:02.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.127+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c113580 con 0x7f167c0fd2f0 2024-04-06T03:48:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.128+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c04c710 con 0x7f167c0fd2f0 2024-04-06T03:48:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.129+0000 7f1678ff9700 1 -- 172.21.15.94:0/2368580736 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f166c00bd60 con 0x7f167c0fd2f0 2024-04-06T03:48:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.129+0000 7f1678ff9700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f166406dca0 0x7f1664070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.129+0000 7f1678ff9700 1 -- 172.21.15.94:0/2368580736 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f166c08e230 con 0x7f167c0fd2f0 2024-04-06T03:48:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.129+0000 7f167b7fe700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f166406dca0 0x7f1664070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:02.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.129+0000 7f167b7fe700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f166406dca0 0x7f1664070150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f16700041d0 tx=0x7f16700043c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.134+0000 7f1678ff9700 1 -- 172.21.15.94:0/2368580736 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f166c05b320 con 0x7f167c0fd2f0 2024-04-06T03:48:02.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:01 smithi094 ceph-mon[94053]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:02.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:01 smithi094 ceph-mon[94053]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:02.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:01 smithi094 ceph-mon[94053]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:02.197 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:01 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:02.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:01 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.kvxcqo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:48:02.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:01 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:02.198 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:01 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/4020287833' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:02.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:01 smithi136 ceph-mon[82061]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:02.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:01 smithi136 ceph-mon[82061]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:02.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:01 smithi136 ceph-mon[82061]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:02.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:01 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:02.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:01 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi094.kvxcqo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:48:02.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:01 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:02.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:01 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/4020287833' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:02.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.362+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f167c1013d0 con 0x7f166406dca0 2024-04-06T03:48:02.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.364+0000 7f1678ff9700 1 -- 172.21.15.94:0/2368580736 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f167c1013d0 con 0x7f166406dca0 2024-04-06T03:48:02.366 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:48:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:48:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:48:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:48:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:48:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:48:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd", 2024-04-06T03:48:02.366 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:48:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:48:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:48:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "13/21 daemons upgraded", 2024-04-06T03:48:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mds daemons", 2024-04-06T03:48:02.367 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:48:02.367 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:48:02.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f166406dca0 msgr2=0x7f1664070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f166406dca0 0x7f1664070150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f16700041d0 tx=0x7f16700043c0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 msgr2=0x7f167c0fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 0x7f167c0fb9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b3b0 tx=0x7f166c009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:02.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 shutdown_connections 2024-04-06T03:48:02.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0fd2f0 0x7f167c0fb9c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f166406dca0 0x7f1664070150 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 --2- 172.21.15.94:0/2368580736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f167c0fdc40 0x7f167c112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 >> 172.21.15.94:0/2368580736 conn(0x7f167c0f6bd0 msgr2=0x7f167c0f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:02.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 shutdown_connections 2024-04-06T03:48:02.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.366+0000 7f168227f700 1 -- 172.21.15.94:0/2368580736 wait complete. 2024-04-06T03:48:02.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.471+0000 7f5b28d90700 1 -- 172.21.15.94:0/1097374647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fe060 msgr2=0x7f5b240fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.471+0000 7f5b28d90700 1 --2- 172.21.15.94:0/1097374647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fe060 0x7f5b240fe4d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b18004940 tx=0x7f5b18009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.471+0000 7f5b28d90700 1 -- 172.21.15.94:0/1097374647 shutdown_connections 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.471+0000 7f5b28d90700 1 --2- 172.21.15.94:0/1097374647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fe060 0x7f5b240fe4d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.471+0000 7f5b28d90700 1 --2- 172.21.15.94:0/1097374647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b240fcef0 0x7f5b240fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.471+0000 7f5b28d90700 1 -- 172.21.15.94:0/1097374647 >> 172.21.15.94:0/1097374647 conn(0x7f5b240f85b0 msgr2=0x7f5b240fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.471+0000 7f5b28d90700 1 -- 172.21.15.94:0/1097374647 shutdown_connections 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.471+0000 7f5b28d90700 1 -- 172.21.15.94:0/1097374647 wait complete. 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b28d90700 1 Processor -- start 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b28d90700 1 -- start start 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b28d90700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fcef0 0x7f5b2410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b28d90700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b240fe060 0x7f5b2410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b28d90700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2410e580 con 0x7f5b240fcef0 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b28d90700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2410edd0 con 0x7f5b240fe060 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b2259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fcef0 0x7f5b2410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b2259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fcef0 0x7f5b2410da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57154/0 (socket says 172.21.15.94:57154) 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b2259c700 1 -- 172.21.15.94:0/293392112 learned_addr learned my addr 172.21.15.94:0/293392112 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.472+0000 7f5b21d9b700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b240fe060 0x7f5b2410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.473+0000 7f5b2259c700 1 -- 172.21.15.94:0/293392112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b240fe060 msgr2=0x7f5b2410df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.473+0000 7f5b2259c700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b240fe060 0x7f5b2410df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.473+0000 7f5b2259c700 1 -- 172.21.15.94:0/293392112 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18005040 con 0x7f5b240fcef0 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.473+0000 7f5b2259c700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fcef0 0x7f5b2410da40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c830 tx=0x7f5b14004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.473+0000 7f5b12ffd700 1 -- 172.21.15.94:0/293392112 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b140106f0 con 0x7f5b240fcef0 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.473+0000 7f5b12ffd700 1 -- 172.21.15.94:0/293392112 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b14010d20 con 0x7f5b240fcef0 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.473+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2410f0b0 con 0x7f5b240fcef0 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.473+0000 7f5b12ffd700 1 -- 172.21.15.94:0/293392112 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b140113a0 con 0x7f5b240fcef0 2024-04-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.473+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2410f5f0 con 0x7f5b240fcef0 2024-04-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.474+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2405f5f0 con 0x7f5b240fcef0 2024-04-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.474+0000 7f5b12ffd700 1 -- 172.21.15.94:0/293392112 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f5b14012400 con 0x7f5b240fcef0 2024-04-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.475+0000 7f5b12ffd700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b0c06dcb0 0x7f5b0c070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.475+0000 7f5b12ffd700 1 -- 172.21.15.94:0/293392112 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5b1408d4b0 con 0x7f5b240fcef0 2024-04-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.475+0000 7f5b21d9b700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b0c06dcb0 0x7f5b0c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.475+0000 7f5b21d9b700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b0c06dcb0 0x7f5b0c070160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009fd0 tx=0x7f5b18009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.479+0000 7f5b12ffd700 1 -- 172.21.15.94:0/293392112 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5b1405a640 con 0x7f5b240fcef0 2024-04-06T03:48:02.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.771+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5b2404bed0 con 0x7f5b240fcef0 2024-04-06T03:48:02.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.771+0000 7f5b12ffd700 1 -- 172.21.15.94:0/293392112 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+155 (secure 0 0 0) 0x7f5b1405a1d0 con 0x7f5b240fcef0 2024-04-06T03:48:02.774 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:48:02.774 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:48:02.774 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in error state 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.773+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b0c06dcb0 msgr2=0x7f5b0c070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.773+0000 7f5b28d90700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b0c06dcb0 0x7f5b0c070160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009fd0 tx=0x7f5b18009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.774+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fcef0 msgr2=0x7f5b2410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.774+0000 7f5b28d90700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fcef0 0x7f5b2410da40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c830 tx=0x7f5b14004470 comp rx=0 tx=0).stop 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.774+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 shutdown_connections 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.774+0000 7f5b28d90700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b240fcef0 0x7f5b2410da40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.774+0000 7f5b28d90700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b0c06dcb0 0x7f5b0c070160 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.774+0000 7f5b28d90700 1 --2- 172.21.15.94:0/293392112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b240fe060 0x7f5b2410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.774+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 >> 172.21.15.94:0/293392112 conn(0x7f5b240f85b0 msgr2=0x7f5b240f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.774+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 shutdown_connections 2024-04-06T03:48:02.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:02.774+0000 7f5b28d90700 1 -- 172.21.15.94:0/293392112 wait complete. 2024-04-06T03:48:03.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:02 smithi136 ceph-mon[82061]: Upgrade: Updating mds.cephfs.smithi094.kvxcqo 2024-04-06T03:48:03.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:02 smithi136 ceph-mon[82061]: Deploying daemon mds.cephfs.smithi094.kvxcqo on smithi094 2024-04-06T03:48:03.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:02 smithi136 ceph-mon[82061]: pgmap v195: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:48:03.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:02 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2246717950' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:48:03.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:02 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/293392112' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:48:03.109 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:02 smithi094 ceph-mon[94053]: Upgrade: Updating mds.cephfs.smithi094.kvxcqo 2024-04-06T03:48:03.110 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:02 smithi094 ceph-mon[94053]: Deploying daemon mds.cephfs.smithi094.kvxcqo on smithi094 2024-04-06T03:48:03.110 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:02 smithi094 ceph-mon[94053]: pgmap v195: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:48:03.110 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:02 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2246717950' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:48:03.110 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:02 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/293392112' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:48:03.965 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:03 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[94049]: 2024-04-06T03:48:03.696+0000 7f71b5c08700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:48:04.098 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:48:03.716+0000 7fa76077b700 -1 bdev(0x563158eaa000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 1 2024-04-06T03:48:04.098 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:48:03.718+0000 7fa765f86700 -1 bdev(0x563158eaa000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 9 2024-04-06T03:48:04.099 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:48:03.718+0000 7fa762f80700 -1 bdev(0x563158eaa000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 8 2024-04-06T03:48:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-mon[82061]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-mon[82061]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-06T03:48:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-mon[82061]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:48:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-mon[82061]: osdmap e80: 6 total, 6 up, 6 in 2024-04-06T03:48:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-mon[82061]: Standby daemon mds.cephfs.smithi136.hbjgtw assigned to filesystem cephfs as rank 0 2024-04-06T03:48:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-mon[82061]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:48:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-mon[82061]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-06T03:48:04.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:03 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi136.hbjgtw=up:replay} 2 up:standby 2024-04-06T03:48:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:03 smithi094 ceph-mon[94053]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:03 smithi094 ceph-mon[94053]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-06T03:48:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:03 smithi094 ceph-mon[94053]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:48:04.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:03 smithi094 ceph-mon[94053]: osdmap e80: 6 total, 6 up, 6 in 2024-04-06T03:48:04.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:03 smithi094 ceph-mon[94053]: Standby daemon mds.cephfs.smithi136.hbjgtw assigned to filesystem cephfs as rank 0 2024-04-06T03:48:04.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:03 smithi094 ceph-mon[94053]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:48:04.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:03 smithi094 ceph-mon[94053]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-06T03:48:04.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:03 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi136.hbjgtw=up:replay} 2 up:standby 2024-04-06T03:48:05.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:04 smithi136 ceph-mon[82061]: pgmap v196: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:48:05.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:04 smithi094 ceph-mon[94053]: pgmap v196: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:48:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:48:06.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:48:06.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:06.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:48:06.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:48:07.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:07 smithi136 ceph-mon[82061]: pgmap v198: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 3 op/s 2024-04-06T03:48:07.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:07 smithi094 ceph-mon[94053]: pgmap v198: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 3 op/s 2024-04-06T03:48:09.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:09 smithi136 ceph-mon[82061]: pgmap v199: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-06T03:48:09.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:09 smithi094 ceph-mon[94053]: pgmap v199: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-06T03:48:11.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:11 smithi136 ceph-mon[82061]: pgmap v200: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-06T03:48:11.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:11 smithi094 ceph-mon[94053]: pgmap v200: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-06T03:48:13.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:13 smithi094 ceph-mon[94053]: pgmap v201: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-06T03:48:13.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:13 smithi094 ceph-mon[94053]: reconnect by client.14524 v1:172.21.15.136:0/1309930737 after 0 2024-04-06T03:48:13.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:13 smithi094 ceph-mon[94053]: reconnect by client.24295 v1:172.21.15.94:0/2208962971 after 0.000999978 2024-04-06T03:48:13.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:13 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:reconnect 2024-04-06T03:48:13.518 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:13 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi136.hbjgtw=up:reconnect} 2 up:standby 2024-04-06T03:48:13.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:13 smithi136 ceph-mon[82061]: pgmap v201: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-06T03:48:13.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:13 smithi136 ceph-mon[82061]: reconnect by client.14524 v1:172.21.15.136:0/1309930737 after 0 2024-04-06T03:48:13.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:13 smithi136 ceph-mon[82061]: reconnect by client.24295 v1:172.21.15.94:0/2208962971 after 0.000999978 2024-04-06T03:48:13.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:13 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:reconnect 2024-04-06T03:48:13.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:13 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi136.hbjgtw=up:reconnect} 2 up:standby 2024-04-06T03:48:14.555 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:14 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:rejoin 2024-04-06T03:48:14.555 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:14 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi136.hbjgtw=up:rejoin} 2 up:standby 2024-04-06T03:48:14.556 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:14 smithi094 ceph-mon[94053]: daemon mds.cephfs.smithi136.hbjgtw is now active in filesystem cephfs as rank 0 2024-04-06T03:48:14.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:14 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:rejoin 2024-04-06T03:48:14.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:14 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi136.hbjgtw=up:rejoin} 2 up:standby 2024-04-06T03:48:14.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:14 smithi136 ceph-mon[82061]: daemon mds.cephfs.smithi136.hbjgtw is now active in filesystem cephfs as rank 0 2024-04-06T03:48:15.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:15 smithi136 ceph-mon[82061]: pgmap v202: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-06T03:48:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:15 smithi136 ceph-mon[82061]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-06T03:48:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:15 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:active 2024-04-06T03:48:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:15 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi136.hbjgtw=up:active} 2 up:standby 2024-04-06T03:48:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:15 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:15 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:15.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:15 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:15.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:15 smithi094 ceph-mon[94053]: pgmap v202: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-06T03:48:15.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:15 smithi094 ceph-mon[94053]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-06T03:48:15.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:15 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.136:6824/838120541,v1:172.21.15.136:6825/838120541] up:active 2024-04-06T03:48:15.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:15 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi136.hbjgtw=up:active} 2 up:standby 2024-04-06T03:48:15.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:15.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:15.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:16.528 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:16 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.94:6828/1650064917,v1:172.21.15.94:6829/1650064917] up:boot 2024-04-06T03:48:16.529 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:16 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi136.hbjgtw=up:active} 3 up:standby 2024-04-06T03:48:16.529 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:48:16.529 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:16 smithi094 ceph-mon[94053]: pgmap v203: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 172 B/s wr, 6 op/s 2024-04-06T03:48:16.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:16 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.94:6828/1650064917,v1:172.21.15.94:6829/1650064917] up:boot 2024-04-06T03:48:16.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:16 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi136.hbjgtw=up:active} 3 up:standby 2024-04-06T03:48:16.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:48:16.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:16 smithi136 ceph-mon[82061]: pgmap v203: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 172 B/s wr, 6 op/s 2024-04-06T03:48:18.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:18 smithi094 ceph-mon[94053]: pgmap v204: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-06T03:48:18.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:18.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:18 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:19.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:18 smithi136 ceph-mon[82061]: pgmap v204: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-06T03:48:19.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:19.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:18 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:20.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:20 smithi094 ceph-mon[94053]: pgmap v205: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-06T03:48:20.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:20.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:20.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:20.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:48:21.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:20 smithi136 ceph-mon[82061]: pgmap v205: 65 pgs: 65 active+clean; 194 MiB data, 843 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-06T03:48:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:21.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:48:22.918 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:22 smithi094 ceph-mon[94053]: pgmap v206: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-06T03:48:23.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:22 smithi136 ceph-mon[82061]: pgmap v206: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: pgmap v207: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.5 KiB/s wr, 5 op/s 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: pgmap v208: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 453 KiB/s rd, 5.5 KiB/s wr, 6 op/s 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:24.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:24.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:24.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:24.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi136.hbjgtw"]}]: dispatch 2024-04-06T03:48:24.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:24 smithi136 ceph-mon[82061]: Upgrade: It appears safe to stop mds.cephfs.smithi136.hbjgtw 2024-04-06T03:48:24.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: pgmap v207: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.5 KiB/s wr, 5 op/s 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: pgmap v208: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 453 KiB/s rd, 5.5 KiB/s wr, 6 op/s 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:24.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi136.hbjgtw"]}]: dispatch 2024-04-06T03:48:24.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:24 smithi094 ceph-mon[94053]: Upgrade: It appears safe to stop mds.cephfs.smithi136.hbjgtw 2024-04-06T03:48:26.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:25 smithi136 ceph-mon[82061]: Upgrade: Updating mds.cephfs.smithi136.hbjgtw 2024-04-06T03:48:26.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:25 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:26.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:25 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.hbjgtw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:48:26.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:25 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:26.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:25 smithi136 ceph-mon[82061]: Deploying daemon mds.cephfs.smithi136.hbjgtw on smithi136 2024-04-06T03:48:26.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:25 smithi136 ceph-mon[82061]: pgmap v209: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 12 KiB/s rd, 5.3 KiB/s wr, 4 op/s 2024-04-06T03:48:26.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:25 smithi094 ceph-mon[94053]: Upgrade: Updating mds.cephfs.smithi136.hbjgtw 2024-04-06T03:48:26.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:25 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:26.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:25 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.hbjgtw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:48:26.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:25 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:26.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:25 smithi094 ceph-mon[94053]: Deploying daemon mds.cephfs.smithi136.hbjgtw on smithi136 2024-04-06T03:48:26.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:25 smithi094 ceph-mon[94053]: pgmap v209: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 12 KiB/s rd, 5.3 KiB/s wr, 4 op/s 2024-04-06T03:48:27.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:26 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[94049]: 2024-04-06T03:48:26.995+0000 7f71b5c08700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:48:27.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:27 smithi094 ceph-mon[94053]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-06T03:48:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:27 smithi094 ceph-mon[94053]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:48:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:27 smithi094 ceph-mon[94053]: osdmap e81: 6 total, 6 up, 6 in 2024-04-06T03:48:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:27 smithi094 ceph-mon[94053]: Standby daemon mds.cephfs.smithi136.trdonf assigned to filesystem cephfs as rank 0 2024-04-06T03:48:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:27 smithi094 ceph-mon[94053]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:48:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:27 smithi094 ceph-mon[94053]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-06T03:48:27.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:27 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi136.trdonf=up:replay} 2 up:standby 2024-04-06T03:48:27.348 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:48:27 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:48:27.014+0000 7fa764783700 -1 bdev(0x563158eaa000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 2 2024-04-06T03:48:27.348 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:48:27 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:48:27.015+0000 7fa761f7e700 -1 bdev(0x563158eaa000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-04-06T03:48:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:27 smithi136 ceph-mon[82061]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-06T03:48:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:27 smithi136 ceph-mon[82061]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:48:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:27 smithi136 ceph-mon[82061]: osdmap e81: 6 total, 6 up, 6 in 2024-04-06T03:48:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:27 smithi136 ceph-mon[82061]: Standby daemon mds.cephfs.smithi136.trdonf assigned to filesystem cephfs as rank 0 2024-04-06T03:48:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:27 smithi136 ceph-mon[82061]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:48:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:27 smithi136 ceph-mon[82061]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-06T03:48:27.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:27 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi136.trdonf=up:replay} 2 up:standby 2024-04-06T03:48:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:28 smithi136 ceph-mon[82061]: pgmap v211: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 6.7 KiB/s wr, 3 op/s 2024-04-06T03:48:28.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:28 smithi094 ceph-mon[94053]: pgmap v211: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 6.7 KiB/s wr, 3 op/s 2024-04-06T03:48:30.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:30 smithi094 ceph-mon[94053]: pgmap v212: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 137 B/s rd, 0 op/s 2024-04-06T03:48:30.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:30 smithi136 ceph-mon[82061]: pgmap v212: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 137 B/s rd, 0 op/s 2024-04-06T03:48:32.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:32 smithi094 ceph-mon[94053]: pgmap v213: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 8.2 MiB/s rd, 3 op/s 2024-04-06T03:48:32.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:32 smithi136 ceph-mon[82061]: pgmap v213: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 8.2 MiB/s rd, 3 op/s 2024-04-06T03:48:32.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.889+0000 7fac1fec1700 1 -- 172.21.15.94:0/4091055024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 msgr2=0x7fac180ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.889+0000 7fac1fec1700 1 --2- 172.21.15.94:0/4091055024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 0x7fac180ffa20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac08004940 tx=0x7fac08009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 -- 172.21.15.94:0/4091055024 shutdown_connections 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 --2- 172.21.15.94:0/4091055024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac180fff60 0x7fac18102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 --2- 172.21.15.94:0/4091055024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 0x7fac180ffa20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 -- 172.21.15.94:0/4091055024 >> 172.21.15.94:0/4091055024 conn(0x7fac180f7430 msgr2=0x7fac180f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 -- 172.21.15.94:0/4091055024 shutdown_connections 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 -- 172.21.15.94:0/4091055024 wait complete. 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 Processor -- start 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 -- start start 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 0x7fac180fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac180fff60 0x7fac180fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac180fbed0 con 0x7fac18066280 2024-04-06T03:48:32.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.890+0000 7fac1fec1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac180fc040 con 0x7fac180fff60 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1dc5d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 0x7fac180fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1dc5d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 0x7fac180fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38212/0 (socket says 172.21.15.94:38212) 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1dc5d700 1 -- 172.21.15.94:0/2562054433 learned_addr learned my addr 172.21.15.94:0/2562054433 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1d45c700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac180fff60 0x7fac180fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1dc5d700 1 -- 172.21.15.94:0/2562054433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac180fff60 msgr2=0x7fac180fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1dc5d700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac180fff60 0x7fac180fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1dc5d700 1 -- 172.21.15.94:0/2562054433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac08005040 con 0x7fac18066280 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1dc5d700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 0x7fac180fd450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fac0800b970 tx=0x7fac08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac167fc700 1 -- 172.21.15.94:0/2562054433 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac0800e040 con 0x7fac18066280 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac167fc700 1 -- 172.21.15.94:0/2562054433 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac0800bbf0 con 0x7fac18066280 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac167fc700 1 -- 172.21.15.94:0/2562054433 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac0801c3f0 con 0x7fac18066280 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac180fc2c0 con 0x7fac18066280 2024-04-06T03:48:32.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.891+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac18115ad0 con 0x7fac18066280 2024-04-06T03:48:32.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.892+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1805f5f0 con 0x7fac18066280 2024-04-06T03:48:32.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.893+0000 7fac167fc700 1 -- 172.21.15.94:0/2562054433 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fac0800bd60 con 0x7fac18066280 2024-04-06T03:48:32.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.893+0000 7fac167fc700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fac0006dcb0 0x7fac00070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:32.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.894+0000 7fac1d45c700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fac0006dcb0 0x7fac00070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:32.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.894+0000 7fac167fc700 1 -- 172.21.15.94:0/2562054433 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fac0808e8d0 con 0x7fac18066280 2024-04-06T03:48:32.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.894+0000 7fac1d45c700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fac0006dcb0 0x7fac00070160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0102e0 tx=0x7fac0c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:32.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:32.898+0000 7fac167fc700 1 -- 172.21.15.94:0/2562054433 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fac08057e80 con 0x7fac18066280 2024-04-06T03:48:33.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.114+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac180f8f70 con 0x7fac0006dcb0 2024-04-06T03:48:33.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.115+0000 7fac167fc700 1 -- 172.21.15.94:0/2562054433 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fac180f8f70 con 0x7fac0006dcb0 2024-04-06T03:48:33.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fac0006dcb0 msgr2=0x7fac00070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fac0006dcb0 0x7fac00070160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0102e0 tx=0x7fac0c00b040 comp rx=0 tx=0).stop 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 msgr2=0x7fac180fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 0x7fac180fd450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fac0800b970 tx=0x7fac08009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 shutdown_connections 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fac18066280 0x7fac180fd450 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fac0006dcb0 0x7fac00070160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 --2- 172.21.15.94:0/2562054433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac180fff60 0x7fac180fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 >> 172.21.15.94:0/2562054433 conn(0x7fac180f7430 msgr2=0x7fac180f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 shutdown_connections 2024-04-06T03:48:33.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.117+0000 7fac1fec1700 1 -- 172.21.15.94:0/2562054433 wait complete. 2024-04-06T03:48:33.130 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:48:33.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 -- 172.21.15.94:0/2125906629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 msgr2=0x7f754c0fe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 --2- 172.21.15.94:0/2125906629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 0x7f754c0fe900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7548004940 tx=0x7f7548009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:33.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 -- 172.21.15.94:0/2125906629 shutdown_connections 2024-04-06T03:48:33.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 --2- 172.21.15.94:0/2125906629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f754c0fee40 0x7f754c0faf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 --2- 172.21.15.94:0/2125906629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 0x7f754c0fe900 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 -- 172.21.15.94:0/2125906629 >> 172.21.15.94:0/2125906629 conn(0x7f754c0f6bd0 msgr2=0x7f754c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 -- 172.21.15.94:0/2125906629 shutdown_connections 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 -- 172.21.15.94:0/2125906629 wait complete. 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 Processor -- start 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 -- start start 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 0x7f754c10d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f754c0fee40 0x7f754c10de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c10e400 con 0x7f754c0fe4f0 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.220+0000 7f7553b2d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c10e570 con 0x7f754c0fee40 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75518c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 0x7f754c10d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75518c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 0x7f754c10d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38240/0 (socket says 172.21.15.94:38240) 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75518c9700 1 -- 172.21.15.94:0/156475785 learned_addr learned my addr 172.21.15.94:0/156475785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75510c8700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f754c0fee40 0x7f754c10de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:33.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75518c9700 1 -- 172.21.15.94:0/156475785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f754c0fee40 msgr2=0x7f754c10de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75518c9700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f754c0fee40 0x7f754c10de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75518c9700 1 -- 172.21.15.94:0/156475785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7548005040 con 0x7f754c0fe4f0 2024-04-06T03:48:33.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75518c9700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 0x7f754c10d8c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7548009150 tx=0x7f7548009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:33.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75427fc700 1 -- 172.21.15.94:0/156475785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f754800e040 con 0x7f754c0fe4f0 2024-04-06T03:48:33.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75427fc700 1 -- 172.21.15.94:0/156475785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7548004290 con 0x7f754c0fe4f0 2024-04-06T03:48:33.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c112f70 con 0x7f754c0fe4f0 2024-04-06T03:48:33.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f75427fc700 1 -- 172.21.15.94:0/156475785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f754801c630 con 0x7f754c0fe4f0 2024-04-06T03:48:33.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.221+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c1134a0 con 0x7f754c0fe4f0 2024-04-06T03:48:33.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.222+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c04c710 con 0x7f754c0fe4f0 2024-04-06T03:48:33.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.223+0000 7f75427fc700 1 -- 172.21.15.94:0/156475785 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f754800bbf0 con 0x7f754c0fe4f0 2024-04-06T03:48:33.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.223+0000 7f75427fc700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f753806dca0 0x7f7538070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:33.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.223+0000 7f75427fc700 1 -- 172.21.15.94:0/156475785 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f754808e180 con 0x7f754c0fe4f0 2024-04-06T03:48:33.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.223+0000 7f75510c8700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f753806dca0 0x7f7538070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:33.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.224+0000 7f75510c8700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f753806dca0 0x7f7538070150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f753c009fd0 tx=0x7f753c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:33.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.228+0000 7f75427fc700 1 -- 172.21.15.94:0/156475785 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f754805b260 con 0x7f754c0fe4f0 2024-04-06T03:48:33.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.440+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f754c102540 con 0x7f753806dca0 2024-04-06T03:48:33.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.441+0000 7f75427fc700 1 -- 172.21.15.94:0/156475785 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f754c102540 con 0x7f753806dca0 2024-04-06T03:48:33.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f753806dca0 msgr2=0x7f7538070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f753806dca0 0x7f7538070150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f753c009fd0 tx=0x7f753c00b040 comp rx=0 tx=0).stop 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 msgr2=0x7f754c10d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 0x7f754c10d8c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7548009150 tx=0x7f7548009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 shutdown_connections 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f754c0fe4f0 0x7f754c10d8c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f753806dca0 0x7f7538070150 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 --2- 172.21.15.94:0/156475785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f754c0fee40 0x7f754c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 >> 172.21.15.94:0/156475785 conn(0x7f754c0f6bd0 msgr2=0x7f754c0f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.443+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 shutdown_connections 2024-04-06T03:48:33.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.444+0000 7f7553b2d700 1 -- 172.21.15.94:0/156475785 wait complete. 2024-04-06T03:48:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.544+0000 7f23b74e4700 1 -- 172.21.15.94:0/2445152236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fcf80 msgr2=0x7f23b00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.544+0000 7f23b74e4700 1 --2- 172.21.15.94:0/2445152236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fcf80 0x7f23b00fd3f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f23ac004940 tx=0x7f23ac009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.544+0000 7f23b74e4700 1 -- 172.21.15.94:0/2445152236 shutdown_connections 2024-04-06T03:48:33.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.544+0000 7f23b74e4700 1 --2- 172.21.15.94:0/2445152236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fcf80 0x7f23b00fd3f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.544+0000 7f23b74e4700 1 --2- 172.21.15.94:0/2445152236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b00fbcd0 0x7f23b00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.544+0000 7f23b74e4700 1 -- 172.21.15.94:0/2445152236 >> 172.21.15.94:0/2445152236 conn(0x7f23b00f7430 msgr2=0x7f23b00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.544+0000 7f23b74e4700 1 -- 172.21.15.94:0/2445152236 shutdown_connections 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.544+0000 7f23b74e4700 1 -- 172.21.15.94:0/2445152236 wait complete. 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b74e4700 1 Processor -- start 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b74e4700 1 -- start start 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b74e4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fbcd0 0x7f23b0066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b74e4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b00fcf80 0x7f23b0114b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b74e4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b010e510 con 0x7f23b00fbcd0 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b74e4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0115070 con 0x7f23b00fcf80 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b5280700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fbcd0 0x7f23b0066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b5280700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fbcd0 0x7f23b0066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38260/0 (socket says 172.21.15.94:38260) 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b5280700 1 -- 172.21.15.94:0/3264629159 learned_addr learned my addr 172.21.15.94:0/3264629159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b4a7f700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b00fcf80 0x7f23b0114b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:33.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b5280700 1 -- 172.21.15.94:0/3264629159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b00fcf80 msgr2=0x7f23b0114b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b5280700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b00fcf80 0x7f23b0114b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b5280700 1 -- 172.21.15.94:0/3264629159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac005040 con 0x7f23b00fbcd0 2024-04-06T03:48:33.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.545+0000 7f23b5280700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fbcd0 0x7f23b0066cf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f23a0002a80 tx=0x7f23a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:33.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.546+0000 7f23a5ffb700 1 -- 172.21.15.94:0/3264629159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23a0007cc0 con 0x7f23b00fbcd0 2024-04-06T03:48:33.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.546+0000 7f23a5ffb700 1 -- 172.21.15.94:0/3264629159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23a000f040 con 0x7f23b00fbcd0 2024-04-06T03:48:33.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.546+0000 7f23a5ffb700 1 -- 172.21.15.94:0/3264629159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23a000daf0 con 0x7f23b00fbcd0 2024-04-06T03:48:33.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.546+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b0115270 con 0x7f23b00fbcd0 2024-04-06T03:48:33.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.546+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b01157e0 con 0x7f23b00fbcd0 2024-04-06T03:48:33.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.547+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b004c710 con 0x7f23b00fbcd0 2024-04-06T03:48:33.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.547+0000 7f23a5ffb700 1 -- 172.21.15.94:0/3264629159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f23a000dc50 con 0x7f23b00fbcd0 2024-04-06T03:48:33.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.548+0000 7f23a5ffb700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f239c06dca0 0x7f239c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:33.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.548+0000 7f23a5ffb700 1 -- 172.21.15.94:0/3264629159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f23a008d2a0 con 0x7f23b00fbcd0 2024-04-06T03:48:33.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.548+0000 7f23b4a7f700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f239c06dca0 0x7f239c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:33.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.548+0000 7f23b4a7f700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f239c06dca0 0x7f239c070150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00b970 tx=0x7f23ac009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:33.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.552+0000 7f23a5ffb700 1 -- 172.21.15.94:0/3264629159 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f23a005a300 con 0x7f23b00fbcd0 2024-04-06T03:48:33.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.762+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f23b0101340 con 0x7f239c06dca0 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.773+0000 7f23a5ffb700 1 -- 172.21.15.94:0/3264629159 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f23b0101340 con 0x7f239c06dca0 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (7m) 15s ago 15m 26.6M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (4m) 15s ago 16m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (4m) 65s ago 15m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 error 15s ago 15m - - 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (19s) 15s ago 10m 13.6M - 19.0.0-2027-g9f192350 a5511c7deba9 bfca2184eb3c 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (47s) 15s ago 10m 14.9M - 19.0.0-2027-g9f192350 a5511c7deba9 c189927827c8 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (10m) 65s ago 10m 15.4M - 17.2.7-819-g6699495c ec300eaa768b 35e8ccc2a25c 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (10m) 65s ago 10m 17.8M - 17.2.7-819-g6699495c ec300eaa768b 668429c9ac5d 2024-04-06T03:48:33.775 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (6m) 15s ago 17m 506M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (5m) 65s ago 15m 451M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (5m) 15s ago 17m 50.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (5m) 65s ago 14m 43.8M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (7m) 15s ago 16m 15.4M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (6m) 65s ago 15m 17.2M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (4m) 15s ago 14m 111M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (3m) 15s ago 13m 121M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (2m) 15s ago 13m 107M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (2m) 65s ago 12m 105M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (101s) 65s ago 12m 102M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 c31367e1c2d5 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (69s) 65s ago 12m 11.7M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 16386e8f8583 2024-04-06T03:48:33.776 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (5m) 15s ago 15m 61.2M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:48:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f239c06dca0 msgr2=0x7f239c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f239c06dca0 0x7f239c070150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00b970 tx=0x7f23ac009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fbcd0 msgr2=0x7f23b0066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fbcd0 0x7f23b0066cf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f23a0002a80 tx=0x7f23a00046b0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 shutdown_connections 2024-04-06T03:48:33.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b00fbcd0 0x7f23b0066cf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f239c06dca0 0x7f239c070150 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 --2- 172.21.15.94:0/3264629159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23b00fcf80 0x7f23b0114b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 >> 172.21.15.94:0/3264629159 conn(0x7f23b00f7430 msgr2=0x7f23b00f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 shutdown_connections 2024-04-06T03:48:33.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.775+0000 7f23b74e4700 1 -- 172.21.15.94:0/3264629159 wait complete. 2024-04-06T03:48:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 -- 172.21.15.94:0/3285951083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fdfe0 msgr2=0x7f6d600fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 --2- 172.21.15.94:0/3285951083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fdfe0 0x7f6d600fe450 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6d50004940 tx=0x7f6d50009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 -- 172.21.15.94:0/3285951083 shutdown_connections 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 --2- 172.21.15.94:0/3285951083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fdfe0 0x7f6d600fe450 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 --2- 172.21.15.94:0/3285951083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d600fce70 0x7f6d600fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 -- 172.21.15.94:0/3285951083 >> 172.21.15.94:0/3285951083 conn(0x7f6d600f8550 msgr2=0x7f6d600fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 -- 172.21.15.94:0/3285951083 shutdown_connections 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 -- 172.21.15.94:0/3285951083 wait complete. 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 Processor -- start 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 -- start start 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fce70 0x7f6d6010fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d600fdfe0 0x7f6d60110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d60110670 con 0x7f6d600fce70 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.875+0000 7f6d67858700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601107e0 con 0x7f6d600fdfe0 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d655f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fce70 0x7f6d6010fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d655f4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fce70 0x7f6d6010fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38284/0 (socket says 172.21.15.94:38284) 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d655f4700 1 -- 172.21.15.94:0/1069653263 learned_addr learned my addr 172.21.15.94:0/1069653263 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d64df3700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d600fdfe0 0x7f6d60110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d655f4700 1 -- 172.21.15.94:0/1069653263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d600fdfe0 msgr2=0x7f6d60110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d655f4700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d600fdfe0 0x7f6d60110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d655f4700 1 -- 172.21.15.94:0/1069653263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50005040 con 0x7f6d600fce70 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d655f4700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fce70 0x7f6d6010fb30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c002a50 tx=0x7f6d5c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d55ffb700 1 -- 172.21.15.94:0/1069653263 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d5c007cc0 con 0x7f6d600fce70 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d55ffb700 1 -- 172.21.15.94:0/1069653263 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d5c00f040 con 0x7f6d600fce70 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d60115240 con 0x7f6d600fce70 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.876+0000 7f6d55ffb700 1 -- 172.21.15.94:0/1069653263 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d5c00da30 con 0x7f6d600fce70 2024-04-06T03:48:33.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.877+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d60115800 con 0x7f6d600fce70 2024-04-06T03:48:33.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.878+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d60101970 con 0x7f6d600fce70 2024-04-06T03:48:33.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.878+0000 7f6d55ffb700 1 -- 172.21.15.94:0/1069653263 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f6d5c00db90 con 0x7f6d600fce70 2024-04-06T03:48:33.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.879+0000 7f6d55ffb700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d4c06dca0 0x7f6d4c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:33.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.879+0000 7f6d55ffb700 1 -- 172.21.15.94:0/1069653263 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6d5c08d360 con 0x7f6d600fce70 2024-04-06T03:48:33.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.879+0000 7f6d64df3700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d4c06dca0 0x7f6d4c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:33.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.879+0000 7f6d64df3700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d4c06dca0 0x7f6d4c070150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000b970 tx=0x7f6d50009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:33.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:33.883+0000 7f6d55ffb700 1 -- 172.21.15.94:0/1069653263 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6d5c05a3c0 con 0x7f6d600fce70 2024-04-06T03:48:34.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.158+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6d6004c710 con 0x7f6d600fce70 2024-04-06T03:48:34.160 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:34 smithi094 ceph-mon[94053]: pgmap v214: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-06T03:48:34.160 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:34 smithi094 ceph-mon[94053]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:34.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.158+0000 7f6d55ffb700 1 -- 172.21.15.94:0/1069653263 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f6d5c059f50 con 0x7f6d600fce70 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 6 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:48:34.161 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:48:34.162 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 1, 2024-04-06T03:48:34.162 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:48:34.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:48:34.162 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:48:34.162 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 17.2.7-819-g6699495c (6699495ce2af437c1ff648f1736d2457f417a1aa) quincy (stable)": 1, 2024-04-06T03:48:34.162 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 12 2024-04-06T03:48:34.162 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:48:34.162 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d4c06dca0 msgr2=0x7f6d4c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d4c06dca0 0x7f6d4c070150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000b970 tx=0x7f6d50009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fce70 msgr2=0x7f6d6010fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fce70 0x7f6d6010fb30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c002a50 tx=0x7f6d5c0046b0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 shutdown_connections 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6d600fce70 0x7f6d6010fb30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d4c06dca0 0x7f6d4c070150 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 --2- 172.21.15.94:0/1069653263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d600fdfe0 0x7f6d60110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 >> 172.21.15.94:0/1069653263 conn(0x7f6d600f8550 msgr2=0x7f6d600f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 shutdown_connections 2024-04-06T03:48:34.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.160+0000 7f6d67858700 1 -- 172.21.15.94:0/1069653263 wait complete. 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 -- 172.21.15.94:0/4215018840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 msgr2=0x7f4d900fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 --2- 172.21.15.94:0/4215018840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 0x7f4d900fd280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c004940 tx=0x7f4d8c009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 -- 172.21.15.94:0/4215018840 shutdown_connections 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 --2- 172.21.15.94:0/4215018840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d900fdfe0 0x7f4d900fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 --2- 172.21.15.94:0/4215018840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 0x7f4d900fd280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 -- 172.21.15.94:0/4215018840 >> 172.21.15.94:0/4215018840 conn(0x7f4d900f8550 msgr2=0x7f4d900fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 -- 172.21.15.94:0/4215018840 shutdown_connections 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 -- 172.21.15.94:0/4215018840 wait complete. 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 Processor -- start 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.261+0000 7f4d97f5e700 1 -- start start 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d97f5e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 0x7f4d9010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:34.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d97f5e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d900fdfe0 0x7f4d9010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d97f5e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9010e490 con 0x7f4d900fce70 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d97f5e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9010e600 con 0x7f4d900fdfe0 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d95cfa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 0x7f4d9010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d95cfa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 0x7f4d9010d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38302/0 (socket says 172.21.15.94:38302) 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d95cfa700 1 -- 172.21.15.94:0/2839521808 learned_addr learned my addr 172.21.15.94:0/2839521808 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d954f9700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d900fdfe0 0x7f4d9010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d95cfa700 1 -- 172.21.15.94:0/2839521808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d900fdfe0 msgr2=0x7f4d9010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d95cfa700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d900fdfe0 0x7f4d9010de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d95cfa700 1 -- 172.21.15.94:0/2839521808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c005040 con 0x7f4d900fce70 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d95cfa700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 0x7f4d9010d950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00b970 tx=0x7f4d8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d867fc700 1 -- 172.21.15.94:0/2839521808 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d8c00e040 con 0x7f4d900fce70 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d867fc700 1 -- 172.21.15.94:0/2839521808 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d8c004290 con 0x7f4d900fce70 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d90113000 con 0x7f4d900fce70 2024-04-06T03:48:34.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.262+0000 7f4d867fc700 1 -- 172.21.15.94:0/2839521808 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d8c01c630 con 0x7f4d900fce70 2024-04-06T03:48:34.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.263+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d901135c0 con 0x7f4d900fce70 2024-04-06T03:48:34.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.263+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9004c710 con 0x7f4d900fce70 2024-04-06T03:48:34.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.264+0000 7f4d867fc700 1 -- 172.21.15.94:0/2839521808 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f4d8c00bbf0 con 0x7f4d900fce70 2024-04-06T03:48:34.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.264+0000 7f4d867fc700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d7c06dca0 0x7f4d7c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:34.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.264+0000 7f4d867fc700 1 -- 172.21.15.94:0/2839521808 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4d8c08e740 con 0x7f4d900fce70 2024-04-06T03:48:34.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.264+0000 7f4d954f9700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d7c06dca0 0x7f4d7c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:34.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.265+0000 7f4d954f9700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d7c06dca0 0x7f4d7c070150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4d80004f40 tx=0x7f4d8000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:34.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.269+0000 7f4d867fc700 1 -- 172.21.15.94:0/2839521808 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4d8c05b8c0 con 0x7f4d900fce70 2024-04-06T03:48:34.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.508+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d90113830 con 0x7f4d900fce70 2024-04-06T03:48:34.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.508+0000 7f4d867fc700 1 -- 172.21.15.94:0/2839521808 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1704 (secure 0 0 0) 0x7f4d90113830 con 0x7f4d900fce70 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout:e26 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 26 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:48:34.511 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:48:27.001819+0000 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 81 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:48:34.512 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24267} 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 0 members: 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{0:24267} state up:replay seq 2 join_fscid=1 addr [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] compat {c=[1],r=[1],i=[7ff]}] 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:48:34.513 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:48:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:48:34.514 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{-1:34400} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:48:34.514 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/1650064917,v1:172.21.15.94:6829/1650064917] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:48:34.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d7c06dca0 msgr2=0x7f4d7c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d7c06dca0 0x7f4d7c070150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4d80004f40 tx=0x7f4d8000b040 comp rx=0 tx=0).stop 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 msgr2=0x7f4d9010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 0x7f4d9010d950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00b970 tx=0x7f4d8c009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 shutdown_connections 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d900fce70 0x7f4d9010d950 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4d7c06dca0 0x7f4d7c070150 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 --2- 172.21.15.94:0/2839521808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d900fdfe0 0x7f4d9010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 >> 172.21.15.94:0/2839521808 conn(0x7f4d900f8550 msgr2=0x7f4d900f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 shutdown_connections 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.510+0000 7f4d97f5e700 1 -- 172.21.15.94:0/2839521808 wait complete. 2024-04-06T03:48:34.515 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-04-06T03:48:34.530 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:34 smithi136 ceph-mon[82061]: pgmap v214: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-04-06T03:48:34.530 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:34 smithi136 ceph-mon[82061]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.612+0000 7f6ae71d2700 1 -- 172.21.15.94:0/1054790484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae0066f10 msgr2=0x7f6ae00ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.612+0000 7f6ae71d2700 1 --2- 172.21.15.94:0/1054790484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae0066f10 0x7f6ae00ffa50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4004940 tx=0x7f6ad4009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.612+0000 7f6ae71d2700 1 -- 172.21.15.94:0/1054790484 shutdown_connections 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.612+0000 7f6ae71d2700 1 --2- 172.21.15.94:0/1054790484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ae00fff90 0x7f6ae0102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.612+0000 7f6ae71d2700 1 --2- 172.21.15.94:0/1054790484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae0066f10 0x7f6ae00ffa50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.612+0000 7f6ae71d2700 1 -- 172.21.15.94:0/1054790484 >> 172.21.15.94:0/1054790484 conn(0x7f6ae00f7480 msgr2=0x7f6ae00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.612+0000 7f6ae71d2700 1 -- 172.21.15.94:0/1054790484 shutdown_connections 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.612+0000 7f6ae71d2700 1 -- 172.21.15.94:0/1054790484 wait complete. 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6ae71d2700 1 Processor -- start 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6ae71d2700 1 -- start start 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6ae71d2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ae0066f10 0x7f6ae010d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6ae71d2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae00fff90 0x7f6ae010de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6ae71d2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae010e410 con 0x7f6ae00fff90 2024-04-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6ae71d2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae010e580 con 0x7f6ae0066f10 2024-04-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6adffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae00fff90 0x7f6ae010de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6adffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae00fff90 0x7f6ae010de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38310/0 (socket says 172.21.15.94:38310) 2024-04-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6adffff700 1 -- 172.21.15.94:0/476393029 learned_addr learned my addr 172.21.15.94:0/476393029 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.613+0000 7f6ae4f6e700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ae0066f10 0x7f6ae010d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.614+0000 7f6adffff700 1 -- 172.21.15.94:0/476393029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ae0066f10 msgr2=0x7f6ae010d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.614+0000 7f6adffff700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ae0066f10 0x7f6ae010d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.614+0000 7f6adffff700 1 -- 172.21.15.94:0/476393029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4005040 con 0x7f6ae00fff90 2024-04-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.614+0000 7f6adffff700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae00fff90 0x7f6ae010de10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6acc002a80 tx=0x7f6acc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:34.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.614+0000 7f6add7fa700 1 -- 172.21.15.94:0/476393029 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6acc007cc0 con 0x7f6ae00fff90 2024-04-06T03:48:34.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.614+0000 7f6add7fa700 1 -- 172.21.15.94:0/476393029 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6acc00f040 con 0x7f6ae00fff90 2024-04-06T03:48:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.614+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae0112fe0 con 0x7f6ae00fff90 2024-04-06T03:48:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.614+0000 7f6add7fa700 1 -- 172.21.15.94:0/476393029 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6acc00dac0 con 0x7f6ae00fff90 2024-04-06T03:48:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.614+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae0113570 con 0x7f6ae00fff90 2024-04-06T03:48:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.615+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae004c710 con 0x7f6ae00fff90 2024-04-06T03:48:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.615+0000 7f6add7fa700 1 -- 172.21.15.94:0/476393029 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f6acc00dc20 con 0x7f6ae00fff90 2024-04-06T03:48:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.616+0000 7f6add7fa700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6ad006dca0 0x7f6ad0070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.616+0000 7f6add7fa700 1 -- 172.21.15.94:0/476393029 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6acc08d310 con 0x7f6ae00fff90 2024-04-06T03:48:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.616+0000 7f6ae4f6e700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6ad006dca0 0x7f6ad0070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.616+0000 7f6ae4f6e700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6ad006dca0 0x7f6ad0070150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400b970 tx=0x7f6ad4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:34.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.621+0000 7f6add7fa700 1 -- 172.21.15.94:0/476393029 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6acc05a370 con 0x7f6ae00fff90 2024-04-06T03:48:34.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.834+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ae00f8f70 con 0x7f6ad006dca0 2024-04-06T03:48:34.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.835+0000 7f6add7fa700 1 -- 172.21.15.94:0/476393029 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6ae00f8f70 con 0x7f6ad006dca0 2024-04-06T03:48:34.837 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "osd", 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "14/21 daemons upgraded", 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mds daemons", 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:48:34.838 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:48:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.838+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6ad006dca0 msgr2=0x7f6ad0070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.838+0000 7f6ae71d2700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6ad006dca0 0x7f6ad0070150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400b970 tx=0x7f6ad4009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.838+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae00fff90 msgr2=0x7f6ae010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.838+0000 7f6ae71d2700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae00fff90 0x7f6ae010de10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6acc002a80 tx=0x7f6acc0046b0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.839+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 shutdown_connections 2024-04-06T03:48:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.839+0000 7f6ae71d2700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6ad006dca0 0x7f6ad0070150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.839+0000 7f6ae71d2700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ae0066f10 0x7f6ae010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.839+0000 7f6ae71d2700 1 --2- 172.21.15.94:0/476393029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ae00fff90 0x7f6ae010de10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.839+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 >> 172.21.15.94:0/476393029 conn(0x7f6ae00f7480 msgr2=0x7f6ae00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.839+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 shutdown_connections 2024-04-06T03:48:34.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.839+0000 7f6ae71d2700 1 -- 172.21.15.94:0/476393029 wait complete. 2024-04-06T03:48:34.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.940+0000 7f35e603d700 1 -- 172.21.15.94:0/658083764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fcf80 msgr2=0x7f35e00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.940+0000 7f35e603d700 1 --2- 172.21.15.94:0/658083764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fcf80 0x7f35e00fd3f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f35c8004940 tx=0x7f35c8009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:34.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.940+0000 7f35e603d700 1 -- 172.21.15.94:0/658083764 shutdown_connections 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.940+0000 7f35e603d700 1 --2- 172.21.15.94:0/658083764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fcf80 0x7f35e00fd3f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.940+0000 7f35e603d700 1 --2- 172.21.15.94:0/658083764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35e00fbcd0 0x7f35e00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.940+0000 7f35e603d700 1 -- 172.21.15.94:0/658083764 >> 172.21.15.94:0/658083764 conn(0x7f35e00f7430 msgr2=0x7f35e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.940+0000 7f35e603d700 1 -- 172.21.15.94:0/658083764 shutdown_connections 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.940+0000 7f35e603d700 1 -- 172.21.15.94:0/658083764 wait complete. 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.940+0000 7f35e603d700 1 Processor -- start 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35e603d700 1 -- start start 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35e603d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fbcd0 0x7f35e010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35e603d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35e00fcf80 0x7f35e010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35e603d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e010e550 con 0x7f35e00fbcd0 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35e603d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e010eda0 con 0x7f35e00fcf80 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35df7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fbcd0 0x7f35e010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35df7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fbcd0 0x7f35e010da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38338/0 (socket says 172.21.15.94:38338) 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35df7fe700 1 -- 172.21.15.94:0/4284032913 learned_addr learned my addr 172.21.15.94:0/4284032913 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:48:34.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35d6ffd700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35e00fcf80 0x7f35e010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:34.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35df7fe700 1 -- 172.21.15.94:0/4284032913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35e00fcf80 msgr2=0x7f35e010df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:34.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35df7fe700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35e00fcf80 0x7f35e010df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:34.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35df7fe700 1 -- 172.21.15.94:0/4284032913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8005040 con 0x7f35e00fbcd0 2024-04-06T03:48:34.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.941+0000 7f35df7fe700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fbcd0 0x7f35e010da10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35d0002a50 tx=0x7f35d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:34.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.942+0000 7f35dcff9700 1 -- 172.21.15.94:0/4284032913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35d0007cc0 con 0x7f35e00fbcd0 2024-04-06T03:48:34.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.942+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e010f080 con 0x7f35e00fbcd0 2024-04-06T03:48:34.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.942+0000 7f35dcff9700 1 -- 172.21.15.94:0/4284032913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35d000f040 con 0x7f35e00fbcd0 2024-04-06T03:48:34.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.942+0000 7f35dcff9700 1 -- 172.21.15.94:0/4284032913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35d000da30 con 0x7f35e00fbcd0 2024-04-06T03:48:34.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.942+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e010f5c0 con 0x7f35e00fbcd0 2024-04-06T03:48:34.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.943+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e0101b10 con 0x7f35e00fbcd0 2024-04-06T03:48:34.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.943+0000 7f35dcff9700 1 -- 172.21.15.94:0/4284032913 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f35d000db90 con 0x7f35e00fbcd0 2024-04-06T03:48:34.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.943+0000 7f35dcff9700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f35cc06dca0 0x7f35cc070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:48:34.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.944+0000 7f35dcff9700 1 -- 172.21.15.94:0/4284032913 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f35d008d360 con 0x7f35e00fbcd0 2024-04-06T03:48:34.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.944+0000 7f35d6ffd700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f35cc06dca0 0x7f35cc070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:48:34.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.944+0000 7f35d6ffd700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f35cc06dca0 0x7f35cc070150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35c800b970 tx=0x7f35c8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:48:34.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:34.948+0000 7f35dcff9700 1 -- 172.21.15.94:0/4284032913 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f35d005a3c0 con 0x7f35e00fbcd0 2024-04-06T03:48:35.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.231+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f35e005f5f0 con 0x7f35e00fbcd0 2024-04-06T03:48:35.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.232+0000 7f35dcff9700 1 -- 172.21.15.94:0/4284032913 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+251 (secure 0 0 0) 0x7f35d0059f50 con 0x7f35e00fbcd0 2024-04-06T03:48:35.234 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s); 1 filesystem is degraded 2024-04-06T03:48:35.234 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:48:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in error state 2024-04-06T03:48:35.234 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-06T03:48:35.234 INFO:teuthology.orchestra.run.smithi094.stdout: fs cephfs is degraded 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f35cc06dca0 msgr2=0x7f35cc070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f35cc06dca0 0x7f35cc070150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35c800b970 tx=0x7f35c8009e30 comp rx=0 tx=0).stop 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fbcd0 msgr2=0x7f35e010da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fbcd0 0x7f35e010da10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35d0002a50 tx=0x7f35d00046b0 comp rx=0 tx=0).stop 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 shutdown_connections 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e00fbcd0 0x7f35e010da10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f35cc06dca0 0x7f35cc070150 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 --2- 172.21.15.94:0/4284032913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35e00fcf80 0x7f35e010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 >> 172.21.15.94:0/4284032913 conn(0x7f35e00f7430 msgr2=0x7f35e0065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:48:35.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 shutdown_connections 2024-04-06T03:48:35.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:48:35.234+0000 7f35e603d700 1 -- 172.21.15.94:0/4284032913 wait complete. 2024-04-06T03:48:35.406 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:35 smithi136 ceph-mon[82061]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:35.406 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:35 smithi136 ceph-mon[82061]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:35.406 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:35 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1069653263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:35.407 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:35 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2839521808' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:48:35.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:35 smithi094 ceph-mon[94053]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:35.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:35 smithi094 ceph-mon[94053]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:35.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:35 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1069653263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:35.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:35 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2839521808' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:48:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:36 smithi094 ceph-mon[94053]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:36 smithi094 ceph-mon[94053]: pgmap v215: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-06T03:48:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:36 smithi094 ceph-mon[94053]: reconnect by client.14524 v1:172.21.15.136:0/1309930737 after 0 2024-04-06T03:48:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:36 smithi094 ceph-mon[94053]: reconnect by client.24295 v1:172.21.15.94:0/2208962971 after 0.000999978 2024-04-06T03:48:36.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:36 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:reconnect 2024-04-06T03:48:36.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:36 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi136.trdonf=up:reconnect} 2 up:standby 2024-04-06T03:48:36.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:36 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/4284032913' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:48:36.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:36.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:48:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:36 smithi136 ceph-mon[82061]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:48:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:36 smithi136 ceph-mon[82061]: pgmap v215: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-06T03:48:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:36 smithi136 ceph-mon[82061]: reconnect by client.14524 v1:172.21.15.136:0/1309930737 after 0 2024-04-06T03:48:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:36 smithi136 ceph-mon[82061]: reconnect by client.24295 v1:172.21.15.94:0/2208962971 after 0.000999978 2024-04-06T03:48:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:36 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:reconnect 2024-04-06T03:48:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:36 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi136.trdonf=up:reconnect} 2 up:standby 2024-04-06T03:48:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:36 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/4284032913' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:48:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:36.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:48:37.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:37 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:rejoin 2024-04-06T03:48:37.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:37 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi136.trdonf=up:rejoin} 2 up:standby 2024-04-06T03:48:37.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:37 smithi094 ceph-mon[94053]: daemon mds.cephfs.smithi136.trdonf is now active in filesystem cephfs as rank 0 2024-04-06T03:48:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:37 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:rejoin 2024-04-06T03:48:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:37 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi136.trdonf=up:rejoin} 2 up:standby 2024-04-06T03:48:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:37 smithi136 ceph-mon[82061]: daemon mds.cephfs.smithi136.trdonf is now active in filesystem cephfs as rank 0 2024-04-06T03:48:38.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:38 smithi094 ceph-mon[94053]: pgmap v216: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-06T03:48:38.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:38 smithi094 ceph-mon[94053]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-06T03:48:38.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:38 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:active 2024-04-06T03:48:38.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:38 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi136.trdonf=up:active} 2 up:standby 2024-04-06T03:48:38.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:38 smithi136 ceph-mon[82061]: pgmap v216: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-06T03:48:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:38 smithi136 ceph-mon[82061]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-06T03:48:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:38 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.136:6826/249801036,v1:172.21.15.136:6827/249801036] up:active 2024-04-06T03:48:38.599 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:38 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi136.trdonf=up:active} 2 up:standby 2024-04-06T03:48:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:40 smithi136 ceph-mon[82061]: pgmap v217: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-06T03:48:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:40 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.136:6824/1500356136,v1:172.21.15.136:6825/1500356136] up:boot 2024-04-06T03:48:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:40 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi136.trdonf=up:active} 3 up:standby 2024-04-06T03:48:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:48:40.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:40.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:40.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:40.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:40 smithi094 ceph-mon[94053]: pgmap v217: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-06T03:48:40.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:40 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.136:6824/1500356136,v1:172.21.15.136:6825/1500356136] up:boot 2024-04-06T03:48:40.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:40 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi136.trdonf=up:active} 3 up:standby 2024-04-06T03:48:40.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:48:40.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:40.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:40.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:42.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:42 smithi094 ceph-mon[94053]: pgmap v218: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 7 op/s 2024-04-06T03:48:42.462 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:42 smithi136 ceph-mon[82061]: pgmap v218: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 7 op/s 2024-04-06T03:48:44.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:43 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:44.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:43 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:44.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:43 smithi094 ceph-mon[94053]: pgmap v219: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-06T03:48:44.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:43 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:44.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:43 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:44.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:43 smithi136 ceph-mon[82061]: pgmap v219: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-06T03:48:45.296 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:45.297 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:45.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:45.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:46.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:46 smithi136 ceph-mon[82061]: pgmap v220: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-06T03:48:46.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:46 smithi094 ceph-mon[94053]: pgmap v220: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-06T03:48:48.386 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: pgmap v221: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-06T03:48:48.386 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:48.386 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:48.386 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:48.386 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:48:48.387 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:48.387 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:48.387 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:48.387 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:48.387 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:48.387 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:48.387 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:48 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi136.trdonf"]}]: dispatch 2024-04-06T03:48:48.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: pgmap v221: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-06T03:48:48.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:48.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:48.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:48:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:48:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:48:48.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:48 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi136.trdonf"]}]: dispatch 2024-04-06T03:48:49.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:49 smithi094 ceph-mon[94053]: Detected new or changed devices on smithi136 2024-04-06T03:48:49.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:49 smithi094 ceph-mon[94053]: pgmap v222: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 18 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-06T03:48:49.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:49 smithi094 ceph-mon[94053]: pgmap v223: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 22 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-04-06T03:48:49.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:49 smithi094 ceph-mon[94053]: Upgrade: It appears safe to stop mds.cephfs.smithi136.trdonf 2024-04-06T03:48:49.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:49 smithi136 ceph-mon[82061]: Detected new or changed devices on smithi136 2024-04-06T03:48:49.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:49 smithi136 ceph-mon[82061]: pgmap v222: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 18 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-06T03:48:49.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:49 smithi136 ceph-mon[82061]: pgmap v223: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 22 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-04-06T03:48:49.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:49 smithi136 ceph-mon[82061]: Upgrade: It appears safe to stop mds.cephfs.smithi136.trdonf 2024-04-06T03:48:50.529 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:50 smithi136 ceph-mon[82061]: Upgrade: Updating mds.cephfs.smithi136.trdonf 2024-04-06T03:48:50.529 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:50.529 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.trdonf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:48:50.529 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:50.529 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:50 smithi136 ceph-mon[82061]: Deploying daemon mds.cephfs.smithi136.trdonf on smithi136 2024-04-06T03:48:50.529 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:50 smithi136 ceph-mon[82061]: pgmap v224: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 362 B/s rd, 5.9 KiB/s wr, 3 op/s 2024-04-06T03:48:50.530 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:50.530 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:48:50.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:50 smithi094 ceph-mon[94053]: Upgrade: Updating mds.cephfs.smithi136.trdonf 2024-04-06T03:48:50.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:50.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi136.trdonf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-06T03:48:50.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:48:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:50 smithi094 ceph-mon[94053]: Deploying daemon mds.cephfs.smithi136.trdonf on smithi136 2024-04-06T03:48:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:50 smithi094 ceph-mon[94053]: pgmap v224: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 362 B/s rd, 5.9 KiB/s wr, 3 op/s 2024-04-06T03:48:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:48:50.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:48:51.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:51 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[94049]: 2024-04-06T03:48:51.384+0000 7f71b5c08700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:48:51.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:51 smithi094 ceph-mon[94053]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-06T03:48:51.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:51 smithi094 ceph-mon[94053]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:48:51.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:51 smithi094 ceph-mon[94053]: osdmap e82: 6 total, 6 up, 6 in 2024-04-06T03:48:51.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:51 smithi094 ceph-mon[94053]: Standby daemon mds.cephfs.smithi094.qogyzw assigned to filesystem cephfs as rank 0 2024-04-06T03:48:51.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:51 smithi094 ceph-mon[94053]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:48:51.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:51 smithi094 ceph-mon[94053]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-06T03:48:51.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:51 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi094.qogyzw=up:replay} 2 up:standby 2024-04-06T03:48:51.848 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:48:51 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:48:51.403+0000 7fa765f86700 -1 bdev(0x563158eaa000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 3 2024-04-06T03:48:51.849 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:48:51 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:48:51.405+0000 7fa762f80700 -1 bdev(0x563158eaa000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 6 2024-04-06T03:48:51.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:51 smithi136 ceph-mon[82061]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-06T03:48:51.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:51 smithi136 ceph-mon[82061]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-06T03:48:51.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:51 smithi136 ceph-mon[82061]: osdmap e82: 6 total, 6 up, 6 in 2024-04-06T03:48:51.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:51 smithi136 ceph-mon[82061]: Standby daemon mds.cephfs.smithi094.qogyzw assigned to filesystem cephfs as rank 0 2024-04-06T03:48:51.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:51 smithi136 ceph-mon[82061]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-06T03:48:51.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:51 smithi136 ceph-mon[82061]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-06T03:48:51.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:51 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi094.qogyzw=up:replay} 2 up:standby 2024-04-06T03:48:52.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:52 smithi136 ceph-mon[82061]: pgmap v226: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s wr, 3 op/s 2024-04-06T03:48:52.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:52 smithi094 ceph-mon[94053]: pgmap v226: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s wr, 3 op/s 2024-04-06T03:48:54.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:54 smithi136 ceph-mon[82061]: pgmap v227: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 1 op/s 2024-04-06T03:48:54.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:54 smithi094 ceph-mon[94053]: pgmap v227: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 1 op/s 2024-04-06T03:48:56.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:56 smithi136 ceph-mon[82061]: pgmap v228: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 4 op/s 2024-04-06T03:48:56.864 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:56 smithi094 ceph-mon[94053]: pgmap v228: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 4 op/s 2024-04-06T03:48:58.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:48:58 smithi094 ceph-mon[94053]: pgmap v229: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-06T03:48:59.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:48:58 smithi136 ceph-mon[82061]: pgmap v229: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-06T03:49:00.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:00 smithi094 ceph-mon[94053]: pgmap v230: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-06T03:49:00.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:00 smithi094 ceph-mon[94053]: reconnect by client.24295 v1:172.21.15.94:0/2208962971 after 0 2024-04-06T03:49:00.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:00 smithi094 ceph-mon[94053]: reconnect by client.14524 v1:172.21.15.136:0/1309930737 after 0.000999983 2024-04-06T03:49:00.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:00 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] up:reconnect 2024-04-06T03:49:00.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:00 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi094.qogyzw=up:reconnect} 2 up:standby 2024-04-06T03:49:01.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:00 smithi136 ceph-mon[82061]: pgmap v230: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-06T03:49:01.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:00 smithi136 ceph-mon[82061]: reconnect by client.24295 v1:172.21.15.94:0/2208962971 after 0 2024-04-06T03:49:01.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:00 smithi136 ceph-mon[82061]: reconnect by client.14524 v1:172.21.15.136:0/1309930737 after 0.000999983 2024-04-06T03:49:01.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:00 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] up:reconnect 2024-04-06T03:49:01.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:00 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi094.qogyzw=up:reconnect} 2 up:standby 2024-04-06T03:49:01.668 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:01 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] up:rejoin 2024-04-06T03:49:01.669 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:01 smithi136 ceph-mon[82061]: fsmap cephfs:1/1 {0=cephfs.smithi094.qogyzw=up:rejoin} 2 up:standby 2024-04-06T03:49:01.669 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:01 smithi136 ceph-mon[82061]: daemon mds.cephfs.smithi094.qogyzw is now active in filesystem cephfs as rank 0 2024-04-06T03:49:01.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:01 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] up:rejoin 2024-04-06T03:49:01.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:01 smithi094 ceph-mon[94053]: fsmap cephfs:1/1 {0=cephfs.smithi094.qogyzw=up:rejoin} 2 up:standby 2024-04-06T03:49:01.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:01 smithi094 ceph-mon[94053]: daemon mds.cephfs.smithi094.qogyzw is now active in filesystem cephfs as rank 0 2024-04-06T03:49:02.817 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:02 smithi136 ceph-mon[82061]: pgmap v231: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-06T03:49:02.817 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:02 smithi136 ceph-mon[82061]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-06T03:49:02.817 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:02 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] up:active 2024-04-06T03:49:02.817 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:02 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 2 up:standby 2024-04-06T03:49:02.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:02 smithi094 ceph-mon[94053]: pgmap v231: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-06T03:49:02.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:02 smithi094 ceph-mon[94053]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-06T03:49:02.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:02 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] up:active 2024-04-06T03:49:02.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:02 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 2 up:standby 2024-04-06T03:49:04.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:04 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:04.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:04 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:04.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:04 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:04.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:04 smithi136 ceph-mon[82061]: mds.? [v2:172.21.15.136:6826/1840231705,v1:172.21.15.136:6827/1840231705] up:boot 2024-04-06T03:49:04.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:04 smithi136 ceph-mon[82061]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:49:04.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:04 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:49:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:04 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:04 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:04 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:04 smithi094 ceph-mon[94053]: mds.? [v2:172.21.15.136:6826/1840231705,v1:172.21.15.136:6827/1840231705] up:boot 2024-04-06T03:49:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:04 smithi094 ceph-mon[94053]: fsmap cephfs:1 {0=cephfs.smithi094.qogyzw=up:active} 3 up:standby 2024-04-06T03:49:04.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:04 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:49:05.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:05 smithi136 ceph-mon[82061]: pgmap v232: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-06T03:49:05.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.385+0000 7f8e2a87b700 1 -- 172.21.15.94:0/1654865781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fcf80 msgr2=0x7f8e240fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:05.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.385+0000 7f8e2a87b700 1 --2- 172.21.15.94:0/1654865781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fcf80 0x7f8e240fd3f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e18004940 tx=0x7f8e18009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:05.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.385+0000 7f8e2a87b700 1 -- 172.21.15.94:0/1654865781 shutdown_connections 2024-04-06T03:49:05.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.385+0000 7f8e2a87b700 1 --2- 172.21.15.94:0/1654865781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fcf80 0x7f8e240fd3f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.385+0000 7f8e2a87b700 1 --2- 172.21.15.94:0/1654865781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e240fbcd0 0x7f8e240fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.385+0000 7f8e2a87b700 1 -- 172.21.15.94:0/1654865781 >> 172.21.15.94:0/1654865781 conn(0x7f8e240f7430 msgr2=0x7f8e240f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:05.388 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:05 smithi094 ceph-mon[94053]: pgmap v232: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-06T03:49:05.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.385+0000 7f8e2a87b700 1 -- 172.21.15.94:0/1654865781 shutdown_connections 2024-04-06T03:49:05.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.385+0000 7f8e2a87b700 1 -- 172.21.15.94:0/1654865781 wait complete. 2024-04-06T03:49:05.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e2a87b700 1 Processor -- start 2024-04-06T03:49:05.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e2a87b700 1 -- start start 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e2a87b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fbcd0 0x7f8e240ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e2a87b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e240fcf80 0x7f8e240fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e2a87b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240fe800 con 0x7f8e240fbcd0 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e2a87b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240fe940 con 0x7f8e240fcf80 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e23fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fbcd0 0x7f8e240ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e23fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fbcd0 0x7f8e240ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58394/0 (socket says 172.21.15.94:58394) 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e23fff700 1 -- 172.21.15.94:0/2332139497 learned_addr learned my addr 172.21.15.94:0/2332139497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.386+0000 7f8e237fe700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e240fcf80 0x7f8e240fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.387+0000 7f8e23fff700 1 -- 172.21.15.94:0/2332139497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e240fcf80 msgr2=0x7f8e240fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.387+0000 7f8e23fff700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e240fcf80 0x7f8e240fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.387+0000 7f8e23fff700 1 -- 172.21.15.94:0/2332139497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e18005040 con 0x7f8e240fbcd0 2024-04-06T03:49:05.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.387+0000 7f8e23fff700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fbcd0 0x7f8e240ffd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8e14002a50 tx=0x7f8e140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:05.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.387+0000 7f8e20ff9700 1 -- 172.21.15.94:0/2332139497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e14007cc0 con 0x7f8e240fbcd0 2024-04-06T03:49:05.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.387+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e240fec20 con 0x7f8e240fbcd0 2024-04-06T03:49:05.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.387+0000 7f8e20ff9700 1 -- 172.21.15.94:0/2332139497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e1400f040 con 0x7f8e240fbcd0 2024-04-06T03:49:05.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.387+0000 7f8e20ff9700 1 -- 172.21.15.94:0/2332139497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e1400da30 con 0x7f8e240fbcd0 2024-04-06T03:49:05.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.387+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e24115ad0 con 0x7f8e240fbcd0 2024-04-06T03:49:05.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.388+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e24101b10 con 0x7f8e240fbcd0 2024-04-06T03:49:05.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.389+0000 7f8e20ff9700 1 -- 172.21.15.94:0/2332139497 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f8e1400db90 con 0x7f8e240fbcd0 2024-04-06T03:49:05.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.389+0000 7f8e20ff9700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8e0c06dca0 0x7f8e0c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:05.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.389+0000 7f8e237fe700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8e0c06dca0 0x7f8e0c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:05.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.389+0000 7f8e20ff9700 1 -- 172.21.15.94:0/2332139497 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f8e1408d3a0 con 0x7f8e240fbcd0 2024-04-06T03:49:05.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.390+0000 7f8e237fe700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8e0c06dca0 0x7f8e0c070150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800b3b0 tx=0x7f8e18009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:05.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.394+0000 7f8e20ff9700 1 -- 172.21.15.94:0/2332139497 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8e14056880 con 0x7f8e240fbcd0 2024-04-06T03:49:05.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.631+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e24104930 con 0x7f8e0c06dca0 2024-04-06T03:49:05.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.632+0000 7f8e20ff9700 1 -- 172.21.15.94:0/2332139497 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8e24104930 con 0x7f8e0c06dca0 2024-04-06T03:49:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8e0c06dca0 msgr2=0x7f8e0c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8e0c06dca0 0x7f8e0c070150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800b3b0 tx=0x7f8e18009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:05.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fbcd0 msgr2=0x7f8e240ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fbcd0 0x7f8e240ffd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8e14002a50 tx=0x7f8e140046b0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 shutdown_connections 2024-04-06T03:49:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e240fbcd0 0x7f8e240ffd50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8e0c06dca0 0x7f8e0c070150 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 --2- 172.21.15.94:0/2332139497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e240fcf80 0x7f8e240fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 >> 172.21.15.94:0/2332139497 conn(0x7f8e240f7430 msgr2=0x7f8e24065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 shutdown_connections 2024-04-06T03:49:05.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.635+0000 7f8e2a87b700 1 -- 172.21.15.94:0/2332139497 wait complete. 2024-04-06T03:49:05.651 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:49:05.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.742+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/13986018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fbcd0 msgr2=0x7f0e180fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:05.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.742+0000 7f0e1e7f2700 1 --2- 172.21.15.94:0/13986018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fbcd0 0x7f0e180fc0e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e08004940 tx=0x7f0e08009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:05.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.742+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/13986018 shutdown_connections 2024-04-06T03:49:05.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.742+0000 7f0e1e7f2700 1 --2- 172.21.15.94:0/13986018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e180fcf80 0x7f0e180fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.742+0000 7f0e1e7f2700 1 --2- 172.21.15.94:0/13986018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fbcd0 0x7f0e180fc0e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.742+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/13986018 >> 172.21.15.94:0/13986018 conn(0x7f0e180f7430 msgr2=0x7f0e180f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:05.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.742+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/13986018 shutdown_connections 2024-04-06T03:49:05.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.742+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/13986018 wait complete. 2024-04-06T03:49:05.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.742+0000 7f0e1e7f2700 1 Processor -- start 2024-04-06T03:49:05.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e1e7f2700 1 -- start start 2024-04-06T03:49:05.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e1e7f2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e180fbcd0 0x7f0e1810dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:05.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e1e7f2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fcf80 0x7f0e1810e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:05.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e1e7f2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1810e620 con 0x7f0e180fcf80 2024-04-06T03:49:05.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e1e7f2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1810edd0 con 0x7f0e180fbcd0 2024-04-06T03:49:05.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e177fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fcf80 0x7f0e1810e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:05.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e177fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fcf80 0x7f0e1810e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58424/0 (socket says 172.21.15.94:58424) 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e177fe700 1 -- 172.21.15.94:0/174079995 learned_addr learned my addr 172.21.15.94:0/174079995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e17fff700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e180fbcd0 0x7f0e1810dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e177fe700 1 -- 172.21.15.94:0/174079995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e180fbcd0 msgr2=0x7f0e1810dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e177fe700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e180fbcd0 0x7f0e1810dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.743+0000 7f0e177fe700 1 -- 172.21.15.94:0/174079995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08005040 con 0x7f0e180fcf80 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.744+0000 7f0e177fe700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fcf80 0x7f0e1810e020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c002a50 tx=0x7f0e0c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.744+0000 7f0e14ff9700 1 -- 172.21.15.94:0/174079995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c007cc0 con 0x7f0e180fcf80 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.744+0000 7f0e14ff9700 1 -- 172.21.15.94:0/174079995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e0c00f040 con 0x7f0e180fcf80 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.744+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1810f0b0 con 0x7f0e180fcf80 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.744+0000 7f0e14ff9700 1 -- 172.21.15.94:0/174079995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c00da30 con 0x7f0e180fcf80 2024-04-06T03:49:05.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.744+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1810f5f0 con 0x7f0e180fcf80 2024-04-06T03:49:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.745+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e18101b10 con 0x7f0e180fcf80 2024-04-06T03:49:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.745+0000 7f0e14ff9700 1 -- 172.21.15.94:0/174079995 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f0e0c00db90 con 0x7f0e180fcf80 2024-04-06T03:49:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.746+0000 7f0e14ff9700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0e0006daa0 0x7f0e0006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.746+0000 7f0e14ff9700 1 -- 172.21.15.94:0/174079995 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f0e0c08d160 con 0x7f0e180fcf80 2024-04-06T03:49:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.746+0000 7f0e17fff700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0e0006daa0 0x7f0e0006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:05.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.746+0000 7f0e17fff700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0e0006daa0 0x7f0e0006ff50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800b970 tx=0x7f0e08009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:05.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.750+0000 7f0e14ff9700 1 -- 172.21.15.94:0/174079995 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0e0c05a190 con 0x7f0e180fcf80 2024-04-06T03:49:05.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.972+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e181048d0 con 0x7f0e0006daa0 2024-04-06T03:49:05.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.973+0000 7f0e14ff9700 1 -- 172.21.15.94:0/174079995 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0e181048d0 con 0x7f0e0006daa0 2024-04-06T03:49:05.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0e0006daa0 msgr2=0x7f0e0006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:05.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0e0006daa0 0x7f0e0006ff50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800b970 tx=0x7f0e08009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:05.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fcf80 msgr2=0x7f0e1810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:05.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fcf80 0x7f0e1810e020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c002a50 tx=0x7f0e0c0046b0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 shutdown_connections 2024-04-06T03:49:05.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f0e0006daa0 0x7f0e0006ff50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e180fbcd0 0x7f0e1810dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 --2- 172.21.15.94:0/174079995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e180fcf80 0x7f0e1810e020 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:05.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 >> 172.21.15.94:0/174079995 conn(0x7f0e180f7430 msgr2=0x7f0e18065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:05.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 shutdown_connections 2024-04-06T03:49:05.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:05.975+0000 7f0e1e7f2700 1 -- 172.21.15.94:0/174079995 wait complete. 2024-04-06T03:49:06.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 -- 172.21.15.94:0/325709388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca80fcf80 msgr2=0x7f4ca80fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 --2- 172.21.15.94:0/325709388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca80fcf80 0x7f4ca80fd3f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c004940 tx=0x7f4c9c009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 -- 172.21.15.94:0/325709388 shutdown_connections 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 --2- 172.21.15.94:0/325709388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca80fcf80 0x7f4ca80fd3f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 --2- 172.21.15.94:0/325709388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ca80fbcd0 0x7f4ca80fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 -- 172.21.15.94:0/325709388 >> 172.21.15.94:0/325709388 conn(0x7f4ca80f7430 msgr2=0x7f4ca80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 -- 172.21.15.94:0/325709388 shutdown_connections 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 -- 172.21.15.94:0/325709388 wait complete. 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 Processor -- start 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 -- start start 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ca80fbcd0 0x7f4ca810c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca80fcf80 0x7f4ca810cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca810d3c0 con 0x7f4ca80fcf80 2024-04-06T03:49:06.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.079+0000 7f4cacd6d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca810dc10 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4ca659c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ca80fbcd0 0x7f4ca810c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4ca5d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca80fcf80 0x7f4ca810cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4ca659c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ca80fbcd0 0x7f4ca810c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.94:37568/0 (socket says 172.21.15.94:37568) 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4ca659c700 1 -- 172.21.15.94:0/827268733 learned_addr learned my addr 172.21.15.94:0/827268733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4ca659c700 1 -- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca80fcf80 msgr2=0x7f4ca810cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4ca659c700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca80fcf80 0x7f4ca810cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4ca659c700 1 -- 172.21.15.94:0/827268733 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c005040 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4ca5d9b700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca80fcf80 0x7f4ca810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4ca659c700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ca80fbcd0 0x7f4ca810c880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800eb40 tx=0x7f4c9800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4c96ffd700 1 -- 172.21.15.94:0/827268733 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c98007cc0 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca810de90 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4c96ffd700 1 -- 172.21.15.94:0/827268733 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c9800f040 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4c96ffd700 1 -- 172.21.15.94:0/827268733 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9800ca10 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.080+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca810e3d0 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.081+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca805f5f0 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.082+0000 7f4c96ffd700 1 -- 172.21.15.94:0/827268733 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f4c9800cc80 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.082+0000 7f4c96ffd700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4c9006dca0 0x7f4c90070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.082+0000 7f4c96ffd700 1 -- 172.21.15.94:0/827268733 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4c9808d260 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.082+0000 7f4ca5d9b700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4c9006dca0 0x7f4c90070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:06.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.083+0000 7f4ca5d9b700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4c9006dca0 0x7f4c90070150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009b30 tx=0x7f4c9c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:06.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.088+0000 7f4c96ffd700 1 -- 172.21.15.94:0/827268733 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4c9805a2c0 con 0x7f4ca80fbcd0 2024-04-06T03:49:06.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.297+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ca810e700 con 0x7f4c9006dca0 2024-04-06T03:49:06.299 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:49:06.299 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:06.299 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:06.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.314+0000 7f4c96ffd700 1 -- 172.21.15.94:0/827268733 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4ca810e700 con 0x7f4c9006dca0 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (8m) 47s ago 16m 26.6M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (5m) 47s ago 16m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (4m) 0s ago 15m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 error 47s ago 16m - - 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (52s) 47s ago 11m 13.6M - 19.0.0-2027-g9f192350 a5511c7deba9 bfca2184eb3c 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (80s) 47s ago 11m 14.9M - 19.0.0-2027-g9f192350 a5511c7deba9 c189927827c8 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (30s) 0s ago 11m 14.8M - 19.0.0-2027-g9f192350 a5511c7deba9 6dd3b4023d80 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (3s) 0s ago 11m 12.7M - 19.0.0-2027-g9f192350 a5511c7deba9 432ef9195992 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (7m) 47s ago 17m 506M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (6m) 0s ago 15m 452M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (6m) 47s ago 17m 50.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (5m) 0s ago 15m 42.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:49:06.317 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (8m) 47s ago 16m 15.4M - 1.5.0 0da6a335fe13 5e0753cdb29c 2024-04-06T03:49:06.317 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (7m) 0s ago 15m 17.0M - 1.5.0 0da6a335fe13 5f0f7b511ee6 2024-04-06T03:49:06.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (4m) 47s ago 14m 111M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:49:06.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (4m) 47s ago 14m 121M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:49:06.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (3m) 47s ago 13m 107M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:49:06.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (2m) 0s ago 13m 123M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:49:06.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (2m) 0s ago 13m 107M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 c31367e1c2d5 2024-04-06T03:49:06.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (101s) 0s ago 12m 109M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 16386e8f8583 2024-04-06T03:49:06.317 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (6m) 47s ago 16m 61.2M - 2.43.0 a07b618ecd1d 37cfd00039d1 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4c9006dca0 msgr2=0x7f4c90070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4c9006dca0 0x7f4c90070150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009b30 tx=0x7f4c9c009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ca80fbcd0 msgr2=0x7f4ca810c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ca80fbcd0 0x7f4ca810c880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800eb40 tx=0x7f4c9800b040 comp rx=0 tx=0).stop 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 shutdown_connections 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4c9006dca0 0x7f4c90070150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ca80fbcd0 0x7f4ca810c880 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 --2- 172.21.15.94:0/827268733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ca80fcf80 0x7f4ca810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 >> 172.21.15.94:0/827268733 conn(0x7f4ca80f7430 msgr2=0x7f4ca80f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 shutdown_connections 2024-04-06T03:49:06.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.316+0000 7f4cacd6d700 1 -- 172.21.15.94:0/827268733 wait complete. 2024-04-06T03:49:06.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:49:06.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:06.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:06.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.417+0000 7f52a3494700 1 -- 172.21.15.94:0/4223640231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 msgr2=0x7f529c0fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.417+0000 7f52a3494700 1 --2- 172.21.15.94:0/4223640231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 0x7f529c0fc100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5298004940 tx=0x7f5298009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.417+0000 7f52a3494700 1 -- 172.21.15.94:0/4223640231 shutdown_connections 2024-04-06T03:49:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.417+0000 7f52a3494700 1 --2- 172.21.15.94:0/4223640231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f529c0fcfa0 0x7f529c0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.417+0000 7f52a3494700 1 --2- 172.21.15.94:0/4223640231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 0x7f529c0fc100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.417+0000 7f52a3494700 1 -- 172.21.15.94:0/4223640231 >> 172.21.15.94:0/4223640231 conn(0x7f529c0f7450 msgr2=0x7f529c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.417+0000 7f52a3494700 1 -- 172.21.15.94:0/4223640231 shutdown_connections 2024-04-06T03:49:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.417+0000 7f52a3494700 1 -- 172.21.15.94:0/4223640231 wait complete. 2024-04-06T03:49:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a3494700 1 Processor -- start 2024-04-06T03:49:06.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a3494700 1 -- start start 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a3494700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 0x7f529c112ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a3494700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f529c0fcfa0 0x7f529c115020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a3494700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c066ad0 con 0x7f529c0fbcf0 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a3494700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c066c10 con 0x7f529c0fcfa0 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a1230700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 0x7f529c112ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a1230700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 0x7f529c112ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58458/0 (socket says 172.21.15.94:58458) 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a1230700 1 -- 172.21.15.94:0/2677009595 learned_addr learned my addr 172.21.15.94:0/2677009595 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a0a2f700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f529c0fcfa0 0x7f529c115020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a1230700 1 -- 172.21.15.94:0/2677009595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f529c0fcfa0 msgr2=0x7f529c115020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a1230700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f529c0fcfa0 0x7f529c115020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a1230700 1 -- 172.21.15.94:0/2677009595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5298005040 con 0x7f529c0fbcf0 2024-04-06T03:49:06.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.418+0000 7f52a1230700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 0x7f529c112ad0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5298013870 tx=0x7f5298009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:06.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.419+0000 7f5291ffb700 1 -- 172.21.15.94:0/2677009595 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5298021070 con 0x7f529c0fbcf0 2024-04-06T03:49:06.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.419+0000 7f5291ffb700 1 -- 172.21.15.94:0/2677009595 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5298009430 con 0x7f529c0fbcf0 2024-04-06T03:49:06.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.419+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c115560 con 0x7f529c0fbcf0 2024-04-06T03:49:06.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.419+0000 7f5291ffb700 1 -- 172.21.15.94:0/2677009595 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f529800e070 con 0x7f529c0fbcf0 2024-04-06T03:49:06.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.419+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c1158b0 con 0x7f529c0fbcf0 2024-04-06T03:49:06.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.420+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c05f5f0 con 0x7f529c0fbcf0 2024-04-06T03:49:06.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.420+0000 7f5291ffb700 1 -- 172.21.15.94:0/2677009595 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f529801c750 con 0x7f529c0fbcf0 2024-04-06T03:49:06.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.421+0000 7f5291ffb700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f528806dc30 0x7f52880700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:06.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.421+0000 7f52a0a2f700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f528806dc30 0x7f52880700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:06.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.421+0000 7f5291ffb700 1 -- 172.21.15.94:0/2677009595 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f529808e7a0 con 0x7f529c0fbcf0 2024-04-06T03:49:06.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.422+0000 7f52a0a2f700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f528806dc30 0x7f52880700e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f528c009b80 tx=0x7f528c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:06.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.425+0000 7f5291ffb700 1 -- 172.21.15.94:0/2677009595 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5298057ca0 con 0x7f529c0fbcf0 2024-04-06T03:49:06.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.725+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f529c115bf0 con 0x7f529c0fbcf0 2024-04-06T03:49:06.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.726+0000 7f5291ffb700 1 -- 172.21.15.94:0/2677009595 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f529805b360 con 0x7f529c0fbcf0 2024-04-06T03:49:06.728 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:49:06.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:49:06.728 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 6 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 4 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 14 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:49:06.729 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f528806dc30 msgr2=0x7f52880700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f528806dc30 0x7f52880700e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f528c009b80 tx=0x7f528c00b040 comp rx=0 tx=0).stop 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 msgr2=0x7f529c112ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 0x7f529c112ad0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5298013870 tx=0x7f5298009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 shutdown_connections 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f529c0fbcf0 0x7f529c112ad0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f528806dc30 0x7f52880700e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 --2- 172.21.15.94:0/2677009595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f529c0fcfa0 0x7f529c115020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 >> 172.21.15.94:0/2677009595 conn(0x7f529c0f7450 msgr2=0x7f529c0f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 shutdown_connections 2024-04-06T03:49:06.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.730+0000 7f52a3494700 1 -- 172.21.15.94:0/2677009595 wait complete. 2024-04-06T03:49:06.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.833+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/3905114974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b58066220 msgr2=0x7f1b580ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.833+0000 7f1b5f7f5700 1 --2- 172.21.15.94:0/3905114974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b58066220 0x7f1b580ffa80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b54004940 tx=0x7f1b54009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.833+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/3905114974 shutdown_connections 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.833+0000 7f1b5f7f5700 1 --2- 172.21.15.94:0/3905114974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b580fffc0 0x7f1b581023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.833+0000 7f1b5f7f5700 1 --2- 172.21.15.94:0/3905114974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b58066220 0x7f1b580ffa80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.833+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/3905114974 >> 172.21.15.94:0/3905114974 conn(0x7f1b580f7430 msgr2=0x7f1b580f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.833+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/3905114974 shutdown_connections 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/3905114974 wait complete. 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5f7f5700 1 Processor -- start 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5f7f5700 1 -- start start 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5f7f5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b58066220 0x7f1b5810da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5f7f5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b580fffc0 0x7f1b5810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5f7f5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5810e5b0 con 0x7f1b580fffc0 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5f7f5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5810edb0 con 0x7f1b58066220 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5cd90700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b580fffc0 0x7f1b5810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:06.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5cd90700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b580fffc0 0x7f1b5810dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58468/0 (socket says 172.21.15.94:58468) 2024-04-06T03:49:06.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5cd90700 1 -- 172.21.15.94:0/1774380402 learned_addr learned my addr 172.21.15.94:0/1774380402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:06.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.834+0000 7f1b5d591700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b58066220 0x7f1b5810da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:06.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.835+0000 7f1b5cd90700 1 -- 172.21.15.94:0/1774380402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b58066220 msgr2=0x7f1b5810da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:06.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.835+0000 7f1b5cd90700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b58066220 0x7f1b5810da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:06.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.835+0000 7f1b5cd90700 1 -- 172.21.15.94:0/1774380402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b54005040 con 0x7f1b580fffc0 2024-04-06T03:49:06.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.835+0000 7f1b5cd90700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b580fffc0 0x7f1b5810dfb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800eb40 tx=0x7f1b4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.835+0000 7f1b4dffb700 1 -- 172.21.15.94:0/1774380402 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b48007cc0 con 0x7f1b580fffc0 2024-04-06T03:49:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.835+0000 7f1b4dffb700 1 -- 172.21.15.94:0/1774380402 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b4800f040 con 0x7f1b580fffc0 2024-04-06T03:49:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.835+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5810f030 con 0x7f1b580fffc0 2024-04-06T03:49:06.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.835+0000 7f1b4dffb700 1 -- 172.21.15.94:0/1774380402 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b4800c9d0 con 0x7f1b580fffc0 2024-04-06T03:49:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.835+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5810f570 con 0x7f1b580fffc0 2024-04-06T03:49:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.836+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5805f5f0 con 0x7f1b580fffc0 2024-04-06T03:49:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.837+0000 7f1b4dffb700 1 -- 172.21.15.94:0/1774380402 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f1b4800cc40 con 0x7f1b580fffc0 2024-04-06T03:49:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.837+0000 7f1b4dffb700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1b4406dca0 0x7f1b44070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.837+0000 7f1b4dffb700 1 -- 172.21.15.94:0/1774380402 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f1b4808d080 con 0x7f1b580fffc0 2024-04-06T03:49:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.837+0000 7f1b5d591700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1b4406dca0 0x7f1b44070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:06.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.838+0000 7f1b5d591700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1b4406dca0 0x7f1b44070150 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1b54004890 tx=0x7f1b54009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:06.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:06.842+0000 7f1b4dffb700 1 -- 172.21.15.94:0/1774380402 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1b4805a140 con 0x7f1b580fffc0 2024-04-06T03:49:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.108+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1b5804bed0 con 0x7f1b580fffc0 2024-04-06T03:49:07.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.109+0000 7f1b4dffb700 1 -- 172.21.15.94:0/1774380402 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1940 (secure 0 0 0) 0x7f1b48059cd0 con 0x7f1b580fffc0 2024-04-06T03:49:07.110 INFO:teuthology.orchestra.run.smithi094.stdout:e36 2024-04-06T03:49:07.110 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:49:07.110 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:49:07.110 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:49:07.110 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 35 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:49:01.609121+0000 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:49:07.111 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 82 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=34400} 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 34400 members: 34400 2024-04-06T03:49:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:34400} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:49:07.113 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:49:07.113 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:49:07.113 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:49:07.113 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:49:07.113 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/1650064917,v1:172.21.15.94:6829/1650064917] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:49:07.113 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.136:6824/1500356136,v1:172.21.15.136:6825/1500356136] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:49:07.113 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.136:6826/1840231705,v1:172.21.15.136:6827/1840231705] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1b4406dca0 msgr2=0x7f1b44070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1b4406dca0 0x7f1b44070150 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1b54004890 tx=0x7f1b54009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b580fffc0 msgr2=0x7f1b5810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b580fffc0 0x7f1b5810dfb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800eb40 tx=0x7f1b4800b040 comp rx=0 tx=0).stop 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 shutdown_connections 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1b4406dca0 0x7f1b44070150 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b58066220 0x7f1b5810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 --2- 172.21.15.94:0/1774380402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b580fffc0 0x7f1b5810dfb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 >> 172.21.15.94:0/1774380402 conn(0x7f1b580f7430 msgr2=0x7f1b580f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 shutdown_connections 2024-04-06T03:49:07.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.112+0000 7f1b5f7f5700 1 -- 172.21.15.94:0/1774380402 wait complete. 2024-04-06T03:49:07.115 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 36 2024-04-06T03:49:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.216+0000 7f27aa28e700 1 -- 172.21.15.94:0/2612216632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 msgr2=0x7f27a4100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.216+0000 7f27aa28e700 1 --2- 172.21.15.94:0/2612216632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 0x7f27a4100710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2794004940 tx=0x7f2794009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.216+0000 7f27aa28e700 1 -- 172.21.15.94:0/2612216632 shutdown_connections 2024-04-06T03:49:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.216+0000 7f27aa28e700 1 --2- 172.21.15.94:0/2612216632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 0x7f27a4100710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.216+0000 7f27aa28e700 1 --2- 172.21.15.94:0/2612216632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27a40feff0 0x7f27a40ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.216+0000 7f27aa28e700 1 -- 172.21.15.94:0/2612216632 >> 172.21.15.94:0/2612216632 conn(0x7f27a40fa950 msgr2=0x7f27a40fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.216+0000 7f27aa28e700 1 -- 172.21.15.94:0/2612216632 shutdown_connections 2024-04-06T03:49:07.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.216+0000 7f27aa28e700 1 -- 172.21.15.94:0/2612216632 wait complete. 2024-04-06T03:49:07.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.216+0000 7f27aa28e700 1 Processor -- start 2024-04-06T03:49:07.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27aa28e700 1 -- start start 2024-04-06T03:49:07.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27aa28e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27a40feff0 0x7f27a410fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:07.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27aa28e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 0x7f27a4110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:07.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27aa28e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a4110740 con 0x7f27a41002a0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27aa28e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a4110f40 con 0x7f27a40feff0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 0x7f27a4110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 0x7f27a4110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58490/0 (socket says 172.21.15.94:58490) 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27a3fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27a40feff0 0x7f27a410fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27a37fe700 1 -- 172.21.15.94:0/3385180036 learned_addr learned my addr 172.21.15.94:0/3385180036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27a37fe700 1 -- 172.21.15.94:0/3385180036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27a40feff0 msgr2=0x7f27a410fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27a37fe700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27a40feff0 0x7f27a410fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27a37fe700 1 -- 172.21.15.94:0/3385180036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794005040 con 0x7f27a41002a0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27a3fff700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27a40feff0 0x7f27a410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.217+0000 7f27a37fe700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 0x7f27a4110140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2794000c00 tx=0x7f2794009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.218+0000 7f27a0ff9700 1 -- 172.21.15.94:0/3385180036 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f279400e070 con 0x7f27a41002a0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.218+0000 7f27a0ff9700 1 -- 172.21.15.94:0/3385180036 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2794009430 con 0x7f27a41002a0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.218+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a41111c0 con 0x7f27a41002a0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.218+0000 7f27a0ff9700 1 -- 172.21.15.94:0/3385180036 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f279400b410 con 0x7f27a41002a0 2024-04-06T03:49:07.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.218+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a41116a0 con 0x7f27a41002a0 2024-04-06T03:49:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.218+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a405f5f0 con 0x7f27a41002a0 2024-04-06T03:49:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.219+0000 7f27a0ff9700 1 -- 172.21.15.94:0/3385180036 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f279401c7c0 con 0x7f27a41002a0 2024-04-06T03:49:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.220+0000 7f27a0ff9700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f278c06dca0 0x7f278c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.220+0000 7f27a3fff700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f278c06dca0 0x7f278c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.220+0000 7f27a0ff9700 1 -- 172.21.15.94:0/3385180036 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f279408ecc0 con 0x7f27a41002a0 2024-04-06T03:49:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.221+0000 7f27a3fff700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f278c06dca0 0x7f278c070150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2798009b80 tx=0x7f279800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:07.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.224+0000 7f27a0ff9700 1 -- 172.21.15.94:0/3385180036 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f279405bd50 con 0x7f27a41002a0 2024-04-06T03:49:07.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:07 smithi136 ceph-mon[82061]: pgmap v233: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-06T03:49:07.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:07 smithi136 ceph-mon[82061]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:07.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:07 smithi136 ceph-mon[82061]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:07.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:07 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:07.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:07 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:07.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:07 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2677009595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:07.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:07 smithi094 ceph-mon[94053]: pgmap v233: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-06T03:49:07.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:07 smithi094 ceph-mon[94053]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:07.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:07 smithi094 ceph-mon[94053]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:07.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:07 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:07.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:07 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:07.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:07 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2677009595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:07.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.446+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27a41118c0 con 0x7f278c06dca0 2024-04-06T03:49:07.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.448+0000 7f27a0ff9700 1 -- 172.21.15.94:0/3385180036 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f27a41118c0 con 0x7f278c06dca0 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "osd", 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "mds", 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "16/21 daemons upgraded", 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading mds daemons", 2024-04-06T03:49:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:49:07.450 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:49:07.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.450+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f278c06dca0 msgr2=0x7f278c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.450+0000 7f27aa28e700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f278c06dca0 0x7f278c070150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2798009b80 tx=0x7f279800b040 comp rx=0 tx=0).stop 2024-04-06T03:49:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.450+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 msgr2=0x7f27a4110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.450+0000 7f27aa28e700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 0x7f27a4110140 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2794000c00 tx=0x7f2794009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.450+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 shutdown_connections 2024-04-06T03:49:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.450+0000 7f27aa28e700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f278c06dca0 0x7f278c070150 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.451+0000 7f27aa28e700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27a40feff0 0x7f27a410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.451+0000 7f27aa28e700 1 --2- 172.21.15.94:0/3385180036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27a41002a0 0x7f27a4110140 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.451+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 >> 172.21.15.94:0/3385180036 conn(0x7f27a40fa950 msgr2=0x7f27a40fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.451+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 shutdown_connections 2024-04-06T03:49:07.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.451+0000 7f27aa28e700 1 -- 172.21.15.94:0/3385180036 wait complete. 2024-04-06T03:49:07.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.553+0000 7fcecd780700 1 -- 172.21.15.94:0/1996973189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec8066b30 msgr2=0x7fcec80ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.553+0000 7fcecd780700 1 --2- 172.21.15.94:0/1996973189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec8066b30 0x7fcec80ffa10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fceb8004940 tx=0x7fceb8009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:07.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.553+0000 7fcecd780700 1 -- 172.21.15.94:0/1996973189 shutdown_connections 2024-04-06T03:49:07.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.553+0000 7fcecd780700 1 --2- 172.21.15.94:0/1996973189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcec80fff50 0x7fcec8102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.553+0000 7fcecd780700 1 --2- 172.21.15.94:0/1996973189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec8066b30 0x7fcec80ffa10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.553+0000 7fcecd780700 1 -- 172.21.15.94:0/1996973189 >> 172.21.15.94:0/1996973189 conn(0x7fcec80f7440 msgr2=0x7fcec80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:07.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.553+0000 7fcecd780700 1 -- 172.21.15.94:0/1996973189 shutdown_connections 2024-04-06T03:49:07.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.553+0000 7fcecd780700 1 -- 172.21.15.94:0/1996973189 wait complete. 2024-04-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcecd780700 1 Processor -- start 2024-04-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcecd780700 1 -- start start 2024-04-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcecd780700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcec8066b30 0x7fcec80fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcecd780700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec80fff50 0x7fcec80fb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcecd780700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80fbea0 con 0x7fcec80fff50 2024-04-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcecd780700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80fbfe0 con 0x7fcec8066b30 2024-04-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcec67fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec80fff50 0x7fcec80fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcec67fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec80fff50 0x7fcec80fb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51106/0 (socket says 172.21.15.94:51106) 2024-04-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcec67fc700 1 -- 172.21.15.94:0/2519071838 learned_addr learned my addr 172.21.15.94:0/2519071838 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcec6ffd700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcec8066b30 0x7fcec80fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcec6ffd700 1 -- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec80fff50 msgr2=0x7fcec80fb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcec6ffd700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec80fff50 0x7fcec80fb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.554+0000 7fcec6ffd700 1 -- 172.21.15.94:0/2519071838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb8005040 con 0x7fcec8066b30 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.555+0000 7fcec67fc700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec80fff50 0x7fcec80fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.555+0000 7fcec6ffd700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcec8066b30 0x7fcec80fd3f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fceb8000c00 tx=0x7fceb8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.555+0000 7fceaf7fe700 1 -- 172.21.15.94:0/2519071838 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceb800e030 con 0x7fcec8066b30 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.555+0000 7fceaf7fe700 1 -- 172.21.15.94:0/2519071838 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fceb8013de0 con 0x7fcec8066b30 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.555+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec80fc260 con 0x7fcec8066b30 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.555+0000 7fceaf7fe700 1 -- 172.21.15.94:0/2519071838 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceb801cdc0 con 0x7fcec8066b30 2024-04-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.555+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec8115aa0 con 0x7fcec8066b30 2024-04-06T03:49:07.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.556+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec805f5f0 con 0x7fcec8066b30 2024-04-06T03:49:07.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.557+0000 7fceaf7fe700 1 -- 172.21.15.94:0/2519071838 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fceb801c650 con 0x7fcec8066b30 2024-04-06T03:49:07.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.557+0000 7fceaf7fe700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fceb006dca0 0x7fceb0070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:07.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.558+0000 7fceaf7fe700 1 -- 172.21.15.94:0/2519071838 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fceb808ed60 con 0x7fcec8066b30 2024-04-06T03:49:07.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.558+0000 7fcec67fc700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fceb006dca0 0x7fceb0070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:07.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.558+0000 7fcec67fc700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fceb006dca0 0x7fceb0070150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcec80fd9d0 tx=0x7fcebc00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:07.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.562+0000 7fceaf7fe700 1 -- 172.21.15.94:0/2519071838 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fceb80582c0 con 0x7fcec8066b30 2024-04-06T03:49:07.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.849+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcec804bed0 con 0x7fcec8066b30 2024-04-06T03:49:07.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.849+0000 7fceaf7fe700 1 -- 172.21.15.94:0/2519071838 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+155 (secure 0 0 0) 0x7fceb805b980 con 0x7fcec8066b30 2024-04-06T03:49:07.850 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:49:07.851 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:49:07.851 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in error state 2024-04-06T03:49:07.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fceb006dca0 msgr2=0x7fceb0070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fceb006dca0 0x7fceb0070150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcec80fd9d0 tx=0x7fcebc00b040 comp rx=0 tx=0).stop 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcec8066b30 msgr2=0x7fcec80fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcec8066b30 0x7fcec80fd3f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fceb8000c00 tx=0x7fceb8009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 shutdown_connections 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fceb006dca0 0x7fceb0070150 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcec8066b30 0x7fcec80fd3f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 --2- 172.21.15.94:0/2519071838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcec80fff50 0x7fcec80fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 >> 172.21.15.94:0/2519071838 conn(0x7fcec80f7440 msgr2=0x7fcec80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 shutdown_connections 2024-04-06T03:49:07.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:07.851+0000 7fcecd780700 1 -- 172.21.15.94:0/2519071838 wait complete. 2024-04-06T03:49:08.100 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:08 smithi094 ceph-mon[94053]: from='client.44347 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:08.100 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:08 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1774380402' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:49:08.100 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:08 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2519071838' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:49:08.373 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:08 smithi136 ceph-mon[82061]: from='client.44347 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:08.373 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:08 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1774380402' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:49:08.373 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:08 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2519071838' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:49:09.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:09 smithi094 ceph-mon[94053]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:09.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:09 smithi094 ceph-mon[94053]: pgmap v234: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 8.9 MiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-04-06T03:49:09.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:09 smithi136 ceph-mon[82061]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:09.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:09 smithi136 ceph-mon[82061]: pgmap v234: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 8.9 MiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: pgmap v235: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 3.9 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: pgmap v236: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: pgmap v237: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 506 KiB/s rd, 6.3 KiB/s wr, 7 op/s 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:10.999 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all mds 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.kvxcqo"}]': finished 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.qogyzw"}]': finished 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:49:11.000 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi136.hbjgtw"}]': finished 2024-04-06T03:49:11.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: pgmap v235: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 3.9 MiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-04-06T03:49:11.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: pgmap v236: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: pgmap v237: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 506 KiB/s rd, 6.3 KiB/s wr, 7 op/s 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all mds 2024-04-06T03:49:11.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.kvxcqo"}]: dispatch 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.kvxcqo"}]': finished 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.qogyzw"}]: dispatch 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi094.qogyzw"}]': finished 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi136.hbjgtw"}]: dispatch 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi136.hbjgtw"}]': finished 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi136.trdonf"}]': finished 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all rgw 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all rbd-mirror 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all ceph-exporter 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all iscsi 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all nfs 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: Upgrade: Setting container_image for all nvmeof 2024-04-06T03:49:11.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:10 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi136.trdonf"}]: dispatch 2024-04-06T03:49:11.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi136.trdonf"}]': finished 2024-04-06T03:49:11.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all rgw 2024-04-06T03:49:11.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all rbd-mirror 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all ceph-exporter 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all iscsi 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all nfs 2024-04-06T03:49:11.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:11.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:11.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: Upgrade: Setting container_image for all nvmeof 2024-04-06T03:49:11.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:10 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:13.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:13 smithi136 ceph-mon[82061]: Upgrade: Updating node-exporter.smithi094 (1/2) 2024-04-06T03:49:13.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:13 smithi136 ceph-mon[82061]: Deploying daemon node-exporter.smithi094 on smithi094 2024-04-06T03:49:13.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:13 smithi136 ceph-mon[82061]: pgmap v238: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 22 KiB/s rd, 6.3 KiB/s wr, 6 op/s 2024-04-06T03:49:13.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:13 smithi094 ceph-mon[94053]: Upgrade: Updating node-exporter.smithi094 (1/2) 2024-04-06T03:49:13.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:13 smithi094 ceph-mon[94053]: Deploying daemon node-exporter.smithi094 on smithi094 2024-04-06T03:49:13.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:13 smithi094 ceph-mon[94053]: pgmap v238: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 22 KiB/s rd, 6.3 KiB/s wr, 6 op/s 2024-04-06T03:49:15.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:15 smithi136 ceph-mon[82061]: pgmap v239: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-06T03:49:15.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:15 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:15.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:15 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:15.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:15 smithi094 ceph-mon[94053]: pgmap v239: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-06T03:49:15.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:15.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:15 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:17.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:17 smithi094 ceph-mon[94053]: pgmap v240: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:17.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:17 smithi094 ceph-mon[94053]: Upgrade: Updating node-exporter.smithi136 (2/2) 2024-04-06T03:49:17.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:17 smithi094 ceph-mon[94053]: Deploying daemon node-exporter.smithi136 on smithi136 2024-04-06T03:49:17.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:17 smithi136 ceph-mon[82061]: pgmap v240: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:17.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:17 smithi136 ceph-mon[82061]: Upgrade: Updating node-exporter.smithi136 (2/2) 2024-04-06T03:49:17.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:17 smithi136 ceph-mon[82061]: Deploying daemon node-exporter.smithi136 on smithi136 2024-04-06T03:49:19.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:19 smithi136 ceph-mon[82061]: pgmap v241: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:19.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:19 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:19.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:19 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:19.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:19 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:19.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:19 smithi094 ceph-mon[94053]: pgmap v241: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:19.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:19 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:19.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:19 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:19.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:19 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:21.318 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:21 smithi094 ceph-mon[94053]: pgmap v242: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:21.318 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:49:21.318 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:21.318 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:21 smithi136 ceph-mon[82061]: pgmap v242: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:49:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:23.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:23 smithi136 ceph-mon[82061]: pgmap v243: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:23.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:23 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:23.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:23 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:23.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:23 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:23.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:23 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:23.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:23 smithi094 ceph-mon[94053]: pgmap v243: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:23.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:23 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:23.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:23 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:23.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:23 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:23.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:23 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:25.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:25.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:24 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:25.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:24 smithi136 ceph-mon[82061]: pgmap v244: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:25.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:25.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:24 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:25.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:24 smithi094 ceph-mon[94053]: pgmap v244: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:26.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:26 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:26.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:26 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:26.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:26 smithi136 ceph-mon[82061]: pgmap v245: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:26.865 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:26 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:26.865 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:26 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:26.865 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:26 smithi094 ceph-mon[94053]: pgmap v245: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: pgmap v246: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: pgmap v247: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.572 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.573 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.573 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.573 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.573 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.573 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.573 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.573 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.573 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:28 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:28.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:28.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:49:28.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:49:28.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: pgmap v246: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:28.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: pgmap v247: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:28.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:28 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:30.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:30 smithi136 ceph-mon[82061]: Upgrade: Updating prometheus.smithi094 2024-04-06T03:49:30.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:30 smithi136 ceph-mon[82061]: pgmap v248: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:30.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:30 smithi136 ceph-mon[82061]: Deploying daemon prometheus.smithi094 on smithi094 2024-04-06T03:49:30.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:30 smithi094 ceph-mon[94053]: Upgrade: Updating prometheus.smithi094 2024-04-06T03:49:30.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:30 smithi094 ceph-mon[94053]: pgmap v248: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:30.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:30 smithi094 ceph-mon[94053]: Deploying daemon prometheus.smithi094 on smithi094 2024-04-06T03:49:32.777 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:32 smithi094 ceph-mon[94053]: pgmap v249: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:33.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:32 smithi136 ceph-mon[82061]: pgmap v249: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:34.686 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:34 smithi094 ceph-mon[94053]: pgmap v250: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:35.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:34 smithi136 ceph-mon[82061]: pgmap v250: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:36.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:36.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:36.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:36.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:36 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:49:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:36.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:36 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:49:37.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:37 smithi136 ceph-mon[82061]: pgmap v251: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:37.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:37 smithi094 ceph-mon[94053]: pgmap v251: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 -- 172.21.15.94:0/487984507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 msgr2=0x7fd5981023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 --2- 172.21.15.94:0/487984507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 0x7fd5981023a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd58c004940 tx=0x7fd58c009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 -- 172.21.15.94:0/487984507 shutdown_connections 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 --2- 172.21.15.94:0/487984507 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 0x7fd5981023a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 --2- 172.21.15.94:0/487984507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd598066220 0x7fd5980ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 -- 172.21.15.94:0/487984507 >> 172.21.15.94:0/487984507 conn(0x7fd5980f7430 msgr2=0x7fd5980f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 -- 172.21.15.94:0/487984507 shutdown_connections 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 -- 172.21.15.94:0/487984507 wait complete. 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 Processor -- start 2024-04-06T03:49:37.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.960+0000 7fd59cd87700 1 -- start start 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd59cd87700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd598066220 0x7fd59810db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd59cd87700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 0x7fd59810e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd59cd87700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59810e6d0 con 0x7fd5980fffc0 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd59cd87700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59810ee70 con 0x7fd598066220 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd595d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 0x7fd59810e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd595d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 0x7fd59810e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36352/0 (socket says 172.21.15.94:36352) 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd595d9b700 1 -- 172.21.15.94:0/2799974261 learned_addr learned my addr 172.21.15.94:0/2799974261 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd59659c700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd598066220 0x7fd59810db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd595d9b700 1 -- 172.21.15.94:0/2799974261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd598066220 msgr2=0x7fd59810db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd595d9b700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd598066220 0x7fd59810db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd595d9b700 1 -- 172.21.15.94:0/2799974261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c005040 con 0x7fd5980fffc0 2024-04-06T03:49:37.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd595d9b700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 0x7fd59810e040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b3b0 tx=0x7fd58c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:37.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd586ffd700 1 -- 172.21.15.94:0/2799974261 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd58c00e040 con 0x7fd5980fffc0 2024-04-06T03:49:37.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd586ffd700 1 -- 172.21.15.94:0/2799974261 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd58c00bcc0 con 0x7fd5980fffc0 2024-04-06T03:49:37.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.961+0000 7fd586ffd700 1 -- 172.21.15.94:0/2799974261 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd58c01c630 con 0x7fd5980fffc0 2024-04-06T03:49:37.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.962+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59810f0f0 con 0x7fd5980fffc0 2024-04-06T03:49:37.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.962+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59810f630 con 0x7fd5980fffc0 2024-04-06T03:49:37.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.963+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59805f5f0 con 0x7fd5980fffc0 2024-04-06T03:49:37.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.964+0000 7fd586ffd700 1 -- 172.21.15.94:0/2799974261 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fd58c0139f0 con 0x7fd5980fffc0 2024-04-06T03:49:37.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.964+0000 7fd586ffd700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd58006dca0 0x7fd580070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:37.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.964+0000 7fd586ffd700 1 -- 172.21.15.94:0/2799974261 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd58c08e2f0 con 0x7fd5980fffc0 2024-04-06T03:49:37.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.964+0000 7fd59659c700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd58006dca0 0x7fd580070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:37.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.965+0000 7fd59659c700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd58006dca0 0x7fd580070150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd588006fd0 tx=0x7fd588008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:37.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:37.973+0000 7fd586ffd700 1 -- 172.21.15.94:0/2799974261 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd58c05b380 con 0x7fd5980fffc0 2024-04-06T03:49:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.224+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd59810f910 con 0x7fd58006dca0 2024-04-06T03:49:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.225+0000 7fd586ffd700 1 -- 172.21.15.94:0/2799974261 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7fd59810f910 con 0x7fd58006dca0 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd58006dca0 msgr2=0x7fd580070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd58006dca0 0x7fd580070150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd588006fd0 tx=0x7fd588008040 comp rx=0 tx=0).stop 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 msgr2=0x7fd59810e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 0x7fd59810e040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b3b0 tx=0x7fd58c009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 shutdown_connections 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd58006dca0 0x7fd580070150 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd598066220 0x7fd59810db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 --2- 172.21.15.94:0/2799974261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5980fffc0 0x7fd59810e040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 >> 172.21.15.94:0/2799974261 conn(0x7fd5980f7430 msgr2=0x7fd5980f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 shutdown_connections 2024-04-06T03:49:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.228+0000 7fd59cd87700 1 -- 172.21.15.94:0/2799974261 wait complete. 2024-04-06T03:49:38.240 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:49:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.330+0000 7f531d61d700 1 -- 172.21.15.94:0/2879024253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180fff00 msgr2=0x7f5318102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.330+0000 7f531d61d700 1 --2- 172.21.15.94:0/2879024253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180fff00 0x7f5318102320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5308004940 tx=0x7f5308009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.331+0000 7f531d61d700 1 -- 172.21.15.94:0/2879024253 shutdown_connections 2024-04-06T03:49:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.331+0000 7f531d61d700 1 --2- 172.21.15.94:0/2879024253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180fff00 0x7f5318102320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.331+0000 7f531d61d700 1 --2- 172.21.15.94:0/2879024253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53180662e0 0x7f53180ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.331+0000 7f531d61d700 1 -- 172.21.15.94:0/2879024253 >> 172.21.15.94:0/2879024253 conn(0x7f53180f7430 msgr2=0x7f53180f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.331+0000 7f531d61d700 1 -- 172.21.15.94:0/2879024253 shutdown_connections 2024-04-06T03:49:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.331+0000 7f531d61d700 1 -- 172.21.15.94:0/2879024253 wait complete. 2024-04-06T03:49:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f531d61d700 1 Processor -- start 2024-04-06T03:49:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f531d61d700 1 -- start start 2024-04-06T03:49:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f531d61d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180662e0 0x7f53180fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f531d61d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53180fff00 0x7f53180fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f531d61d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53180fbee0 con 0x7f53180662e0 2024-04-06T03:49:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f531d61d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53180fc020 con 0x7f53180fff00 2024-04-06T03:49:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f5316ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180662e0 0x7f53180fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f5316ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180662e0 0x7f53180fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36376/0 (socket says 172.21.15.94:36376) 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f5316ffd700 1 -- 172.21.15.94:0/489875370 learned_addr learned my addr 172.21.15.94:0/489875370 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f53167fc700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53180fff00 0x7f53180fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f5316ffd700 1 -- 172.21.15.94:0/489875370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53180fff00 msgr2=0x7f53180fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f5316ffd700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53180fff00 0x7f53180fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f5316ffd700 1 -- 172.21.15.94:0/489875370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308005040 con 0x7f53180662e0 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.332+0000 7f5316ffd700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180662e0 0x7f53180fd3a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b790 tx=0x7f530c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.333+0000 7f52ff7fe700 1 -- 172.21.15.94:0/489875370 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f530c00f6f0 con 0x7f53180662e0 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.333+0000 7f52ff7fe700 1 -- 172.21.15.94:0/489875370 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f530c00fd20 con 0x7f53180662e0 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.333+0000 7f52ff7fe700 1 -- 172.21.15.94:0/489875370 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f530c0103a0 con 0x7f53180662e0 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.333+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53180fc300 con 0x7f53180662e0 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.333+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5318115af0 con 0x7f53180662e0 2024-04-06T03:49:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.334+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531805f5f0 con 0x7f53180662e0 2024-04-06T03:49:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.334+0000 7f52ff7fe700 1 -- 172.21.15.94:0/489875370 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f530c011400 con 0x7f53180662e0 2024-04-06T03:49:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.335+0000 7f52ff7fe700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f530006dca0 0x7f5300070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.335+0000 7f52ff7fe700 1 -- 172.21.15.94:0/489875370 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f530c08d410 con 0x7f53180662e0 2024-04-06T03:49:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.335+0000 7f53167fc700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f530006dca0 0x7f5300070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.335+0000 7f53167fc700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f530006dca0 0x7f5300070150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f53180fd980 tx=0x7f5308009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:38.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.339+0000 7f52ff7fe700 1 -- 172.21.15.94:0/489875370 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f530c056970 con 0x7f53180662e0 2024-04-06T03:49:38.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.618+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53180f8f70 con 0x7f530006dca0 2024-04-06T03:49:38.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.619+0000 7f52ff7fe700 1 -- 172.21.15.94:0/489875370 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f53180f8f70 con 0x7f530006dca0 2024-04-06T03:49:38.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f530006dca0 msgr2=0x7f5300070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f530006dca0 0x7f5300070150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f53180fd980 tx=0x7f5308009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:38.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180662e0 msgr2=0x7f53180fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180662e0 0x7f53180fd3a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b790 tx=0x7f530c009220 comp rx=0 tx=0).stop 2024-04-06T03:49:38.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 shutdown_connections 2024-04-06T03:49:38.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53180662e0 0x7f53180fd3a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f530006dca0 0x7f5300070150 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 --2- 172.21.15.94:0/489875370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53180fff00 0x7f53180fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 >> 172.21.15.94:0/489875370 conn(0x7f53180f7430 msgr2=0x7f53180f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 shutdown_connections 2024-04-06T03:49:38.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.622+0000 7f531d61d700 1 -- 172.21.15.94:0/489875370 wait complete. 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.724+0000 7f393436e700 1 -- 172.21.15.94:0/32838870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 msgr2=0x7f392c1023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.724+0000 7f393436e700 1 --2- 172.21.15.94:0/32838870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 0x7f392c1023a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f391c004940 tx=0x7f391c009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.724+0000 7f393436e700 1 -- 172.21.15.94:0/32838870 shutdown_connections 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.724+0000 7f393436e700 1 --2- 172.21.15.94:0/32838870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 0x7f392c1023a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.724+0000 7f393436e700 1 --2- 172.21.15.94:0/32838870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f392c066220 0x7f392c0ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.724+0000 7f393436e700 1 -- 172.21.15.94:0/32838870 >> 172.21.15.94:0/32838870 conn(0x7f392c0f7430 msgr2=0x7f392c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.724+0000 7f393436e700 1 -- 172.21.15.94:0/32838870 shutdown_connections 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.724+0000 7f393436e700 1 -- 172.21.15.94:0/32838870 wait complete. 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f393436e700 1 Processor -- start 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f393436e700 1 -- start start 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f393436e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f392c066220 0x7f392c10fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f393436e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 0x7f392c110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f393436e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c110850 con 0x7f392c0fffc0 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f393436e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c110f90 con 0x7f392c066220 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f3931909700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 0x7f392c110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f3931909700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 0x7f392c110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36400/0 (socket says 172.21.15.94:36400) 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f3931909700 1 -- 172.21.15.94:0/2558223159 learned_addr learned my addr 172.21.15.94:0/2558223159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f3931909700 1 -- 172.21.15.94:0/2558223159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f392c066220 msgr2=0x7f392c10fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f393210a700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f392c066220 0x7f392c10fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.725+0000 7f3931909700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f392c066220 0x7f392c10fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.726+0000 7f3931909700 1 -- 172.21.15.94:0/2558223159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c005040 con 0x7f392c0fffc0 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.726+0000 7f3931909700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 0x7f392c110250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f391c00b3b0 tx=0x7f391c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.726+0000 7f3922ffd700 1 -- 172.21.15.94:0/2558223159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f391c00e040 con 0x7f392c0fffc0 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.726+0000 7f3922ffd700 1 -- 172.21.15.94:0/2558223159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f391c00bcc0 con 0x7f392c0fffc0 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.726+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c111210 con 0x7f392c0fffc0 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.726+0000 7f3922ffd700 1 -- 172.21.15.94:0/2558223159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f391c01c630 con 0x7f392c0fffc0 2024-04-06T03:49:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.726+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c1117d0 con 0x7f392c0fffc0 2024-04-06T03:49:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.727+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c05f5f0 con 0x7f392c0fffc0 2024-04-06T03:49:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.727+0000 7f3922ffd700 1 -- 172.21.15.94:0/2558223159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f391c0139f0 con 0x7f392c0fffc0 2024-04-06T03:49:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.728+0000 7f3922ffd700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f391806dca0 0x7f3918070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.728+0000 7f393210a700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f391806dca0 0x7f3918070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.728+0000 7f3922ffd700 1 -- 172.21.15.94:0/2558223159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f391c08e610 con 0x7f392c0fffc0 2024-04-06T03:49:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.728+0000 7f393210a700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f391806dca0 0x7f3918070150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f392c065f60 tx=0x7f392800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:38.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.732+0000 7f3922ffd700 1 -- 172.21.15.94:0/2558223159 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f391c05b740 con 0x7f392c0fffc0 2024-04-06T03:49:38.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.976+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f392c111c00 con 0x7f391806dca0 2024-04-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.991+0000 7f3922ffd700 1 -- 172.21.15.94:0/2558223159 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f392c111c00 con 0x7f391806dca0 2024-04-06T03:49:38.992 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (8m) 16s ago 16m 26.6M - 0.25.0 c8568f914cd2 6937459ab73c 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (5m) 16s ago 17m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (5m) 17s ago 16m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 error 16s ago 16m - - 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (84s) 16s ago 11m 15.2M - 19.0.0-2027-g9f192350 a5511c7deba9 bfca2184eb3c 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (112s) 16s ago 11m 83.0M - 19.0.0-2027-g9f192350 a5511c7deba9 c189927827c8 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (63s) 17s ago 11m 14.9M - 19.0.0-2027-g9f192350 a5511c7deba9 6dd3b4023d80 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (36s) 17s ago 11m 14.0M - 19.0.0-2027-g9f192350 a5511c7deba9 432ef9195992 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (7m) 16s ago 18m 508M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (6m) 17s ago 16m 452M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (6m) 16s ago 18m 54.3M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (6m) 17s ago 16m 44.0M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (24s) 16s ago 17m 7902k - 1.5.0 0da6a335fe13 21b4e6161925 2024-04-06T03:49:38.993 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (20s) 17s ago 16m 3594k - 1.5.0 0da6a335fe13 a1730b2e350a 2024-04-06T03:49:38.994 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (5m) 16s ago 15m 111M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:49:38.994 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (4m) 16s ago 14m 123M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:49:38.994 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (3m) 16s ago 14m 107M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:49:38.994 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (3m) 17s ago 13m 123M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:49:38.994 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (2m) 17s ago 13m 108M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 c31367e1c2d5 2024-04-06T03:49:38.994 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (2m) 17s ago 13m 109M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 16386e8f8583 2024-04-06T03:49:38.994 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 starting - - - - 2024-04-06T03:49:38.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.993+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f391806dca0 msgr2=0x7f3918070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.993+0000 7f393436e700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f391806dca0 0x7f3918070150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f392c065f60 tx=0x7f392800b040 comp rx=0 tx=0).stop 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.994+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 msgr2=0x7f392c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.994+0000 7f393436e700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 0x7f392c110250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f391c00b3b0 tx=0x7f391c009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.994+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 shutdown_connections 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.994+0000 7f393436e700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f391806dca0 0x7f3918070150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.994+0000 7f393436e700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f392c066220 0x7f392c10fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.994+0000 7f393436e700 1 --2- 172.21.15.94:0/2558223159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f392c0fffc0 0x7f392c110250 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.994+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 >> 172.21.15.94:0/2558223159 conn(0x7f392c0f7430 msgr2=0x7f392c0fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.994+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 shutdown_connections 2024-04-06T03:49:38.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:38.994+0000 7f393436e700 1 -- 172.21.15.94:0/2558223159 wait complete. 2024-04-06T03:49:39.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.111+0000 7f30371fe700 1 -- 172.21.15.94:0/3370419883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 msgr2=0x7f30300ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.111+0000 7f30371fe700 1 --2- 172.21.15.94:0/3370419883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 0x7f30300ffb10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3024004940 tx=0x7f3024009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.111+0000 7f30371fe700 1 -- 172.21.15.94:0/3370419883 shutdown_connections 2024-04-06T03:49:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.111+0000 7f30371fe700 1 --2- 172.21.15.94:0/3370419883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3030100050 0x7f3030102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.111+0000 7f30371fe700 1 --2- 172.21.15.94:0/3370419883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 0x7f30300ffb10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.111+0000 7f30371fe700 1 -- 172.21.15.94:0/3370419883 >> 172.21.15.94:0/3370419883 conn(0x7f30300f74e0 msgr2=0x7f30300f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.111+0000 7f30371fe700 1 -- 172.21.15.94:0/3370419883 shutdown_connections 2024-04-06T03:49:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.111+0000 7f30371fe700 1 -- 172.21.15.94:0/3370419883 wait complete. 2024-04-06T03:49:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.112+0000 7f30371fe700 1 Processor -- start 2024-04-06T03:49:39.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.112+0000 7f30371fe700 1 -- start start 2024-04-06T03:49:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.112+0000 7f30371fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 0x7f303010dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.112+0000 7f30371fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3030100050 0x7f303010e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.112+0000 7f30371fe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303010e620 con 0x7f3030066ab0 2024-04-06T03:49:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.112+0000 7f30371fe700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303010ee70 con 0x7f3030100050 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.112+0000 7f3034f9a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 0x7f303010dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.112+0000 7f3034f9a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 0x7f303010dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36416/0 (socket says 172.21.15.94:36416) 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.112+0000 7f3034f9a700 1 -- 172.21.15.94:0/772976496 learned_addr learned my addr 172.21.15.94:0/772976496 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f302ffff700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3030100050 0x7f303010e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f3034f9a700 1 -- 172.21.15.94:0/772976496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3030100050 msgr2=0x7f303010e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f3034f9a700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3030100050 0x7f303010e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f3034f9a700 1 -- 172.21.15.94:0/772976496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024005040 con 0x7f3030066ab0 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f302ffff700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3030100050 0x7f303010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f3034f9a700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 0x7f303010dae0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f302400b970 tx=0x7f3024009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f302d7fa700 1 -- 172.21.15.94:0/772976496 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302400e040 con 0x7f3030066ab0 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303010f0f0 con 0x7f3030066ab0 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f302d7fa700 1 -- 172.21.15.94:0/772976496 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f302400bbf0 con 0x7f3030066ab0 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f302d7fa700 1 -- 172.21.15.94:0/772976496 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302401c630 con 0x7f3030066ab0 2024-04-06T03:49:39.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.113+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303010f630 con 0x7f3030066ab0 2024-04-06T03:49:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.114+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303005f5f0 con 0x7f3030066ab0 2024-04-06T03:49:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.115+0000 7f302d7fa700 1 -- 172.21.15.94:0/772976496 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f302400bd60 con 0x7f3030066ab0 2024-04-06T03:49:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.115+0000 7f302d7fa700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f30200720c0 0x7f3020074570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.115+0000 7f302d7fa700 1 -- 172.21.15.94:0/772976496 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f302408e9c0 con 0x7f3030066ab0 2024-04-06T03:49:39.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.115+0000 7f302ffff700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f30200720c0 0x7f3020074570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:39.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.116+0000 7f302ffff700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f30200720c0 0x7f3020074570 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f301c009fd0 tx=0x7f301c009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:39.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.124+0000 7f302d7fa700 1 -- 172.21.15.94:0/772976496 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f302405ba50 con 0x7f3030066ab0 2024-04-06T03:49:39.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:39 smithi136 ceph-mon[82061]: pgmap v252: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:39.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:39 smithi136 ceph-mon[82061]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:39.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:39 smithi094 ceph-mon[94053]: pgmap v252: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:39.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:39 smithi094 ceph-mon[94053]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.534+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f303010f9b0 con 0x7f3030066ab0 2024-04-06T03:49:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.535+0000 7f302d7fa700 1 -- 172.21.15.94:0/772976496 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f302405b5e0 con 0x7f3030066ab0 2024-04-06T03:49:39.536 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 6 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 4 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:49:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 14 2024-04-06T03:49:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:49:39.538 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:49:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f30200720c0 msgr2=0x7f3020074570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f30200720c0 0x7f3020074570 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f301c009fd0 tx=0x7f301c009ee0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 msgr2=0x7f303010dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 0x7f303010dae0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f302400b970 tx=0x7f3024009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 shutdown_connections 2024-04-06T03:49:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3030066ab0 0x7f303010dae0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f30200720c0 0x7f3020074570 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 --2- 172.21.15.94:0/772976496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3030100050 0x7f303010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 >> 172.21.15.94:0/772976496 conn(0x7f30300f74e0 msgr2=0x7f30300f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 shutdown_connections 2024-04-06T03:49:39.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.537+0000 7f30371fe700 1 -- 172.21.15.94:0/772976496 wait complete. 2024-04-06T03:49:39.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.647+0000 7fbabc35f700 1 -- 172.21.15.94:0/1976104124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 msgr2=0x7fbab40ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:39.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.647+0000 7fbabc35f700 1 --2- 172.21.15.94:0/1976104124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 0x7fbab40ff9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbab0004940 tx=0x7fbab0009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:39.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.647+0000 7fbabc35f700 1 -- 172.21.15.94:0/1976104124 shutdown_connections 2024-04-06T03:49:39.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.647+0000 7fbabc35f700 1 --2- 172.21.15.94:0/1976104124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbab40fff00 0x7fbab4102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.647+0000 7fbabc35f700 1 --2- 172.21.15.94:0/1976104124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 0x7fbab40ff9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.647+0000 7fbabc35f700 1 -- 172.21.15.94:0/1976104124 >> 172.21.15.94:0/1976104124 conn(0x7fbab40f7430 msgr2=0x7fbab40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:39.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.647+0000 7fbabc35f700 1 -- 172.21.15.94:0/1976104124 shutdown_connections 2024-04-06T03:49:39.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.647+0000 7fbabc35f700 1 -- 172.21.15.94:0/1976104124 wait complete. 2024-04-06T03:49:39.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.648+0000 7fbabc35f700 1 Processor -- start 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.648+0000 7fbabc35f700 1 -- start start 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.648+0000 7fbabc35f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 0x7fbab410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.648+0000 7fbabc35f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbab40fff00 0x7fbab410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.648+0000 7fbabc35f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab410e4c0 con 0x7fbab4066ef0 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.648+0000 7fbabc35f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab410e630 con 0x7fbab40fff00 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.648+0000 7fbaba0fb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 0x7fbab410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.648+0000 7fbaba0fb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 0x7fbab410d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36426/0 (socket says 172.21.15.94:36426) 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.648+0000 7fbaba0fb700 1 -- 172.21.15.94:0/2231531718 learned_addr learned my addr 172.21.15.94:0/2231531718 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbab98fa700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbab40fff00 0x7fbab410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbaba0fb700 1 -- 172.21.15.94:0/2231531718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbab40fff00 msgr2=0x7fbab410dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbaba0fb700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbab40fff00 0x7fbab410dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbaba0fb700 1 -- 172.21.15.94:0/2231531718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0005040 con 0x7fbab4066ef0 2024-04-06T03:49:39.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbaba0fb700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 0x7fbab410d980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbab000b3b0 tx=0x7fbab0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:39.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbaaaffd700 1 -- 172.21.15.94:0/2231531718 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab000e040 con 0x7fbab4066ef0 2024-04-06T03:49:39.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab4113030 con 0x7fbab4066ef0 2024-04-06T03:49:39.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbaaaffd700 1 -- 172.21.15.94:0/2231531718 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbab000bbf0 con 0x7fbab4066ef0 2024-04-06T03:49:39.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbaaaffd700 1 -- 172.21.15.94:0/2231531718 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab001c630 con 0x7fbab4066ef0 2024-04-06T03:49:39.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.649+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab41135f0 con 0x7fbab4066ef0 2024-04-06T03:49:39.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.650+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab404c710 con 0x7fbab4066ef0 2024-04-06T03:49:39.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.651+0000 7fbaaaffd700 1 -- 172.21.15.94:0/2231531718 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fbab000bd60 con 0x7fbab4066ef0 2024-04-06T03:49:39.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.651+0000 7fbaaaffd700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbaa006dca0 0x7fbaa0070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:39.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.651+0000 7fbaaaffd700 1 -- 172.21.15.94:0/2231531718 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fbab008e1b0 con 0x7fbab4066ef0 2024-04-06T03:49:39.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.651+0000 7fbab98fa700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbaa006dca0 0x7fbaa0070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:39.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.652+0000 7fbab98fa700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbaa006dca0 0x7fbaa0070150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4004f40 tx=0x7fbaa400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:39.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.656+0000 7fbaaaffd700 1 -- 172.21.15.94:0/2231531718 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbab005b240 con 0x7fbab4066ef0 2024-04-06T03:49:39.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.920+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbab4113ad0 con 0x7fbab4066ef0 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.920+0000 7fbaaaffd700 1 -- 172.21.15.94:0/2231531718 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1940 (secure 0 0 0) 0x7fbab005add0 con 0x7fbab4066ef0 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stdout:e36 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 35 2024-04-06T03:49:39.923 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:49:01.609121+0000 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 82 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:49:39.924 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=34400} 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 34400 members: 34400 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:34400} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:49:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:49:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:49:39.926 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/1650064917,v1:172.21.15.94:6829/1650064917] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:49:39.926 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.136:6824/1500356136,v1:172.21.15.136:6825/1500356136] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:49:39.926 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.136:6826/1840231705,v1:172.21.15.136:6827/1840231705] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:49:39.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.923+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbaa006dca0 msgr2=0x7fbaa0070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.923+0000 7fbabc35f700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbaa006dca0 0x7fbaa0070150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4004f40 tx=0x7fbaa400b040 comp rx=0 tx=0).stop 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.923+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 msgr2=0x7fbab410d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.923+0000 7fbabc35f700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 0x7fbab410d980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbab000b3b0 tx=0x7fbab0009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.924+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 shutdown_connections 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.924+0000 7fbabc35f700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbab4066ef0 0x7fbab410d980 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.924+0000 7fbabc35f700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fbaa006dca0 0x7fbaa0070150 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.924+0000 7fbabc35f700 1 --2- 172.21.15.94:0/2231531718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbab40fff00 0x7fbab410dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.924+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 >> 172.21.15.94:0/2231531718 conn(0x7fbab40f7430 msgr2=0x7fbab40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.924+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 shutdown_connections 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:39.924+0000 7fbabc35f700 1 -- 172.21.15.94:0/2231531718 wait complete. 2024-04-06T03:49:39.927 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 36 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.028+0000 7f98c029f700 1 -- 172.21.15.94:0/1358785166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 msgr2=0x7f98b80fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.028+0000 7f98c029f700 1 --2- 172.21.15.94:0/1358785166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 0x7f98b80fd280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f98b4004940 tx=0x7f98b4009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.028+0000 7f98c029f700 1 -- 172.21.15.94:0/1358785166 shutdown_connections 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.028+0000 7f98c029f700 1 --2- 172.21.15.94:0/1358785166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b80fdfe0 0x7f98b80fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.028+0000 7f98c029f700 1 --2- 172.21.15.94:0/1358785166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 0x7f98b80fd280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.028+0000 7f98c029f700 1 -- 172.21.15.94:0/1358785166 >> 172.21.15.94:0/1358785166 conn(0x7f98b80f8550 msgr2=0x7f98b80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.028+0000 7f98c029f700 1 -- 172.21.15.94:0/1358785166 shutdown_connections 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.028+0000 7f98c029f700 1 -- 172.21.15.94:0/1358785166 wait complete. 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98c029f700 1 Processor -- start 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98c029f700 1 -- start start 2024-04-06T03:49:40.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98c029f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 0x7f98b810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:40.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98c029f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b80fdfe0 0x7f98b810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:40.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98c029f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b810e510 con 0x7f98b80fce70 2024-04-06T03:49:40.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98c029f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b8112eb0 con 0x7f98b80fdfe0 2024-04-06T03:49:40.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98be03b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 0x7f98b810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:40.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98be03b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 0x7f98b810d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36446/0 (socket says 172.21.15.94:36446) 2024-04-06T03:49:40.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98be03b700 1 -- 172.21.15.94:0/1273905525 learned_addr learned my addr 172.21.15.94:0/1273905525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:40.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98bd83a700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b80fdfe0 0x7f98b810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:40.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98be03b700 1 -- 172.21.15.94:0/1273905525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b80fdfe0 msgr2=0x7f98b810de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98be03b700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b80fdfe0 0x7f98b810de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98be03b700 1 -- 172.21.15.94:0/1273905525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b4005040 con 0x7f98b80fce70 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.029+0000 7f98bd83a700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b80fdfe0 0x7f98b810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.030+0000 7f98be03b700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 0x7f98b810d940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98b400b3b0 tx=0x7f98b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.030+0000 7f98aeffd700 1 -- 172.21.15.94:0/1273905525 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b400e040 con 0x7f98b80fce70 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.030+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b81130b0 con 0x7f98b80fce70 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.030+0000 7f98aeffd700 1 -- 172.21.15.94:0/1273905525 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98b400bcc0 con 0x7f98b80fce70 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.030+0000 7f98aeffd700 1 -- 172.21.15.94:0/1273905525 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b401c630 con 0x7f98b80fce70 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.030+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b8113620 con 0x7f98b80fce70 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.031+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b804c710 con 0x7f98b80fce70 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.031+0000 7f98aeffd700 1 -- 172.21.15.94:0/1273905525 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f98b4013980 con 0x7f98b80fce70 2024-04-06T03:49:40.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.032+0000 7f98aeffd700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f98a406dca0 0x7f98a4070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:40.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.032+0000 7f98aeffd700 1 -- 172.21.15.94:0/1273905525 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f98b408e330 con 0x7f98b80fce70 2024-04-06T03:49:40.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.032+0000 7f98bd83a700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f98a406dca0 0x7f98a4070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:40.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.032+0000 7f98bd83a700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f98a406dca0 0x7f98a4070150 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98a8006fd0 tx=0x7f98a8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:40.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.037+0000 7f98aeffd700 1 -- 172.21.15.94:0/1273905525 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f98b405b340 con 0x7f98b80fce70 2024-04-06T03:49:40.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.323+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98b81022c0 con 0x7f98a406dca0 2024-04-06T03:49:40.325 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:40 smithi094 ceph-mon[94053]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:40.325 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:40 smithi094 ceph-mon[94053]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:40.326 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:40.326 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:40 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:40.326 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:40 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/772976496' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:40.326 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:40 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2231531718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:49:40.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.325+0000 7f98aeffd700 1 -- 172.21.15.94:0/1273905525 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f98b81022c0 con 0x7f98a406dca0 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "osd", 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mds", 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "16/21 daemons upgraded", 2024-04-06T03:49:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading prometheus daemons", 2024-04-06T03:49:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:49:40.328 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f98a406dca0 msgr2=0x7f98a4070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f98a406dca0 0x7f98a4070150 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98a8006fd0 tx=0x7f98a8008040 comp rx=0 tx=0).stop 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 msgr2=0x7f98b810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 0x7f98b810d940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98b400b3b0 tx=0x7f98b4009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 shutdown_connections 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98b80fce70 0x7f98b810d940 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f98a406dca0 0x7f98a4070150 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 --2- 172.21.15.94:0/1273905525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98b80fdfe0 0x7f98b810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 >> 172.21.15.94:0/1273905525 conn(0x7f98b80f8550 msgr2=0x7f98b80f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 shutdown_connections 2024-04-06T03:49:40.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.328+0000 7f98c029f700 1 -- 172.21.15.94:0/1273905525 wait complete. 2024-04-06T03:49:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:40 smithi136 ceph-mon[82061]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:40 smithi136 ceph-mon[82061]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:40 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:40 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/772976496' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:40.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:40 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2231531718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:49:40.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.440+0000 7fb8f5e25700 1 -- 172.21.15.94:0/1312048266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 msgr2=0x7fb8f00ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.440+0000 7fb8f5e25700 1 --2- 172.21.15.94:0/1312048266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 0x7fb8f00ffa20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0004940 tx=0x7fb8e0009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.440+0000 7fb8f5e25700 1 -- 172.21.15.94:0/1312048266 shutdown_connections 2024-04-06T03:49:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.440+0000 7fb8f5e25700 1 --2- 172.21.15.94:0/1312048266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8f00fff60 0x7fb8f0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.440+0000 7fb8f5e25700 1 --2- 172.21.15.94:0/1312048266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 0x7fb8f00ffa20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.440+0000 7fb8f5e25700 1 -- 172.21.15.94:0/1312048266 >> 172.21.15.94:0/1312048266 conn(0x7fb8f00f7430 msgr2=0x7fb8f00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.440+0000 7fb8f5e25700 1 -- 172.21.15.94:0/1312048266 shutdown_connections 2024-04-06T03:49:40.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.440+0000 7fb8f5e25700 1 -- 172.21.15.94:0/1312048266 wait complete. 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.440+0000 7fb8f5e25700 1 Processor -- start 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8f5e25700 1 -- start start 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8f5e25700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 0x7fb8f010da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8f5e25700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8f00fff60 0x7fb8f010dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8f5e25700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f010e5c0 con 0x7fb8f0066280 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8f5e25700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f010edc0 con 0x7fb8f00fff60 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8ef7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 0x7fb8f010da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8ef7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 0x7fb8f010da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36458/0 (socket says 172.21.15.94:36458) 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8ef7fe700 1 -- 172.21.15.94:0/2507847179 learned_addr learned my addr 172.21.15.94:0/2507847179 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8eeffd700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8f00fff60 0x7fb8f010dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8ef7fe700 1 -- 172.21.15.94:0/2507847179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8f00fff60 msgr2=0x7fb8f010dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8ef7fe700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8f00fff60 0x7fb8f010dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.441+0000 7fb8ef7fe700 1 -- 172.21.15.94:0/2507847179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0005040 con 0x7fb8f0066280 2024-04-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.442+0000 7fb8ef7fe700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 0x7fb8f010da80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.442+0000 7fb8d7fff700 1 -- 172.21.15.94:0/2507847179 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e000e030 con 0x7fb8f0066280 2024-04-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.442+0000 7fb8d7fff700 1 -- 172.21.15.94:0/2507847179 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8e0013de0 con 0x7fb8f0066280 2024-04-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.442+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f010f040 con 0x7fb8f0066280 2024-04-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.442+0000 7fb8d7fff700 1 -- 172.21.15.94:0/2507847179 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e001ce50 con 0x7fb8f0066280 2024-04-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.442+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f010f580 con 0x7fb8f0066280 2024-04-06T03:49:40.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.443+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f005f5f0 con 0x7fb8f0066280 2024-04-06T03:49:40.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.444+0000 7fb8d7fff700 1 -- 172.21.15.94:0/2507847179 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fb8e001c400 con 0x7fb8f0066280 2024-04-06T03:49:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.445+0000 7fb8d7fff700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8d806dc30 0x7fb8d80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:49:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.445+0000 7fb8d7fff700 1 -- 172.21.15.94:0/2507847179 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fb8e008f360 con 0x7fb8f0066280 2024-04-06T03:49:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.445+0000 7fb8eeffd700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8d806dc30 0x7fb8d80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:49:40.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.446+0000 7fb8eeffd700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8d806dc30 0x7fb8d80700e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009b80 tx=0x7fb8e400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:49:40.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.449+0000 7fb8d7fff700 1 -- 172.21.15.94:0/2507847179 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb8e005c3f0 con 0x7fb8f0066280 2024-04-06T03:49:40.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.741+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb8f004bed0 con 0x7fb8f0066280 2024-04-06T03:49:40.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.742+0000 7fb8d7fff700 1 -- 172.21.15.94:0/2507847179 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+155 (secure 0 0 0) 0x7fb8e005bf80 con 0x7fb8f0066280 2024-04-06T03:49:40.744 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:49:40.744 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:49:40.744 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in error state 2024-04-06T03:49:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.744+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8d806dc30 msgr2=0x7fb8d80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.744+0000 7fb8f5e25700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8d806dc30 0x7fb8d80700e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009b80 tx=0x7fb8e400b040 comp rx=0 tx=0).stop 2024-04-06T03:49:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.744+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 msgr2=0x7fb8f010da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:49:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.744+0000 7fb8f5e25700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 0x7fb8f010da80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c00 tx=0x7fb8e0009e30 comp rx=0 tx=0).stop 2024-04-06T03:49:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.744+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 shutdown_connections 2024-04-06T03:49:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.744+0000 7fb8f5e25700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8f0066280 0x7fb8f010da80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.744+0000 7fb8f5e25700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8d806dc30 0x7fb8d80700e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.744+0000 7fb8f5e25700 1 --2- 172.21.15.94:0/2507847179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8f00fff60 0x7fb8f010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:49:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.744+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 >> 172.21.15.94:0/2507847179 conn(0x7fb8f00f7430 msgr2=0x7fb8f00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:49:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.745+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 shutdown_connections 2024-04-06T03:49:40.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:49:40.745+0000 7fb8f5e25700 1 -- 172.21.15.94:0/2507847179 wait complete. 2024-04-06T03:49:41.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:41 smithi094 ceph-mon[94053]: pgmap v253: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:41.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:41 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:41.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:41 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:41.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:41 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2507847179' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:49:41.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:41 smithi136 ceph-mon[82061]: pgmap v253: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:41.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:41 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:41.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:41 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:41.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:41 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2507847179' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:49:42.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:42 smithi136 ceph-mon[82061]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:42.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:42 smithi094 ceph-mon[94053]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:49:43.405 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:43 smithi094 ceph-mon[94053]: pgmap v254: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:43.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:43 smithi136 ceph-mon[82061]: pgmap v254: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:45.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: pgmap v255: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: pgmap v256: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: pgmap v257: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.099 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.100 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:44 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: pgmap v255: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:45.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:45.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:45.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:49:45.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: pgmap v256: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: pgmap v257: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.168 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:45.169 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:44 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:49:47.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:46 smithi094 ceph-mon[94053]: Upgrade: Updating alertmanager.smithi094 2024-04-06T03:49:47.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:46 smithi094 ceph-mon[94053]: Deploying daemon alertmanager.smithi094 on smithi094 2024-04-06T03:49:47.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:46 smithi094 ceph-mon[94053]: pgmap v258: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:47.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:46 smithi136 ceph-mon[82061]: Upgrade: Updating alertmanager.smithi094 2024-04-06T03:49:47.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:46 smithi136 ceph-mon[82061]: Deploying daemon alertmanager.smithi094 on smithi094 2024-04-06T03:49:47.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:46 smithi136 ceph-mon[82061]: pgmap v258: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:49.162 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:48 smithi094 ceph-mon[94053]: pgmap v259: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:49.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:48 smithi136 ceph-mon[82061]: pgmap v259: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:50.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:50 smithi094 ceph-mon[94053]: pgmap v260: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:50.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:49:51.150 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:50 smithi136 ceph-mon[82061]: pgmap v260: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:51.150 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:49:52.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:52 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:52.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:52 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:52.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:52 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:52.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:52 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:52.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:52 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:52.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:52 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:49:53.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:53 smithi094 ceph-mon[94053]: pgmap v261: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:53.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:53 smithi136 ceph-mon[82061]: pgmap v261: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:55.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:55 smithi136 ceph-mon[82061]: pgmap v262: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:55.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:55 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:55.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:55 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:55.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:55 smithi094 ceph-mon[94053]: pgmap v262: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:55.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:55 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:55.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:55 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:57.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:57 smithi094 ceph-mon[94053]: pgmap v263: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:57.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:57 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:57.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:57 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:57.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:57 smithi136 ceph-mon[82061]: pgmap v263: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:57.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:57 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:57.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:57 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:49:59.319 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:49:59 smithi094 ceph-mon[94053]: pgmap v264: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:49:59.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:49:59 smithi136 ceph-mon[82061]: pgmap v264: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:01.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: pgmap v265: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: pgmap v266: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: pgmap v267: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: overall HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.350 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:00 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: pgmap v265: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:01.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:01.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: pgmap v266: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: pgmap v267: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: overall HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.418 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:01.419 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:00 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:03.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:03 smithi094 ceph-mon[94053]: Upgrade: Updating grafana.smithi094 2024-04-06T03:50:03.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:03 smithi094 ceph-mon[94053]: Deploying daemon grafana.smithi094 on smithi094 2024-04-06T03:50:03.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:03 smithi094 ceph-mon[94053]: pgmap v268: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:03.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:03 smithi136 ceph-mon[82061]: Upgrade: Updating grafana.smithi094 2024-04-06T03:50:03.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:03 smithi136 ceph-mon[82061]: Deploying daemon grafana.smithi094 on smithi094 2024-04-06T03:50:03.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:03 smithi136 ceph-mon[82061]: pgmap v268: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:05.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:05 smithi136 ceph-mon[82061]: pgmap v269: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:05.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:05 smithi094 ceph-mon[94053]: pgmap v269: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:06.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:50:06.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:06.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:06.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:06 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:06.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:50:06.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:06.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:06.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:06 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:07.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:07 smithi136 ceph-mon[82061]: pgmap v270: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:07.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:07 smithi094 ceph-mon[94053]: pgmap v270: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:09.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:09 smithi136 ceph-mon[82061]: pgmap v271: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:09.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:09 smithi094 ceph-mon[94053]: pgmap v271: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:10.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.855+0000 7fe4cf598700 1 -- 172.21.15.94:0/503321855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c8066b70 msgr2=0x7fe4c80ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:10.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.855+0000 7fe4cf598700 1 --2- 172.21.15.94:0/503321855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c8066b70 0x7fe4c80ffa50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4004940 tx=0x7fe4c4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:10.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 -- 172.21.15.94:0/503321855 shutdown_connections 2024-04-06T03:50:10.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 --2- 172.21.15.94:0/503321855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c80fff90 0x7fe4c81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 --2- 172.21.15.94:0/503321855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c8066b70 0x7fe4c80ffa50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 -- 172.21.15.94:0/503321855 >> 172.21.15.94:0/503321855 conn(0x7fe4c80f74e0 msgr2=0x7fe4c80f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 -- 172.21.15.94:0/503321855 shutdown_connections 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 -- 172.21.15.94:0/503321855 wait complete. 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 Processor -- start 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 -- start start 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c8066b70 0x7fe4c810dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c80fff90 0x7fe4c810e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c810e6b0 con 0x7fe4c80fff90 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.856+0000 7fe4cf598700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c810ee00 con 0x7fe4c8066b70 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4ccb33700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c80fff90 0x7fe4c810e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4ccb33700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c80fff90 0x7fe4c810e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34532/0 (socket says 172.21.15.94:34532) 2024-04-06T03:50:10.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4ccb33700 1 -- 172.21.15.94:0/3992222727 learned_addr learned my addr 172.21.15.94:0/3992222727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:10.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4cd334700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c8066b70 0x7fe4c810dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:10.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4ccb33700 1 -- 172.21.15.94:0/3992222727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c8066b70 msgr2=0x7fe4c810dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:10.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4ccb33700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c8066b70 0x7fe4c810dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:10.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4ccb33700 1 -- 172.21.15.94:0/3992222727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c4005040 con 0x7fe4c80fff90 2024-04-06T03:50:10.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4ccb33700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c80fff90 0x7fe4c810e020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800b790 tx=0x7fe4b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:10.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4bdffb700 1 -- 172.21.15.94:0/3992222727 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b800f6f0 con 0x7fe4c80fff90 2024-04-06T03:50:10.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4bdffb700 1 -- 172.21.15.94:0/3992222727 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4b800fd20 con 0x7fe4c80fff90 2024-04-06T03:50:10.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4bdffb700 1 -- 172.21.15.94:0/3992222727 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b80103a0 con 0x7fe4c80fff90 2024-04-06T03:50:10.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.857+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c810f0e0 con 0x7fe4c80fff90 2024-04-06T03:50:10.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.858+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c810f6a0 con 0x7fe4c80fff90 2024-04-06T03:50:10.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.859+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c805f5f0 con 0x7fe4c80fff90 2024-04-06T03:50:10.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.859+0000 7fe4bdffb700 1 -- 172.21.15.94:0/3992222727 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fe4b8011400 con 0x7fe4c80fff90 2024-04-06T03:50:10.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.860+0000 7fe4bdffb700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe4b406dca0 0x7fe4b4070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:10.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.860+0000 7fe4bdffb700 1 -- 172.21.15.94:0/3992222727 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe4b808d2c0 con 0x7fe4c80fff90 2024-04-06T03:50:10.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.860+0000 7fe4cd334700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe4b406dca0 0x7fe4b4070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:10.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.860+0000 7fe4cd334700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe4b406dca0 0x7fe4b4070150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400b970 tx=0x7fe4c4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:10.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:10.868+0000 7fe4bdffb700 1 -- 172.21.15.94:0/3992222727 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe4b805a3f0 con 0x7fe4c80fff90 2024-04-06T03:50:11.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:11 smithi094 ceph-mon[94053]: pgmap v272: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:11.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:11.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:11 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.178+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4c810f9e0 con 0x7fe4b406dca0 2024-04-06T03:50:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.179+0000 7fe4bdffb700 1 -- 172.21.15.94:0/3992222727 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7fe4c810f9e0 con 0x7fe4b406dca0 2024-04-06T03:50:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.181+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe4b406dca0 msgr2=0x7fe4b4070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.181+0000 7fe4cf598700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe4b406dca0 0x7fe4b4070150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400b970 tx=0x7fe4c4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.182+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c80fff90 msgr2=0x7fe4c810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.182+0000 7fe4cf598700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c80fff90 0x7fe4c810e020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800b790 tx=0x7fe4b8009250 comp rx=0 tx=0).stop 2024-04-06T03:50:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.182+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 shutdown_connections 2024-04-06T03:50:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.182+0000 7fe4cf598700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe4b406dca0 0x7fe4b4070150 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.182+0000 7fe4cf598700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4c8066b70 0x7fe4c810dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.182+0000 7fe4cf598700 1 --2- 172.21.15.94:0/3992222727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4c80fff90 0x7fe4c810e020 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.182+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 >> 172.21.15.94:0/3992222727 conn(0x7fe4c80f74e0 msgr2=0x7fe4c80f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.182+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 shutdown_connections 2024-04-06T03:50:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.182+0000 7fe4cf598700 1 -- 172.21.15.94:0/3992222727 wait complete. 2024-04-06T03:50:11.194 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:50:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.291+0000 7f6dba03e700 1 -- 172.21.15.94:0/3602979463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 msgr2=0x7f6db40fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.291+0000 7f6dba03e700 1 --2- 172.21.15.94:0/3602979463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 0x7f6db40fc0e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6da4004940 tx=0x7f6da4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.291+0000 7f6dba03e700 1 -- 172.21.15.94:0/3602979463 shutdown_connections 2024-04-06T03:50:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.291+0000 7f6dba03e700 1 --2- 172.21.15.94:0/3602979463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6db40fcf80 0x7f6db40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.291+0000 7f6dba03e700 1 --2- 172.21.15.94:0/3602979463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 0x7f6db40fc0e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.291+0000 7f6dba03e700 1 -- 172.21.15.94:0/3602979463 >> 172.21.15.94:0/3602979463 conn(0x7f6db40f7430 msgr2=0x7f6db40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.291+0000 7f6dba03e700 1 -- 172.21.15.94:0/3602979463 shutdown_connections 2024-04-06T03:50:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.291+0000 7f6dba03e700 1 -- 172.21.15.94:0/3602979463 wait complete. 2024-04-06T03:50:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6dba03e700 1 Processor -- start 2024-04-06T03:50:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6dba03e700 1 -- start start 2024-04-06T03:50:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6dba03e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 0x7f6db4110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6dba03e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6db40fcf80 0x7f6db4112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6dba03e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4066370 con 0x7f6db40fbcd0 2024-04-06T03:50:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6dba03e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40664b0 con 0x7f6db40fcf80 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6db37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 0x7f6db4110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6db37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 0x7f6db4110670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34546/0 (socket says 172.21.15.94:34546) 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6db37fe700 1 -- 172.21.15.94:0/4251548428 learned_addr learned my addr 172.21.15.94:0/4251548428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.292+0000 7f6db2ffd700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6db40fcf80 0x7f6db4112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.293+0000 7f6db37fe700 1 -- 172.21.15.94:0/4251548428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6db40fcf80 msgr2=0x7f6db4112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.293+0000 7f6db37fe700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6db40fcf80 0x7f6db4112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.293+0000 7f6db37fe700 1 -- 172.21.15.94:0/4251548428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da4005040 con 0x7f6db40fbcd0 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.293+0000 7f6db37fe700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 0x7f6db4110670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009150 tx=0x7f6da4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.293+0000 7f6d9bfff700 1 -- 172.21.15.94:0/4251548428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6da400e040 con 0x7f6db40fbcd0 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.293+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db4113160 con 0x7f6db40fbcd0 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.293+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db41136f0 con 0x7f6db40fbcd0 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.293+0000 7f6d9bfff700 1 -- 172.21.15.94:0/4251548428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6da4004290 con 0x7f6db40fbcd0 2024-04-06T03:50:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.293+0000 7f6d9bfff700 1 -- 172.21.15.94:0/4251548428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6da401c630 con 0x7f6db40fbcd0 2024-04-06T03:50:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.294+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da00052e0 con 0x7f6db40fbcd0 2024-04-06T03:50:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.295+0000 7f6d9bfff700 1 -- 172.21.15.94:0/4251548428 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f6da400bbf0 con 0x7f6db40fbcd0 2024-04-06T03:50:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.295+0000 7f6d9bfff700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d9c06dca0 0x7f6d9c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.296+0000 7f6db2ffd700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d9c06dca0 0x7f6d9c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.296+0000 7f6d9bfff700 1 -- 172.21.15.94:0/4251548428 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6da408e090 con 0x7f6db40fbcd0 2024-04-06T03:50:11.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.296+0000 7f6db2ffd700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d9c06dca0 0x7f6d9c070150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009fd0 tx=0x7f6da800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:11.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.304+0000 7f6d9bfff700 1 -- 172.21.15.94:0/4251548428 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6da4057690 con 0x7f6db40fbcd0 2024-04-06T03:50:11.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:11 smithi136 ceph-mon[82061]: pgmap v272: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:11.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:11 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:11.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:11 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:11.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.532+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6da0000bf0 con 0x7f6d9c06dca0 2024-04-06T03:50:11.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.533+0000 7f6d9bfff700 1 -- 172.21.15.94:0/4251548428 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f6da0000bf0 con 0x7f6d9c06dca0 2024-04-06T03:50:11.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d9c06dca0 msgr2=0x7f6d9c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d9c06dca0 0x7f6d9c070150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009fd0 tx=0x7f6da800b040 comp rx=0 tx=0).stop 2024-04-06T03:50:11.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 msgr2=0x7f6db4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 0x7f6db4110670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009150 tx=0x7f6da4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:11.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 shutdown_connections 2024-04-06T03:50:11.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6db40fbcd0 0x7f6db4110670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6d9c06dca0 0x7f6d9c070150 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 --2- 172.21.15.94:0/4251548428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6db40fcf80 0x7f6db4112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 >> 172.21.15.94:0/4251548428 conn(0x7f6db40f7430 msgr2=0x7f6db40f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:11.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.536+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 shutdown_connections 2024-04-06T03:50:11.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.537+0000 7f6dba03e700 1 -- 172.21.15.94:0/4251548428 wait complete. 2024-04-06T03:50:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.650+0000 7f3cf362d700 1 -- 172.21.15.94:0/2063003662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 msgr2=0x7f3cec0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.650+0000 7f3cf362d700 1 --2- 172.21.15.94:0/2063003662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 0x7f3cec0fc0e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8004940 tx=0x7f3ce8009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.651+0000 7f3cf362d700 1 -- 172.21.15.94:0/2063003662 shutdown_connections 2024-04-06T03:50:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.651+0000 7f3cf362d700 1 --2- 172.21.15.94:0/2063003662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cec0fcf80 0x7f3cec0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.651+0000 7f3cf362d700 1 --2- 172.21.15.94:0/2063003662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 0x7f3cec0fc0e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.651+0000 7f3cf362d700 1 -- 172.21.15.94:0/2063003662 >> 172.21.15.94:0/2063003662 conn(0x7f3cec0f7430 msgr2=0x7f3cec0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.651+0000 7f3cf362d700 1 -- 172.21.15.94:0/2063003662 shutdown_connections 2024-04-06T03:50:11.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.651+0000 7f3cf362d700 1 -- 172.21.15.94:0/2063003662 wait complete. 2024-04-06T03:50:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.652+0000 7f3cf362d700 1 Processor -- start 2024-04-06T03:50:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.652+0000 7f3cf362d700 1 -- start start 2024-04-06T03:50:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.652+0000 7f3cf362d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 0x7f3cec066d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.652+0000 7f3cf362d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cec0fcf80 0x7f3cec1126f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:11.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.652+0000 7f3cf362d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec112c30 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.652+0000 7f3cf362d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec112d70 con 0x7f3cec0fcf80 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3cf13c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 0x7f3cec066d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3cf13c9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 0x7f3cec066d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34562/0 (socket says 172.21.15.94:34562) 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3cf13c9700 1 -- 172.21.15.94:0/914411793 learned_addr learned my addr 172.21.15.94:0/914411793 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3cf0bc8700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cec0fcf80 0x7f3cec1126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3cf13c9700 1 -- 172.21.15.94:0/914411793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cec0fcf80 msgr2=0x7f3cec1126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3cf13c9700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cec0fcf80 0x7f3cec1126f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3cf13c9700 1 -- 172.21.15.94:0/914411793 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce8005040 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3cf13c9700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 0x7f3cec066d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800b3b0 tx=0x7f3ce8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3ce1ffb700 1 -- 172.21.15.94:0/914411793 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce800e040 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cec112ff0 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3ce1ffb700 1 -- 172.21.15.94:0/914411793 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ce800bbf0 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.653+0000 7f3ce1ffb700 1 -- 172.21.15.94:0/914411793 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce801c630 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.654+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec113520 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.655+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cec04c710 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.656+0000 7f3ce1ffb700 1 -- 172.21.15.94:0/914411793 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f3ce800bd60 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.656+0000 7f3ce1ffb700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3cd806dca0 0x7f3cd8070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.656+0000 7f3ce1ffb700 1 -- 172.21.15.94:0/914411793 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f3ce808e0f0 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.656+0000 7f3cf0bc8700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3cd806dca0 0x7f3cd8070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:11.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.657+0000 7f3cf0bc8700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3cd806dca0 0x7f3cd8070150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc004f40 tx=0x7f3cdc00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:11.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.665+0000 7f3ce1ffb700 1 -- 172.21.15.94:0/914411793 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3ce805b220 con 0x7f3cec0fbcd0 2024-04-06T03:50:11.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.878+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cec1012e0 con 0x7f3cd806dca0 2024-04-06T03:50:11.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.888+0000 7f3ce1ffb700 1 -- 172.21.15.94:0/914411793 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f3cec1012e0 con 0x7f3cd806dca0 2024-04-06T03:50:11.890 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:50:11.890 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (21s) 1s ago 17m 17.1M - 0.25.0 c8568f914cd2 f3dd69f56752 2024-04-06T03:50:11.890 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (6m) 1s ago 17m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:50:11.890 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (6m) 50s ago 16m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:50:11.890 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (6s) 1s ago 17m 52.9M - 9.4.12 2bacad6d85d8 550bb3828233 2024-04-06T03:50:11.890 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (117s) 1s ago 12m 15.5M - 19.0.0-2027-g9f192350 a5511c7deba9 bfca2184eb3c 2024-04-06T03:50:11.890 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (2m) 1s ago 12m 83.0M - 19.0.0-2027-g9f192350 a5511c7deba9 c189927827c8 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (96s) 50s ago 12m 14.9M - 19.0.0-2027-g9f192350 a5511c7deba9 6dd3b4023d80 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (69s) 50s ago 12m 14.0M - 19.0.0-2027-g9f192350 a5511c7deba9 432ef9195992 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (8m) 1s ago 18m 516M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (7m) 50s ago 16m 452M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (7m) 1s ago 18m 55.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (6m) 50s ago 16m 44.0M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (57s) 1s ago 17m 13.5M - 1.5.0 0da6a335fe13 21b4e6161925 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (53s) 50s ago 16m 3594k - 1.5.0 0da6a335fe13 a1730b2e350a 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (5m) 1s ago 15m 112M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:50:11.891 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (5m) 1s ago 15m 124M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:50:11.892 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (4m) 1s ago 14m 108M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:50:11.892 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (3m) 50s ago 14m 123M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:50:11.892 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (3m) 50s ago 14m 108M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 c31367e1c2d5 2024-04-06T03:50:11.892 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (2m) 50s ago 13m 109M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 16386e8f8583 2024-04-06T03:50:11.892 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (37s) 1s ago 17m 49.1M - 2.43.0 a07b618ecd1d 4cad242accb3 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.890+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3cd806dca0 msgr2=0x7f3cd8070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.890+0000 7f3cf362d700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3cd806dca0 0x7f3cd8070150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc004f40 tx=0x7f3cdc00b040 comp rx=0 tx=0).stop 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.890+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 msgr2=0x7f3cec066d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.890+0000 7f3cf362d700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 0x7f3cec066d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800b3b0 tx=0x7f3ce8009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.891+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 shutdown_connections 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.891+0000 7f3cf362d700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cec0fbcd0 0x7f3cec066d50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.891+0000 7f3cf362d700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3cd806dca0 0x7f3cd8070150 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.891+0000 7f3cf362d700 1 --2- 172.21.15.94:0/914411793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cec0fcf80 0x7f3cec1126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.891+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 >> 172.21.15.94:0/914411793 conn(0x7f3cec0f7430 msgr2=0x7f3cec0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.891+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 shutdown_connections 2024-04-06T03:50:11.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.891+0000 7f3cf362d700 1 -- 172.21.15.94:0/914411793 wait complete. 2024-04-06T03:50:11.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.997+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/2835326614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40662e0 msgr2=0x7ff7b40ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:11.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.997+0000 7ff7b9d2a700 1 --2- 172.21.15.94:0/2835326614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40662e0 0x7ff7b40ff9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4004940 tx=0x7ff7a4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:11.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.997+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/2835326614 shutdown_connections 2024-04-06T03:50:11.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.997+0000 7ff7b9d2a700 1 --2- 172.21.15.94:0/2835326614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7b40fff00 0x7ff7b4102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.997+0000 7ff7b9d2a700 1 --2- 172.21.15.94:0/2835326614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40662e0 0x7ff7b40ff9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.997+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/2835326614 >> 172.21.15.94:0/2835326614 conn(0x7ff7b40f7430 msgr2=0x7ff7b40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:12.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.997+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/2835326614 shutdown_connections 2024-04-06T03:50:12.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.997+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/2835326614 wait complete. 2024-04-06T03:50:12.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.997+0000 7ff7b9d2a700 1 Processor -- start 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b9d2a700 1 -- start start 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b9d2a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7b40662e0 0x7ff7b40fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b9d2a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40fff00 0x7ff7b40fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b9d2a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40fbe70 con 0x7ff7b40fff00 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b9d2a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40fbfe0 con 0x7ff7b40662e0 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40fff00 0x7ff7b40fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40fff00 0x7ff7b40fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34576/0 (socket says 172.21.15.94:34576) 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b2ffd700 1 -- 172.21.15.94:0/881295843 learned_addr learned my addr 172.21.15.94:0/881295843 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b37fe700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7b40662e0 0x7ff7b40fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b2ffd700 1 -- 172.21.15.94:0/881295843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7b40662e0 msgr2=0x7ff7b40fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b2ffd700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7b40662e0 0x7ff7b40fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.998+0000 7ff7b2ffd700 1 -- 172.21.15.94:0/881295843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4005040 con 0x7ff7b40fff00 2024-04-06T03:50:12.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.999+0000 7ff7b2ffd700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40fff00 0x7ff7b40fb8c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8002a50 tx=0x7ff7a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:12.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.999+0000 7ff79bfff700 1 -- 172.21.15.94:0/881295843 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7a8007cc0 con 0x7ff7b40fff00 2024-04-06T03:50:12.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.999+0000 7ff79bfff700 1 -- 172.21.15.94:0/881295843 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7a800f040 con 0x7ff7b40fff00 2024-04-06T03:50:12.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.999+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b40fc2c0 con 0x7ff7b40fff00 2024-04-06T03:50:12.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.999+0000 7ff79bfff700 1 -- 172.21.15.94:0/881295843 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7a800da30 con 0x7ff7b40fff00 2024-04-06T03:50:12.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:11.999+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b4115a60 con 0x7ff7b40fff00 2024-04-06T03:50:12.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.000+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b405f5f0 con 0x7ff7b40fff00 2024-04-06T03:50:12.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.000+0000 7ff79bfff700 1 -- 172.21.15.94:0/881295843 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7ff7a800db90 con 0x7ff7b40fff00 2024-04-06T03:50:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.001+0000 7ff79bfff700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff79c06dca0 0x7ff79c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.001+0000 7ff79bfff700 1 -- 172.21.15.94:0/881295843 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff7a808d380 con 0x7ff7b40fff00 2024-04-06T03:50:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.001+0000 7ff7b37fe700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff79c06dca0 0x7ff79c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.001+0000 7ff7b37fe700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff79c06dca0 0x7ff79c070150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40fd180 tx=0x7ff7a4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:12.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.010+0000 7ff79bfff700 1 -- 172.21.15.94:0/881295843 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff7a8056860 con 0x7ff7b40fff00 2024-04-06T03:50:12.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:12 smithi094 ceph-mon[94053]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:12.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:12.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:12 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:12.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:12 smithi136 ceph-mon[82061]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:12.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:12 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:12.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:12 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.370+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff7b4115c80 con 0x7ff7b40fff00 2024-04-06T03:50:12.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.370+0000 7ff79bfff700 1 -- 172.21.15.94:0/881295843 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff7a8059f20 con 0x7ff7b40fff00 2024-04-06T03:50:12.372 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:50:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:50:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:50:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 6 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 4 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 14 2024-04-06T03:50:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:50:12.374 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:50:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff79c06dca0 msgr2=0x7ff79c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff79c06dca0 0x7ff79c070150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40fd180 tx=0x7ff7a4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40fff00 msgr2=0x7ff7b40fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40fff00 0x7ff7b40fb8c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8002a50 tx=0x7ff7a80046b0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 shutdown_connections 2024-04-06T03:50:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff79c06dca0 0x7ff79c070150 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7b40662e0 0x7ff7b40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 --2- 172.21.15.94:0/881295843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b40fff00 0x7ff7b40fb8c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 >> 172.21.15.94:0/881295843 conn(0x7ff7b40f7430 msgr2=0x7ff7b40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 shutdown_connections 2024-04-06T03:50:12.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.373+0000 7ff7b9d2a700 1 -- 172.21.15.94:0/881295843 wait complete. 2024-04-06T03:50:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.478+0000 7f81cfef6700 1 -- 172.21.15.94:0/2951519432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fd2c0 msgr2=0x7f81c80fd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.478+0000 7f81cfef6700 1 --2- 172.21.15.94:0/2951519432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fd2c0 0x7f81c80fd6d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f81c4004940 tx=0x7f81c4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 -- 172.21.15.94:0/2951519432 shutdown_connections 2024-04-06T03:50:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 --2- 172.21.15.94:0/2951519432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c80fdc10 0x7f81c80fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 --2- 172.21.15.94:0/2951519432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fd2c0 0x7f81c80fd6d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 -- 172.21.15.94:0/2951519432 >> 172.21.15.94:0/2951519432 conn(0x7f81c80f6ba0 msgr2=0x7f81c80f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 -- 172.21.15.94:0/2951519432 shutdown_connections 2024-04-06T03:50:12.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 -- 172.21.15.94:0/2951519432 wait complete. 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 Processor -- start 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 -- start start 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c80fd2c0 0x7f81c8110760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fdc10 0x7f81c8112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c80fbcc0 con 0x7f81c80fdc10 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.479+0000 7f81cfef6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c80fbe00 con 0x7f81c80fd2c0 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cd491700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fdc10 0x7f81c8112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cd491700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fdc10 0x7f81c8112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34604/0 (socket says 172.21.15.94:34604) 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cd491700 1 -- 172.21.15.94:0/1868419917 learned_addr learned my addr 172.21.15.94:0/1868419917 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:12.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cdc92700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c80fd2c0 0x7f81c8110760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:12.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cd491700 1 -- 172.21.15.94:0/1868419917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c80fd2c0 msgr2=0x7f81c8110760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:12.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cd491700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c80fd2c0 0x7f81c8110760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cd491700 1 -- 172.21.15.94:0/1868419917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c4005040 con 0x7f81c80fdc10 2024-04-06T03:50:12.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cd491700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fdc10 0x7f81c8112cb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f81b8002a50 tx=0x7f81b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:12.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81be7fc700 1 -- 172.21.15.94:0/1868419917 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b8007cc0 con 0x7f81c80fdc10 2024-04-06T03:50:12.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81be7fc700 1 -- 172.21.15.94:0/1868419917 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81b800f040 con 0x7f81c80fdc10 2024-04-06T03:50:12.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c81132b0 con 0x7f81c80fdc10 2024-04-06T03:50:12.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81be7fc700 1 -- 172.21.15.94:0/1868419917 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b800d9c0 con 0x7f81c80fdc10 2024-04-06T03:50:12.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.480+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c8113870 con 0x7f81c80fdc10 2024-04-06T03:50:12.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.481+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c805f5f0 con 0x7f81c80fdc10 2024-04-06T03:50:12.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.482+0000 7f81be7fc700 1 -- 172.21.15.94:0/1868419917 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f81b8009730 con 0x7f81c80fdc10 2024-04-06T03:50:12.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.482+0000 7f81be7fc700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f81b406dca0 0x7f81b4070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:12.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.483+0000 7f81cdc92700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f81b406dca0 0x7f81b4070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:12.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.483+0000 7f81be7fc700 1 -- 172.21.15.94:0/1868419917 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f81b808d360 con 0x7f81c80fdc10 2024-04-06T03:50:12.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.483+0000 7f81cdc92700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f81b406dca0 0x7f81b4070150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f81c4004890 tx=0x7f81c4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:12.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.487+0000 7f81be7fc700 1 -- 172.21.15.94:0/1868419917 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f81b8056840 con 0x7f81c80fdc10 2024-04-06T03:50:12.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.742+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f81c804bed0 con 0x7f81c80fdc10 2024-04-06T03:50:12.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.742+0000 7f81be7fc700 1 -- 172.21.15.94:0/1868419917 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1940 (secure 0 0 0) 0x7f81b8059f00 con 0x7f81c80fdc10 2024-04-06T03:50:12.744 INFO:teuthology.orchestra.run.smithi094.stdout:e36 2024-04-06T03:50:12.744 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:50:12.744 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:50:12.744 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:50:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 35 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:49:01.609121+0000 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:50:12.745 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 82 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=34400} 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:50:12.746 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:50:12.747 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 34400 members: 34400 2024-04-06T03:50:12.747 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:34400} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:50:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:50:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:50:12.747 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:50:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:50:12.747 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/1650064917,v1:172.21.15.94:6829/1650064917] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:50:12.747 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.136:6824/1500356136,v1:172.21.15.136:6825/1500356136] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:50:12.747 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.136:6826/1840231705,v1:172.21.15.136:6827/1840231705] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:50:12.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f81b406dca0 msgr2=0x7f81b4070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:12.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f81b406dca0 0x7f81b4070150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f81c4004890 tx=0x7f81c4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:12.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fdc10 msgr2=0x7f81c8112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:12.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fdc10 0x7f81c8112cb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f81b8002a50 tx=0x7f81b8004600 comp rx=0 tx=0).stop 2024-04-06T03:50:12.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 shutdown_connections 2024-04-06T03:50:12.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f81b406dca0 0x7f81b4070150 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81c80fd2c0 0x7f81c8110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 --2- 172.21.15.94:0/1868419917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c80fdc10 0x7f81c8112cb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 >> 172.21.15.94:0/1868419917 conn(0x7f81c80f6ba0 msgr2=0x7f81c80f82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:12.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 shutdown_connections 2024-04-06T03:50:12.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.745+0000 7f81cfef6700 1 -- 172.21.15.94:0/1868419917 wait complete. 2024-04-06T03:50:12.749 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 36 2024-04-06T03:50:12.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.855+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2780937970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4066b10 msgr2=0x7fd5d40ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.855+0000 7fd5d8f24700 1 --2- 172.21.15.94:0/2780937970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4066b10 0x7fd5d40ffab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4004940 tx=0x7fd5c4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.855+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2780937970 shutdown_connections 2024-04-06T03:50:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.855+0000 7fd5d8f24700 1 --2- 172.21.15.94:0/2780937970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5d40ffff0 0x7fd5d41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.855+0000 7fd5d8f24700 1 --2- 172.21.15.94:0/2780937970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d4066b10 0x7fd5d40ffab0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.855+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2780937970 >> 172.21.15.94:0/2780937970 conn(0x7fd5d40f74e0 msgr2=0x7fd5d40f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.855+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2780937970 shutdown_connections 2024-04-06T03:50:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.855+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2780937970 wait complete. 2024-04-06T03:50:12.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.855+0000 7fd5d8f24700 1 Processor -- start 2024-04-06T03:50:12.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d8f24700 1 -- start start 2024-04-06T03:50:12.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d8f24700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5d4066b10 0x7fd5d410dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:12.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d8f24700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d40ffff0 0x7fd5d410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:12.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d8f24700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d410e6b0 con 0x7fd5d40ffff0 2024-04-06T03:50:12.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d8f24700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d410ee50 con 0x7fd5d4066b10 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d1d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d40ffff0 0x7fd5d410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d1d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d40ffff0 0x7fd5d410e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34620/0 (socket says 172.21.15.94:34620) 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d1d9b700 1 -- 172.21.15.94:0/2679580997 learned_addr learned my addr 172.21.15.94:0/2679580997 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d259c700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5d4066b10 0x7fd5d410dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d1d9b700 1 -- 172.21.15.94:0/2679580997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5d4066b10 msgr2=0x7fd5d410dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d1d9b700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5d4066b10 0x7fd5d410dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d1d9b700 1 -- 172.21.15.94:0/2679580997 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4005040 con 0x7fd5d40ffff0 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d259c700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5d4066b10 0x7fd5d410dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.856+0000 7fd5d1d9b700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d40ffff0 0x7fd5d410e020 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800b790 tx=0x7fd5c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.857+0000 7fd5c2ffd700 1 -- 172.21.15.94:0/2679580997 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5c800f770 con 0x7fd5d40ffff0 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.857+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d410f130 con 0x7fd5d40ffff0 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.857+0000 7fd5c2ffd700 1 -- 172.21.15.94:0/2679580997 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5c800fda0 con 0x7fd5d40ffff0 2024-04-06T03:50:12.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.857+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d410f6f0 con 0x7fd5d40ffff0 2024-04-06T03:50:12.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.857+0000 7fd5c2ffd700 1 -- 172.21.15.94:0/2679580997 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5c800d310 con 0x7fd5d40ffff0 2024-04-06T03:50:12.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.858+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b40052e0 con 0x7fd5d40ffff0 2024-04-06T03:50:12.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.859+0000 7fd5c2ffd700 1 -- 172.21.15.94:0/2679580997 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fd5c800f8d0 con 0x7fd5d40ffff0 2024-04-06T03:50:12.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.859+0000 7fd5c2ffd700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd5bc06daa0 0x7fd5bc06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:12.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.859+0000 7fd5c2ffd700 1 -- 172.21.15.94:0/2679580997 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd5c808d660 con 0x7fd5d40ffff0 2024-04-06T03:50:12.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.859+0000 7fd5d259c700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd5bc06daa0 0x7fd5bc06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:12.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.860+0000 7fd5d259c700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd5bc06daa0 0x7fd5bc06ff50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b3b0 tx=0x7fd5c4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:12.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:12.868+0000 7fd5c2ffd700 1 -- 172.21.15.94:0/2679580997 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd5c805a6f0 con 0x7fd5d40ffff0 2024-04-06T03:50:13.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.089+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5b4000bf0 con 0x7fd5bc06daa0 2024-04-06T03:50:13.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.090+0000 7fd5c2ffd700 1 -- 172.21.15.94:0/2679580997 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7fd5b4000bf0 con 0x7fd5bc06daa0 2024-04-06T03:50:13.092 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:50:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c", 2024-04-06T03:50:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": true, 2024-04-06T03:50:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-06T03:50:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [ 2024-04-06T03:50:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-06T03:50:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "osd", 2024-04-06T03:50:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr", 2024-04-06T03:50:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "mds", 2024-04-06T03:50:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "mon" 2024-04-06T03:50:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-06T03:50:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": "16/21 daemons upgraded", 2024-04-06T03:50:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "Currently upgrading grafana daemons", 2024-04-06T03:50:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:50:13.093 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:50:13.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.092+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd5bc06daa0 msgr2=0x7fd5bc06ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:13.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.092+0000 7fd5d8f24700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd5bc06daa0 0x7fd5bc06ff50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b3b0 tx=0x7fd5c4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:13.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.092+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d40ffff0 msgr2=0x7fd5d410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.092+0000 7fd5d8f24700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d40ffff0 0x7fd5d410e020 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800b790 tx=0x7fd5c8009250 comp rx=0 tx=0).stop 2024-04-06T03:50:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.092+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 shutdown_connections 2024-04-06T03:50:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.092+0000 7fd5d8f24700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd5bc06daa0 0x7fd5bc06ff50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.092+0000 7fd5d8f24700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5d4066b10 0x7fd5d410dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.092+0000 7fd5d8f24700 1 --2- 172.21.15.94:0/2679580997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d40ffff0 0x7fd5d410e020 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.092+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 >> 172.21.15.94:0/2679580997 conn(0x7fd5d40f74e0 msgr2=0x7fd5d40f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.093+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 shutdown_connections 2024-04-06T03:50:13.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.093+0000 7fd5d8f24700 1 -- 172.21.15.94:0/2679580997 wait complete. 2024-04-06T03:50:13.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:13 smithi094 ceph-mon[94053]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:13.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:13 smithi094 ceph-mon[94053]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:13.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:13 smithi094 ceph-mon[94053]: pgmap v273: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:13.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:13 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/881295843' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:13.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:13 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1868419917' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:50:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/1658411435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0066280 msgr2=0x7fb8c0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 --2- 172.21.15.94:0/1658411435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0066280 0x7fb8c0101e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4004940 tx=0x7fb8b4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/1658411435 shutdown_connections 2024-04-06T03:50:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 --2- 172.21.15.94:0/1658411435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8c0102360 0x7fb8c0104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 --2- 172.21.15.94:0/1658411435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0066280 0x7fb8c0101e20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/1658411435 >> 172.21.15.94:0/1658411435 conn(0x7fb8c00f7430 msgr2=0x7fb8c00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/1658411435 shutdown_connections 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/1658411435 wait complete. 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 Processor -- start 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.197+0000 7fb8c6b0b700 1 -- start start 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8c6b0b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8c0066280 0x7fb8c010fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8c6b0b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0102360 0x7fb8c0110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8c6b0b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c0110850 con 0x7fb8c0102360 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8c6b0b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c0110f90 con 0x7fb8c0066280 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8bffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0102360 0x7fb8c0110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8bffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0102360 0x7fb8c0110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34628/0 (socket says 172.21.15.94:34628) 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8bffff700 1 -- 172.21.15.94:0/732340401 learned_addr learned my addr 172.21.15.94:0/732340401 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8c48a7700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8c0066280 0x7fb8c010fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:13.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8bffff700 1 -- 172.21.15.94:0/732340401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8c0066280 msgr2=0x7fb8c010fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:13.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8bffff700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8c0066280 0x7fb8c010fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8bffff700 1 -- 172.21.15.94:0/732340401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4005040 con 0x7fb8c0102360 2024-04-06T03:50:13.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.198+0000 7fb8bffff700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0102360 0x7fb8c0110250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac002a50 tx=0x7fb8ac0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:13.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.199+0000 7fb8bd7fa700 1 -- 172.21.15.94:0/732340401 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8ac007cc0 con 0x7fb8c0102360 2024-04-06T03:50:13.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.199+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c0111270 con 0x7fb8c0102360 2024-04-06T03:50:13.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.199+0000 7fb8bd7fa700 1 -- 172.21.15.94:0/732340401 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8ac00f040 con 0x7fb8c0102360 2024-04-06T03:50:13.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.199+0000 7fb8bd7fa700 1 -- 172.21.15.94:0/732340401 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8ac00da30 con 0x7fb8c0102360 2024-04-06T03:50:13.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.199+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c0111830 con 0x7fb8c0102360 2024-04-06T03:50:13.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.200+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c005f5f0 con 0x7fb8c0102360 2024-04-06T03:50:13.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.200+0000 7fb8bd7fa700 1 -- 172.21.15.94:0/732340401 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fb8ac00db90 con 0x7fb8c0102360 2024-04-06T03:50:13.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.200+0000 7fb8bd7fa700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8b006dca0 0x7fb8b0070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:13.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.201+0000 7fb8bd7fa700 1 -- 172.21.15.94:0/732340401 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fb8ac08d3b0 con 0x7fb8c0102360 2024-04-06T03:50:13.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.201+0000 7fb8c48a7700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8b006dca0 0x7fb8b0070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:13.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.201+0000 7fb8c48a7700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8b006dca0 0x7fb8b0070150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4004890 tx=0x7fb8b4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:13.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.205+0000 7fb8bd7fa700 1 -- 172.21.15.94:0/732340401 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb8ac05a3c0 con 0x7fb8c0102360 2024-04-06T03:50:13.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:13 smithi136 ceph-mon[82061]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:13.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:13 smithi136 ceph-mon[82061]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:13.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:13 smithi136 ceph-mon[82061]: pgmap v273: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:13.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:13 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/881295843' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:13.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:13 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1868419917' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:50:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.537+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb8c0111af0 con 0x7fb8c0102360 2024-04-06T03:50:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.537+0000 7fb8bd7fa700 1 -- 172.21.15.94:0/732340401 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+155 (secure 0 0 0) 0x7fb8ac059f50 con 0x7fb8c0102360 2024-04-06T03:50:13.539 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-06T03:50:13.539 INFO:teuthology.orchestra.run.smithi094.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-06T03:50:13.540 INFO:teuthology.orchestra.run.smithi094.stdout: daemon grafana.smithi094 on smithi094 is in error state 2024-04-06T03:50:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.539+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8b006dca0 msgr2=0x7fb8b0070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.539+0000 7fb8c6b0b700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8b006dca0 0x7fb8b0070150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4004890 tx=0x7fb8b4009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.540+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0102360 msgr2=0x7fb8c0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.540+0000 7fb8c6b0b700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0102360 0x7fb8c0110250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac002a50 tx=0x7fb8ac0046b0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.540+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 shutdown_connections 2024-04-06T03:50:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.540+0000 7fb8c6b0b700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fb8b006dca0 0x7fb8b0070150 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.540+0000 7fb8c6b0b700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8c0066280 0x7fb8c010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.540+0000 7fb8c6b0b700 1 --2- 172.21.15.94:0/732340401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8c0102360 0x7fb8c0110250 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.540+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 >> 172.21.15.94:0/732340401 conn(0x7fb8c00f7430 msgr2=0x7fb8c00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.540+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 shutdown_connections 2024-04-06T03:50:13.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:13.540+0000 7fb8c6b0b700 1 -- 172.21.15.94:0/732340401 wait complete. 2024-04-06T03:50:14.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:14 smithi136 ceph-mon[82061]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:14.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:14 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/732340401' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:50:14.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:14 smithi094 ceph-mon[94053]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:14.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:14 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/732340401' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:50:15.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:15 smithi094 ceph-mon[94053]: pgmap v274: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:15.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:15 smithi136 ceph-mon[82061]: pgmap v274: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:16.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: pgmap v275: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.668 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: Upgrade: Finalizing container_image settings 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-06T03:50:16.669 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.670 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.671 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: Upgrade: Complete! 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:50:16.672 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.851 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:50:16.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:50:16.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: pgmap v275: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:16.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:50:16.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-06T03:50:16.852 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:16.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.853 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.854 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.854 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.854 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.854 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.854 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.855 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.855 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.855 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.855 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.855 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.855 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.855 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:16.856 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: Upgrade: Finalizing container_image settings 2024-04-06T03:50:16.856 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.856 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-06T03:50:16.856 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-06T03:50:16.856 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.857 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-06T03:50:16.857 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-06T03:50:16.857 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-06T03:50:16.857 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-06T03:50:16.857 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-06T03:50:16.857 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-06T03:50:16.858 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-06T03:50:16.858 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-06T03:50:16.858 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-06T03:50:16.858 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-06T03:50:16.858 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-06T03:50:16.858 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.858 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-06T03:50:16.859 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-06T03:50:16.859 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-06T03:50:16.859 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-06T03:50:16.859 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-06T03:50:16.859 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-06T03:50:16.859 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-06T03:50:16.860 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-06T03:50:16.860 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.860 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.860 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.860 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.860 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.860 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-06T03:50:16.861 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: Upgrade: Complete! 2024-04-06T03:50:16.861 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-06T03:50:16.861 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-06T03:50:16.861 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:16.861 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:50:16.861 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:50:16.861 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:16.862 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:50:16.862 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:50:16.862 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:50:16.862 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:17.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:17 smithi094 ceph-mon[94053]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-06T03:50:17.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:17 smithi094 ceph-mon[94053]: Cluster is now healthy 2024-04-06T03:50:17.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:17 smithi136 ceph-mon[82061]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-06T03:50:17.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:17 smithi136 ceph-mon[82061]: Cluster is now healthy 2024-04-06T03:50:18.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:18 smithi094 ceph-mon[94053]: pgmap v276: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:18.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:18 smithi136 ceph-mon[82061]: pgmap v276: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:21.551 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:21 smithi136 ceph-mon[82061]: pgmap v277: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:21.551 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:21.552 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:50:21.552 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:21 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:21.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:21 smithi094 ceph-mon[94053]: pgmap v277: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:21.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:21.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:50:21.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:21 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:50:23.247 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:23 smithi094 ceph-mon[94053]: pgmap v278: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:23.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:23 smithi136 ceph-mon[82061]: pgmap v278: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:25.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:25 smithi136 ceph-mon[82061]: pgmap v279: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:25.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:25 smithi094 ceph-mon[94053]: pgmap v279: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:27.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:27 smithi136 ceph-mon[82061]: pgmap v280: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:27.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:27 smithi094 ceph-mon[94053]: pgmap v280: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:29.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:29 smithi136 ceph-mon[82061]: pgmap v281: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:29.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:29 smithi094 ceph-mon[94053]: pgmap v281: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:31.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:31 smithi136 ceph-mon[82061]: pgmap v282: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:31.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:31 smithi094 ceph-mon[94053]: pgmap v282: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:33.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:33 smithi136 ceph-mon[82061]: pgmap v283: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:33.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:33 smithi094 ceph-mon[94053]: pgmap v283: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:35.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:35 smithi136 ceph-mon[82061]: pgmap v284: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:35.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:35 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:50:35.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:35 smithi094 ceph-mon[94053]: pgmap v284: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:35.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:35 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:50:37.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:37 smithi136 ceph-mon[82061]: pgmap v285: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:37.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:37 smithi094 ceph-mon[94053]: pgmap v285: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:39.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:39 smithi136 ceph-mon[82061]: pgmap v286: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:39.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:39 smithi094 ceph-mon[94053]: pgmap v286: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:41.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:41 smithi136 ceph-mon[82061]: pgmap v287: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:41.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:41 smithi094 ceph-mon[94053]: pgmap v287: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:43.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:43 smithi136 ceph-mon[82061]: pgmap v288: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:43.646 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:43 smithi094 ceph-mon[94053]: pgmap v288: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.644+0000 7f007e9d9700 1 -- 172.21.15.94:0/3048886754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 msgr2=0x7f00780ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.644+0000 7f007e9d9700 1 --2- 172.21.15.94:0/3048886754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 0x7f00780ffa30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0068004940 tx=0x7f0068009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.644+0000 7f007e9d9700 1 -- 172.21.15.94:0/3048886754 shutdown_connections 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.644+0000 7f007e9d9700 1 --2- 172.21.15.94:0/3048886754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00780fff70 0x7f0078102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.644+0000 7f007e9d9700 1 --2- 172.21.15.94:0/3048886754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 0x7f00780ffa30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.644+0000 7f007e9d9700 1 -- 172.21.15.94:0/3048886754 >> 172.21.15.94:0/3048886754 conn(0x7f00780f7480 msgr2=0x7f00780f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.644+0000 7f007e9d9700 1 -- 172.21.15.94:0/3048886754 shutdown_connections 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.644+0000 7f007e9d9700 1 -- 172.21.15.94:0/3048886754 wait complete. 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f007e9d9700 1 Processor -- start 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f007e9d9700 1 -- start start 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f007e9d9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 0x7f007810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f007e9d9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00780fff70 0x7f007810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:43.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f007e9d9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007810e5a0 con 0x7f0078066b10 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f007e9d9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007810edf0 con 0x7f00780fff70 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f0077fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 0x7f007810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f0077fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 0x7f007810da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41128/0 (socket says 172.21.15.94:41128) 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f0077fff700 1 -- 172.21.15.94:0/1772797249 learned_addr learned my addr 172.21.15.94:0/1772797249 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.645+0000 7f00777fe700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00780fff70 0x7f007810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f0077fff700 1 -- 172.21.15.94:0/1772797249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00780fff70 msgr2=0x7f007810dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f0077fff700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00780fff70 0x7f007810dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f0077fff700 1 -- 172.21.15.94:0/1772797249 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068005040 con 0x7f0078066b10 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f00777fe700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00780fff70 0x7f007810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f0077fff700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 0x7f007810da60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f006800b3b0 tx=0x7f0068009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f0074ff9700 1 -- 172.21.15.94:0/1772797249 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006800e040 con 0x7f0078066b10 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f0074ff9700 1 -- 172.21.15.94:0/1772797249 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f006800bcc0 con 0x7f0078066b10 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007810f070 con 0x7f0078066b10 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f0074ff9700 1 -- 172.21.15.94:0/1772797249 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006801c630 con 0x7f0078066b10 2024-04-06T03:50:43.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.646+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007810f5b0 con 0x7f0078066b10 2024-04-06T03:50:43.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.647+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007805f5f0 con 0x7f0078066b10 2024-04-06T03:50:43.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.648+0000 7f0074ff9700 1 -- 172.21.15.94:0/1772797249 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f00680139f0 con 0x7f0078066b10 2024-04-06T03:50:43.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.648+0000 7f0074ff9700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f006006dca0 0x7f0060070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:43.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.648+0000 7f00777fe700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f006006dca0 0x7f0060070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:43.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.648+0000 7f0074ff9700 1 -- 172.21.15.94:0/1772797249 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f006808e300 con 0x7f0078066b10 2024-04-06T03:50:43.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.649+0000 7f00777fe700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f006006dca0 0x7f0060070150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f006c0041d0 tx=0x7f006c009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.656+0000 7f0074ff9700 1 -- 172.21.15.94:0/1772797249 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f006805b390 con 0x7f0078066b10 2024-04-06T03:50:43.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.876+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f007810f880 con 0x7f006006dca0 2024-04-06T03:50:43.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.877+0000 7f0074ff9700 1 -- 172.21.15.94:0/1772797249 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f007810f880 con 0x7f006006dca0 2024-04-06T03:50:43.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.879+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f006006dca0 msgr2=0x7f0060070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:43.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.879+0000 7f007e9d9700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f006006dca0 0x7f0060070150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f006c0041d0 tx=0x7f006c009040 comp rx=0 tx=0).stop 2024-04-06T03:50:43.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.879+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 msgr2=0x7f007810da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:43.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.879+0000 7f007e9d9700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 0x7f007810da60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f006800b3b0 tx=0x7f0068009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:43.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.879+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 shutdown_connections 2024-04-06T03:50:43.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.879+0000 7f007e9d9700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0078066b10 0x7f007810da60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:43.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.879+0000 7f007e9d9700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f006006dca0 0x7f0060070150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:43.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.879+0000 7f007e9d9700 1 --2- 172.21.15.94:0/1772797249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00780fff70 0x7f007810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:43.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.879+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 >> 172.21.15.94:0/1772797249 conn(0x7f00780f7480 msgr2=0x7f00780f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:43.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.880+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 shutdown_connections 2024-04-06T03:50:43.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:43.880+0000 7f007e9d9700 1 -- 172.21.15.94:0/1772797249 wait complete. 2024-04-06T03:50:44.670 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph orch ps' 2024-04-06T03:50:44.990 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:50:45.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:45 smithi094 ceph-mon[94053]: pgmap v289: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:45.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:45 smithi094 ceph-mon[94053]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:45.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:45 smithi136 ceph-mon[82061]: pgmap v289: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:45.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:45 smithi136 ceph-mon[82061]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:46.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.596+0000 7ff5788ee700 1 -- 172.21.15.94:0/3581342845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5741026c0 msgr2=0x7ff5740fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:46.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.596+0000 7ff5788ee700 1 --2- 172.21.15.94:0/3581342845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5741026c0 0x7ff5740fbd30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff568004940 tx=0x7ff568009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.596+0000 7ff5788ee700 1 -- 172.21.15.94:0/3581342845 shutdown_connections 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.597+0000 7ff5788ee700 1 --2- 172.21.15.94:0/3581342845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5741026c0 0x7ff5740fbd30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.597+0000 7ff5788ee700 1 --2- 172.21.15.94:0/3581342845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff574101d20 0x7ff5741020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.597+0000 7ff5788ee700 1 -- 172.21.15.94:0/3581342845 >> 172.21.15.94:0/3581342845 conn(0x7ff5740f7870 msgr2=0x7ff5740f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.597+0000 7ff5788ee700 1 -- 172.21.15.94:0/3581342845 shutdown_connections 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.597+0000 7ff5788ee700 1 -- 172.21.15.94:0/3581342845 wait complete. 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.597+0000 7ff5788ee700 1 Processor -- start 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.597+0000 7ff5788ee700 1 -- start start 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff5788ee700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574101d20 0x7ff574104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff5788ee700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5741026c0 0x7ff574102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff5788ee700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574103150 con 0x7ff574101d20 2024-04-06T03:50:46.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff5788ee700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741032c0 con 0x7ff5741026c0 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff57259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574101d20 0x7ff574104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff57259c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574101d20 0x7ff574104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41148/0 (socket says 172.21.15.94:41148) 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff57259c700 1 -- 172.21.15.94:0/472098677 learned_addr learned my addr 172.21.15.94:0/472098677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff571d9b700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5741026c0 0x7ff574102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff57259c700 1 -- 172.21.15.94:0/472098677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5741026c0 msgr2=0x7ff574102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff57259c700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5741026c0 0x7ff574102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff57259c700 1 -- 172.21.15.94:0/472098677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff568005040 con 0x7ff574101d20 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff57259c700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574101d20 0x7ff574104610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff56400b790 tx=0x7ff564009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff562ffd700 1 -- 172.21.15.94:0/472098677 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff56400f6f0 con 0x7ff574101d20 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5741035a0 con 0x7ff574101d20 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff562ffd700 1 -- 172.21.15.94:0/472098677 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff56400fd20 con 0x7ff574101d20 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.598+0000 7ff562ffd700 1 -- 172.21.15.94:0/472098677 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5640103a0 con 0x7ff574101d20 2024-04-06T03:50:46.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.599+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57411e690 con 0x7ff574101d20 2024-04-06T03:50:46.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.599+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5740fd470 con 0x7ff574101d20 2024-04-06T03:50:46.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.600+0000 7ff562ffd700 1 -- 172.21.15.94:0/472098677 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7ff564011400 con 0x7ff574101d20 2024-04-06T03:50:46.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.600+0000 7ff562ffd700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff55c06dca0 0x7ff55c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:46.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.600+0000 7ff562ffd700 1 -- 172.21.15.94:0/472098677 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff56408d420 con 0x7ff574101d20 2024-04-06T03:50:46.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.601+0000 7ff571d9b700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff55c06dca0 0x7ff55c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:46.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.601+0000 7ff571d9b700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff55c06dca0 0x7ff55c070150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5741042b0 tx=0x7ff568009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:46.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.605+0000 7ff562ffd700 1 -- 172.21.15.94:0/472098677 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff56405a4b0 con 0x7ff574101d20 2024-04-06T03:50:46.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.815+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5740669d0 con 0x7ff55c06dca0 2024-04-06T03:50:46.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.826+0000 7ff562ffd700 1 -- 172.21.15.94:0/472098677 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff5740669d0 con 0x7ff55c06dca0 2024-04-06T03:50:46.827 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (56s) 36s ago 18m 17.1M - 0.25.0 c8568f914cd2 f3dd69f56752 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (6m) 36s ago 18m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (6m) 85s ago 17m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (41s) 36s ago 17m 52.9M - 9.4.12 2bacad6d85d8 550bb3828233 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (2m) 36s ago 12m 15.5M - 19.0.0-2027-g9f192350 a5511c7deba9 bfca2184eb3c 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (3m) 36s ago 12m 83.0M - 19.0.0-2027-g9f192350 a5511c7deba9 c189927827c8 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (2m) 85s ago 12m 14.9M - 19.0.0-2027-g9f192350 a5511c7deba9 6dd3b4023d80 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (104s) 85s ago 12m 14.0M - 19.0.0-2027-g9f192350 a5511c7deba9 432ef9195992 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (8m) 36s ago 19m 516M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (8m) 85s ago 17m 452M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (7m) 36s ago 19m 55.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (7m) 85s ago 17m 44.0M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (92s) 36s ago 18m 13.5M - 1.5.0 0da6a335fe13 21b4e6161925 2024-04-06T03:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (88s) 85s ago 17m 3594k - 1.5.0 0da6a335fe13 a1730b2e350a 2024-04-06T03:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (6m) 36s ago 16m 112M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (5m) 36s ago 15m 124M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (5m) 36s ago 15m 108M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (4m) 85s ago 15m 123M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (3m) 85s ago 14m 108M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 c31367e1c2d5 2024-04-06T03:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (3m) 85s ago 14m 109M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 16386e8f8583 2024-04-06T03:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (72s) 36s ago 17m 49.1M - 2.43.0 a07b618ecd1d 4cad242accb3 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff55c06dca0 msgr2=0x7ff55c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff55c06dca0 0x7ff55c070150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5741042b0 tx=0x7ff568009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574101d20 msgr2=0x7ff574104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574101d20 0x7ff574104610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff56400b790 tx=0x7ff564009250 comp rx=0 tx=0).stop 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 shutdown_connections 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff574101d20 0x7ff574104610 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff55c06dca0 0x7ff55c070150 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 --2- 172.21.15.94:0/472098677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5741026c0 0x7ff574102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 >> 172.21.15.94:0/472098677 conn(0x7ff5740f7870 msgr2=0x7ff5740662c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 shutdown_connections 2024-04-06T03:50:46.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:46.828+0000 7ff5788ee700 1 -- 172.21.15.94:0/472098677 wait complete. 2024-04-06T03:50:47.511 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph orch upgrade status' 2024-04-06T03:50:47.524 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:47 smithi094 ceph-mon[94053]: pgmap v290: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:47.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:47 smithi136 ceph-mon[82061]: pgmap v290: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:47.799 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:50:48.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:48 smithi094 ceph-mon[94053]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:48.597 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:48 smithi136 ceph-mon[82061]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:49.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.468+0000 7f417af95700 1 -- 172.21.15.94:0/113653211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 msgr2=0x7f41741031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:49.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.468+0000 7f417af95700 1 --2- 172.21.15.94:0/113653211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 0x7f41741031d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4168004940 tx=0x7f4168009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:49.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.468+0000 7f417af95700 1 -- 172.21.15.94:0/113653211 shutdown_connections 2024-04-06T03:50:49.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.468+0000 7f417af95700 1 --2- 172.21.15.94:0/113653211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41741037a0 0x7f41740fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:49.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.468+0000 7f417af95700 1 --2- 172.21.15.94:0/113653211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 0x7f41741031d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:49.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.468+0000 7f417af95700 1 -- 172.21.15.94:0/113653211 >> 172.21.15.94:0/113653211 conn(0x7f41740f89b0 msgr2=0x7f41740fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:49.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.468+0000 7f417af95700 1 -- 172.21.15.94:0/113653211 shutdown_connections 2024-04-06T03:50:49.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.468+0000 7f417af95700 1 -- 172.21.15.94:0/113653211 wait complete. 2024-04-06T03:50:49.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f417af95700 1 Processor -- start 2024-04-06T03:50:49.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f417af95700 1 -- start start 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f417af95700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 0x7f4174113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f417af95700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41741037a0 0x7f4174113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f417af95700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4174114520 con 0x7f4174102e00 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f417af95700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417410db70 con 0x7f41741037a0 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f4178d31700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 0x7f4174113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f4178d31700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 0x7f4174113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58358/0 (socket says 172.21.15.94:58358) 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f4178d31700 1 -- 172.21.15.94:0/774274746 learned_addr learned my addr 172.21.15.94:0/774274746 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.469+0000 7f4173fff700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41741037a0 0x7f4174113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.470+0000 7f4178d31700 1 -- 172.21.15.94:0/774274746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41741037a0 msgr2=0x7f4174113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.470+0000 7f4178d31700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41741037a0 0x7f4174113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.470+0000 7f4178d31700 1 -- 172.21.15.94:0/774274746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4168005040 con 0x7f4174102e00 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.470+0000 7f4178d31700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 0x7f4174113990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f416800b3b0 tx=0x7f4168009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.470+0000 7f41717fa700 1 -- 172.21.15.94:0/774274746 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f416800e040 con 0x7f4174102e00 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.470+0000 7f41717fa700 1 -- 172.21.15.94:0/774274746 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4168013420 con 0x7f4174102e00 2024-04-06T03:50:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.470+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417410ddf0 con 0x7f4174102e00 2024-04-06T03:50:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.470+0000 7f41717fa700 1 -- 172.21.15.94:0/774274746 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f416801c630 con 0x7f4174102e00 2024-04-06T03:50:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.470+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417410e3b0 con 0x7f4174102e00 2024-04-06T03:50:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.471+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4174100110 con 0x7f4174102e00 2024-04-06T03:50:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.471+0000 7f41717fa700 1 -- 172.21.15.94:0/774274746 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f4168013590 con 0x7f4174102e00 2024-04-06T03:50:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.472+0000 7f41717fa700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f416406dca0 0x7f4164070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.472+0000 7f41717fa700 1 -- 172.21.15.94:0/774274746 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f416808e7f0 con 0x7f4174102e00 2024-04-06T03:50:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.472+0000 7f4173fff700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f416406dca0 0x7f4164070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.472+0000 7f4173fff700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f416406dca0 0x7f4164070150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f417410ece0 tx=0x7f416000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:49.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.477+0000 7f41717fa700 1 -- 172.21.15.94:0/774274746 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f416805b880 con 0x7f4174102e00 2024-04-06T03:50:49.597 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:49 smithi136 ceph-mon[82061]: pgmap v291: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:49.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:49 smithi094 ceph-mon[94053]: pgmap v291: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:49.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.785+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f417410f050 con 0x7f416406dca0 2024-04-06T03:50:49.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.786+0000 7f41717fa700 1 -- 172.21.15.94:0/774274746 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f417410f050 con 0x7f416406dca0 2024-04-06T03:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "target_image": null, 2024-04-06T03:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "in_progress": false, 2024-04-06T03:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "which": "", 2024-04-06T03:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "services_complete": [], 2024-04-06T03:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "progress": null, 2024-04-06T03:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "message": "", 2024-04-06T03:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "is_paused": false 2024-04-06T03:50:49.788 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f416406dca0 msgr2=0x7f4164070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f416406dca0 0x7f4164070150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f417410ece0 tx=0x7f416000b040 comp rx=0 tx=0).stop 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 msgr2=0x7f4174113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 0x7f4174113990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f416800b3b0 tx=0x7f4168009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 shutdown_connections 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4174102e00 0x7f4174113990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f416406dca0 0x7f4164070150 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 --2- 172.21.15.94:0/774274746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41741037a0 0x7f4174113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 >> 172.21.15.94:0/774274746 conn(0x7f41740f89b0 msgr2=0x7f41741073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:49.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 shutdown_connections 2024-04-06T03:50:49.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:49.788+0000 7f417af95700 1 -- 172.21.15.94:0/774274746 wait complete. 2024-04-06T03:50:50.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:50:50.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph health detail' 2024-04-06T03:50:50.597 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:50:50.868 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:50:51.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:51 smithi136 ceph-mon[82061]: pgmap v292: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:51.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:51 smithi136 ceph-mon[82061]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:51.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:51 smithi094 ceph-mon[94053]: pgmap v292: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:51.667 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:51 smithi094 ceph-mon[94053]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:50:52.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.320+0000 7f951865d700 1 -- 172.21.15.94:0/744806439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95101037a0 msgr2=0x7f95100fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:52.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.320+0000 7f951865d700 1 --2- 172.21.15.94:0/744806439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95101037a0 0x7f95100fce10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f950c004940 tx=0x7f950c009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.320+0000 7f951865d700 1 -- 172.21.15.94:0/744806439 shutdown_connections 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.320+0000 7f951865d700 1 --2- 172.21.15.94:0/744806439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f95101037a0 0x7f95100fce10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.320+0000 7f951865d700 1 --2- 172.21.15.94:0/744806439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9510102e00 0x7f95101031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.320+0000 7f951865d700 1 -- 172.21.15.94:0/744806439 >> 172.21.15.94:0/744806439 conn(0x7f95100f89b0 msgr2=0x7f95100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.320+0000 7f951865d700 1 -- 172.21.15.94:0/744806439 shutdown_connections 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.320+0000 7f951865d700 1 -- 172.21.15.94:0/744806439 wait complete. 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f951865d700 1 Processor -- start 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f951865d700 1 -- start start 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f951865d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9510102e00 0x7f9510113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f951865d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95101037a0 0x7f95101140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f951865d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9510114700 con 0x7f9510102e00 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f951865d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951010daf0 con 0x7f95101037a0 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f95163f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9510102e00 0x7f9510113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:52.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f95163f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9510102e00 0x7f9510113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58370/0 (socket says 172.21.15.94:58370) 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f95163f9700 1 -- 172.21.15.94:0/3404090070 learned_addr learned my addr 172.21.15.94:0/3404090070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f95163f9700 1 -- 172.21.15.94:0/3404090070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95101037a0 msgr2=0x7f95101140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f95163f9700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95101037a0 0x7f95101140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f95163f9700 1 -- 172.21.15.94:0/3404090070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c005040 con 0x7f9510102e00 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.321+0000 7f95163f9700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9510102e00 0x7f9510113b70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9500002a50 tx=0x7f95000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.322+0000 7f9506ffd700 1 -- 172.21.15.94:0/3404090070 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9500007cc0 con 0x7f9510102e00 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.322+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951010ddd0 con 0x7f9510102e00 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.322+0000 7f9506ffd700 1 -- 172.21.15.94:0/3404090070 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f950000f040 con 0x7f9510102e00 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.322+0000 7f9506ffd700 1 -- 172.21.15.94:0/3404090070 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9500009640 con 0x7f9510102e00 2024-04-06T03:50:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.322+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951010e390 con 0x7f9510102e00 2024-04-06T03:50:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.323+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95101052d0 con 0x7f9510102e00 2024-04-06T03:50:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.323+0000 7f9506ffd700 1 -- 172.21.15.94:0/3404090070 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f95000097a0 con 0x7f9510102e00 2024-04-06T03:50:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.324+0000 7f9506ffd700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f94fc06dca0 0x7f94fc070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.324+0000 7f9515bf8700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f94fc06dca0 0x7f94fc070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.324+0000 7f9506ffd700 1 -- 172.21.15.94:0/3404090070 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f950008d340 con 0x7f9510102e00 2024-04-06T03:50:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.324+0000 7f9515bf8700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f94fc06dca0 0x7f94fc070150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f950c00b3b0 tx=0x7f950c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:52.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.333+0000 7f9506ffd700 1 -- 172.21.15.94:0/3404090070 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f950005a3d0 con 0x7f9510102e00 2024-04-06T03:50:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.616+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9510109ee0 con 0x7f9510102e00 2024-04-06T03:50:52.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.616+0000 7f9506ffd700 1 -- 172.21.15.94:0/3404090070 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9500056030 con 0x7f9510102e00 2024-04-06T03:50:52.618 INFO:teuthology.orchestra.run.smithi094.stdout:HEALTH_OK 2024-04-06T03:50:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.618+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f94fc06dca0 msgr2=0x7f94fc070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.618+0000 7f951865d700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f94fc06dca0 0x7f94fc070150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f950c00b3b0 tx=0x7f950c009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.618+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9510102e00 msgr2=0x7f9510113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.618+0000 7f951865d700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9510102e00 0x7f9510113b70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9500002a50 tx=0x7f95000046b0 comp rx=0 tx=0).stop 2024-04-06T03:50:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.619+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 shutdown_connections 2024-04-06T03:50:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.619+0000 7f951865d700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9510102e00 0x7f9510113b70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:52.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.619+0000 7f951865d700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f94fc06dca0 0x7f94fc070150 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:52.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.619+0000 7f951865d700 1 --2- 172.21.15.94:0/3404090070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95101037a0 0x7f95101140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:52.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.619+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 >> 172.21.15.94:0/3404090070 conn(0x7f95100f89b0 msgr2=0x7f9510102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:52.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.619+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 shutdown_connections 2024-04-06T03:50:52.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:52.619+0000 7f951865d700 1 -- 172.21.15.94:0/3404090070 wait complete. 2024-04-06T03:50:53.283 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph versions' 2024-04-06T03:50:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:50:53.597 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:53 smithi136 ceph-mon[82061]: pgmap v293: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:53.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:53 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3404090070' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:50:53.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:53 smithi094 ceph-mon[94053]: pgmap v293: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:53.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:53 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3404090070' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-06T03:50:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.158+0000 7fd885f4c700 1 -- 172.21.15.94:0/2847496151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102710 msgr2=0x7fd8800fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.158+0000 7fd885f4c700 1 --2- 172.21.15.94:0/2847496151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102710 0x7fd8800fbd80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd868004940 tx=0x7fd868009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.158+0000 7fd885f4c700 1 -- 172.21.15.94:0/2847496151 shutdown_connections 2024-04-06T03:50:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.158+0000 7fd885f4c700 1 --2- 172.21.15.94:0/2847496151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102710 0x7fd8800fbd80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.158+0000 7fd885f4c700 1 --2- 172.21.15.94:0/2847496151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880101d70 0x7fd880102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:55.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.158+0000 7fd885f4c700 1 -- 172.21.15.94:0/2847496151 >> 172.21.15.94:0/2847496151 conn(0x7fd880072bc0 msgr2=0x7fd880072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.158+0000 7fd885f4c700 1 -- 172.21.15.94:0/2847496151 shutdown_connections 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.158+0000 7fd885f4c700 1 -- 172.21.15.94:0/2847496151 wait complete. 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd885f4c700 1 Processor -- start 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd885f4c700 1 -- start start 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd885f4c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880101d70 0x7fd88010eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd885f4c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102710 0x7fd88010f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd885f4c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88010f6d0 con 0x7fd880101d70 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd885f4c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88010f810 con 0x7fd880102710 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880101d70 0x7fd88010eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880101d70 0x7fd88010eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58392/0 (socket says 172.21.15.94:58392) 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd87f7fe700 1 -- 172.21.15.94:0/2917691590 learned_addr learned my addr 172.21.15.94:0/2917691590 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.159+0000 7fd877fff700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102710 0x7fd88010f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:55.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.160+0000 7fd87f7fe700 1 -- 172.21.15.94:0/2917691590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102710 msgr2=0x7fd88010f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.160+0000 7fd87f7fe700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102710 0x7fd88010f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.160+0000 7fd87f7fe700 1 -- 172.21.15.94:0/2917691590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868005040 con 0x7fd880101d70 2024-04-06T03:50:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.160+0000 7fd87f7fe700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880101d70 0x7fd88010eb40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd87000c8c0 tx=0x7fd8700044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.160+0000 7fd87cff9700 1 -- 172.21.15.94:0/2917691590 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd87000a980 con 0x7fd880101d70 2024-04-06T03:50:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.160+0000 7fd87cff9700 1 -- 172.21.15.94:0/2917691590 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd87000ce60 con 0x7fd880101d70 2024-04-06T03:50:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.160+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd880117bb0 con 0x7fd880101d70 2024-04-06T03:50:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.160+0000 7fd87cff9700 1 -- 172.21.15.94:0/2917691590 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8700115e0 con 0x7fd880101d70 2024-04-06T03:50:55.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.160+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd880118110 con 0x7fd880101d70 2024-04-06T03:50:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.161+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd880104240 con 0x7fd880101d70 2024-04-06T03:50:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.161+0000 7fd87cff9700 1 -- 172.21.15.94:0/2917691590 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fd87000aae0 con 0x7fd880101d70 2024-04-06T03:50:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.162+0000 7fd87cff9700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd86c06dca0 0x7fd86c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:50:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.162+0000 7fd87cff9700 1 -- 172.21.15.94:0/2917691590 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd87008d710 con 0x7fd880101d70 2024-04-06T03:50:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.162+0000 7fd877fff700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd86c06dca0 0x7fd86c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:50:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.162+0000 7fd877fff700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd86c06dca0 0x7fd86c070150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd86800b3b0 tx=0x7fd868009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:50:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.170+0000 7fd87cff9700 1 -- 172.21.15.94:0/2917691590 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd870056c70 con 0x7fd880101d70 2024-04-06T03:50:55.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.457+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd880108ee0 con 0x7fd880101d70 2024-04-06T03:50:55.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.458+0000 7fd87cff9700 1 -- 172.21.15.94:0/2917691590 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd87005a330 con 0x7fd880101d70 2024-04-06T03:50:55.459 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 6 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 4 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:50:55.460 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:50:55.461 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 14 2024-04-06T03:50:55.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:50:55.461 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd86c06dca0 msgr2=0x7fd86c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd86c06dca0 0x7fd86c070150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd86800b3b0 tx=0x7fd868009e30 comp rx=0 tx=0).stop 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880101d70 msgr2=0x7fd88010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880101d70 0x7fd88010eb40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd87000c8c0 tx=0x7fd8700044c0 comp rx=0 tx=0).stop 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 shutdown_connections 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880101d70 0x7fd88010eb40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd86c06dca0 0x7fd86c070150 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 --2- 172.21.15.94:0/2917691590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd880102710 0x7fd88010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 >> 172.21.15.94:0/2917691590 conn(0x7fd880072bc0 msgr2=0x7fd8800fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 shutdown_connections 2024-04-06T03:50:55.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:50:55.460+0000 7fd885f4c700 1 -- 172.21.15.94:0/2917691590 wait complete. 2024-04-06T03:50:55.597 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:55 smithi136 ceph-mon[82061]: pgmap v294: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:55.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:55 smithi094 ceph-mon[94053]: pgmap v294: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:56.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-06T03:50:56.337 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:50:56.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:56 smithi094 ceph-mon[94053]: pgmap v295: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:56.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:56 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2917691590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:56.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:56 smithi136 ceph-mon[82061]: pgmap v295: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:56.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:56 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2917691590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:50:57.984 INFO:teuthology.orchestra.run.smithi094.stdout:wait for servicemap items w/ changing names to refresh 2024-04-06T03:50:58.503 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:50:58 smithi094 ceph-mon[94053]: pgmap v296: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:58.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'sleep 60' 2024-04-06T03:50:58.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:50:58 smithi136 ceph-mon[82061]: pgmap v296: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:50:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:51:00.666 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:00 smithi094 ceph-mon[94053]: pgmap v297: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:00.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:00 smithi136 ceph-mon[82061]: pgmap v297: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:02.847 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:02 smithi136 ceph-mon[82061]: pgmap v298: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:02.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:02 smithi094 ceph-mon[94053]: pgmap v298: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:04.719 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:04 smithi094 ceph-mon[94053]: pgmap v299: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:04.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:04 smithi136 ceph-mon[82061]: pgmap v299: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:05.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:05 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:51:05.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:05 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:51:06.847 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:06 smithi136 ceph-mon[82061]: pgmap v300: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:06.865 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:06 smithi094 ceph-mon[94053]: pgmap v300: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:08.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:08 smithi136 ceph-mon[82061]: pgmap v301: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:08.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:08 smithi094 ceph-mon[94053]: pgmap v301: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:10.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:10 smithi136 ceph-mon[82061]: pgmap v302: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:10.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:10 smithi094 ceph-mon[94053]: pgmap v302: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:12.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:12 smithi136 ceph-mon[82061]: pgmap v303: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:12.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:12 smithi094 ceph-mon[94053]: pgmap v303: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:14.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:14 smithi136 ceph-mon[82061]: pgmap v304: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:14.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:14 smithi094 ceph-mon[94053]: pgmap v304: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:16.849 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:16 smithi136 ceph-mon[82061]: pgmap v305: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:16.850 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:51:16.865 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:16 smithi094 ceph-mon[94053]: pgmap v305: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:16.865 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:51:17.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:51:17.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:51:17.848 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:51:17.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:51:17.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:51:17.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:51:19.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:18 smithi136 ceph-mon[82061]: pgmap v306: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:19.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:18 smithi094 ceph-mon[94053]: pgmap v306: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:21.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:20 smithi136 ceph-mon[82061]: pgmap v307: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:21.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:51:21.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:20 smithi094 ceph-mon[94053]: pgmap v307: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:21.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:51:23.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:22 smithi136 ceph-mon[82061]: pgmap v308: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:23.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:22 smithi094 ceph-mon[94053]: pgmap v308: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:25.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:24 smithi136 ceph-mon[82061]: pgmap v309: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:25.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:24 smithi094 ceph-mon[94053]: pgmap v309: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:27.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:26 smithi136 ceph-mon[82061]: pgmap v310: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:27.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:26 smithi094 ceph-mon[94053]: pgmap v310: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:29.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:28 smithi136 ceph-mon[82061]: pgmap v311: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:29.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:28 smithi094 ceph-mon[94053]: pgmap v311: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:31.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:30 smithi136 ceph-mon[82061]: pgmap v312: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:31.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:30 smithi094 ceph-mon[94053]: pgmap v312: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:33.097 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:32 smithi136 ceph-mon[82061]: pgmap v313: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:33.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:32 smithi094 ceph-mon[94053]: pgmap v313: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:35.097 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:34 smithi136 ceph-mon[82061]: pgmap v314: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:35.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:34 smithi094 ceph-mon[94053]: pgmap v314: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:36.097 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:35 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:51:36.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:35 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:51:37.097 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:36 smithi136 ceph-mon[82061]: pgmap v315: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:37.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:36 smithi094 ceph-mon[94053]: pgmap v315: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:39.097 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:38 smithi136 ceph-mon[82061]: pgmap v316: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:39.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:38 smithi094 ceph-mon[94053]: pgmap v316: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:41.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:40 smithi094 ceph-mon[94053]: pgmap v317: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:41.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:40 smithi136 ceph-mon[82061]: pgmap v317: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:43.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:42 smithi094 ceph-mon[94053]: pgmap v318: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:43.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:42 smithi136 ceph-mon[82061]: pgmap v318: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:45.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:44 smithi094 ceph-mon[94053]: pgmap v319: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:45.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:44 smithi136 ceph-mon[82061]: pgmap v319: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:47.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:46 smithi094 ceph-mon[94053]: pgmap v320: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:47.349 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:46 smithi136 ceph-mon[82061]: pgmap v320: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:49.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:48 smithi094 ceph-mon[94053]: pgmap v321: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:49.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:48 smithi136 ceph-mon[82061]: pgmap v321: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:51.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:50 smithi094 ceph-mon[94053]: pgmap v322: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:51.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:50 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:51:51.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:50 smithi136 ceph-mon[82061]: pgmap v322: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:51.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:50 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:51:53.150 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:52 smithi136 ceph-mon[82061]: pgmap v323: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:53.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:52 smithi094 ceph-mon[94053]: pgmap v323: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:55.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:54 smithi094 ceph-mon[94053]: pgmap v324: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:55.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:54 smithi136 ceph-mon[82061]: pgmap v324: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:57.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:56 smithi094 ceph-mon[94053]: pgmap v325: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:57.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:56 smithi136 ceph-mon[82061]: pgmap v325: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:59.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:51:58 smithi094 ceph-mon[94053]: pgmap v326: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:51:59.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:51:58 smithi136 ceph-mon[82061]: pgmap v326: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:01.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:00 smithi094 ceph-mon[94053]: pgmap v327: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:01.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph orch ps' 2024-04-06T03:52:01.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:00 smithi136 ceph-mon[82061]: pgmap v327: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:01.510 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:52:02.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 -- 172.21.15.94:0/416117107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544102710 msgr2=0x7ff5440fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:02.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 --2- 172.21.15.94:0/416117107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544102710 0x7ff5440fbd80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff530004940 tx=0x7ff530009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:02.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 -- 172.21.15.94:0/416117107 shutdown_connections 2024-04-06T03:52:02.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 --2- 172.21.15.94:0/416117107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544102710 0x7ff5440fbd80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:02.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 --2- 172.21.15.94:0/416117107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff544101d70 0x7ff544102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:02.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 -- 172.21.15.94:0/416117107 >> 172.21.15.94:0/416117107 conn(0x7ff544072bc0 msgr2=0x7ff544072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:02.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 -- 172.21.15.94:0/416117107 shutdown_connections 2024-04-06T03:52:02.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 -- 172.21.15.94:0/416117107 wait complete. 2024-04-06T03:52:02.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 Processor -- start 2024-04-06T03:52:02.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.952+0000 7ff54b031700 1 -- start start 2024-04-06T03:52:02.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.953+0000 7ff54b031700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544101d70 0x7ff5440fe190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:02.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.953+0000 7ff54b031700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff544102710 0x7ff5440fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:02.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.953+0000 7ff54b031700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5440fec60 con 0x7ff544101d70 2024-04-06T03:52:02.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.953+0000 7ff54b031700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5440feda0 con 0x7ff544102710 2024-04-06T03:52:02.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.953+0000 7ff548dcd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544101d70 0x7ff5440fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:02.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.953+0000 7ff548dcd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544101d70 0x7ff5440fe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54316/0 (socket says 172.21.15.94:54316) 2024-04-06T03:52:02.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.953+0000 7ff548dcd700 1 -- 172.21.15.94:0/2019649611 learned_addr learned my addr 172.21.15.94:0/2019649611 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:02.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.953+0000 7ff543fff700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff544102710 0x7ff5440fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:02.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.953+0000 7ff548dcd700 1 -- 172.21.15.94:0/2019649611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff544102710 msgr2=0x7ff5440fe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:02.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.954+0000 7ff548dcd700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff544102710 0x7ff5440fe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:02.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.954+0000 7ff548dcd700 1 -- 172.21.15.94:0/2019649611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff530005040 con 0x7ff544101d70 2024-04-06T03:52:02.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.954+0000 7ff543fff700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff544102710 0x7ff5440fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:52:02.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.954+0000 7ff548dcd700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544101d70 0x7ff5440fe190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff538002a50 tx=0x7ff5380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:02.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.954+0000 7ff5417fa700 1 -- 172.21.15.94:0/2019649611 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff538007cc0 con 0x7ff544101d70 2024-04-06T03:52:02.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.954+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff544119ea0 con 0x7ff544101d70 2024-04-06T03:52:02.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.954+0000 7ff5417fa700 1 -- 172.21.15.94:0/2019649611 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff53800f040 con 0x7ff544101d70 2024-04-06T03:52:02.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.954+0000 7ff5417fa700 1 -- 172.21.15.94:0/2019649611 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff538009640 con 0x7ff544101d70 2024-04-06T03:52:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.954+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54411a320 con 0x7ff544101d70 2024-04-06T03:52:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.955+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5440fd4c0 con 0x7ff544101d70 2024-04-06T03:52:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.955+0000 7ff5417fa700 1 -- 172.21.15.94:0/2019649611 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7ff5380097c0 con 0x7ff544101d70 2024-04-06T03:52:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.956+0000 7ff5417fa700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff53406dca0 0x7ff534070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.956+0000 7ff543fff700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff53406dca0 0x7ff534070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.956+0000 7ff5417fa700 1 -- 172.21.15.94:0/2019649611 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff53808d300 con 0x7ff544101d70 2024-04-06T03:52:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.956+0000 7ff543fff700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff53406dca0 0x7ff534070150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff530009fd0 tx=0x7ff530009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:02.961+0000 7ff5417fa700 1 -- 172.21.15.94:0/2019649611 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff538056860 con 0x7ff544101d70 2024-04-06T03:52:03.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:02 smithi094 ceph-mon[94053]: pgmap v328: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:03.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.178+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 --> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5440ff880 con 0x7ff53406dca0 2024-04-06T03:52:03.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.189+0000 7ff5417fa700 1 -- 172.21.15.94:0/2019649611 <== mgr.34100 v2:172.21.15.94:6800/3117305505 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff5440ff880 con 0x7ff53406dca0 2024-04-06T03:52:03.190 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-06T03:52:03.190 INFO:teuthology.orchestra.run.smithi094.stdout:alertmanager.smithi094 smithi094 *:9093,9094 running (2m) 112s ago 19m 17.1M - 0.25.0 c8568f914cd2 f3dd69f56752 2024-04-06T03:52:03.190 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi094 smithi094 running (8m) 112s ago 19m 6660k - 19.0.0-2027-g9f192350 a5511c7deba9 155409986bc5 2024-04-06T03:52:03.190 INFO:teuthology.orchestra.run.smithi094.stdout:crash.smithi136 smithi136 running (7m) 2m ago 18m 6656k - 19.0.0-2027-g9f192350 a5511c7deba9 ba1138e56112 2024-04-06T03:52:03.190 INFO:teuthology.orchestra.run.smithi094.stdout:grafana.smithi094 smithi094 *:3000 running (117s) 112s ago 19m 52.9M - 9.4.12 2bacad6d85d8 550bb3828233 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.kvxcqo smithi094 running (3m) 112s ago 14m 15.5M - 19.0.0-2027-g9f192350 a5511c7deba9 bfca2184eb3c 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi094.qogyzw smithi094 running (4m) 112s ago 14m 83.0M - 19.0.0-2027-g9f192350 a5511c7deba9 c189927827c8 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.hbjgtw smithi136 running (3m) 2m ago 14m 14.9M - 19.0.0-2027-g9f192350 a5511c7deba9 6dd3b4023d80 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:mds.cephfs.smithi136.trdonf smithi136 running (3m) 2m ago 14m 14.0M - 19.0.0-2027-g9f192350 a5511c7deba9 432ef9195992 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi094.purlzp smithi094 *:8443,9283,8765 running (10m) 112s ago 20m 516M - 19.0.0-2027-g9f192350 a5511c7deba9 bc2e52ccba3c 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.smithi136.jagvwt smithi136 *:8443,9283,8765 running (9m) 2m ago 18m 452M - 19.0.0-2027-g9f192350 a5511c7deba9 018a0bf963f1 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi094 smithi094 running (9m) 112s ago 20m 55.5M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 02f9626f6514 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:mon.smithi136 smithi136 running (8m) 2m ago 18m 44.0M 2048M 19.0.0-2027-g9f192350 a5511c7deba9 46983eca715e 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi094 smithi094 *:9100 running (2m) 112s ago 19m 13.5M - 1.5.0 0da6a335fe13 21b4e6161925 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:node-exporter.smithi136 smithi136 *:9100 running (2m) 2m ago 18m 3594k - 1.5.0 0da6a335fe13 a1730b2e350a 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7m) 112s ago 17m 112M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 38a0da79352f 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi094 running (6m) 112s ago 17m 124M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 108ce348aea2 2024-04-06T03:52:03.191 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi094 running (6m) 112s ago 16m 108M 1557M 19.0.0-2027-g9f192350 a5511c7deba9 eb8d21d9c0dc 2024-04-06T03:52:03.192 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi136 running (5m) 2m ago 16m 123M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 0bdf47d78dc3 2024-04-06T03:52:03.192 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi136 running (5m) 2m ago 15m 108M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 c31367e1c2d5 2024-04-06T03:52:03.192 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi136 running (4m) 2m ago 15m 109M 2581M 19.0.0-2027-g9f192350 a5511c7deba9 16386e8f8583 2024-04-06T03:52:03.192 INFO:teuthology.orchestra.run.smithi094.stdout:prometheus.smithi094 smithi094 *:9095 running (2m) 112s ago 18m 49.1M - 2.43.0 a07b618ecd1d 4cad242accb3 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff53406dca0 msgr2=0x7ff534070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff53406dca0 0x7ff534070150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff530009fd0 tx=0x7ff530009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544101d70 msgr2=0x7ff5440fe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544101d70 0x7ff5440fe190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff538002a50 tx=0x7ff5380046b0 comp rx=0 tx=0).stop 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 shutdown_connections 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff544101d70 0x7ff5440fe190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff53406dca0 0x7ff534070150 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 --2- 172.21.15.94:0/2019649611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff544102710 0x7ff5440fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 >> 172.21.15.94:0/2019649611 conn(0x7ff544072bc0 msgr2=0x7ff544100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:03.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 shutdown_connections 2024-04-06T03:52:03.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:03.191+0000 7ff54b031700 1 -- 172.21.15.94:0/2019649611 wait complete. 2024-04-06T03:52:03.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:02 smithi136 ceph-mon[82061]: pgmap v328: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:03.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph versions' 2024-04-06T03:52:04.099 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:52:04.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:03 smithi094 ceph-mon[94053]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:52:04.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:03 smithi136 ceph-mon[82061]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-06T03:52:05.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:04 smithi094 ceph-mon[94053]: pgmap v329: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:05.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:04 smithi136 ceph-mon[82061]: pgmap v329: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:05.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.641+0000 7ff5aa966700 1 -- 172.21.15.94:0/1226051171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fde00 msgr2=0x7ff5a40fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:05.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.641+0000 7ff5aa966700 1 --2- 172.21.15.94:0/1226051171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fde00 0x7ff5a40fe1d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff594004940 tx=0x7ff594009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:05.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.641+0000 7ff5aa966700 1 -- 172.21.15.94:0/1226051171 shutdown_connections 2024-04-06T03:52:05.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.641+0000 7ff5aa966700 1 --2- 172.21.15.94:0/1226051171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a40fe7a0 0x7ff5a4106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:05.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.641+0000 7ff5aa966700 1 --2- 172.21.15.94:0/1226051171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fde00 0x7ff5a40fe1d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:05.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.641+0000 7ff5aa966700 1 -- 172.21.15.94:0/1226051171 >> 172.21.15.94:0/1226051171 conn(0x7ff5a40f78d0 msgr2=0x7ff5a40f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:05.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.641+0000 7ff5aa966700 1 -- 172.21.15.94:0/1226051171 shutdown_connections 2024-04-06T03:52:05.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.641+0000 7ff5aa966700 1 -- 172.21.15.94:0/1226051171 wait complete. 2024-04-06T03:52:05.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.642+0000 7ff5aa966700 1 Processor -- start 2024-04-06T03:52:05.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.642+0000 7ff5aa966700 1 -- start start 2024-04-06T03:52:05.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5aa966700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a40fde00 0x7ff5a4113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5aa966700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fe7a0 0x7ff5a41140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5aa966700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4114700 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5aa966700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a410daf0 con 0x7ff5a40fde00 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fe7a0 0x7ff5a41140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a37fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fe7a0 0x7ff5a41140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54332/0 (socket says 172.21.15.94:54332) 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a37fe700 1 -- 172.21.15.94:0/3344535813 learned_addr learned my addr 172.21.15.94:0/3344535813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a3fff700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a40fde00 0x7ff5a4113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a37fe700 1 -- 172.21.15.94:0/3344535813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a40fde00 msgr2=0x7ff5a4113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a37fe700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a40fde00 0x7ff5a4113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a37fe700 1 -- 172.21.15.94:0/3344535813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594005040 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a37fe700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fe7a0 0x7ff5a41140b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c8c0 tx=0x7ff5980044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a0ff9700 1 -- 172.21.15.94:0/3344535813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff59800a760 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a0ff9700 1 -- 172.21.15.94:0/3344535813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff59800ad90 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a410ddd0 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5a0ff9700 1 -- 172.21.15.94:0/3344535813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5980113a0 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.643+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a410e310 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.644+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a4066770 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.645+0000 7ff5a0ff9700 1 -- 172.21.15.94:0/3344535813 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7ff598012400 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.645+0000 7ff5a0ff9700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff58c06dca0 0x7ff58c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:05.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.645+0000 7ff5a3fff700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff58c06dca0 0x7ff58c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:05.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.645+0000 7ff5a0ff9700 1 -- 172.21.15.94:0/3344535813 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff59808d620 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.646+0000 7ff5a3fff700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff58c06dca0 0x7ff58c070150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff594004f90 tx=0x7ff594009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:05.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.650+0000 7ff5a0ff9700 1 -- 172.21.15.94:0/3344535813 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff59805a6b0 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.939+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff5a4109e90 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.939+0000 7ff5a0ff9700 1 -- 172.21.15.94:0/3344535813 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff59805a240 con 0x7ff5a40fe7a0 2024-04-06T03:52:05.940 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-06T03:52:05.940 INFO:teuthology.orchestra.run.smithi094.stdout: "mon": { 2024-04-06T03:52:05.940 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:52:05.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:52:05.940 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": { 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 2 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": { 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 6 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 4 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: "overall": { 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)": 14 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-06T03:52:05.941 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-06T03:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.941+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff58c06dca0 msgr2=0x7ff58c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.941+0000 7ff5aa966700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff58c06dca0 0x7ff58c070150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff594004f90 tx=0x7ff594009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.941+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fe7a0 msgr2=0x7ff5a41140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.941+0000 7ff5aa966700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fe7a0 0x7ff5a41140b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c8c0 tx=0x7ff5980044c0 comp rx=0 tx=0).stop 2024-04-06T03:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.942+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 shutdown_connections 2024-04-06T03:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.942+0000 7ff5aa966700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff58c06dca0 0x7ff58c070150 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.942+0000 7ff5aa966700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a40fde00 0x7ff5a4113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:05.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.942+0000 7ff5aa966700 1 --2- 172.21.15.94:0/3344535813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5a40fe7a0 0x7ff5a41140b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.942+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 >> 172.21.15.94:0/3344535813 conn(0x7ff5a40f78d0 msgr2=0x7ff5a40f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.942+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 shutdown_connections 2024-04-06T03:52:05.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:05.942+0000 7ff5aa966700 1 -- 172.21.15.94:0/3344535813 wait complete. 2024-04-06T03:52:06.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:05 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:52:06.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:05 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:52:06.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-06T03:52:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:52:07.132 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:06 smithi094 ceph-mon[94053]: pgmap v330: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:07.132 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:06 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3344535813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:52:07.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:06 smithi136 ceph-mon[82061]: pgmap v330: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:07.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:06 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3344535813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:52:08.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.402+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/643508978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8102710 msgr2=0x7fd8b80fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:08.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.402+0000 7fd8bdb6d700 1 --2- 172.21.15.94:0/643508978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8102710 0x7fd8b80fbd80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac004940 tx=0x7fd8ac009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:08.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/643508978 shutdown_connections 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 --2- 172.21.15.94:0/643508978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8102710 0x7fd8b80fbd80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 --2- 172.21.15.94:0/643508978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8b8101d70 0x7fd8b8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/643508978 >> 172.21.15.94:0/643508978 conn(0x7fd8b8072bc0 msgr2=0x7fd8b8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/643508978 shutdown_connections 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/643508978 wait complete. 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 Processor -- start 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 -- start start 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8101d70 0x7fd8b810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8b8102710 0x7fd8b810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b810f6d0 con 0x7fd8b8101d70 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.403+0000 7fd8bdb6d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b810f810 con 0x7fd8b8102710 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8b77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8101d70 0x7fd8b810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8b77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8101d70 0x7fd8b810eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49730/0 (socket says 172.21.15.94:49730) 2024-04-06T03:52:08.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8b77fe700 1 -- 172.21.15.94:0/313266133 learned_addr learned my addr 172.21.15.94:0/313266133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8b6ffd700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8b8102710 0x7fd8b810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8b77fe700 1 -- 172.21.15.94:0/313266133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8b8102710 msgr2=0x7fd8b810f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8b77fe700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8b8102710 0x7fd8b810f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8b77fe700 1 -- 172.21.15.94:0/313266133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac005040 con 0x7fd8b8101d70 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8b77fe700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8101d70 0x7fd8b810eb40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8002a50 tx=0x7fd8a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd89ffff700 1 -- 172.21.15.94:0/313266133 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a8007cc0 con 0x7fd8b8101d70 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b8117bb0 con 0x7fd8b8101d70 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd89ffff700 1 -- 172.21.15.94:0/313266133 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8a800f040 con 0x7fd8b8101d70 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd89ffff700 1 -- 172.21.15.94:0/313266133 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a800da30 con 0x7fd8b8101d70 2024-04-06T03:52:08.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.404+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b8118110 con 0x7fd8b8101d70 2024-04-06T03:52:08.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.405+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b8104240 con 0x7fd8b8101d70 2024-04-06T03:52:08.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.406+0000 7fd89ffff700 1 -- 172.21.15.94:0/313266133 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fd8a800db90 con 0x7fd8b8101d70 2024-04-06T03:52:08.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.407+0000 7fd89ffff700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd8a006dcb0 0x7fd8a0070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:08.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.407+0000 7fd8b6ffd700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd8a006dcb0 0x7fd8a0070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:08.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.407+0000 7fd89ffff700 1 -- 172.21.15.94:0/313266133 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd8a808da00 con 0x7fd8b8101d70 2024-04-06T03:52:08.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.407+0000 7fd8b6ffd700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd8a006dcb0 0x7fd8a0070160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8066000 tx=0x7fd8ac009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:08.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.411+0000 7fd89ffff700 1 -- 172.21.15.94:0/313266133 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd8a8057010 con 0x7fd8b8101d70 2024-04-06T03:52:08.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.717+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd8b804bed0 con 0x7fd8b8101d70 2024-04-06T03:52:08.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.717+0000 7fd89ffff700 1 -- 172.21.15.94:0/313266133 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd8a805a6d0 con 0x7fd8b8101d70 2024-04-06T03:52:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.719+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd8a006dcb0 msgr2=0x7fd8a0070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.719+0000 7fd8bdb6d700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd8a006dcb0 0x7fd8a0070160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8066000 tx=0x7fd8ac009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.719+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8101d70 msgr2=0x7fd8b810eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.719+0000 7fd8bdb6d700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8101d70 0x7fd8b810eb40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8002a50 tx=0x7fd8a80046b0 comp rx=0 tx=0).stop 2024-04-06T03:52:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.720+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 shutdown_connections 2024-04-06T03:52:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.720+0000 7fd8bdb6d700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd8b8101d70 0x7fd8b810eb40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:08.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.720+0000 7fd8bdb6d700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fd8a006dcb0 0x7fd8a0070160 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:08.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.720+0000 7fd8bdb6d700 1 --2- 172.21.15.94:0/313266133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8b8102710 0x7fd8b810f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:08.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.720+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 >> 172.21.15.94:0/313266133 conn(0x7fd8b8072bc0 msgr2=0x7fd8b80fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:08.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.720+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 shutdown_connections 2024-04-06T03:52:08.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:08.720+0000 7fd8bdb6d700 1 -- 172.21.15.94:0/313266133 wait complete. 2024-04-06T03:52:08.731 INFO:teuthology.orchestra.run.smithi094.stdout:true 2024-04-06T03:52:09.018 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:08 smithi094 ceph-mon[94053]: pgmap v331: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:09.018 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:08 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/313266133' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:52:09.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:08 smithi136 ceph-mon[82061]: pgmap v331: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:09.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:08 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/313266133' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:52:09.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -e sha1=9f192350ba5fe6d0a365b251a8c2af61f56a5fa6 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-06T03:52:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:52:11.006 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:10 smithi094 ceph-mon[94053]: pgmap v332: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:11.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.217+0000 7f2f7866c700 1 -- 172.21.15.94:0/4276348997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 msgr2=0x7f2f700fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.217+0000 7f2f7866c700 1 --2- 172.21.15.94:0/4276348997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 0x7f2f700fe260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c004940 tx=0x7f2f6c009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.217+0000 7f2f7866c700 1 -- 172.21.15.94:0/4276348997 shutdown_connections 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.217+0000 7f2f7866c700 1 --2- 172.21.15.94:0/4276348997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f700fe830 0x7f2f70106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.217+0000 7f2f7866c700 1 --2- 172.21.15.94:0/4276348997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 0x7f2f700fe260 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.217+0000 7f2f7866c700 1 -- 172.21.15.94:0/4276348997 >> 172.21.15.94:0/4276348997 conn(0x7f2f70072bc0 msgr2=0x7f2f70072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.217+0000 7f2f7866c700 1 -- 172.21.15.94:0/4276348997 shutdown_connections 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.217+0000 7f2f7866c700 1 -- 172.21.15.94:0/4276348997 wait complete. 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f7866c700 1 Processor -- start 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f7866c700 1 -- start start 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f7866c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 0x7f2f700fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f7866c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f700fe830 0x7f2f700fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f7866c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700fc820 con 0x7f2f700fde90 2024-04-06T03:52:11.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f7866c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700fc960 con 0x7f2f700fe830 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f76408700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 0x7f2f700fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f76408700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 0x7f2f700fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49762/0 (socket says 172.21.15.94:49762) 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f76408700 1 -- 172.21.15.94:0/3450806560 learned_addr learned my addr 172.21.15.94:0/3450806560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f76408700 1 -- 172.21.15.94:0/3450806560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f700fe830 msgr2=0x7f2f700fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f76408700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f700fe830 0x7f2f700fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.218+0000 7f2f76408700 1 -- 172.21.15.94:0/3450806560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c005040 con 0x7f2f700fde90 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.219+0000 7f2f76408700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 0x7f2f700fbd50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009390 tx=0x7f2f6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.219+0000 7f2f66ffd700 1 -- 172.21.15.94:0/3450806560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c00e040 con 0x7f2f700fde90 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.219+0000 7f2f66ffd700 1 -- 172.21.15.94:0/3450806560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f6c009af0 con 0x7f2f700fde90 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.219+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f700fcbb0 con 0x7f2f700fde90 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.219+0000 7f2f66ffd700 1 -- 172.21.15.94:0/3450806560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c01c3f0 con 0x7f2f700fde90 2024-04-06T03:52:11.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.219+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7011a280 con 0x7f2f700fde90 2024-04-06T03:52:11.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.220+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f70065e30 con 0x7f2f700fde90 2024-04-06T03:52:11.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.221+0000 7f2f66ffd700 1 -- 172.21.15.94:0/3450806560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f2f6c013840 con 0x7f2f700fde90 2024-04-06T03:52:11.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.221+0000 7f2f66ffd700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f2f5c06dca0 0x7f2f5c070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:11.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.221+0000 7f2f66ffd700 1 -- 172.21.15.94:0/3450806560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f2f6c08e7c0 con 0x7f2f700fde90 2024-04-06T03:52:11.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.221+0000 7f2f75c07700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f2f5c06dca0 0x7f2f5c070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:11.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.222+0000 7f2f75c07700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f2f5c06dca0 0x7f2f5c070150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f700fd690 tx=0x7f2f60009ce0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:11.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.226+0000 7f2f66ffd700 1 -- 172.21.15.94:0/3450806560 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2f6c057dc0 con 0x7f2f700fde90 2024-04-06T03:52:11.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:10 smithi136 ceph-mon[82061]: pgmap v332: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:11.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.506+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2f7004bed0 con 0x7f2f700fde90 2024-04-06T03:52:11.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.506+0000 7f2f66ffd700 1 -- 172.21.15.94:0/3450806560 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2f6c05b480 con 0x7f2f700fde90 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f2f5c06dca0 msgr2=0x7f2f5c070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f2f5c06dca0 0x7f2f5c070150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f700fd690 tx=0x7f2f60009ce0 comp rx=0 tx=0).stop 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 msgr2=0x7f2f700fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 0x7f2f700fbd50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009390 tx=0x7f2f6c009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 shutdown_connections 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2f700fde90 0x7f2f700fbd50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f2f5c06dca0 0x7f2f5c070150 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 --2- 172.21.15.94:0/3450806560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f700fe830 0x7f2f700fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 >> 172.21.15.94:0/3450806560 conn(0x7f2f70072bc0 msgr2=0x7f2f700ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 shutdown_connections 2024-04-06T03:52:11.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:11.509+0000 7f2f7866c700 1 -- 172.21.15.94:0/3450806560 wait complete. 2024-04-06T03:52:11.521 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph version 19.0.0-2027-g9f192350 (9f192350ba5fe6d0a365b251a8c2af61f56a5fa6) squid (dev)" 2024-04-06T03:52:12.132 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:11 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3450806560' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:52:12.156 DEBUG:teuthology.parallel:result is None 2024-04-06T03:52:12.156 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-06T03:52:12.172 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:52:12.173 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-06T03:52:12.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:11 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3450806560' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-06T03:52:12.437 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/mon.smithi094/config 2024-04-06T03:52:13.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:12 smithi094 ceph-mon[94053]: pgmap v333: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:13.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:12 smithi136 ceph-mon[82061]: pgmap v333: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:14.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.231+0000 7f1900231700 1 -- 172.21.15.94:0/1645890199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fb4d0 msgr2=0x7f18f80fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:14.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.231+0000 7f1900231700 1 --2- 172.21.15.94:0/1645890199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fb4d0 0x7f18f80fb940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f18e8004940 tx=0x7f18e8009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:14.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.231+0000 7f1900231700 1 -- 172.21.15.94:0/1645890199 shutdown_connections 2024-04-06T03:52:14.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.231+0000 7f1900231700 1 --2- 172.21.15.94:0/1645890199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fb4d0 0x7f18f80fb940 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:14.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.231+0000 7f1900231700 1 --2- 172.21.15.94:0/1645890199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f80fdcc0 0x7f18f80fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:14.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.231+0000 7f1900231700 1 -- 172.21.15.94:0/1645890199 >> 172.21.15.94:0/1645890199 conn(0x7f18f80714c0 msgr2=0x7f18f80718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:14.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.231+0000 7f1900231700 1 -- 172.21.15.94:0/1645890199 shutdown_connections 2024-04-06T03:52:14.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.231+0000 7f1900231700 1 -- 172.21.15.94:0/1645890199 wait complete. 2024-04-06T03:52:14.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.232+0000 7f1900231700 1 Processor -- start 2024-04-06T03:52:14.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.232+0000 7f1900231700 1 -- start start 2024-04-06T03:52:14.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.232+0000 7f1900231700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f80fb4d0 0x7f18f8115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:14.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.232+0000 7f1900231700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fdcc0 0x7f18f8116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:14.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.232+0000 7f1900231700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81167b0 con 0x7f18f80fdcc0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.232+0000 7f1900231700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f810fd90 con 0x7f18f80fb4d0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.232+0000 7f18fd7cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fdcc0 0x7f18f8116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.232+0000 7f18fd7cc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fdcc0 0x7f18f8116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49780/0 (socket says 172.21.15.94:49780) 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.232+0000 7f18fd7cc700 1 -- 172.21.15.94:0/2595002108 learned_addr learned my addr 172.21.15.94:0/2595002108 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f18fdfcd700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f80fb4d0 0x7f18f8115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f18fd7cc700 1 -- 172.21.15.94:0/2595002108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f80fb4d0 msgr2=0x7f18f8115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f18fd7cc700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f80fb4d0 0x7f18f8115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f18fd7cc700 1 -- 172.21.15.94:0/2595002108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8005040 con 0x7f18f80fdcc0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f18fd7cc700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fdcc0 0x7f18f8116160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f18e800b3b0 tx=0x7f18e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f18ee7fc700 1 -- 172.21.15.94:0/2595002108 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e800e040 con 0x7f18f80fdcc0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f8110010 con 0x7f18f80fdcc0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f18ee7fc700 1 -- 172.21.15.94:0/2595002108 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18e800bbc0 con 0x7f18f80fdcc0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f18ee7fc700 1 -- 172.21.15.94:0/2595002108 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e801c3f0 con 0x7f18f80fdcc0 2024-04-06T03:52:14.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.233+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f8110550 con 0x7f18f80fdcc0 2024-04-06T03:52:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.234+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f8107380 con 0x7f18f80fdcc0 2024-04-06T03:52:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.235+0000 7f18ee7fc700 1 -- 172.21.15.94:0/2595002108 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f18e800bd30 con 0x7f18f80fdcc0 2024-04-06T03:52:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.235+0000 7f18ee7fc700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f18e406dca0 0x7f18e4070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:14.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.235+0000 7f18ee7fc700 1 -- 172.21.15.94:0/2595002108 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f18e808e8d0 con 0x7f18f80fdcc0 2024-04-06T03:52:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.235+0000 7f18fdfcd700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f18e406dca0 0x7f18e4070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:14.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.235+0000 7f18fdfcd700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f18e406dca0 0x7f18e4070150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18f4004990 tx=0x7f18f4009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:14.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.240+0000 7f18ee7fc700 1 -- 172.21.15.94:0/2595002108 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f18e805b8e0 con 0x7f18f80fdcc0 2024-04-06T03:52:14.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.484+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f18f8111270 con 0x7f18f80fdcc0 2024-04-06T03:52:14.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.484+0000 7f18ee7fc700 1 -- 172.21.15.94:0/2595002108 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1940 (secure 0 0 0) 0x7f18e805b470 con 0x7f18f80fdcc0 2024-04-06T03:52:14.486 INFO:teuthology.orchestra.run.smithi094.stdout:e36 2024-04-06T03:52:14.486 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-06T03:52:14.486 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-06T03:52:14.486 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-06T03:52:14.486 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:14.486 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-06T03:52:14.486 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-06T03:52:14.486 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 35 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-06T03:37:49.396246+0000 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-06T03:49:01.609121+0000 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 60 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 82 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-06T03:52:14.487 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=34400} 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 1 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 34400 members: 34400 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.qogyzw{0:34400} state up:active seq 21 join_fscid=1 addr [v2:172.21.15.94:6826/1447944673,v1:172.21.15.94:6827/1447944673] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:14.488 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-06T03:52:14.489 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:14.489 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi094.kvxcqo{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.94:6828/1650064917,v1:172.21.15.94:6829/1650064917] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:52:14.489 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.hbjgtw{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.136:6824/1500356136,v1:172.21.15.136:6825/1500356136] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:52:14.489 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.cephfs.smithi136.trdonf{-1:44335} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.136:6826/1840231705,v1:172.21.15.136:6827/1840231705] compat {c=[1],r=[1],i=[1fff]}] 2024-04-06T03:52:14.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f18e406dca0 msgr2=0x7f18e4070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f18e406dca0 0x7f18e4070150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18f4004990 tx=0x7f18f4009ee0 comp rx=0 tx=0).stop 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fdcc0 msgr2=0x7f18f8116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fdcc0 0x7f18f8116160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f18e800b3b0 tx=0x7f18e8009e30 comp rx=0 tx=0).stop 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 shutdown_connections 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f18e406dca0 0x7f18e4070150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f80fb4d0 0x7f18f8115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 --2- 172.21.15.94:0/2595002108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18f80fdcc0 0x7f18f8116160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 >> 172.21.15.94:0/2595002108 conn(0x7f18f80714c0 msgr2=0x7f18f80f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 shutdown_connections 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:14.487+0000 7f1900231700 1 -- 172.21.15.94:0/2595002108 wait complete. 2024-04-06T03:52:14.490 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 36 2024-04-06T03:52:15.104 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-06T03:52:15.115 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-06T03:52:15.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:14 smithi094 ceph-mon[94053]: pgmap v334: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:15.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:14 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2595002108' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.216+0000 7f7e328ee640 1 -- 172.21.15.94:0/2080953812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 msgr2=0x7f7e2c0fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.216+0000 7f7e328ee640 1 --2- 172.21.15.94:0/2080953812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 0x7f7e2c0fc1e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7e200048d0 tx=0x7f7e20009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.216+0000 7f7e328ee640 1 -- 172.21.15.94:0/2080953812 shutdown_connections 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.216+0000 7f7e328ee640 1 --2- 172.21.15.94:0/2080953812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 0x7f7e2c0fc1e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.216+0000 7f7e328ee640 1 --2- 172.21.15.94:0/2080953812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e2c1021b0 0x7f7e2c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.216+0000 7f7e328ee640 1 -- 172.21.15.94:0/2080953812 >> 172.21.15.94:0/2080953812 conn(0x7f7e2c0f7da0 msgr2=0x7f7e2c0fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.216+0000 7f7e328ee640 1 -- 172.21.15.94:0/2080953812 shutdown_connections 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.216+0000 7f7e328ee640 1 -- 172.21.15.94:0/2080953812 wait complete. 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.216+0000 7f7e328ee640 1 Processor -- start 2024-04-06T03:52:15.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e328ee640 1 -- start start 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e328ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e2c1021b0 0x7f7e2c112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e328ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 0x7f7e2c113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e328ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c113840 con 0x7f7e2c102b50 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e328ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c10ce90 con 0x7f7e2c1021b0 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 0x7f7e2c113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 0x7f7e2c113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49798/0 (socket says 172.21.15.94:49798) 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e2b7fe640 1 -- 172.21.15.94:0/2266906662 learned_addr learned my addr 172.21.15.94:0/2266906662 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e2bfff640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e2c1021b0 0x7f7e2c112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e2b7fe640 1 -- 172.21.15.94:0/2266906662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e2c1021b0 msgr2=0x7f7e2c112d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e2b7fe640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e2c1021b0 0x7f7e2c112d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e2b7fe640 1 -- 172.21.15.94:0/2266906662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20005040 con 0x7f7e2c102b50 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e2bfff640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e2c1021b0 0x7f7e2c112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e2b7fe640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 0x7f7e2c113240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7e20004930 tx=0x7f7e20009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e28ff9640 1 -- 172.21.15.94:0/2266906662 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e200094c0 con 0x7f7e2c102b50 2024-04-06T03:52:15.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c10d110 con 0x7f7e2c102b50 2024-04-06T03:52:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e28ff9640 1 -- 172.21.15.94:0/2266906662 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e20032e50 con 0x7f7e2c102b50 2024-04-06T03:52:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.217+0000 7f7e28ff9640 1 -- 172.21.15.94:0/2266906662 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e200433b0 con 0x7f7e2c102b50 2024-04-06T03:52:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.218+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c10d5d0 con 0x7f7e2c102b50 2024-04-06T03:52:15.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.218+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c0fd920 con 0x7f7e2c102b50 2024-04-06T03:52:15.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.219+0000 7f7e28ff9640 1 -- 172.21.15.94:0/2266906662 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f7e2002f070 con 0x7f7e2c102b50 2024-04-06T03:52:15.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.219+0000 7f7e28ff9640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7e1406dc10 0x7f7e140700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:15.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.219+0000 7f7e28ff9640 1 -- 172.21.15.94:0/2266906662 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7e200af670 con 0x7f7e2c102b50 2024-04-06T03:52:15.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.219+0000 7f7e2bfff640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7e1406dc10 0x7f7e140700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:15.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.220+0000 7f7e2bfff640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7e1406dc10 0x7f7e140700c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0041d0 tx=0x7f7e1c009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:15.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.228+0000 7f7e28ff9640 1 -- 172.21.15.94:0/2266906662 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7e20078b50 con 0x7f7e2c102b50 2024-04-06T03:52:15.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:14 smithi136 ceph-mon[82061]: pgmap v334: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:15.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:14 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2595002108' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-06T03:52:15.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.410+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e2c1092f0 con 0x7f7e2c102b50 2024-04-06T03:52:15.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.411+0000 7f7e28ff9640 1 -- 172.21.15.94:0/2266906662 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f7e2007c210 con 0x7f7e2c102b50 2024-04-06T03:52:15.413 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:15.413 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":82,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","created":"2024-04-06T03:31:12.411006+0000","modified":"2024-04-06T03:48:51.384943+0000","last_up_change":"2024-04-06T03:47:32.894139+0000","last_in_change":"2024-04-06T03:36:17.627724+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-06T03:35:23.271684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-06T03:37:48.333560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-06T03:37:48.624904+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f51dba0-a74e-444c-b4ab-d5f1fa68ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6803","nonce":95929004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6805","nonce":95929004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6809","nonce":95929004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6807","nonce":95929004}]},"public_addr":"172.21.15.94:6803/95929004","cluster_addr":"172.21.15.94:6805/95929004","heartbeat_back_addr":"172.21.15.94:6809/95929004","heartbeat_front_addr":"172.21.15.94:6807/95929004","state":["exists","up"]},{"osd":1,"uuid":"2dd01097-c6d6-45e9-acee-ca5ea5f4e166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6811","nonce":3698645734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6813","nonce":3698645734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6817","nonce":3698645734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6815","nonce":3698645734}]},"public_addr":"172.21.15.94:6811/3698645734","cluster_addr":"172.21.15.94:6813/3698645734","heartbeat_back_addr":"172.21.15.94:6817/3698645734","heartbeat_front_addr":"172.21.15.94:6815/3698645734","state":["exists","up"]},{"osd":2,"uuid":"9645da7c-ffbc-4415-8fa0-0fcb07c53f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6819","nonce":1970793192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6821","nonce":1970793192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6825","nonce":1970793192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6823","nonce":1970793192}]},"public_addr":"172.21.15.94:6819/1970793192","cluster_addr":"172.21.15.94:6821/1970793192","heartbeat_back_addr":"172.21.15.94:6825/1970793192","heartbeat_front_addr":"172.21.15.94:6823/1970793192","state":["exists","up"]},{"osd":3,"uuid":"43321ebd-f195-4738-a2a5-f4d55a375bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6801","nonce":2267317275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6803","nonce":2267317275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6807","nonce":2267317275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6805","nonce":2267317275}]},"public_addr":"172.21.15.136:6801/2267317275","cluster_addr":"172.21.15.136:6803/2267317275","heartbeat_back_addr":"172.21.15.136:6807/2267317275","heartbeat_front_addr":"172.21.15.136:6805/2267317275","state":["exists","up"]},{"osd":4,"uuid":"572f9cc0-e83c-4354-babb-519be04069c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6809","nonce":3530002524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6811","nonce":3530002524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6815","nonce":3530002524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6813","nonce":3530002524}]},"public_addr":"172.21.15.136:6809/3530002524","cluster_addr":"172.21.15.136:6811/3530002524","heartbeat_back_addr":"172.21.15.136:6815/3530002524","heartbeat_front_addr":"172.21.15.136:6813/3530002524","state":["exists","up"]},{"osd":5,"uuid":"a38e98d8-dfb7-4083-aae8-e5853a896d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6817","nonce":853857034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6819","nonce":853857034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6823","nonce":853857034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6821","nonce":853857034}]},"public_addr":"172.21.15.136:6817/853857034","cluster_addr":"172.21.15.136:6819/853857034","heartbeat_back_addr":"172.21.15.136:6823/853857034","heartbeat_front_addr":"172.21.15.136:6821/853857034","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:25.784070+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:53.656615+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:20.675464+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:44.290139+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:09.122272+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:34.741771+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.136:6826/249801036":"2024-04-07T03:48:51.384914+0000","172.21.15.136:6827/249801036":"2024-04-07T03:48:51.384914+0000","172.21.15.136:6825/838120541":"2024-04-07T03:48:26.996253+0000","172.21.15.136:6824/838120541":"2024-04-07T03:48:26.996253+0000","172.21.15.94:6828/3794043365":"2024-04-07T03:48:03.697599+0000","172.21.15.94:6827/1913624529":"2024-04-07T03:47:37.708879+0000","172.21.15.94:0/2779591805":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6801/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:0/4131567976":"2024-04-07T03:40:06.067489+0000","172.21.15.94:6800/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:0/1084447585":"2024-04-07T03:32:10.456958+0000","172.21.15.94:0/2795266101":"2024-04-07T03:43:05.151475+0000","172.21.15.94:6800/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1319147349":"2024-04-07T03:40:06.067489+0000","172.21.15.94:0/1247691830":"2024-04-07T03:33:12.953920+0000","172.21.15.136:6829/614857857":"2024-04-07T03:42:13.227829+0000","172.21.15.94:0/457112173":"2024-04-07T03:40:06.067489+0000","172.21.15.136:0/2374087532":"2024-04-07T03:42:13.227829+0000","172.21.15.94:6801/4259523022":"2024-04-07T03:43:05.151475+0000","172.21.15.94:6801/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1085015108":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/852522634":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/3195038611":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/2412091380":"2024-04-07T03:33:12.953920+0000","172.21.15.94:6800/4259523022":"2024-04-07T03:43:05.151475+0000","172.21.15.94:0/3423462064":"2024-04-07T03:43:05.151475+0000","172.21.15.136:0/445150575":"2024-04-07T03:42:13.227829+0000","172.21.15.94:0/414290682":"2024-04-07T03:43:05.151475+0000","172.21.15.136:0/299229849":"2024-04-07T03:42:13.227829+0000","172.21.15.94:6800/3627366865":"2024-04-07T03:40:06.067489+0000","172.21.15.94:0/3705686069":"2024-04-07T03:40:06.067489+0000","172.21.15.94:6800/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6829/3794043365":"2024-04-07T03:48:03.697599+0000","172.21.15.94:0/3724179185":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/3039501657":"2024-04-07T03:43:05.151475+0000","172.21.15.136:6828/614857857":"2024-04-07T03:42:13.227829+0000","172.21.15.94:6801/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6801/3627366865":"2024-04-07T03:40:06.067489+0000","172.21.15.136:0/3970723326":"2024-04-07T03:42:13.227829+0000","172.21.15.94:0/1674904333":"2024-04-07T03:43:05.151475+0000","172.21.15.94:0/873173111":"2024-04-07T03:43:05.151475+0000","172.21.15.94:0/1082665650":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6826/1913624529":"2024-04-07T03:47:37.708879+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.413+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7e1406dc10 msgr2=0x7f7e140700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.413+0000 7f7e328ee640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7e1406dc10 0x7f7e140700c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0041d0 tx=0x7f7e1c009040 comp rx=0 tx=0).stop 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.413+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 msgr2=0x7f7e2c113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.413+0000 7f7e328ee640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 0x7f7e2c113240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7e20004930 tx=0x7f7e20009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.413+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 shutdown_connections 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.413+0000 7f7e328ee640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7e1406dc10 0x7f7e140700c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.413+0000 7f7e328ee640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e2c102b50 0x7f7e2c113240 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.413+0000 7f7e328ee640 1 --2- 172.21.15.94:0/2266906662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e2c1021b0 0x7f7e2c112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.413+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 >> 172.21.15.94:0/2266906662 conn(0x7f7e2c0f7da0 msgr2=0x7f7e2c0f8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.414+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 shutdown_connections 2024-04-06T03:52:15.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.414+0000 7f7e328ee640 1 -- 172.21.15.94:0/2266906662 wait complete. 2024-04-06T03:52:15.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-06T03:35:23.271684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-06T03:37:48.333560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-06T03:37:48.624904+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-06T03:52:15.426 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-06T03:52:15.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.532+0000 7f099359d640 1 -- 172.21.15.94:0/604243418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c100f10 msgr2=0x7f098c1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.532+0000 7f099359d640 1 --2- 172.21.15.94:0/604243418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c100f10 0x7f098c1012e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f09880048d0 tx=0x7f0988009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.532+0000 7f099359d640 1 -- 172.21.15.94:0/604243418 shutdown_connections 2024-04-06T03:52:15.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.532+0000 7f099359d640 1 --2- 172.21.15.94:0/604243418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f098c1018b0 0x7f098c0faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.532+0000 7f099359d640 1 --2- 172.21.15.94:0/604243418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c100f10 0x7f098c1012e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.532+0000 7f099359d640 1 -- 172.21.15.94:0/604243418 >> 172.21.15.94:0/604243418 conn(0x7f098c071e80 msgr2=0x7f098c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.532+0000 7f099359d640 1 -- 172.21.15.94:0/604243418 shutdown_connections 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.532+0000 7f099359d640 1 -- 172.21.15.94:0/604243418 wait complete. 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f099359d640 1 Processor -- start 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f099359d640 1 -- start start 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f099359d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f098c100f10 0x7f098c112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f099359d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1018b0 0x7f098c113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f099359d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c1137c0 con 0x7f098c1018b0 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f099359d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c10cd80 con 0x7f098c100f10 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f0991d9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1018b0 0x7f098c113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f0991d9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1018b0 0x7f098c113130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49816/0 (socket says 172.21.15.94:49816) 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f0991d9a640 1 -- 172.21.15.94:0/1640874074 learned_addr learned my addr 172.21.15.94:0/1640874074 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f099259b640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f098c100f10 0x7f098c112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f0991d9a640 1 -- 172.21.15.94:0/1640874074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f098c100f10 msgr2=0x7f098c112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f0991d9a640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f098c100f10 0x7f098c112bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f0991d9a640 1 -- 172.21.15.94:0/1640874074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0988005040 con 0x7f098c1018b0 2024-04-06T03:52:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.533+0000 7f0991d9a640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1018b0 0x7f098c113130 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f097c00b790 tx=0x7f097c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.534+0000 7f0982ffd640 1 -- 172.21.15.94:0/1640874074 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f097c00c700 con 0x7f098c1018b0 2024-04-06T03:52:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.534+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c10d060 con 0x7f098c1018b0 2024-04-06T03:52:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.534+0000 7f0982ffd640 1 -- 172.21.15.94:0/1640874074 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f097c00cd30 con 0x7f098c1018b0 2024-04-06T03:52:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.534+0000 7f0982ffd640 1 -- 172.21.15.94:0/1640874074 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f097c013370 con 0x7f098c1018b0 2024-04-06T03:52:15.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.534+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c10d580 con 0x7f098c1018b0 2024-04-06T03:52:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.535+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c0fc680 con 0x7f098c1018b0 2024-04-06T03:52:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.535+0000 7f0982ffd640 1 -- 172.21.15.94:0/1640874074 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f097c012070 con 0x7f098c1018b0 2024-04-06T03:52:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.535+0000 7f0982ffd640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f097806dc10 0x7f09780700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.536+0000 7f0982ffd640 1 -- 172.21.15.94:0/1640874074 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f097c08d280 con 0x7f098c1018b0 2024-04-06T03:52:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.536+0000 7f099259b640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f097806dc10 0x7f09780700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.536+0000 7f099259b640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f097806dc10 0x7f09780700c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0988004540 tx=0x7f0988009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:15.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.540+0000 7f0982ffd640 1 -- 172.21.15.94:0/1640874074 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f097c056760 con 0x7f098c1018b0 2024-04-06T03:52:15.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.818+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f098c108100 con 0x7f098c1018b0 2024-04-06T03:52:15.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.819+0000 7f0982ffd640 1 -- 172.21.15.94:0/1640874074 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f097c059e20 con 0x7f098c1018b0 2024-04-06T03:52:15.820 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-06T03:52:15.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f097806dc10 msgr2=0x7f09780700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f097806dc10 0x7f09780700c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0988004540 tx=0x7f0988009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1018b0 msgr2=0x7f098c113130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1018b0 0x7f098c113130 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f097c00b790 tx=0x7f097c0091d0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 shutdown_connections 2024-04-06T03:52:15.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f097806dc10 0x7f09780700c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1018b0 0x7f098c113130 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 --2- 172.21.15.94:0/1640874074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f098c100f10 0x7f098c112bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 >> 172.21.15.94:0/1640874074 conn(0x7f098c071e80 msgr2=0x7f098c0f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:15.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 shutdown_connections 2024-04-06T03:52:15.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.821+0000 7f099359d640 1 -- 172.21.15.94:0/1640874074 wait complete. 2024-04-06T03:52:15.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-06T03:52:15.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.937+0000 7f64660ee640 1 -- 172.21.15.94:0/3497961141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6460102240 msgr2=0x7f6460102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.937+0000 7f64660ee640 1 --2- 172.21.15.94:0/3497961141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6460102240 0x7f6460102610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f64500048d0 tx=0x7f6450009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.937+0000 7f64660ee640 1 -- 172.21.15.94:0/3497961141 shutdown_connections 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.937+0000 7f64660ee640 1 --2- 172.21.15.94:0/3497961141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64600643a0 0x7f64600647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.937+0000 7f64660ee640 1 --2- 172.21.15.94:0/3497961141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6460102240 0x7f6460102610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.937+0000 7f64660ee640 1 -- 172.21.15.94:0/3497961141 >> 172.21.15.94:0/3497961141 conn(0x7f64600710b0 msgr2=0x7f64600714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.937+0000 7f64660ee640 1 -- 172.21.15.94:0/3497961141 shutdown_connections 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.937+0000 7f64660ee640 1 -- 172.21.15.94:0/3497961141 wait complete. 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.937+0000 7f64660ee640 1 Processor -- start 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f64660ee640 1 -- start start 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f64660ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64600643a0 0x7f6460112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f64660ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6460102240 0x7f6460113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f64660ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64601138f0 con 0x7f64600643a0 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f64660ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646010ceb0 con 0x7f6460102240 2024-04-06T03:52:15.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f645f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64600643a0 0x7f6460112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f645f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64600643a0 0x7f6460112d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49826/0 (socket says 172.21.15.94:49826) 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f645f7fe640 1 -- 172.21.15.94:0/866620121 learned_addr learned my addr 172.21.15.94:0/866620121 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f645f7fe640 1 -- 172.21.15.94:0/866620121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6460102240 msgr2=0x7f6460113260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f645f7fe640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6460102240 0x7f6460113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f645f7fe640 1 -- 172.21.15.94:0/866620121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6450005040 con 0x7f64600643a0 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f645f7fe640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64600643a0 0x7f6460112d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6450004930 tx=0x7f6450009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f6447fff640 1 -- 172.21.15.94:0/866620121 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64500094c0 con 0x7f64600643a0 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f6447fff640 1 -- 172.21.15.94:0/866620121 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6450032e50 con 0x7f64600643a0 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646010d130 con 0x7f64600643a0 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.938+0000 7f6447fff640 1 -- 172.21.15.94:0/866620121 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64500433b0 con 0x7f64600643a0 2024-04-06T03:52:15.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.939+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646010d5f0 con 0x7f64600643a0 2024-04-06T03:52:15.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.940+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64600fd940 con 0x7f64600643a0 2024-04-06T03:52:15.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.940+0000 7f6447fff640 1 -- 172.21.15.94:0/866620121 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f645002f070 con 0x7f64600643a0 2024-04-06T03:52:15.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.940+0000 7f6447fff640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f644806dc10 0x7f64480700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:15.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.940+0000 7f6447fff640 1 -- 172.21.15.94:0/866620121 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f64500af520 con 0x7f64600643a0 2024-04-06T03:52:15.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.940+0000 7f645effd640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f644806dc10 0x7f64480700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:15.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.941+0000 7f645effd640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f644806dc10 0x7f64480700c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f646010e5e0 tx=0x7f6454006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:15.945+0000 7f6447fff640 1 -- 172.21.15.94:0/866620121 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6450078a00 con 0x7f64600643a0 2024-04-06T03:52:16.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.125+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f64601093c0 con 0x7f64600643a0 2024-04-06T03:52:16.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.125+0000 7f6447fff640 1 -- 172.21.15.94:0/866620121 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f645007c0c0 con 0x7f64600643a0 2024-04-06T03:52:16.127 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f644806dc10 msgr2=0x7f64480700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f644806dc10 0x7f64480700c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f646010e5e0 tx=0x7f6454006d20 comp rx=0 tx=0).stop 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64600643a0 msgr2=0x7f6460112d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64600643a0 0x7f6460112d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6450004930 tx=0x7f6450009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 shutdown_connections 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f644806dc10 0x7f64480700c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6460102240 0x7f6460113260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 --2- 172.21.15.94:0/866620121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64600643a0 0x7f6460112d20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 >> 172.21.15.94:0/866620121 conn(0x7f64600710b0 msgr2=0x7f64600f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 shutdown_connections 2024-04-06T03:52:16.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.127+0000 7f64660ee640 1 -- 172.21.15.94:0/866620121 wait complete. 2024-04-06T03:52:16.138 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:15 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2266906662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:52:16.138 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:15 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1640874074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:52:16.140 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-06T03:52:16.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.239+0000 7f279579d640 1 -- 172.21.15.94:0/532073044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fb9e0 msgr2=0x7f27900fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.239+0000 7f279579d640 1 --2- 172.21.15.94:0/532073044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fb9e0 0x7f27900fbdb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f27800048d0 tx=0x7f2780009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.239+0000 7f279579d640 1 -- 172.21.15.94:0/532073044 shutdown_connections 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.239+0000 7f279579d640 1 --2- 172.21.15.94:0/532073044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27900fc2f0 0x7f2790108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.239+0000 7f279579d640 1 --2- 172.21.15.94:0/532073044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fb9e0 0x7f27900fbdb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.239+0000 7f279579d640 1 -- 172.21.15.94:0/532073044 >> 172.21.15.94:0/532073044 conn(0x7f2790071e80 msgr2=0x7f2790072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.239+0000 7f279579d640 1 -- 172.21.15.94:0/532073044 shutdown_connections 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.239+0000 7f279579d640 1 -- 172.21.15.94:0/532073044 wait complete. 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f279579d640 1 Processor -- start 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f279579d640 1 -- start start 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f279579d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27900fb9e0 0x7f2790114d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f279579d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fc2f0 0x7f27901152a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f279579d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27901158a0 con 0x7f27900fc2f0 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f279579d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279010eef0 con 0x7f27900fb9e0 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f278f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fc2f0 0x7f27901152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f278f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fc2f0 0x7f27901152a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49836/0 (socket says 172.21.15.94:49836) 2024-04-06T03:52:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f278f7fe640 1 -- 172.21.15.94:0/2780180084 learned_addr learned my addr 172.21.15.94:0/2780180084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f278ffff640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27900fb9e0 0x7f2790114d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f278f7fe640 1 -- 172.21.15.94:0/2780180084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27900fb9e0 msgr2=0x7f2790114d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f278f7fe640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27900fb9e0 0x7f2790114d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.240+0000 7f278f7fe640 1 -- 172.21.15.94:0/2780180084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2780005040 con 0x7f27900fc2f0 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.241+0000 7f278f7fe640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fc2f0 0x7f27901152a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f277800c790 tx=0x7f27780043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.241+0000 7f278cff9640 1 -- 172.21.15.94:0/2780180084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277800a820 con 0x7f27900fc2f0 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.241+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279010f1d0 con 0x7f27900fc2f0 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.241+0000 7f278cff9640 1 -- 172.21.15.94:0/2780180084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f277800ae50 con 0x7f27900fc2f0 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.241+0000 7f278cff9640 1 -- 172.21.15.94:0/2780180084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27780115e0 con 0x7f27900fc2f0 2024-04-06T03:52:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.241+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279010f6f0 con 0x7f27900fc2f0 2024-04-06T03:52:16.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.242+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27900fb580 con 0x7f27900fc2f0 2024-04-06T03:52:16.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.242+0000 7f278cff9640 1 -- 172.21.15.94:0/2780180084 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f2778012400 con 0x7f27900fc2f0 2024-04-06T03:52:16.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.243+0000 7f278cff9640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f277c06dc10 0x7f277c0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:16.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.243+0000 7f278cff9640 1 -- 172.21.15.94:0/2780180084 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f277808d7d0 con 0x7f27900fc2f0 2024-04-06T03:52:16.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.243+0000 7f278ffff640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f277c06dc10 0x7f277c0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:16.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.243+0000 7f278ffff640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f277c06dc10 0x7f277c0700c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2780003080 tx=0x7f2780009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:16.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.252+0000 7f278cff9640 1 -- 172.21.15.94:0/2780180084 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2778056d30 con 0x7f27900fc2f0 2024-04-06T03:52:16.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:15 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2266906662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:52:16.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:15 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1640874074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:52:16.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.433+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f27901103d0 con 0x7f27900fc2f0 2024-04-06T03:52:16.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.433+0000 7f278cff9640 1 -- 172.21.15.94:0/2780180084 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f277805a3f0 con 0x7f27900fc2f0 2024-04-06T03:52:16.435 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f277c06dc10 msgr2=0x7f277c0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f277c06dc10 0x7f277c0700c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2780003080 tx=0x7f2780009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fc2f0 msgr2=0x7f27901152a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fc2f0 0x7f27901152a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f277800c790 tx=0x7f27780043c0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 shutdown_connections 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f277c06dc10 0x7f277c0700c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27900fc2f0 0x7f27901152a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 --2- 172.21.15.94:0/2780180084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27900fb9e0 0x7f2790114d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 >> 172.21.15.94:0/2780180084 conn(0x7f2790071e80 msgr2=0x7f2790103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 shutdown_connections 2024-04-06T03:52:16.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.435+0000 7f279579d640 1 -- 172.21.15.94:0/2780180084 wait complete. 2024-04-06T03:52:16.447 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-06T03:52:16.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 -- 172.21.15.94:0/4152024206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f8102240 msgr2=0x7f36f8102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 --2- 172.21.15.94:0/4152024206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f8102240 0x7f36f8102610 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f36e80048d0 tx=0x7f36e8009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 -- 172.21.15.94:0/4152024206 shutdown_connections 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 --2- 172.21.15.94:0/4152024206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36f80643a0 0x7f36f80647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 --2- 172.21.15.94:0/4152024206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f8102240 0x7f36f8102610 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 -- 172.21.15.94:0/4152024206 >> 172.21.15.94:0/4152024206 conn(0x7f36f8071130 msgr2=0x7f36f8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 -- 172.21.15.94:0/4152024206 shutdown_connections 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 -- 172.21.15.94:0/4152024206 wait complete. 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 Processor -- start 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 -- start start 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f80643a0 0x7f36f8112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36f8102240 0x7f36f81131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f81137d0 con 0x7f36f80643a0 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36fccee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f810ce20 con 0x7f36f8102240 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.545+0000 7f36f659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f80643a0 0x7f36f8112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:16.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36f659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f80643a0 0x7f36f8112c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49852/0 (socket says 172.21.15.94:49852) 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36f659c640 1 -- 172.21.15.94:0/3965375500 learned_addr learned my addr 172.21.15.94:0/3965375500 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36effff640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36f8102240 0x7f36f81131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36f659c640 1 -- 172.21.15.94:0/3965375500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36f8102240 msgr2=0x7f36f81131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36f659c640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36f8102240 0x7f36f81131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36f659c640 1 -- 172.21.15.94:0/3965375500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8005040 con 0x7f36f80643a0 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36f659c640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f80643a0 0x7f36f8112c90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f36e8004930 tx=0x7f36e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36eeffd640 1 -- 172.21.15.94:0/3965375500 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36e80094c0 con 0x7f36f80643a0 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36eeffd640 1 -- 172.21.15.94:0/3965375500 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36e8032e50 con 0x7f36f80643a0 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f810d0a0 con 0x7f36f80643a0 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36eeffd640 1 -- 172.21.15.94:0/3965375500 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36e80433b0 con 0x7f36f80643a0 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.546+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f810d560 con 0x7f36f80643a0 2024-04-06T03:52:16.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.547+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f80fd940 con 0x7f36f80643a0 2024-04-06T03:52:16.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.548+0000 7f36eeffd640 1 -- 172.21.15.94:0/3965375500 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f36e802f070 con 0x7f36f80643a0 2024-04-06T03:52:16.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.548+0000 7f36eeffd640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f36e406dc10 0x7f36e40700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:16.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.549+0000 7f36eeffd640 1 -- 172.21.15.94:0/3965375500 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f36e80af640 con 0x7f36f80643a0 2024-04-06T03:52:16.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.549+0000 7f36effff640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f36e406dc10 0x7f36e40700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:16.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.549+0000 7f36effff640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f36e406dc10 0x7f36e40700c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f36f810e550 tx=0x7f36e0009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:16.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.553+0000 7f36eeffd640 1 -- 172.21.15.94:0/3965375500 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f36e8078b20 con 0x7f36f80643a0 2024-04-06T03:52:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.811+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36f810e300 con 0x7f36f80643a0 2024-04-06T03:52:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.811+0000 7f36eeffd640 1 -- 172.21.15.94:0/3965375500 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5213 (secure 0 0 0) 0x7f36e807c1e0 con 0x7f36f80643a0 2024-04-06T03:52:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:16.813 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44331,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6825/1500356136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1500356136},{"type":"v1","addr":"172.21.15.136:6825","nonce":1500356136}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44335,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6827/1840231705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1840231705},{"type":"v1","addr":"172.21.15.136:6827","nonce":1840231705}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:49:01.609121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":0,"incarnation":32,"state":"up:active","state_seq":21,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34400,"qdb_cluster":[34400]},"id":1}]} 2024-04-06T03:52:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 36 2024-04-06T03:52:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f36e406dc10 msgr2=0x7f36e40700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f36e406dc10 0x7f36e40700c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f36f810e550 tx=0x7f36e0009040 comp rx=0 tx=0).stop 2024-04-06T03:52:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f80643a0 msgr2=0x7f36f8112c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f80643a0 0x7f36f8112c90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f36e8004930 tx=0x7f36e8009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 shutdown_connections 2024-04-06T03:52:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f36e406dc10 0x7f36e40700c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36f8102240 0x7f36f81131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 --2- 172.21.15.94:0/3965375500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36f80643a0 0x7f36f8112c90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 >> 172.21.15.94:0/3965375500 conn(0x7f36f8071130 msgr2=0x7f36f80f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:16.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 shutdown_connections 2024-04-06T03:52:16.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.813+0000 7f36fccee640 1 -- 172.21.15.94:0/3965375500 wait complete. 2024-04-06T03:52:16.826 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 11, 'max_mds': 1, 'flags': 18} 2024-04-06T03:52:16.826 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 12 2024-04-06T03:52:16.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.926+0000 7f6f70d9d640 1 -- 172.21.15.94:0/1450660654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 msgr2=0x7f6f6c1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.926+0000 7f6f70d9d640 1 --2- 172.21.15.94:0/1450660654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 0x7f6f6c1012e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0048d0 tx=0x7f6f5c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.926+0000 7f6f70d9d640 1 -- 172.21.15.94:0/1450660654 shutdown_connections 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.926+0000 7f6f70d9d640 1 --2- 172.21.15.94:0/1450660654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f6c1018b0 0x7f6f6c0faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.926+0000 7f6f70d9d640 1 --2- 172.21.15.94:0/1450660654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 0x7f6f6c1012e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.926+0000 7f6f70d9d640 1 -- 172.21.15.94:0/1450660654 >> 172.21.15.94:0/1450660654 conn(0x7f6f6c071e80 msgr2=0x7f6f6c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.926+0000 7f6f70d9d640 1 -- 172.21.15.94:0/1450660654 shutdown_connections 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.926+0000 7f6f70d9d640 1 -- 172.21.15.94:0/1450660654 wait complete. 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f70d9d640 1 Processor -- start 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f70d9d640 1 -- start start 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f70d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 0x7f6f6c10dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f70d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f6c1018b0 0x7f6f6c10e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f70d9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0652c0 con 0x7f6f6c100f10 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f70d9d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c10e7f0 con 0x7f6f6c1018b0 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 0x7f6f6c10dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:16.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 0x7f6f6c10dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49870/0 (socket says 172.21.15.94:49870) 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f6b7fe640 1 -- 172.21.15.94:0/3424187705 learned_addr learned my addr 172.21.15.94:0/3424187705 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f6affd640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f6c1018b0 0x7f6f6c10e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f6b7fe640 1 -- 172.21.15.94:0/3424187705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f6c1018b0 msgr2=0x7f6f6c10e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.927+0000 7f6f6b7fe640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f6c1018b0 0x7f6f6c10e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.928+0000 7f6f6b7fe640 1 -- 172.21.15.94:0/3424187705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c005040 con 0x7f6f6c100f10 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.928+0000 7f6f6b7fe640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 0x7f6f6c10dd70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c034040 tx=0x7f6f5c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.928+0000 7f6f53fff640 1 -- 172.21.15.94:0/3424187705 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f5c0092b0 con 0x7f6f6c100f10 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.928+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6c10ea20 con 0x7f6f6c100f10 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.928+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c116f80 con 0x7f6f6c100f10 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.928+0000 7f6f53fff640 1 -- 172.21.15.94:0/3424187705 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f5c03b400 con 0x7f6f6c100f10 2024-04-06T03:52:16.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.928+0000 7f6f53fff640 1 -- 172.21.15.94:0/3424187705 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f5c0435f0 con 0x7f6f6c100f10 2024-04-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.929+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f58005210 con 0x7f6f6c100f10 2024-04-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.929+0000 7f6f53fff640 1 -- 172.21.15.94:0/3424187705 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f6f5c02f070 con 0x7f6f6c100f10 2024-04-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.930+0000 7f6f53fff640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6f5406dc10 0x7f6f540700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.930+0000 7f6f53fff640 1 -- 172.21.15.94:0/3424187705 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6f5c0af310 con 0x7f6f6c100f10 2024-04-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.930+0000 7f6f6affd640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6f5406dc10 0x7f6f540700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:16.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.930+0000 7f6f6affd640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6f5406dc10 0x7f6f540700c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c064b10 tx=0x7f6f6000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:16.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:16.934+0000 7f6f53fff640 1 -- 172.21.15.94:0/3424187705 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6f5c0788a0 con 0x7f6f6c100f10 2024-04-06T03:52:17.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:16 smithi094 ceph-mon[94053]: pgmap v335: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:17.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:16 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/866620121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-06T03:52:17.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:16 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2780180084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-06T03:52:17.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:16 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:52:17.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:16 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3965375500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-06T03:52:17.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.181+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 12, "format": "json"} v 0) v1 -- 0x7f6f580054c0 con 0x7f6f6c100f10 2024-04-06T03:52:17.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.182+0000 7f6f53fff640 1 -- 172.21.15.94:0/3424187705 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 12, "format": "json"}]=0 dumped fsmap epoch 12 v36) v1 ==== 107+0+4855 (secure 0 0 0) 0x7f6f5c078090 con 0x7f6f6c100f10 2024-04-06T03:52:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:17.183 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.94:6829/3794043365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3794043365},{"type":"v1","addr":"172.21.15.94:6829","nonce":3794043365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:38:04.308946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14460":{"gid":14460,"name":"cephfs.smithi094.qogyzw","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.94:6827/1913624529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1913624529},{"type":"v1","addr":"172.21.15.94:6827","nonce":1913624529}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:17.184 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-06T03:52:17.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6f5406dc10 msgr2=0x7f6f540700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6f5406dc10 0x7f6f540700c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c064b10 tx=0x7f6f6000b040 comp rx=0 tx=0).stop 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 msgr2=0x7f6f6c10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 0x7f6f6c10dd70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c034040 tx=0x7f6f5c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 shutdown_connections 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f6f5406dc10 0x7f6f540700c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f6c1018b0 0x7f6f6c10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 --2- 172.21.15.94:0/3424187705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f6c100f10 0x7f6f6c10dd70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 >> 172.21.15.94:0/3424187705 conn(0x7f6f6c071e80 msgr2=0x7f6f6c103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 shutdown_connections 2024-04-06T03:52:17.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.184+0000 7f6f70d9d640 1 -- 172.21.15.94:0/3424187705 wait complete. 2024-04-06T03:52:17.200 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 13 2024-04-06T03:52:17.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.303+0000 7f496f59d640 1 -- 172.21.15.94:0/639931619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4968100f10 msgr2=0x7f49681012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.303+0000 7f496f59d640 1 --2- 172.21.15.94:0/639931619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4968100f10 0x7f49681012e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f49640048d0 tx=0x7f4964009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.303+0000 7f496f59d640 1 -- 172.21.15.94:0/639931619 shutdown_connections 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.303+0000 7f496f59d640 1 --2- 172.21.15.94:0/639931619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49681018b0 0x7f49680faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.303+0000 7f496f59d640 1 --2- 172.21.15.94:0/639931619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4968100f10 0x7f49681012e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.303+0000 7f496f59d640 1 -- 172.21.15.94:0/639931619 >> 172.21.15.94:0/639931619 conn(0x7f4968071e80 msgr2=0x7f4968072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.303+0000 7f496f59d640 1 -- 172.21.15.94:0/639931619 shutdown_connections 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.303+0000 7f496f59d640 1 -- 172.21.15.94:0/639931619 wait complete. 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496f59d640 1 Processor -- start 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496f59d640 1 -- start start 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496f59d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4968100f10 0x7f496810dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496f59d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49681018b0 0x7f496810e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496f59d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49680652c0 con 0x7f49681018b0 2024-04-06T03:52:17.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496f59d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496810e7f0 con 0x7f4968100f10 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496dd9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49681018b0 0x7f496810e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496dd9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49681018b0 0x7f496810e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49890/0 (socket says 172.21.15.94:49890) 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496dd9a640 1 -- 172.21.15.94:0/1387432501 learned_addr learned my addr 172.21.15.94:0/1387432501 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496e59b640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4968100f10 0x7f496810dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496dd9a640 1 -- 172.21.15.94:0/1387432501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4968100f10 msgr2=0x7f496810dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496dd9a640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4968100f10 0x7f496810dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496dd9a640 1 -- 172.21.15.94:0/1387432501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4964005040 con 0x7f49681018b0 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.304+0000 7f496dd9a640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49681018b0 0x7f496810e2b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4958002a50 tx=0x7f4958004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.305+0000 7f495effd640 1 -- 172.21.15.94:0/1387432501 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4958007c30 con 0x7f49681018b0 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.305+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496810ea80 con 0x7f49681018b0 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.305+0000 7f495effd640 1 -- 172.21.15.94:0/1387432501 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f495800f040 con 0x7f49681018b0 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.305+0000 7f495effd640 1 -- 172.21.15.94:0/1387432501 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495800db50 con 0x7f49681018b0 2024-04-06T03:52:17.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.305+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4968117030 con 0x7f49681018b0 2024-04-06T03:52:17.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.306+0000 7f495effd640 1 -- 172.21.15.94:0/1387432501 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f495801c400 con 0x7f49681018b0 2024-04-06T03:52:17.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.306+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49680fc680 con 0x7f49681018b0 2024-04-06T03:52:17.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.306+0000 7f495effd640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f495406dc10 0x7f49540700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.307+0000 7f495effd640 1 -- 172.21.15.94:0/1387432501 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f495808cf20 con 0x7f49681018b0 2024-04-06T03:52:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.307+0000 7f496e59b640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f495406dc10 0x7f49540700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:17.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.307+0000 7f496e59b640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f495406dc10 0x7f49540700c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4964003080 tx=0x7f4964009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:17.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.311+0000 7f495effd640 1 -- 172.21.15.94:0/1387432501 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4958056410 con 0x7f49681018b0 2024-04-06T03:52:17.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:16 smithi136 ceph-mon[82061]: pgmap v335: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:17.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:16 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/866620121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-06T03:52:17.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:16 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2780180084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-06T03:52:17.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:16 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-06T03:52:17.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:16 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3965375500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-06T03:52:17.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.552+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 13, "format": "json"} v 0) v1 -- 0x7f4968117220 con 0x7f49681018b0 2024-04-06T03:52:17.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.553+0000 7f495effd640 1 -- 172.21.15.94:0/1387432501 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 13, "format": "json"}]=0 dumped fsmap epoch 13 v36) v1 ==== 107+0+4067 (secure 0 0 0) 0x7f4958059ad0 con 0x7f49681018b0 2024-04-06T03:52:17.554 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:17.554 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14472,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.94:6829/3794043365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3794043365},{"type":"v1","addr":"172.21.15.94:6829","nonce":3794043365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:47:37.708895+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:17.555 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 13 2024-04-06T03:52:17.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.555+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f495406dc10 msgr2=0x7f49540700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.555+0000 7f496f59d640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f495406dc10 0x7f49540700c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4964003080 tx=0x7f4964009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.555+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49681018b0 msgr2=0x7f496810e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.555+0000 7f496f59d640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49681018b0 0x7f496810e2b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4958002a50 tx=0x7f4958004600 comp rx=0 tx=0).stop 2024-04-06T03:52:17.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.556+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 shutdown_connections 2024-04-06T03:52:17.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.556+0000 7f496f59d640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f495406dc10 0x7f49540700c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.556+0000 7f496f59d640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49681018b0 0x7f496810e2b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.556+0000 7f496f59d640 1 --2- 172.21.15.94:0/1387432501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4968100f10 0x7f496810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.556+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 >> 172.21.15.94:0/1387432501 conn(0x7f4968071e80 msgr2=0x7f4968103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:17.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.556+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 shutdown_connections 2024-04-06T03:52:17.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.556+0000 7f496f59d640 1 -- 172.21.15.94:0/1387432501 wait complete. 2024-04-06T03:52:17.568 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-04-06T03:52:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.667+0000 7f91e9aee640 1 -- 172.21.15.94:0/3148554327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 msgr2=0x7f91e4100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.667+0000 7f91e9aee640 1 --2- 172.21.15.94:0/3148554327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 0x7f91e4100c80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f91d40048d0 tx=0x7f91d4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.667+0000 7f91e9aee640 1 -- 172.21.15.94:0/3148554327 shutdown_connections 2024-04-06T03:52:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.667+0000 7f91e9aee640 1 --2- 172.21.15.94:0/3148554327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91e40fa8a0 0x7f91e40facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.667+0000 7f91e9aee640 1 --2- 172.21.15.94:0/3148554327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 0x7f91e4100c80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.667+0000 7f91e9aee640 1 -- 172.21.15.94:0/3148554327 >> 172.21.15.94:0/3148554327 conn(0x7f91e4070880 msgr2=0x7f91e4070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.667+0000 7f91e9aee640 1 -- 172.21.15.94:0/3148554327 shutdown_connections 2024-04-06T03:52:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.667+0000 7f91e9aee640 1 -- 172.21.15.94:0/3148554327 wait complete. 2024-04-06T03:52:17.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e9aee640 1 Processor -- start 2024-04-06T03:52:17.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e9aee640 1 -- start start 2024-04-06T03:52:17.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e9aee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91e40fa8a0 0x7f91e4111bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:17.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e9aee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 0x7f91e4112100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e9aee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4112700 con 0x7f91e41008b0 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e9aee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e410bd50 con 0x7f91e40fa8a0 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 0x7f91e4112100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 0x7f91e4112100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48332/0 (socket says 172.21.15.94:48332) 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e2ffd640 1 -- 172.21.15.94:0/3535645264 learned_addr learned my addr 172.21.15.94:0/3535645264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.668+0000 7f91e37fe640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91e40fa8a0 0x7f91e4111bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.669+0000 7f91e2ffd640 1 -- 172.21.15.94:0/3535645264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91e40fa8a0 msgr2=0x7f91e4111bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.669+0000 7f91e2ffd640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91e40fa8a0 0x7f91e4111bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.669+0000 7f91e2ffd640 1 -- 172.21.15.94:0/3535645264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d4005040 con 0x7f91e41008b0 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.669+0000 7f91e2ffd640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 0x7f91e4112100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f91d800eb70 tx=0x7f91d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.669+0000 7f91cbfff640 1 -- 172.21.15.94:0/3535645264 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91d8007c30 con 0x7f91e41008b0 2024-04-06T03:52:17.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.669+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e410bfd0 con 0x7f91e41008b0 2024-04-06T03:52:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.669+0000 7f91cbfff640 1 -- 172.21.15.94:0/3535645264 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91d800f040 con 0x7f91e41008b0 2024-04-06T03:52:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.669+0000 7f91cbfff640 1 -- 172.21.15.94:0/3535645264 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91d80095b0 con 0x7f91e41008b0 2024-04-06T03:52:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.669+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e410c4f0 con 0x7f91e41008b0 2024-04-06T03:52:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.670+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e40fbf90 con 0x7f91e41008b0 2024-04-06T03:52:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.670+0000 7f91cbfff640 1 -- 172.21.15.94:0/3535645264 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f91d8009730 con 0x7f91e41008b0 2024-04-06T03:52:17.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.671+0000 7f91cbfff640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f91cc06dc10 0x7f91cc0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:17.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.671+0000 7f91cbfff640 1 -- 172.21.15.94:0/3535645264 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f91d808d2f0 con 0x7f91e41008b0 2024-04-06T03:52:17.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.671+0000 7f91e37fe640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f91cc06dc10 0x7f91cc0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:17.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.671+0000 7f91e37fe640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f91cc06dc10 0x7f91cc0700c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f91d4003080 tx=0x7f91d4009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:17.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.676+0000 7f91cbfff640 1 -- 172.21.15.94:0/3535645264 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f91d8056880 con 0x7f91e41008b0 2024-04-06T03:52:17.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.945+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7f91e4108230 con 0x7f91e41008b0 2024-04-06T03:52:17.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.946+0000 7f91cbfff640 1 -- 172.21.15.94:0/3535645264 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v36) v1 ==== 107+0+4078 (secure 0 0 0) 0x7f91d8059f40 con 0x7f91e41008b0 2024-04-06T03:52:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:17.947 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:47:37.713536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi094.kvxcqo","rank":0,"incarnation":14,"state":"up:replay","state_seq":2,"addr":"172.21.15.94:6829/3794043365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3794043365},{"type":"v1","addr":"172.21.15.94:6829","nonce":3794043365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:17.948 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 14 2024-04-06T03:52:17.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.947+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f91cc06dc10 msgr2=0x7f91cc0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.947+0000 7f91e9aee640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f91cc06dc10 0x7f91cc0700c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f91d4003080 tx=0x7f91d4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.947+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 msgr2=0x7f91e4112100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:17.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.948+0000 7f91e9aee640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 0x7f91e4112100 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f91d800eb70 tx=0x7f91d800b040 comp rx=0 tx=0).stop 2024-04-06T03:52:17.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.948+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 shutdown_connections 2024-04-06T03:52:17.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.948+0000 7f91e9aee640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f91cc06dc10 0x7f91cc0700c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.948+0000 7f91e9aee640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91e41008b0 0x7f91e4112100 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.948+0000 7f91e9aee640 1 --2- 172.21.15.94:0/3535645264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91e40fa8a0 0x7f91e4111bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:17.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.948+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 >> 172.21.15.94:0/3535645264 conn(0x7f91e4070880 msgr2=0x7f91e40f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:17.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.948+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 shutdown_connections 2024-04-06T03:52:17.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:17.948+0000 7f91e9aee640 1 -- 172.21.15.94:0/3535645264 wait complete. 2024-04-06T03:52:17.960 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-04-06T03:52:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.059+0000 7fe328aee640 1 -- 172.21.15.94:0/3048938355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe324102240 msgr2=0x7fe324102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.059+0000 7fe328aee640 1 --2- 172.21.15.94:0/3048938355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe324102240 0x7fe324102610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe3140048d0 tx=0x7fe314009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.059+0000 7fe328aee640 1 -- 172.21.15.94:0/3048938355 shutdown_connections 2024-04-06T03:52:18.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.059+0000 7fe328aee640 1 --2- 172.21.15.94:0/3048938355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3240643a0 0x7fe3240647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.059+0000 7fe328aee640 1 --2- 172.21.15.94:0/3048938355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe324102240 0x7fe324102610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.059+0000 7fe328aee640 1 -- 172.21.15.94:0/3048938355 >> 172.21.15.94:0/3048938355 conn(0x7fe3240710b0 msgr2=0x7fe3240714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.059+0000 7fe328aee640 1 -- 172.21.15.94:0/3048938355 shutdown_connections 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.059+0000 7fe328aee640 1 -- 172.21.15.94:0/3048938355 wait complete. 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe328aee640 1 Processor -- start 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe328aee640 1 -- start start 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe328aee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3240643a0 0x7fe324112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe328aee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe324102240 0x7fe324113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe328aee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe324113850 con 0x7fe3240643a0 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe328aee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32410cea0 con 0x7fe324102240 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe32259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3240643a0 0x7fe324112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe32259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3240643a0 0x7fe324112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48346/0 (socket says 172.21.15.94:48346) 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe32259c640 1 -- 172.21.15.94:0/1655518885 learned_addr learned my addr 172.21.15.94:0/1655518885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:18.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe319d9b640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe324102240 0x7fe324113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe32259c640 1 -- 172.21.15.94:0/1655518885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe324102240 msgr2=0x7fe324113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe32259c640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe324102240 0x7fe324113250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe32259c640 1 -- 172.21.15.94:0/1655518885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe314005040 con 0x7fe3240643a0 2024-04-06T03:52:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe32259c640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3240643a0 0x7fe324112d10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe314004930 tx=0x7fe314009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe31b7fe640 1 -- 172.21.15.94:0/1655518885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3140094c0 con 0x7fe3240643a0 2024-04-06T03:52:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe31b7fe640 1 -- 172.21.15.94:0/1655518885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe314032e50 con 0x7fe3240643a0 2024-04-06T03:52:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32410d120 con 0x7fe3240643a0 2024-04-06T03:52:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.060+0000 7fe31b7fe640 1 -- 172.21.15.94:0/1655518885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3140433b0 con 0x7fe3240643a0 2024-04-06T03:52:18.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.061+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32410d5e0 con 0x7fe3240643a0 2024-04-06T03:52:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.062+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3240fd940 con 0x7fe3240643a0 2024-04-06T03:52:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.062+0000 7fe31b7fe640 1 -- 172.21.15.94:0/1655518885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fe31402f070 con 0x7fe3240643a0 2024-04-06T03:52:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.062+0000 7fe31b7fe640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe31006dc10 0x7fe3100700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.063+0000 7fe31b7fe640 1 -- 172.21.15.94:0/1655518885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe3140af4f0 con 0x7fe3240643a0 2024-04-06T03:52:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.063+0000 7fe319d9b640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe31006dc10 0x7fe3100700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.063+0000 7fe319d9b640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe31006dc10 0x7fe3100700c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe32410e5d0 tx=0x7fe30c009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:18.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.067+0000 7fe31b7fe640 1 -- 172.21.15.94:0/1655518885 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe3140789d0 con 0x7fe3240643a0 2024-04-06T03:52:18.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:17 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3424187705' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 12, "format": "json"}]: dispatch 2024-04-06T03:52:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:52:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:52:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:17 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1387432501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-04-06T03:52:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:52:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:52:18.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:17 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:52:18.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.286+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7fe32410e380 con 0x7fe3240643a0 2024-04-06T03:52:18.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.287+0000 7fe31b7fe640 1 -- 172.21.15.94:0/1655518885 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v36) v1 ==== 107+0+4083 (secure 0 0 0) 0x7fe31407c090 con 0x7fe3240643a0 2024-04-06T03:52:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:18.288 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:47:45.736683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi094.kvxcqo","rank":0,"incarnation":14,"state":"up:reconnect","state_seq":148,"addr":"172.21.15.94:6829/3794043365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3794043365},{"type":"v1","addr":"172.21.15.94:6829","nonce":3794043365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:18.289 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 15 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe31006dc10 msgr2=0x7fe3100700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe31006dc10 0x7fe3100700c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe32410e5d0 tx=0x7fe30c009040 comp rx=0 tx=0).stop 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3240643a0 msgr2=0x7fe324112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3240643a0 0x7fe324112d10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe314004930 tx=0x7fe314009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 shutdown_connections 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe31006dc10 0x7fe3100700c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe324102240 0x7fe324113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 --2- 172.21.15.94:0/1655518885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3240643a0 0x7fe324112d10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 >> 172.21.15.94:0/1655518885 conn(0x7fe3240710b0 msgr2=0x7fe3240f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 shutdown_connections 2024-04-06T03:52:18.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.289+0000 7fe328aee640 1 -- 172.21.15.94:0/1655518885 wait complete. 2024-04-06T03:52:18.301 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-06T03:52:18.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:17 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3424187705' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 12, "format": "json"}]: dispatch 2024-04-06T03:52:18.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:52:18.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:52:18.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:17 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1387432501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-04-06T03:52:18.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-06T03:52:18.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-06T03:52:18.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:17 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.400+0000 7efc1dd9d640 1 -- 172.21.15.94:0/2089401495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 msgr2=0x7efc181012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.400+0000 7efc1dd9d640 1 --2- 172.21.15.94:0/2089401495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 0x7efc181012e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0048d0 tx=0x7efc0c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.400+0000 7efc1dd9d640 1 -- 172.21.15.94:0/2089401495 shutdown_connections 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.400+0000 7efc1dd9d640 1 --2- 172.21.15.94:0/2089401495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc181018b0 0x7efc180faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.400+0000 7efc1dd9d640 1 --2- 172.21.15.94:0/2089401495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 0x7efc181012e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.400+0000 7efc1dd9d640 1 -- 172.21.15.94:0/2089401495 >> 172.21.15.94:0/2089401495 conn(0x7efc18071e80 msgr2=0x7efc18072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.400+0000 7efc1dd9d640 1 -- 172.21.15.94:0/2089401495 shutdown_connections 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.400+0000 7efc1dd9d640 1 -- 172.21.15.94:0/2089401495 wait complete. 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc1dd9d640 1 Processor -- start 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc1dd9d640 1 -- start start 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc1dd9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 0x7efc1810ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:18.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc1dd9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc181018b0 0x7efc1810e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc1dd9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1810e830 con 0x7efc18100f10 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc1dd9d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1810e9a0 con 0x7efc181018b0 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc1cd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 0x7efc1810ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc1cd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 0x7efc1810ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48374/0 (socket says 172.21.15.94:48374) 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc1cd9b640 1 -- 172.21.15.94:0/236610072 learned_addr learned my addr 172.21.15.94:0/236610072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.401+0000 7efc17fff640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc181018b0 0x7efc1810e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.402+0000 7efc1cd9b640 1 -- 172.21.15.94:0/236610072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc181018b0 msgr2=0x7efc1810e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.402+0000 7efc1cd9b640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc181018b0 0x7efc1810e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.402+0000 7efc1cd9b640 1 -- 172.21.15.94:0/236610072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c005040 con 0x7efc18100f10 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.402+0000 7efc1cd9b640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 0x7efc1810ddb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0047f0 tx=0x7efc0c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.402+0000 7efc157fa640 1 -- 172.21.15.94:0/236610072 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0c0094c0 con 0x7efc18100f10 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.402+0000 7efc157fa640 1 -- 172.21.15.94:0/236610072 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc0c032e50 con 0x7efc18100f10 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.402+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1810ec20 con 0x7efc18100f10 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.402+0000 7efc157fa640 1 -- 172.21.15.94:0/236610072 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0c0435f0 con 0x7efc18100f10 2024-04-06T03:52:18.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.402+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc18117070 con 0x7efc18100f10 2024-04-06T03:52:18.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.403+0000 7efc157fa640 1 -- 172.21.15.94:0/236610072 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7efc0c02f070 con 0x7efc18100f10 2024-04-06T03:52:18.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.403+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc180fc680 con 0x7efc18100f10 2024-04-06T03:52:18.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.404+0000 7efc157fa640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7efc0806dc20 0x7efc080700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.404+0000 7efc157fa640 1 -- 172.21.15.94:0/236610072 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7efc0c0af320 con 0x7efc18100f10 2024-04-06T03:52:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.404+0000 7efc17fff640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7efc0806dc20 0x7efc080700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:18.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.404+0000 7efc17fff640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7efc0806dc20 0x7efc080700d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efc04006fd0 tx=0x7efc04008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:18.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.408+0000 7efc157fa640 1 -- 172.21.15.94:0/236610072 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efc0c078800 con 0x7efc18100f10 2024-04-06T03:52:18.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.625+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7efc1804bcc0 con 0x7efc18100f10 2024-04-06T03:52:18.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.626+0000 7efc157fa640 1 -- 172.21.15.94:0/236610072 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v36) v1 ==== 107+0+4080 (secure 0 0 0) 0x7efc0c07bec0 con 0x7efc18100f10 2024-04-06T03:52:18.627 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:18.627 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:47:45.797069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi094.kvxcqo","rank":0,"incarnation":14,"state":"up:rejoin","state_seq":149,"addr":"172.21.15.94:6829/3794043365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3794043365},{"type":"v1","addr":"172.21.15.94:6829","nonce":3794043365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:18.628 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 16 2024-04-06T03:52:18.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7efc0806dc20 msgr2=0x7efc080700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7efc0806dc20 0x7efc080700d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efc04006fd0 tx=0x7efc04008040 comp rx=0 tx=0).stop 2024-04-06T03:52:18.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 msgr2=0x7efc1810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 0x7efc1810ddb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0047f0 tx=0x7efc0c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 shutdown_connections 2024-04-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7efc0806dc20 0x7efc080700d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc181018b0 0x7efc1810e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 --2- 172.21.15.94:0/236610072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc18100f10 0x7efc1810ddb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 >> 172.21.15.94:0/236610072 conn(0x7efc18071e80 msgr2=0x7efc18103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 shutdown_connections 2024-04-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.628+0000 7efc1dd9d640 1 -- 172.21.15.94:0/236610072 wait complete. 2024-04-06T03:52:18.640 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-06T03:52:18.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.738+0000 7f5bb219d640 1 -- 172.21.15.94:0/1848541750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac100f10 msgr2=0x7f5bac1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.738+0000 7f5bb219d640 1 --2- 172.21.15.94:0/1848541750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac100f10 0x7f5bac1012e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80048d0 tx=0x7f5ba8009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.738+0000 7f5bb219d640 1 -- 172.21.15.94:0/1848541750 shutdown_connections 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.738+0000 7f5bb219d640 1 --2- 172.21.15.94:0/1848541750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bac1018b0 0x7f5bac0faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.738+0000 7f5bb219d640 1 --2- 172.21.15.94:0/1848541750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac100f10 0x7f5bac1012e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.738+0000 7f5bb219d640 1 -- 172.21.15.94:0/1848541750 >> 172.21.15.94:0/1848541750 conn(0x7f5bac071e80 msgr2=0x7f5bac072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.738+0000 7f5bb219d640 1 -- 172.21.15.94:0/1848541750 shutdown_connections 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.738+0000 7f5bb219d640 1 -- 172.21.15.94:0/1848541750 wait complete. 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb219d640 1 Processor -- start 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb219d640 1 -- start start 2024-04-06T03:52:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb219d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bac100f10 0x7f5bac111a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb219d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac1018b0 0x7f5bac111f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb219d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac112590 con 0x7f5bac1018b0 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb219d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac10bc30 con 0x7f5bac100f10 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb099a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac1018b0 0x7f5bac111f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb099a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac1018b0 0x7f5bac111f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48394/0 (socket says 172.21.15.94:48394) 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb099a640 1 -- 172.21.15.94:0/652344555 learned_addr learned my addr 172.21.15.94:0/652344555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.739+0000 7f5bb119b640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bac100f10 0x7f5bac111a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.740+0000 7f5bb099a640 1 -- 172.21.15.94:0/652344555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bac100f10 msgr2=0x7f5bac111a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.740+0000 7f5bb099a640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bac100f10 0x7f5bac111a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.740+0000 7f5bb099a640 1 -- 172.21.15.94:0/652344555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8005040 con 0x7f5bac1018b0 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.740+0000 7f5bb099a640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac1018b0 0x7f5bac111f90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c002a50 tx=0x7f5b9c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.740+0000 7f5ba1ffb640 1 -- 172.21.15.94:0/652344555 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b9c007c30 con 0x7f5bac1018b0 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.740+0000 7f5ba1ffb640 1 -- 172.21.15.94:0/652344555 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b9c00f040 con 0x7f5bac1018b0 2024-04-06T03:52:18.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.740+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac10bf10 con 0x7f5bac1018b0 2024-04-06T03:52:18.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.740+0000 7f5ba1ffb640 1 -- 172.21.15.94:0/652344555 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b9c00db50 con 0x7f5bac1018b0 2024-04-06T03:52:18.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.740+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac10c430 con 0x7f5bac1018b0 2024-04-06T03:52:18.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.741+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac0fc680 con 0x7f5bac1018b0 2024-04-06T03:52:18.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.741+0000 7f5ba1ffb640 1 -- 172.21.15.94:0/652344555 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f5b9c00dcf0 con 0x7f5bac1018b0 2024-04-06T03:52:18.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.742+0000 7f5ba1ffb640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b9806dc10 0x7f5b980700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:18.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.742+0000 7f5bb119b640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b9806dc10 0x7f5b980700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:18.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.742+0000 7f5ba1ffb640 1 -- 172.21.15.94:0/652344555 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f5b9c08ce60 con 0x7f5bac1018b0 2024-04-06T03:52:18.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.742+0000 7f5bb119b640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b9806dc10 0x7f5b980700c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8003080 tx=0x7f5ba8009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:18.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.746+0000 7f5ba1ffb640 1 -- 172.21.15.94:0/652344555 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5b9c0563c0 con 0x7f5bac1018b0 2024-04-06T03:52:18.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.983+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f5bac108070 con 0x7f5bac1018b0 2024-04-06T03:52:18.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.983+0000 7f5ba1ffb640 1 -- 172.21.15.94:0/652344555 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v36) v1 ==== 107+0+4089 (secure 0 0 0) 0x7f5b9c059a80 con 0x7f5bac1018b0 2024-04-06T03:52:18.985 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:18.985 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:47:47.799730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi094.kvxcqo","rank":0,"incarnation":14,"state":"up:active","state_seq":150,"addr":"172.21.15.94:6829/3794043365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3794043365},{"type":"v1","addr":"172.21.15.94:6829","nonce":3794043365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[14472]},"id":1}]} 2024-04-06T03:52:18.986 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 17 2024-04-06T03:52:18.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.985+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b9806dc10 msgr2=0x7f5b980700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.985+0000 7f5bb219d640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b9806dc10 0x7f5b980700c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8003080 tx=0x7f5ba8009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.985+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac1018b0 msgr2=0x7f5bac111f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:18.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.985+0000 7f5bb219d640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac1018b0 0x7f5bac111f90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c002a50 tx=0x7f5b9c004600 comp rx=0 tx=0).stop 2024-04-06T03:52:18.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.986+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 shutdown_connections 2024-04-06T03:52:18.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.986+0000 7f5bb219d640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f5b9806dc10 0x7f5b980700c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.986+0000 7f5bb219d640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bac1018b0 0x7f5bac111f90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.986+0000 7f5bb219d640 1 --2- 172.21.15.94:0/652344555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bac100f10 0x7f5bac111a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:18.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.986+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 >> 172.21.15.94:0/652344555 conn(0x7f5bac071e80 msgr2=0x7f5bac103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:18.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.986+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 shutdown_connections 2024-04-06T03:52:18.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:18.986+0000 7f5bb219d640 1 -- 172.21.15.94:0/652344555 wait complete. 2024-04-06T03:52:18.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-06T03:52:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 -- 172.21.15.94:0/1657483414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77500fb110 msgr2=0x7f77500fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 --2- 172.21.15.94:0/1657483414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77500fb110 0x7f77500fb560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f77440048d0 tx=0x7f7744009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 -- 172.21.15.94:0/1657483414 shutdown_connections 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 --2- 172.21.15.94:0/1657483414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77500fb110 0x7f77500fb560 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 --2- 172.21.15.94:0/1657483414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7750101140 0x7f7750101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 -- 172.21.15.94:0/1657483414 >> 172.21.15.94:0/1657483414 conn(0x7f77500710b0 msgr2=0x7f77500714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 -- 172.21.15.94:0/1657483414 shutdown_connections 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 -- 172.21.15.94:0/1657483414 wait complete. 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 Processor -- start 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.095+0000 7f7754cee640 1 -- start start 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f7754cee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77500fb110 0x7f775010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f7754cee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7750101140 0x7f775010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f7754cee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750065330 con 0x7f7750101140 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f7754cee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775010e960 con 0x7f77500fb110 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f774dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7750101140 0x7f775010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f774dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7750101140 0x7f775010e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48422/0 (socket says 172.21.15.94:48422) 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f774dd9b640 1 -- 172.21.15.94:0/1148392050 learned_addr learned my addr 172.21.15.94:0/1148392050 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:19.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f774e59c640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77500fb110 0x7f775010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f774dd9b640 1 -- 172.21.15.94:0/1148392050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77500fb110 msgr2=0x7f775010deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f774dd9b640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77500fb110 0x7f775010deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f774dd9b640 1 -- 172.21.15.94:0/1148392050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744005040 con 0x7f7750101140 2024-04-06T03:52:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f774dd9b640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7750101140 0x7f775010e3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f77440041d0 tx=0x7f7744009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f773effd640 1 -- 172.21.15.94:0/1148392050 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77440092e0 con 0x7f7750101140 2024-04-06T03:52:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775010ebe0 con 0x7f7750101140 2024-04-06T03:52:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f773effd640 1 -- 172.21.15.94:0/1148392050 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7744009440 con 0x7f7750101140 2024-04-06T03:52:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f773effd640 1 -- 172.21.15.94:0/1148392050 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7744043640 con 0x7f7750101140 2024-04-06T03:52:19.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.096+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7750117170 con 0x7f7750101140 2024-04-06T03:52:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.098+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77500fc800 con 0x7f7750101140 2024-04-06T03:52:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.098+0000 7f773effd640 1 -- 172.21.15.94:0/1148392050 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f774403b9f0 con 0x7f7750101140 2024-04-06T03:52:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.098+0000 7f773effd640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f773806dc10 0x7f77380700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.098+0000 7f773effd640 1 -- 172.21.15.94:0/1148392050 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f77440b03f0 con 0x7f7750101140 2024-04-06T03:52:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.098+0000 7f774e59c640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f773806dc10 0x7f77380700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:19.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.099+0000 7f774e59c640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f773806dc10 0x7f77380700c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f77400041d0 tx=0x7f7740009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:19.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.103+0000 7f773effd640 1 -- 172.21.15.94:0/1148392050 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7744079950 con 0x7f7750101140 2024-04-06T03:52:19.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:18 smithi094 ceph-mon[94053]: pgmap v336: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:19.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:18 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3535645264' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-06T03:52:19.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:18 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1655518885' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-06T03:52:19.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:18 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/236610072' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-06T03:52:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.344+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f775004bcc0 con 0x7f7750101140 2024-04-06T03:52:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.345+0000 7f773effd640 1 -- 172.21.15.94:0/1148392050 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v36) v1 ==== 107+0+4936 (secure 0 0 0) 0x7f774407d010 con 0x7f7750101140 2024-04-06T03:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:19.346 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:47:47.799730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi094.kvxcqo","rank":0,"incarnation":14,"state":"up:active","state_seq":150,"addr":"172.21.15.94:6829/3794043365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3794043365},{"type":"v1","addr":"172.21.15.94:6829","nonce":3794043365}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14472,"qdb_cluster":[14472]},"id":1}]} 2024-04-06T03:52:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 18 2024-04-06T03:52:19.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:18 smithi136 ceph-mon[82061]: pgmap v336: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:19.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:18 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3535645264' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-06T03:52:19.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:18 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1655518885' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-06T03:52:19.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:18 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/236610072' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-06T03:52:19.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f773806dc10 msgr2=0x7f77380700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f773806dc10 0x7f77380700c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f77400041d0 tx=0x7f7740009040 comp rx=0 tx=0).stop 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7750101140 msgr2=0x7f775010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7750101140 0x7f775010e3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f77440041d0 tx=0x7f7744009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 shutdown_connections 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f773806dc10 0x7f77380700c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7750101140 0x7f775010e3f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 --2- 172.21.15.94:0/1148392050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77500fb110 0x7f775010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 >> 172.21.15.94:0/1148392050 conn(0x7f77500710b0 msgr2=0x7f77500f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 shutdown_connections 2024-04-06T03:52:19.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.347+0000 7f7754cee640 1 -- 172.21.15.94:0/1148392050 wait complete. 2024-04-06T03:52:19.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-06T03:52:19.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.457+0000 7f54a5eee640 1 -- 172.21.15.94:0/4025435148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 msgr2=0x7f54a0102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.457+0000 7f54a5eee640 1 --2- 172.21.15.94:0/4025435148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 0x7f54a0102580 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f54900048d0 tx=0x7f5490009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.457+0000 7f54a5eee640 1 -- 172.21.15.94:0/4025435148 shutdown_connections 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.457+0000 7f54a5eee640 1 --2- 172.21.15.94:0/4025435148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54a0102b50 0x7f54a00fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.457+0000 7f54a5eee640 1 --2- 172.21.15.94:0/4025435148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 0x7f54a0102580 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.457+0000 7f54a5eee640 1 -- 172.21.15.94:0/4025435148 >> 172.21.15.94:0/4025435148 conn(0x7f54a00f7dc0 msgr2=0x7f54a00fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.457+0000 7f54a5eee640 1 -- 172.21.15.94:0/4025435148 shutdown_connections 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.457+0000 7f54a5eee640 1 -- 172.21.15.94:0/4025435148 wait complete. 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.458+0000 7f54a5eee640 1 Processor -- start 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.458+0000 7f54a5eee640 1 -- start start 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.458+0000 7f54a5eee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 0x7f54a006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.458+0000 7f54a5eee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54a0102b50 0x7f54a006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.458+0000 7f54a5eee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a0071740 con 0x7f54a01021b0 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.458+0000 7f54a5eee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a0070250 con 0x7f54a0102b50 2024-04-06T03:52:19.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.458+0000 7f549f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 0x7f54a006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.458+0000 7f549f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 0x7f54a006f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48432/0 (socket says 172.21.15.94:48432) 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.458+0000 7f549f7fe640 1 -- 172.21.15.94:0/4058886076 learned_addr learned my addr 172.21.15.94:0/4058886076 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f549effd640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54a0102b50 0x7f54a006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f549f7fe640 1 -- 172.21.15.94:0/4058886076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54a0102b50 msgr2=0x7f54a006fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f549f7fe640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54a0102b50 0x7f54a006fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f549f7fe640 1 -- 172.21.15.94:0/4058886076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490005040 con 0x7f54a01021b0 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f549f7fe640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 0x7f54a006f7d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5490004930 tx=0x7f5490009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f5487fff640 1 -- 172.21.15.94:0/4058886076 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54900094c0 con 0x7f54a01021b0 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a0070470 con 0x7f54a01021b0 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f5487fff640 1 -- 172.21.15.94:0/4058886076 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5490032e50 con 0x7f54a01021b0 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f5487fff640 1 -- 172.21.15.94:0/4058886076 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54900433b0 con 0x7f54a01021b0 2024-04-06T03:52:19.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.459+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a0119430 con 0x7f54a01021b0 2024-04-06T03:52:19.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.460+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a00fd920 con 0x7f54a01021b0 2024-04-06T03:52:19.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.461+0000 7f5487fff640 1 -- 172.21.15.94:0/4058886076 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f549002f070 con 0x7f54a01021b0 2024-04-06T03:52:19.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.461+0000 7f5487fff640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f548806dc10 0x7f54880700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:19.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.461+0000 7f5487fff640 1 -- 172.21.15.94:0/4058886076 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f54900af5a0 con 0x7f54a01021b0 2024-04-06T03:52:19.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.461+0000 7f549effd640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f548806dc10 0x7f54880700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:19.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.462+0000 7f549effd640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f548806dc10 0x7f54880700c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54a0070f00 tx=0x7f5494008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:19.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.468+0000 7f5487fff640 1 -- 172.21.15.94:0/4058886076 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5490078a80 con 0x7f54a01021b0 2024-04-06T03:52:19.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.681+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f54a004bcc0 con 0x7f54a01021b0 2024-04-06T03:52:19.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.681+0000 7f5487fff640 1 -- 172.21.15.94:0/4058886076 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v36) v1 ==== 107+0+4135 (secure 0 0 0) 0x7f549007c140 con 0x7f54a01021b0 2024-04-06T03:52:19.683 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:19.683 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:03.697842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:19.683 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f548806dc10 msgr2=0x7f54880700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f548806dc10 0x7f54880700c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54a0070f00 tx=0x7f5494008040 comp rx=0 tx=0).stop 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 msgr2=0x7f54a006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 0x7f54a006f7d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5490004930 tx=0x7f5490009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 shutdown_connections 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f548806dc10 0x7f54880700c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54a0102b50 0x7f54a006fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 --2- 172.21.15.94:0/4058886076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54a01021b0 0x7f54a006f7d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 >> 172.21.15.94:0/4058886076 conn(0x7f54a00f7dc0 msgr2=0x7f54a00f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 shutdown_connections 2024-04-06T03:52:19.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.683+0000 7f54a5eee640 1 -- 172.21.15.94:0/4058886076 wait complete. 2024-04-06T03:52:19.696 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-06T03:52:19.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.793+0000 7f77e18ee640 1 -- 172.21.15.94:0/1922374994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc0fb110 msgr2=0x7f77dc0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.793+0000 7f77e18ee640 1 --2- 172.21.15.94:0/1922374994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc0fb110 0x7f77dc0fb560 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0048d0 tx=0x7f77cc009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.793+0000 7f77e18ee640 1 -- 172.21.15.94:0/1922374994 shutdown_connections 2024-04-06T03:52:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.793+0000 7f77e18ee640 1 --2- 172.21.15.94:0/1922374994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc0fb110 0x7f77dc0fb560 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.793+0000 7f77e18ee640 1 --2- 172.21.15.94:0/1922374994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77dc101140 0x7f77dc101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.793+0000 7f77e18ee640 1 -- 172.21.15.94:0/1922374994 >> 172.21.15.94:0/1922374994 conn(0x7f77dc0710b0 msgr2=0x7f77dc0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.793+0000 7f77e18ee640 1 -- 172.21.15.94:0/1922374994 shutdown_connections 2024-04-06T03:52:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.793+0000 7f77e18ee640 1 -- 172.21.15.94:0/1922374994 wait complete. 2024-04-06T03:52:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.794+0000 7f77e18ee640 1 Processor -- start 2024-04-06T03:52:19.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.794+0000 7f77e18ee640 1 -- start start 2024-04-06T03:52:19.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.794+0000 7f77e18ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77dc0fb110 0x7f77dc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77e18ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc101140 0x7f77dc10e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77e18ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc065330 con 0x7f77dc101140 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77e18ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc10e960 con 0x7f77dc0fb110 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77da7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc101140 0x7f77dc10e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77da7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc101140 0x7f77dc10e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48444/0 (socket says 172.21.15.94:48444) 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77da7fc640 1 -- 172.21.15.94:0/3212513159 learned_addr learned my addr 172.21.15.94:0/3212513159 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77daffd640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77dc0fb110 0x7f77dc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77da7fc640 1 -- 172.21.15.94:0/3212513159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77dc0fb110 msgr2=0x7f77dc10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77da7fc640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77dc0fb110 0x7f77dc10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77da7fc640 1 -- 172.21.15.94:0/3212513159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc005040 con 0x7f77dc101140 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77da7fc640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc101140 0x7f77dc10e3f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0041d0 tx=0x7f77cc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77c37fe640 1 -- 172.21.15.94:0/3212513159 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77cc030870 con 0x7f77dc101140 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc10ebe0 con 0x7f77dc101140 2024-04-06T03:52:19.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77c37fe640 1 -- 172.21.15.94:0/3212513159 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77cc0092e0 con 0x7f77dc101140 2024-04-06T03:52:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77c37fe640 1 -- 172.21.15.94:0/3212513159 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77cc043400 con 0x7f77dc101140 2024-04-06T03:52:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.795+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc117170 con 0x7f77dc101140 2024-04-06T03:52:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.796+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc0fc800 con 0x7f77dc101140 2024-04-06T03:52:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.797+0000 7f77c37fe640 1 -- 172.21.15.94:0/3212513159 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f77cc03b420 con 0x7f77dc101140 2024-04-06T03:52:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.797+0000 7f77c37fe640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f77c406dc10 0x7f77c40700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.797+0000 7f77c37fe640 1 -- 172.21.15.94:0/3212513159 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f77cc0b0410 con 0x7f77dc101140 2024-04-06T03:52:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.797+0000 7f77daffd640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f77c406dc10 0x7f77c40700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:19.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.798+0000 7f77daffd640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f77c406dc10 0x7f77c40700c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77d0007fd0 tx=0x7f77d0009e90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:19.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:19.802+0000 7f77c37fe640 1 -- 172.21.15.94:0/3212513159 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f77cc079970 con 0x7f77dc101140 2024-04-06T03:52:20.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.020+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f77dc04bcc0 con 0x7f77dc101140 2024-04-06T03:52:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.020+0000 7f77c37fe640 1 -- 172.21.15.94:0/3212513159 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v36) v1 ==== 107+0+4146 (secure 0 0 0) 0x7f77cc07d030 con 0x7f77dc101140 2024-04-06T03:52:20.022 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:20.022 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:03.702027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":0,"incarnation":20,"state":"up:replay","state_seq":3,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:20.022 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-04-06T03:52:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.022+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f77c406dc10 msgr2=0x7f77c40700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.022+0000 7f77e18ee640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f77c406dc10 0x7f77c40700c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77d0007fd0 tx=0x7f77d0009e90 comp rx=0 tx=0).stop 2024-04-06T03:52:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.022+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc101140 msgr2=0x7f77dc10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.022+0000 7f77e18ee640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc101140 0x7f77dc10e3f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0041d0 tx=0x7f77cc009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.022+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 shutdown_connections 2024-04-06T03:52:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.022+0000 7f77e18ee640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f77c406dc10 0x7f77c40700c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.022+0000 7f77e18ee640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77dc101140 0x7f77dc10e3f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.022+0000 7f77e18ee640 1 --2- 172.21.15.94:0/3212513159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77dc0fb110 0x7f77dc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.022+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 >> 172.21.15.94:0/3212513159 conn(0x7f77dc0710b0 msgr2=0x7f77dc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:20.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.023+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 shutdown_connections 2024-04-06T03:52:20.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.023+0000 7f77e18ee640 1 -- 172.21.15.94:0/3212513159 wait complete. 2024-04-06T03:52:20.037 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:19 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/652344555' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-06T03:52:20.037 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:19 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1148392050' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-06T03:52:20.037 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:19 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/4058886076' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-06T03:52:20.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-06T03:52:20.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.136+0000 7f173219d640 1 -- 172.21.15.94:0/1310690778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 msgr2=0x7f172c0faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.136+0000 7f173219d640 1 --2- 172.21.15.94:0/1310690778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 0x7f172c0faf40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f171c0048d0 tx=0x7f171c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.136+0000 7f173219d640 1 -- 172.21.15.94:0/1310690778 shutdown_connections 2024-04-06T03:52:20.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.136+0000 7f173219d640 1 --2- 172.21.15.94:0/1310690778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 0x7f172c0faf40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.136+0000 7f173219d640 1 --2- 172.21.15.94:0/1310690778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c100f10 0x7f172c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.136+0000 7f173219d640 1 -- 172.21.15.94:0/1310690778 >> 172.21.15.94:0/1310690778 conn(0x7f172c071e80 msgr2=0x7f172c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:20.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.136+0000 7f173219d640 1 -- 172.21.15.94:0/1310690778 shutdown_connections 2024-04-06T03:52:20.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.136+0000 7f173219d640 1 -- 172.21.15.94:0/1310690778 wait complete. 2024-04-06T03:52:20.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173219d640 1 Processor -- start 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173219d640 1 -- start start 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173219d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c100f10 0x7f172c10bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173219d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 0x7f172c10c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173219d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0651e0 con 0x7f172c1018b0 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173219d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c065320 con 0x7f172c100f10 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173099a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 0x7f172c10c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173099a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 0x7f172c10c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48464/0 (socket says 172.21.15.94:48464) 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173099a640 1 -- 172.21.15.94:0/1503374623 learned_addr learned my addr 172.21.15.94:0/1503374623 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173119b640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c100f10 0x7f172c10bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173099a640 1 -- 172.21.15.94:0/1503374623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c100f10 msgr2=0x7f172c10bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173099a640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c100f10 0x7f172c10bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173099a640 1 -- 172.21.15.94:0/1503374623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c005040 con 0x7f172c1018b0 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.137+0000 7f173119b640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c100f10 0x7f172c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:52:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.138+0000 7f173099a640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 0x7f172c10c040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f171c004930 tx=0x7f171c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.138+0000 7f1721ffb640 1 -- 172.21.15.94:0/1503374623 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f171c0094c0 con 0x7f172c1018b0 2024-04-06T03:52:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.138+0000 7f1721ffb640 1 -- 172.21.15.94:0/1503374623 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f171c032e50 con 0x7f172c1018b0 2024-04-06T03:52:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.138+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c10c670 con 0x7f172c1018b0 2024-04-06T03:52:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.138+0000 7f1721ffb640 1 -- 172.21.15.94:0/1503374623 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f171c0433b0 con 0x7f172c1018b0 2024-04-06T03:52:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.138+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c119310 con 0x7f172c1018b0 2024-04-06T03:52:20.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.139+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172c0fc680 con 0x7f172c1018b0 2024-04-06T03:52:20.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.139+0000 7f1721ffb640 1 -- 172.21.15.94:0/1503374623 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f171c02f070 con 0x7f172c1018b0 2024-04-06T03:52:20.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.140+0000 7f1721ffb640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f171806dc10 0x7f17180700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:20.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.140+0000 7f1721ffb640 1 -- 172.21.15.94:0/1503374623 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f171c0af570 con 0x7f172c1018b0 2024-04-06T03:52:20.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.140+0000 7f173119b640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f171806dc10 0x7f17180700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:20.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.140+0000 7f173119b640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f171806dc10 0x7f17180700c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f17240041d0 tx=0x7f1724009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:20.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.145+0000 7f1721ffb640 1 -- 172.21.15.94:0/1503374623 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f171c078a50 con 0x7f172c1018b0 2024-04-06T03:52:20.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:19 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/652344555' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-06T03:52:20.351 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:19 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1148392050' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-06T03:52:20.352 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:19 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/4058886076' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-06T03:52:20.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.459+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f172c1080a0 con 0x7f172c1018b0 2024-04-06T03:52:20.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.459+0000 7f1721ffb640 1 -- 172.21.15.94:0/1503374623 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v36) v1 ==== 107+0+4151 (secure 0 0 0) 0x7f171c07c110 con 0x7f172c1018b0 2024-04-06T03:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:20.461 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:11.501083+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":0,"incarnation":20,"state":"up:reconnect","state_seq":155,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:20.461 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-04-06T03:52:20.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.461+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f171806dc10 msgr2=0x7f17180700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.461+0000 7f173219d640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f171806dc10 0x7f17180700c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f17240041d0 tx=0x7f1724009040 comp rx=0 tx=0).stop 2024-04-06T03:52:20.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.461+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 msgr2=0x7f172c10c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.461+0000 7f173219d640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 0x7f172c10c040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f171c004930 tx=0x7f171c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.461+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 shutdown_connections 2024-04-06T03:52:20.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.461+0000 7f173219d640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f171806dc10 0x7f17180700c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.461+0000 7f173219d640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c1018b0 0x7f172c10c040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.461+0000 7f173219d640 1 --2- 172.21.15.94:0/1503374623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c100f10 0x7f172c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.461+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 >> 172.21.15.94:0/1503374623 conn(0x7f172c071e80 msgr2=0x7f172c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.462+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 shutdown_connections 2024-04-06T03:52:20.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.462+0000 7f173219d640 1 -- 172.21.15.94:0/1503374623 wait complete. 2024-04-06T03:52:20.476 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-06T03:52:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3127028989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc1018b0 msgr2=0x7fc4bc0faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 --2- 172.21.15.94:0/3127028989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc1018b0 0x7fc4bc0faf40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40048d0 tx=0x7fc4a4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3127028989 shutdown_connections 2024-04-06T03:52:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 --2- 172.21.15.94:0/3127028989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc1018b0 0x7fc4bc0faf40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 --2- 172.21.15.94:0/3127028989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4bc100f10 0x7fc4bc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3127028989 >> 172.21.15.94:0/3127028989 conn(0x7fc4bc071e80 msgr2=0x7fc4bc072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3127028989 shutdown_connections 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3127028989 wait complete. 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 Processor -- start 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 -- start start 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc100f10 0x7fc4bc10dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4bc1018b0 0x7fc4bc10e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc065310 con 0x7fc4bc100f10 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.576+0000 7fc4c09fe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc10e790 con 0x7fc4bc1018b0 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4bad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc100f10 0x7fc4bc10dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4bad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc100f10 0x7fc4bc10dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48490/0 (socket says 172.21.15.94:48490) 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4bad9d640 1 -- 172.21.15.94:0/3033265218 learned_addr learned my addr 172.21.15.94:0/3033265218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4b259c640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4bc1018b0 0x7fc4bc10e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4bad9d640 1 -- 172.21.15.94:0/3033265218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4bc1018b0 msgr2=0x7fc4bc10e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4bad9d640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4bc1018b0 0x7fc4bc10e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4bad9d640 1 -- 172.21.15.94:0/3033265218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a4005040 con 0x7fc4bc100f10 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4bad9d640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc100f10 0x7fc4bc10dd10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac002a50 tx=0x7fc4ac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4b3fff640 1 -- 172.21.15.94:0/3033265218 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4ac007c30 con 0x7fc4bc100f10 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc10ea70 con 0x7fc4bc100f10 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4b3fff640 1 -- 172.21.15.94:0/3033265218 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4ac00f040 con 0x7fc4bc100f10 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4b3fff640 1 -- 172.21.15.94:0/3033265218 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4ac00db50 con 0x7fc4bc100f10 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.577+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc117030 con 0x7fc4bc100f10 2024-04-06T03:52:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.578+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc0fc680 con 0x7fc4bc100f10 2024-04-06T03:52:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.579+0000 7fc4b3fff640 1 -- 172.21.15.94:0/3033265218 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fc4ac01c400 con 0x7fc4bc100f10 2024-04-06T03:52:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.579+0000 7fc4b3fff640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc4a806dc10 0x7fc4a80700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.579+0000 7fc4b3fff640 1 -- 172.21.15.94:0/3033265218 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fc4ac08cf20 con 0x7fc4bc100f10 2024-04-06T03:52:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.579+0000 7fc4b259c640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc4a806dc10 0x7fc4a80700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.579+0000 7fc4b259c640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc4a806dc10 0x7fc4a80700c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc064b60 tx=0x7fc4a4009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:20.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.583+0000 7fc4b3fff640 1 -- 172.21.15.94:0/3033265218 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc4ac0c1280 con 0x7fc4bc100f10 2024-04-06T03:52:20.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.804+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fc4bc04bcc0 con 0x7fc4bc100f10 2024-04-06T03:52:20.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.804+0000 7fc4b3fff640 1 -- 172.21.15.94:0/3033265218 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v36) v1 ==== 107+0+4148 (secure 0 0 0) 0x7fc4ac056400 con 0x7fc4bc100f10 2024-04-06T03:52:20.806 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:20.806 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:12.258213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":0,"incarnation":20,"state":"up:rejoin","state_seq":156,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:20.806 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 22 2024-04-06T03:52:20.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc4a806dc10 msgr2=0x7fc4a80700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc4a806dc10 0x7fc4a80700c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc064b60 tx=0x7fc4a4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc100f10 msgr2=0x7fc4bc10dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc100f10 0x7fc4bc10dd10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac002a50 tx=0x7fc4ac004600 comp rx=0 tx=0).stop 2024-04-06T03:52:20.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 shutdown_connections 2024-04-06T03:52:20.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc4a806dc10 0x7fc4a80700c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4bc1018b0 0x7fc4bc10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 --2- 172.21.15.94:0/3033265218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4bc100f10 0x7fc4bc10dd10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 >> 172.21.15.94:0/3033265218 conn(0x7fc4bc071e80 msgr2=0x7fc4bc0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:20.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 shutdown_connections 2024-04-06T03:52:20.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.807+0000 7fc4c09fe640 1 -- 172.21.15.94:0/3033265218 wait complete. 2024-04-06T03:52:20.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-06T03:52:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 -- 172.21.15.94:0/2615668943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 msgr2=0x7f96681012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 --2- 172.21.15.94:0/2615668943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 0x7f96681012e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f96640048d0 tx=0x7f9664009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 -- 172.21.15.94:0/2615668943 shutdown_connections 2024-04-06T03:52:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 --2- 172.21.15.94:0/2615668943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96681018b0 0x7f96680faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 --2- 172.21.15.94:0/2615668943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 0x7f96681012e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 -- 172.21.15.94:0/2615668943 >> 172.21.15.94:0/2615668943 conn(0x7f9668071e80 msgr2=0x7f9668072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 -- 172.21.15.94:0/2615668943 shutdown_connections 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 -- 172.21.15.94:0/2615668943 wait complete. 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 Processor -- start 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.919+0000 7f966ed9d640 1 -- start start 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 0x7f966810dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96681018b0 0x7f966810e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966ed9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96680652e0 con 0x7f9668100f10 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966ed9d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966810e7d0 con 0x7f96681018b0 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 0x7f966810dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 0x7f966810dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48504/0 (socket says 172.21.15.94:48504) 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966dd9b640 1 -- 172.21.15.94:0/2403889887 learned_addr learned my addr 172.21.15.94:0/2403889887 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966d59a640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96681018b0 0x7f966810e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966dd9b640 1 -- 172.21.15.94:0/2403889887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96681018b0 msgr2=0x7f966810e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:20.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966dd9b640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96681018b0 0x7f966810e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966dd9b640 1 -- 172.21.15.94:0/2403889887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664005040 con 0x7f9668100f10 2024-04-06T03:52:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966dd9b640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 0x7f966810dd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9664034040 tx=0x7f9664009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f965e7fc640 1 -- 172.21.15.94:0/2403889887 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96640092b0 con 0x7f9668100f10 2024-04-06T03:52:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f965e7fc640 1 -- 172.21.15.94:0/2403889887 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f966403b400 con 0x7f9668100f10 2024-04-06T03:52:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966810ea20 con 0x7f9668100f10 2024-04-06T03:52:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f965e7fc640 1 -- 172.21.15.94:0/2403889887 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96640435d0 con 0x7f9668100f10 2024-04-06T03:52:20.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.920+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9668116f60 con 0x7f9668100f10 2024-04-06T03:52:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.922+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96680fc680 con 0x7f9668100f10 2024-04-06T03:52:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.922+0000 7f965e7fc640 1 -- 172.21.15.94:0/2403889887 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f966402f070 con 0x7f9668100f10 2024-04-06T03:52:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.923+0000 7f965e7fc640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f965406dc10 0x7f96540700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.923+0000 7f965e7fc640 1 -- 172.21.15.94:0/2403889887 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f96640af300 con 0x7f9668100f10 2024-04-06T03:52:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.923+0000 7f966d59a640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f965406dc10 0x7f96540700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:20.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.923+0000 7f966d59a640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f965406dc10 0x7f96540700c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9668064b30 tx=0x7f965800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:20.927+0000 7f965e7fc640 1 -- 172.21.15.94:0/2403889887 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9664078890 con 0x7f9668100f10 2024-04-06T03:52:21.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.146+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f9668064940 con 0x7f9668100f10 2024-04-06T03:52:21.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.147+0000 7f965e7fc640 1 -- 172.21.15.94:0/2403889887 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v36) v1 ==== 107+0+4157 (secure 0 0 0) 0x7f966407bf50 con 0x7f9668100f10 2024-04-06T03:52:21.149 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:21.149 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:14.261787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":0,"incarnation":20,"state":"up:active","state_seq":157,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24255,"qdb_cluster":[24255]},"id":1}]} 2024-04-06T03:52:21.149 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 23 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f965406dc10 msgr2=0x7f96540700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f965406dc10 0x7f96540700c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9668064b30 tx=0x7f965800b040 comp rx=0 tx=0).stop 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 msgr2=0x7f966810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 0x7f966810dd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9664034040 tx=0x7f9664009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 shutdown_connections 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f965406dc10 0x7f96540700c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96681018b0 0x7f966810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 --2- 172.21.15.94:0/2403889887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9668100f10 0x7f966810dd50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 >> 172.21.15.94:0/2403889887 conn(0x7f9668071e80 msgr2=0x7f9668103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:21.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 shutdown_connections 2024-04-06T03:52:21.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.149+0000 7f966ed9d640 1 -- 172.21.15.94:0/2403889887 wait complete. 2024-04-06T03:52:21.164 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:20 smithi094 ceph-mon[94053]: pgmap v337: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:21.164 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:20 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3212513159' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-06T03:52:21.164 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:20 smithi094 ceph-mon[94053]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:52:21.164 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:20 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1503374623' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-06T03:52:21.164 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:20 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3033265218' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-06T03:52:21.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-06T03:52:21.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 -- 172.21.15.94:0/3152559079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 msgr2=0x7f3a4c0faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 --2- 172.21.15.94:0/3152559079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 0x7f3a4c0faf40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0048d0 tx=0x7f3a3c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 -- 172.21.15.94:0/3152559079 shutdown_connections 2024-04-06T03:52:21.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 --2- 172.21.15.94:0/3152559079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 0x7f3a4c0faf40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 --2- 172.21.15.94:0/3152559079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c100f10 0x7f3a4c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 -- 172.21.15.94:0/3152559079 >> 172.21.15.94:0/3152559079 conn(0x7f3a4c071e80 msgr2=0x7f3a4c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:21.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 -- 172.21.15.94:0/3152559079 shutdown_connections 2024-04-06T03:52:21.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 -- 172.21.15.94:0/3152559079 wait complete. 2024-04-06T03:52:21.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 Processor -- start 2024-04-06T03:52:21.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.263+0000 7f3a5279d640 1 -- start start 2024-04-06T03:52:21.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a5279d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c100f10 0x7f3a4c10dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:21.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a5279d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 0x7f3a4c10e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:21.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a5279d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c065310 con 0x7f3a4c1018b0 2024-04-06T03:52:21.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a5279d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c10e790 con 0x7f3a4c100f10 2024-04-06T03:52:21.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a50f9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 0x7f3a4c10e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:21.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a50f9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 0x7f3a4c10e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48524/0 (socket says 172.21.15.94:48524) 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a50f9a640 1 -- 172.21.15.94:0/2872119695 learned_addr learned my addr 172.21.15.94:0/2872119695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a5179b640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c100f10 0x7f3a4c10dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a50f9a640 1 -- 172.21.15.94:0/2872119695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c100f10 msgr2=0x7f3a4c10dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a50f9a640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c100f10 0x7f3a4c10dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a50f9a640 1 -- 172.21.15.94:0/2872119695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c005040 con 0x7f3a4c1018b0 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a50f9a640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 0x7f3a4c10e250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c034040 tx=0x7f3a3c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a41ffb640 1 -- 172.21.15.94:0/2872119695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a3c0092b0 con 0x7f3a4c1018b0 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c10ea10 con 0x7f3a4c1018b0 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a41ffb640 1 -- 172.21.15.94:0/2872119695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a3c03b400 con 0x7f3a4c1018b0 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.264+0000 7f3a41ffb640 1 -- 172.21.15.94:0/2872119695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a3c0435d0 con 0x7f3a4c1018b0 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.265+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c116f70 con 0x7f3a4c1018b0 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.266+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c0fc680 con 0x7f3a4c1018b0 2024-04-06T03:52:21.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.266+0000 7f3a41ffb640 1 -- 172.21.15.94:0/2872119695 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f3a3c02f070 con 0x7f3a4c1018b0 2024-04-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.266+0000 7f3a41ffb640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3a3806dc10 0x7f3a380700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.267+0000 7f3a5179b640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3a3806dc10 0x7f3a380700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.267+0000 7f3a41ffb640 1 -- 172.21.15.94:0/2872119695 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f3a3c0af390 con 0x7f3a4c1018b0 2024-04-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.267+0000 7f3a5179b640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3a3806dc10 0x7f3a380700c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a44004a70 tx=0x7f3a4400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:21.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.271+0000 7f3a41ffb640 1 -- 172.21.15.94:0/2872119695 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3a3c078870 con 0x7f3a4c1018b0 2024-04-06T03:52:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:20 smithi136 ceph-mon[82061]: pgmap v337: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:20 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3212513159' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-06T03:52:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:20 smithi136 ceph-mon[82061]: from='mgr.34100 172.21.15.94:0/3717731252' entity='mgr.smithi094.purlzp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-06T03:52:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:20 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1503374623' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-06T03:52:21.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:20 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3033265218' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-06T03:52:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.489+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f3a4c04bcc0 con 0x7f3a4c1018b0 2024-04-06T03:52:21.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.490+0000 7f3a41ffb640 1 -- 172.21.15.94:0/2872119695 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v36) v1 ==== 107+0+5004 (secure 0 0 0) 0x7f3a3c07bf30 con 0x7f3a4c1018b0 2024-04-06T03:52:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:21.492 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:14.261787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi136.hbjgtw","rank":0,"incarnation":20,"state":"up:active","state_seq":157,"addr":"172.21.15.136:6825/838120541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":838120541},{"type":"v1","addr":"172.21.15.136:6825","nonce":838120541}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24255,"qdb_cluster":[24255]},"id":1}]} 2024-04-06T03:52:21.492 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 24 2024-04-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3a3806dc10 msgr2=0x7f3a380700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3a3806dc10 0x7f3a380700c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a44004a70 tx=0x7f3a4400b040 comp rx=0 tx=0).stop 2024-04-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 msgr2=0x7f3a4c10e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 0x7f3a4c10e250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c034040 tx=0x7f3a3c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 shutdown_connections 2024-04-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3a3806dc10 0x7f3a380700c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a4c1018b0 0x7f3a4c10e250 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 --2- 172.21.15.94:0/2872119695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a4c100f10 0x7f3a4c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 >> 172.21.15.94:0/2872119695 conn(0x7f3a4c071e80 msgr2=0x7f3a4c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 shutdown_connections 2024-04-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.492+0000 7f3a5279d640 1 -- 172.21.15.94:0/2872119695 wait complete. 2024-04-06T03:52:21.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-06T03:52:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1286642485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 msgr2=0x7ff1b4101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 --2- 172.21.15.94:0/1286642485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 0x7ff1b4101510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00048d0 tx=0x7ff1b0009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1286642485 shutdown_connections 2024-04-06T03:52:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 --2- 172.21.15.94:0/1286642485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b40fb110 0x7ff1b40fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 --2- 172.21.15.94:0/1286642485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 0x7ff1b4101510 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1286642485 >> 172.21.15.94:0/1286642485 conn(0x7ff1b40710b0 msgr2=0x7ff1b40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1286642485 shutdown_connections 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1286642485 wait complete. 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 Processor -- start 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.609+0000 7ff1bb4ee640 1 -- start start 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1bb4ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b40fb110 0x7ff1b410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1bb4ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 0x7ff1b410e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1bb4ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b4065310 con 0x7ff1b4101140 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1bb4ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b410e900 con 0x7ff1b40fb110 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1b8a89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 0x7ff1b410e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1b8a89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 0x7ff1b410e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48540/0 (socket says 172.21.15.94:48540) 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1b8a89640 1 -- 172.21.15.94:0/1969312433 learned_addr learned my addr 172.21.15.94:0/1969312433 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1b928a640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b40fb110 0x7ff1b410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1b8a89640 1 -- 172.21.15.94:0/1969312433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b40fb110 msgr2=0x7ff1b410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1b8a89640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b40fb110 0x7ff1b410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1b8a89640 1 -- 172.21.15.94:0/1969312433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b0005040 con 0x7ff1b4101140 2024-04-06T03:52:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1b8a89640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 0x7ff1b410e3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4002a50 tx=0x7ff1a4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1a9ffb640 1 -- 172.21.15.94:0/1969312433 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a4007c30 con 0x7ff1b4101140 2024-04-06T03:52:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1a9ffb640 1 -- 172.21.15.94:0/1969312433 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1a400f040 con 0x7ff1b4101140 2024-04-06T03:52:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b410ebe0 con 0x7ff1b4101140 2024-04-06T03:52:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.610+0000 7ff1a9ffb640 1 -- 172.21.15.94:0/1969312433 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a400db50 con 0x7ff1b4101140 2024-04-06T03:52:21.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.611+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b41171a0 con 0x7ff1b4101140 2024-04-06T03:52:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.611+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b40fc800 con 0x7ff1b4101140 2024-04-06T03:52:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.612+0000 7ff1a9ffb640 1 -- 172.21.15.94:0/1969312433 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7ff1a401c400 con 0x7ff1b4101140 2024-04-06T03:52:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.612+0000 7ff1a9ffb640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff1a006dc10 0x7ff1a00700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.612+0000 7ff1a9ffb640 1 -- 172.21.15.94:0/1969312433 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff1a408ce70 con 0x7ff1b4101140 2024-04-06T03:52:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.612+0000 7ff1b928a640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff1a006dc10 0x7ff1a00700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:21.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.613+0000 7ff1b928a640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff1a006dc10 0x7ff1a00700c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0000c00 tx=0x7ff1b0009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:21.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.617+0000 7ff1a9ffb640 1 -- 172.21.15.94:0/1969312433 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff1a4056400 con 0x7ff1b4101140 2024-04-06T03:52:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.843+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7ff1b404bcc0 con 0x7ff1b4101140 2024-04-06T03:52:21.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.844+0000 7ff1a9ffb640 1 -- 172.21.15.94:0/1969312433 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v36) v1 ==== 107+0+4203 (secure 0 0 0) 0x7ff1a4059ac0 con 0x7ff1b4101140 2024-04-06T03:52:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:21.846 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:26.996566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:21.846 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-04-06T03:52:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff1a006dc10 msgr2=0x7ff1a00700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff1a006dc10 0x7ff1a00700c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0000c00 tx=0x7ff1b0009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 msgr2=0x7ff1b410e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 0x7ff1b410e3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4002a50 tx=0x7ff1a4004600 comp rx=0 tx=0).stop 2024-04-06T03:52:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 shutdown_connections 2024-04-06T03:52:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff1a006dc10 0x7ff1a00700c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1b4101140 0x7ff1b410e3c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 --2- 172.21.15.94:0/1969312433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b40fb110 0x7ff1b410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 >> 172.21.15.94:0/1969312433 conn(0x7ff1b40710b0 msgr2=0x7ff1b40f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 shutdown_connections 2024-04-06T03:52:21.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.846+0000 7ff1bb4ee640 1 -- 172.21.15.94:0/1969312433 wait complete. 2024-04-06T03:52:21.859 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-06T03:52:21.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 -- 172.21.15.94:0/1206242479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db00fb110 msgr2=0x7f3db00fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 --2- 172.21.15.94:0/1206242479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db00fb110 0x7f3db00fb560 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3da40048d0 tx=0x7f3da4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 -- 172.21.15.94:0/1206242479 shutdown_connections 2024-04-06T03:52:21.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 --2- 172.21.15.94:0/1206242479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db00fb110 0x7f3db00fb560 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 --2- 172.21.15.94:0/1206242479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db0101140 0x7f3db0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 -- 172.21.15.94:0/1206242479 >> 172.21.15.94:0/1206242479 conn(0x7f3db00710b0 msgr2=0x7f3db00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 -- 172.21.15.94:0/1206242479 shutdown_connections 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 -- 172.21.15.94:0/1206242479 wait complete. 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 Processor -- start 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 -- start start 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00fb110 0x7f3db0112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db0101140 0x7f3db01132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0113970 con 0x7f3db0101140 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.959+0000 7f3db58ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db010cf30 con 0x7f3db00fb110 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3dae7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db0101140 0x7f3db01132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3dae7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db0101140 0x7f3db01132e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48558/0 (socket says 172.21.15.94:48558) 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3dae7fc640 1 -- 172.21.15.94:0/3031508381 learned_addr learned my addr 172.21.15.94:0/3031508381 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3daeffd640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00fb110 0x7f3db0112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3dae7fc640 1 -- 172.21.15.94:0/3031508381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00fb110 msgr2=0x7f3db0112da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:21.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3dae7fc640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00fb110 0x7f3db0112da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3dae7fc640 1 -- 172.21.15.94:0/3031508381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da4005040 con 0x7f3db0101140 2024-04-06T03:52:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3dae7fc640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db0101140 0x7f3db01132e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3da40041d0 tx=0x7f3da4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3d977fe640 1 -- 172.21.15.94:0/3031508381 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da40092e0 con 0x7f3db0101140 2024-04-06T03:52:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3d977fe640 1 -- 172.21.15.94:0/3031508381 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3da4009440 con 0x7f3db0101140 2024-04-06T03:52:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db010d1b0 con 0x7f3db0101140 2024-04-06T03:52:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3d977fe640 1 -- 172.21.15.94:0/3031508381 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da4043640 con 0x7f3db0101140 2024-04-06T03:52:21.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.960+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db010d6d0 con 0x7f3db0101140 2024-04-06T03:52:21.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.961+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db00fc800 con 0x7f3db0101140 2024-04-06T03:52:21.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.962+0000 7f3d977fe640 1 -- 172.21.15.94:0/3031508381 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f3da403b9f0 con 0x7f3db0101140 2024-04-06T03:52:21.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.962+0000 7f3d977fe640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d9806dc10 0x7f3d980700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:21.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.962+0000 7f3daeffd640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d9806dc10 0x7f3d980700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:21.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.962+0000 7f3d977fe640 1 -- 172.21.15.94:0/3031508381 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f3da40b03b0 con 0x7f3db0101140 2024-04-06T03:52:21.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.963+0000 7f3daeffd640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d9806dc10 0x7f3d980700c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3da0006fd0 tx=0x7f3da0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:21.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:21.969+0000 7f3d977fe640 1 -- 172.21.15.94:0/3031508381 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3da4079910 con 0x7f3db0101140 2024-04-06T03:52:22.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:21 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2403889887' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-06T03:52:22.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:21 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2872119695' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-06T03:52:22.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:21 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1969312433' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-06T03:52:22.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.185+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f3db01082e0 con 0x7f3db0101140 2024-04-06T03:52:22.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.186+0000 7f3d977fe640 1 -- 172.21.15.94:0/3031508381 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v36) v1 ==== 107+0+4214 (secure 0 0 0) 0x7f3da407cfd0 con 0x7f3db0101140 2024-04-06T03:52:22.188 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:22.188 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:27.001819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":0,"incarnation":26,"state":"up:replay","state_seq":2,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:22.188 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 26 2024-04-06T03:52:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d9806dc10 msgr2=0x7f3d980700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d9806dc10 0x7f3d980700c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3da0006fd0 tx=0x7f3da0008040 comp rx=0 tx=0).stop 2024-04-06T03:52:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db0101140 msgr2=0x7f3db01132e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db0101140 0x7f3db01132e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3da40041d0 tx=0x7f3da4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 shutdown_connections 2024-04-06T03:52:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d9806dc10 0x7f3d980700c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3db0101140 0x7f3db01132e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 --2- 172.21.15.94:0/3031508381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00fb110 0x7f3db0112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 >> 172.21.15.94:0/3031508381 conn(0x7f3db00710b0 msgr2=0x7f3db00f9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:22.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 shutdown_connections 2024-04-06T03:52:22.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.188+0000 7f3db58ee640 1 -- 172.21.15.94:0/3031508381 wait complete. 2024-04-06T03:52:22.201 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-06T03:52:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.299+0000 7fafca99d640 1 -- 172.21.15.94:0/2169465443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc41018b0 msgr2=0x7fafc40faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.299+0000 7fafca99d640 1 --2- 172.21.15.94:0/2169465443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc41018b0 0x7fafc40faf40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fafb40048d0 tx=0x7fafb4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.299+0000 7fafca99d640 1 -- 172.21.15.94:0/2169465443 shutdown_connections 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.299+0000 7fafca99d640 1 --2- 172.21.15.94:0/2169465443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc41018b0 0x7fafc40faf40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.299+0000 7fafca99d640 1 --2- 172.21.15.94:0/2169465443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4100f10 0x7fafc41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.299+0000 7fafca99d640 1 -- 172.21.15.94:0/2169465443 >> 172.21.15.94:0/2169465443 conn(0x7fafc4071e80 msgr2=0x7fafc4072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.299+0000 7fafca99d640 1 -- 172.21.15.94:0/2169465443 shutdown_connections 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.299+0000 7fafca99d640 1 -- 172.21.15.94:0/2169465443 wait complete. 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafca99d640 1 Processor -- start 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafca99d640 1 -- start start 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafca99d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc4100f10 0x7fafc4112b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafca99d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc41018b0 0x7fafc4113080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:22.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafca99d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc4113680 con 0x7fafc4100f10 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafca99d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc410ccd0 con 0x7fafc41018b0 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafc999b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc4100f10 0x7fafc4112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafc999b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc4100f10 0x7fafc4112b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48570/0 (socket says 172.21.15.94:48570) 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafc999b640 1 -- 172.21.15.94:0/3682774812 learned_addr learned my addr 172.21.15.94:0/3682774812 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafc999b640 1 -- 172.21.15.94:0/3682774812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc41018b0 msgr2=0x7fafc4113080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafc999b640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc41018b0 0x7fafc4113080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafc999b640 1 -- 172.21.15.94:0/3682774812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4005040 con 0x7fafc4100f10 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.300+0000 7fafc999b640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc4100f10 0x7fafc4112b40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fafc0002a50 tx=0x7fafc0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.301+0000 7fafba7fc640 1 -- 172.21.15.94:0/3682774812 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc0007c30 con 0x7fafc4100f10 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.301+0000 7fafba7fc640 1 -- 172.21.15.94:0/3682774812 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafc000f040 con 0x7fafc4100f10 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.301+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc410cfb0 con 0x7fafc4100f10 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.301+0000 7fafba7fc640 1 -- 172.21.15.94:0/3682774812 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc000db50 con 0x7fafc4100f10 2024-04-06T03:52:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.301+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc410d4d0 con 0x7fafc4100f10 2024-04-06T03:52:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.302+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc40fc680 con 0x7fafc4100f10 2024-04-06T03:52:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.302+0000 7fafba7fc640 1 -- 172.21.15.94:0/3682774812 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fafc000dcf0 con 0x7fafc4100f10 2024-04-06T03:52:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.303+0000 7fafba7fc640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fafb006dc10 0x7fafb00700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.303+0000 7fafba7fc640 1 -- 172.21.15.94:0/3682774812 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fafc008ce60 con 0x7fafc4100f10 2024-04-06T03:52:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.303+0000 7fafc919a640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fafb006dc10 0x7fafb00700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:22.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.303+0000 7fafc919a640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fafb006dc10 0x7fafb00700c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fafb40043d0 tx=0x7fafb4009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.310+0000 7fafba7fc640 1 -- 172.21.15.94:0/3682774812 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fafc00563c0 con 0x7fafc4100f10 2024-04-06T03:52:22.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:21 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2403889887' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-06T03:52:22.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:21 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2872119695' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-06T03:52:22.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:21 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1969312433' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-06T03:52:22.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.527+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fafc404bcc0 con 0x7fafc4100f10 2024-04-06T03:52:22.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.528+0000 7fafba7fc640 1 -- 172.21.15.94:0/3682774812 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v36) v1 ==== 107+0+4219 (secure 0 0 0) 0x7fafc0059a80 con 0x7fafc4100f10 2024-04-06T03:52:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:22.530 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:35.026834+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":160,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:22.530 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fafb006dc10 msgr2=0x7fafb00700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fafb006dc10 0x7fafb00700c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fafb40043d0 tx=0x7fafb4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc4100f10 msgr2=0x7fafc4112b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc4100f10 0x7fafc4112b40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fafc0002a50 tx=0x7fafc0004600 comp rx=0 tx=0).stop 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 shutdown_connections 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fafb006dc10 0x7fafb00700c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc41018b0 0x7fafc4113080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 --2- 172.21.15.94:0/3682774812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafc4100f10 0x7fafc4112b40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 >> 172.21.15.94:0/3682774812 conn(0x7fafc4071e80 msgr2=0x7fafc4064370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:22.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 shutdown_connections 2024-04-06T03:52:22.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.530+0000 7fafca99d640 1 -- 172.21.15.94:0/3682774812 wait complete. 2024-04-06T03:52:22.543 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-06T03:52:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.642+0000 7f7b1b39d640 1 -- 172.21.15.94:0/2669776084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b141018b0 msgr2=0x7f7b140faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.642+0000 7f7b1b39d640 1 --2- 172.21.15.94:0/2669776084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b141018b0 0x7f7b140faf40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b040048d0 tx=0x7f7b04009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.642+0000 7f7b1b39d640 1 -- 172.21.15.94:0/2669776084 shutdown_connections 2024-04-06T03:52:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.642+0000 7f7b1b39d640 1 --2- 172.21.15.94:0/2669776084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b141018b0 0x7f7b140faf40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.642+0000 7f7b1b39d640 1 --2- 172.21.15.94:0/2669776084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b14100f10 0x7f7b141012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.642+0000 7f7b1b39d640 1 -- 172.21.15.94:0/2669776084 >> 172.21.15.94:0/2669776084 conn(0x7f7b14071e80 msgr2=0x7f7b14072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.642+0000 7f7b1b39d640 1 -- 172.21.15.94:0/2669776084 shutdown_connections 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.642+0000 7f7b1b39d640 1 -- 172.21.15.94:0/2669776084 wait complete. 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1b39d640 1 Processor -- start 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1b39d640 1 -- start start 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1b39d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b14100f10 0x7f7b1410dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1b39d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b141018b0 0x7f7b1410e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1b39d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14065330 con 0x7f7b14100f10 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1b39d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1410e7a0 con 0x7f7b141018b0 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1a39b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b14100f10 0x7f7b1410dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1a39b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b14100f10 0x7f7b1410dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48594/0 (socket says 172.21.15.94:48594) 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1a39b640 1 -- 172.21.15.94:0/1204608595 learned_addr learned my addr 172.21.15.94:0/1204608595 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b19b9a640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b141018b0 0x7f7b1410e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1a39b640 1 -- 172.21.15.94:0/1204608595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b141018b0 msgr2=0x7f7b1410e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1a39b640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b141018b0 0x7f7b1410e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1a39b640 1 -- 172.21.15.94:0/1204608595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04005040 con 0x7f7b14100f10 2024-04-06T03:52:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.643+0000 7f7b1a39b640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b14100f10 0x7f7b1410dcf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b10002a50 tx=0x7f7b10004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.644+0000 7f7b0affd640 1 -- 172.21.15.94:0/1204608595 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b10007c30 con 0x7f7b14100f10 2024-04-06T03:52:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.644+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1410ea80 con 0x7f7b14100f10 2024-04-06T03:52:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.644+0000 7f7b0affd640 1 -- 172.21.15.94:0/1204608595 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b1000f040 con 0x7f7b14100f10 2024-04-06T03:52:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.644+0000 7f7b0affd640 1 -- 172.21.15.94:0/1204608595 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1000db50 con 0x7f7b14100f10 2024-04-06T03:52:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.644+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b14117010 con 0x7f7b14100f10 2024-04-06T03:52:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.645+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b140fc680 con 0x7f7b14100f10 2024-04-06T03:52:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.645+0000 7f7b0affd640 1 -- 172.21.15.94:0/1204608595 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f7b1001c400 con 0x7f7b14100f10 2024-04-06T03:52:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.645+0000 7f7b0affd640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7b0006dc10 0x7f7b000700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.646+0000 7f7b0affd640 1 -- 172.21.15.94:0/1204608595 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7b1008cf70 con 0x7f7b14100f10 2024-04-06T03:52:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.646+0000 7f7b19b9a640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7b0006dc10 0x7f7b000700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.646+0000 7f7b19b9a640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7b0006dc10 0x7f7b000700c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b14064b80 tx=0x7f7b04009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:22.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.654+0000 7f7b0affd640 1 -- 172.21.15.94:0/1204608595 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7b100c1280 con 0x7f7b14100f10 2024-04-06T03:52:22.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.876+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f7b141080d0 con 0x7f7b14100f10 2024-04-06T03:52:22.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.877+0000 7f7b0affd640 1 -- 172.21.15.94:0/1204608595 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v36) v1 ==== 107+0+4216 (secure 0 0 0) 0x7f7b10056450 con 0x7f7b14100f10 2024-04-06T03:52:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:22.878 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:35.120591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":161,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:22.879 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 28 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.879+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7b0006dc10 msgr2=0x7f7b000700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.879+0000 7f7b1b39d640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7b0006dc10 0x7f7b000700c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b14064b80 tx=0x7f7b04009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.879+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b14100f10 msgr2=0x7f7b1410dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.879+0000 7f7b1b39d640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b14100f10 0x7f7b1410dcf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b10002a50 tx=0x7f7b10004600 comp rx=0 tx=0).stop 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.879+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 shutdown_connections 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.879+0000 7f7b1b39d640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f7b0006dc10 0x7f7b000700c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.879+0000 7f7b1b39d640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b141018b0 0x7f7b1410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.879+0000 7f7b1b39d640 1 --2- 172.21.15.94:0/1204608595 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b14100f10 0x7f7b1410dcf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.879+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 >> 172.21.15.94:0/1204608595 conn(0x7f7b14071e80 msgr2=0x7f7b140feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:22.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.880+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 shutdown_connections 2024-04-06T03:52:22.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.880+0000 7f7b1b39d640 1 -- 172.21.15.94:0/1204608595 wait complete. 2024-04-06T03:52:22.892 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.991+0000 7f1de78ee640 1 -- 172.21.15.94:0/3314023805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de00fb110 msgr2=0x7f1de00fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.991+0000 7f1de78ee640 1 --2- 172.21.15.94:0/3314023805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de00fb110 0x7f1de00fb560 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00048d0 tx=0x7f1dd0009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.991+0000 7f1de78ee640 1 -- 172.21.15.94:0/3314023805 shutdown_connections 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.991+0000 7f1de78ee640 1 --2- 172.21.15.94:0/3314023805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de00fb110 0x7f1de00fb560 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.991+0000 7f1de78ee640 1 --2- 172.21.15.94:0/3314023805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de0101140 0x7f1de0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.991+0000 7f1de78ee640 1 -- 172.21.15.94:0/3314023805 >> 172.21.15.94:0/3314023805 conn(0x7f1de00710b0 msgr2=0x7f1de00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.991+0000 7f1de78ee640 1 -- 172.21.15.94:0/3314023805 shutdown_connections 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.991+0000 7f1de78ee640 1 -- 172.21.15.94:0/3314023805 wait complete. 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de78ee640 1 Processor -- start 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de78ee640 1 -- start start 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de78ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de00fb110 0x7f1de0111c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:22.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de78ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de0101140 0x7f1de0112140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de78ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0112740 con 0x7f1de0101140 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de78ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de010bd90 con 0x7f1de00fb110 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de4e89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de0101140 0x7f1de0112140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de4e89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de0101140 0x7f1de0112140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48620/0 (socket says 172.21.15.94:48620) 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de4e89640 1 -- 172.21.15.94:0/3446614916 learned_addr learned my addr 172.21.15.94:0/3446614916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de4e89640 1 -- 172.21.15.94:0/3446614916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de00fb110 msgr2=0x7f1de0111c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de568a640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de00fb110 0x7f1de0111c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de4e89640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de00fb110 0x7f1de0111c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de4e89640 1 -- 172.21.15.94:0/3446614916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd0005040 con 0x7f1de0101140 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de4e89640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de0101140 0x7f1de0112140 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00041d0 tx=0x7f1dd0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1dd5ffb640 1 -- 172.21.15.94:0/3446614916 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd0030870 con 0x7f1de0101140 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1dd5ffb640 1 -- 172.21.15.94:0/3446614916 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dd00092e0 con 0x7f1de0101140 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de010c010 con 0x7f1de0101140 2024-04-06T03:52:22.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.992+0000 7f1dd5ffb640 1 -- 172.21.15.94:0/3446614916 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd0043400 con 0x7f1de0101140 2024-04-06T03:52:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.993+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de010c530 con 0x7f1de0101140 2024-04-06T03:52:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.994+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de00fc800 con 0x7f1de0101140 2024-04-06T03:52:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.994+0000 7f1dd5ffb640 1 -- 172.21.15.94:0/3446614916 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f1dd003b420 con 0x7f1de0101140 2024-04-06T03:52:22.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.995+0000 7f1dd5ffb640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1dcc06dc10 0x7f1dcc0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.995+0000 7f1de568a640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1dcc06dc10 0x7f1dcc0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.995+0000 7f1dd5ffb640 1 -- 172.21.15.94:0/3446614916 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f1dd00b0420 con 0x7f1de0101140 2024-04-06T03:52:22.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.995+0000 7f1de568a640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1dcc06dc10 0x7f1dcc0700c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc004ce0 tx=0x7f1ddc00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:23.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:22.999+0000 7f1dd5ffb640 1 -- 172.21.15.94:0/3446614916 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1dd0079980 con 0x7f1de0101140 2024-04-06T03:52:23.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:22 smithi094 ceph-mon[94053]: pgmap v338: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:23.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:22 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3031508381' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-06T03:52:23.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:22 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3682774812' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-06T03:52:23.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:22 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1204608595' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-06T03:52:23.249 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:22 smithi136 ceph-mon[82061]: pgmap v338: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:23.249 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:22 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3031508381' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-06T03:52:23.249 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:22 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3682774812' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-06T03:52:23.249 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:22 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1204608595' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-06T03:52:23.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.266+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f1de0108250 con 0x7f1de0101140 2024-04-06T03:52:23.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.267+0000 7f1dd5ffb640 1 -- 172.21.15.94:0/3446614916 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v36) v1 ==== 107+0+4225 (secure 0 0 0) 0x7f1dd007d040 con 0x7f1de0101140 2024-04-06T03:52:23.269 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:23.269 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:37.123790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":0,"incarnation":26,"state":"up:active","state_seq":162,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24267,"qdb_cluster":[24267]},"id":1}]} 2024-04-06T03:52:23.269 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 29 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1dcc06dc10 msgr2=0x7f1dcc0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1dcc06dc10 0x7f1dcc0700c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc004ce0 tx=0x7f1ddc00b040 comp rx=0 tx=0).stop 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de0101140 msgr2=0x7f1de0112140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de0101140 0x7f1de0112140 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00041d0 tx=0x7f1dd0009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 shutdown_connections 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f1dcc06dc10 0x7f1dcc0700c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1de0101140 0x7f1de0112140 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 --2- 172.21.15.94:0/3446614916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1de00fb110 0x7f1de0111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 >> 172.21.15.94:0/3446614916 conn(0x7f1de00710b0 msgr2=0x7f1de00f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 shutdown_connections 2024-04-06T03:52:23.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.269+0000 7f1de78ee640 1 -- 172.21.15.94:0/3446614916 wait complete. 2024-04-06T03:52:23.281 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-06T03:52:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 -- 172.21.15.94:0/913554750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 msgr2=0x7ff3ac0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 --2- 172.21.15.94:0/913554750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 0x7ff3ac0fb560 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff3940048d0 tx=0x7ff394009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 -- 172.21.15.94:0/913554750 shutdown_connections 2024-04-06T03:52:23.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 --2- 172.21.15.94:0/913554750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 0x7ff3ac0fb560 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 --2- 172.21.15.94:0/913554750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3ac101140 0x7ff3ac101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 -- 172.21.15.94:0/913554750 >> 172.21.15.94:0/913554750 conn(0x7ff3ac0710b0 msgr2=0x7ff3ac0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 -- 172.21.15.94:0/913554750 shutdown_connections 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 -- 172.21.15.94:0/913554750 wait complete. 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 Processor -- start 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.379+0000 7ff3b08ee640 1 -- start start 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3b08ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 0x7ff3ac112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3b08ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3ac101140 0x7ff3ac113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3b08ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac113840 con 0x7ff3ac0fb110 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3b08ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac10ce90 con 0x7ff3ac101140 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3aa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 0x7ff3ac112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3aa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 0x7ff3ac112d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48632/0 (socket says 172.21.15.94:48632) 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3aa59c640 1 -- 172.21.15.94:0/2827657060 learned_addr learned my addr 172.21.15.94:0/2827657060 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3a3fff640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3ac101140 0x7ff3ac113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3aa59c640 1 -- 172.21.15.94:0/2827657060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3ac101140 msgr2=0x7ff3ac113240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3aa59c640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3ac101140 0x7ff3ac113240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3aa59c640 1 -- 172.21.15.94:0/2827657060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff394005040 con 0x7ff3ac0fb110 2024-04-06T03:52:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3aa59c640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 0x7ff3ac112d00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff39c002a50 tx=0x7ff39c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3a2ffd640 1 -- 172.21.15.94:0/2827657060 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff39c007c30 con 0x7ff3ac0fb110 2024-04-06T03:52:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac10d170 con 0x7ff3ac0fb110 2024-04-06T03:52:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3a2ffd640 1 -- 172.21.15.94:0/2827657060 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff39c00f040 con 0x7ff3ac0fb110 2024-04-06T03:52:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.380+0000 7ff3a2ffd640 1 -- 172.21.15.94:0/2827657060 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff39c00db50 con 0x7ff3ac0fb110 2024-04-06T03:52:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.381+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac10d660 con 0x7ff3ac0fb110 2024-04-06T03:52:23.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.381+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac0fc800 con 0x7ff3ac0fb110 2024-04-06T03:52:23.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.382+0000 7ff3a2ffd640 1 -- 172.21.15.94:0/2827657060 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7ff39c01c400 con 0x7ff3ac0fb110 2024-04-06T03:52:23.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.382+0000 7ff3a2ffd640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff39806dc10 0x7ff3980700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:23.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.383+0000 7ff3a3fff640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff39806dc10 0x7ff3980700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:23.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.383+0000 7ff3a2ffd640 1 -- 172.21.15.94:0/2827657060 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff39c08ce80 con 0x7ff3ac0fb110 2024-04-06T03:52:23.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.383+0000 7ff3a3fff640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff39806dc10 0x7ff3980700c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff394000c00 tx=0x7ff394009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:23.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.387+0000 7ff3a2ffd640 1 -- 172.21.15.94:0/2827657060 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff39c056410 con 0x7ff3ac0fb110 2024-04-06T03:52:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.606+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7ff3ac10e370 con 0x7ff3ac0fb110 2024-04-06T03:52:23.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.606+0000 7ff3a2ffd640 1 -- 172.21.15.94:0/2827657060 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v36) v1 ==== 107+0+5075 (secure 0 0 0) 0x7ff39c059ad0 con 0x7ff3ac0fb110 2024-04-06T03:52:23.608 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:23.608 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44331,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6825/1500356136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1500356136},{"type":"v1","addr":"172.21.15.136:6825","nonce":1500356136}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:37.123790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi136.trdonf","rank":0,"incarnation":26,"state":"up:active","state_seq":162,"addr":"172.21.15.136:6827/249801036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":249801036},{"type":"v1","addr":"172.21.15.136:6827","nonce":249801036}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24267,"qdb_cluster":[24267]},"id":1}]} 2024-04-06T03:52:23.608 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 30 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff39806dc10 msgr2=0x7ff3980700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff39806dc10 0x7ff3980700c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff394000c00 tx=0x7ff394009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 msgr2=0x7ff3ac112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 0x7ff3ac112d00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff39c002a50 tx=0x7ff39c004600 comp rx=0 tx=0).stop 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 shutdown_connections 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7ff39806dc10 0x7ff3980700c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3ac101140 0x7ff3ac113240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 --2- 172.21.15.94:0/2827657060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3ac0fb110 0x7ff3ac112d00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 >> 172.21.15.94:0/2827657060 conn(0x7ff3ac0710b0 msgr2=0x7ff3ac0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:23.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 shutdown_connections 2024-04-06T03:52:23.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.608+0000 7ff3b08ee640 1 -- 172.21.15.94:0/2827657060 wait complete. 2024-04-06T03:52:23.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-06T03:52:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 -- 172.21.15.94:0/1391717778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2814101140 msgr2=0x7f2814101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 --2- 172.21.15.94:0/1391717778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2814101140 0x7f2814101510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f28080048d0 tx=0x7f2808009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 -- 172.21.15.94:0/1391717778 shutdown_connections 2024-04-06T03:52:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 --2- 172.21.15.94:0/1391717778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28140fb110 0x7f28140fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 --2- 172.21.15.94:0/1391717778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2814101140 0x7f2814101510 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 -- 172.21.15.94:0/1391717778 >> 172.21.15.94:0/1391717778 conn(0x7f28140710b0 msgr2=0x7f28140714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:23.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 -- 172.21.15.94:0/1391717778 shutdown_connections 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 -- 172.21.15.94:0/1391717778 wait complete. 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 Processor -- start 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.722+0000 7f281aeee640 1 -- start start 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f281aeee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28140fb110 0x7f281410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f281aeee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2814101140 0x7f281410e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f281aeee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814065310 con 0x7f28140fb110 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f281aeee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281410e900 con 0x7f2814101140 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f2818c8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28140fb110 0x7f281410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f2818c8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28140fb110 0x7f281410de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48638/0 (socket says 172.21.15.94:48638) 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f2818c8a640 1 -- 172.21.15.94:0/584586099 learned_addr learned my addr 172.21.15.94:0/584586099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f2813fff640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2814101140 0x7f281410e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f2818c8a640 1 -- 172.21.15.94:0/584586099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2814101140 msgr2=0x7f281410e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f2818c8a640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2814101140 0x7f281410e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f2818c8a640 1 -- 172.21.15.94:0/584586099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808005040 con 0x7f28140fb110 2024-04-06T03:52:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f2813fff640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2814101140 0x7f281410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:52:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f2818c8a640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28140fb110 0x7f281410de80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f28080041d0 tx=0x7f2808009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f28117fa640 1 -- 172.21.15.94:0/584586099 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2808030870 con 0x7f28140fb110 2024-04-06T03:52:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f28117fa640 1 -- 172.21.15.94:0/584586099 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28080092e0 con 0x7f28140fb110 2024-04-06T03:52:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f28117fa640 1 -- 172.21.15.94:0/584586099 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2808043400 con 0x7f28140fb110 2024-04-06T03:52:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281410eb80 con 0x7f28140fb110 2024-04-06T03:52:23.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.723+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2814117140 con 0x7f28140fb110 2024-04-06T03:52:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.724+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28140fc800 con 0x7f28140fb110 2024-04-06T03:52:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.725+0000 7f28117fa640 1 -- 172.21.15.94:0/584586099 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f280803b420 con 0x7f28140fb110 2024-04-06T03:52:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.725+0000 7f28117fa640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f280406dc10 0x7f28040700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.725+0000 7f28117fa640 1 -- 172.21.15.94:0/584586099 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f28080b0410 con 0x7f28140fb110 2024-04-06T03:52:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.725+0000 7f2813fff640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f280406dc10 0x7f28040700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:23.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.726+0000 7f2813fff640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f280406dc10 0x7f28040700c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2800007fd0 tx=0x7f2800009e90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:23.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.734+0000 7f28117fa640 1 -- 172.21.15.94:0/584586099 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2808079970 con 0x7f28140fb110 2024-04-06T03:52:23.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.952+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f281404bcc0 con 0x7f28140fb110 2024-04-06T03:52:23.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.952+0000 7f28117fa640 1 -- 172.21.15.94:0/584586099 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v36) v1 ==== 107+0+4274 (secure 0 0 0) 0x7f280807d030 con 0x7f28140fb110 2024-04-06T03:52:23.954 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:23.954 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44331,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6825/1500356136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1500356136},{"type":"v1","addr":"172.21.15.136:6825","nonce":1500356136}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:51.385187+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:23.954 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 31 2024-04-06T03:52:23.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.954+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f280406dc10 msgr2=0x7f28040700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.954+0000 7f281aeee640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f280406dc10 0x7f28040700c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2800007fd0 tx=0x7f2800009e90 comp rx=0 tx=0).stop 2024-04-06T03:52:23.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.954+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28140fb110 msgr2=0x7f281410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:23.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.954+0000 7f281aeee640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28140fb110 0x7f281410de80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f28080041d0 tx=0x7f2808009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.954+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 shutdown_connections 2024-04-06T03:52:23.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.954+0000 7f281aeee640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f280406dc10 0x7f28040700c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.955+0000 7f281aeee640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2814101140 0x7f281410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.955+0000 7f281aeee640 1 --2- 172.21.15.94:0/584586099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28140fb110 0x7f281410de80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:23.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.955+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 >> 172.21.15.94:0/584586099 conn(0x7f28140710b0 msgr2=0x7f28140f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:23.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.955+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 shutdown_connections 2024-04-06T03:52:23.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:23.955+0000 7f281aeee640 1 -- 172.21.15.94:0/584586099 wait complete. 2024-04-06T03:52:23.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-06T03:52:24.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 -- 172.21.15.94:0/3957914455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 msgr2=0x7f4a680fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 --2- 172.21.15.94:0/3957914455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 0x7f4a680fb560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a580048d0 tx=0x7f4a58009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 -- 172.21.15.94:0/3957914455 shutdown_connections 2024-04-06T03:52:24.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 --2- 172.21.15.94:0/3957914455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 0x7f4a680fb560 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 --2- 172.21.15.94:0/3957914455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a68101140 0x7f4a68101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 -- 172.21.15.94:0/3957914455 >> 172.21.15.94:0/3957914455 conn(0x7f4a680710b0 msgr2=0x7f4a680714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:24.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 -- 172.21.15.94:0/3957914455 shutdown_connections 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 -- 172.21.15.94:0/3957914455 wait complete. 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 Processor -- start 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.066+0000 7f4a6faee640 1 -- start start 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6faee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 0x7f4a68112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6faee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a68101140 0x7f4a68113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6faee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a68113820 con 0x7f4a680fb110 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6faee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6810ce70 con 0x7f4a68101140 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6d88a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 0x7f4a68112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6d88a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 0x7f4a68112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48654/0 (socket says 172.21.15.94:48654) 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6d88a640 1 -- 172.21.15.94:0/3814723337 learned_addr learned my addr 172.21.15.94:0/3814723337 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6d089640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a68101140 0x7f4a68113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6d88a640 1 -- 172.21.15.94:0/3814723337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a68101140 msgr2=0x7f4a68113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6d88a640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a68101140 0x7f4a68113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6d88a640 1 -- 172.21.15.94:0/3814723337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a58005040 con 0x7f4a680fb110 2024-04-06T03:52:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6d88a640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 0x7f4a68112ce0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c830 tx=0x7f4a64004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a5e7fc640 1 -- 172.21.15.94:0/3814723337 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a6400a850 con 0x7f4a680fb110 2024-04-06T03:52:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6810d150 con 0x7f4a680fb110 2024-04-06T03:52:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a5e7fc640 1 -- 172.21.15.94:0/3814723337 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a6400ae80 con 0x7f4a680fb110 2024-04-06T03:52:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a5e7fc640 1 -- 172.21.15.94:0/3814723337 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a64012ea0 con 0x7f4a680fb110 2024-04-06T03:52:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.067+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6810d640 con 0x7f4a680fb110 2024-04-06T03:52:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.069+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a680fc800 con 0x7f4a680fb110 2024-04-06T03:52:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.069+0000 7f4a5e7fc640 1 -- 172.21.15.94:0/3814723337 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f4a64012400 con 0x7f4a680fb110 2024-04-06T03:52:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.069+0000 7f4a5e7fc640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4a5406dc10 0x7f4a540700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.069+0000 7f4a5e7fc640 1 -- 172.21.15.94:0/3814723337 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4a6408d7f0 con 0x7f4a680fb110 2024-04-06T03:52:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.069+0000 7f4a6d089640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4a5406dc10 0x7f4a540700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.070+0000 7f4a6d089640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4a5406dc10 0x7f4a540700c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4a680fb070 tx=0x7f4a58009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:24.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.077+0000 7f4a5e7fc640 1 -- 172.21.15.94:0/3814723337 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4a64056d50 con 0x7f4a680fb110 2024-04-06T03:52:24.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.297+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f4a6804bcc0 con 0x7f4a680fb110 2024-04-06T03:52:24.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.297+0000 7f4a5e7fc640 1 -- 172.21.15.94:0/3814723337 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v36) v1 ==== 107+0+4353 (secure 0 0 0) 0x7f4a640126b0 con 0x7f4a680fb110 2024-04-06T03:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:24.299 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44331,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6825/1500356136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1500356136},{"type":"v1","addr":"172.21.15.136:6825","nonce":1500356136}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:51.390178+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:24.299 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 32 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4a5406dc10 msgr2=0x7f4a540700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4a5406dc10 0x7f4a540700c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4a680fb070 tx=0x7f4a58009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 msgr2=0x7f4a68112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 0x7f4a68112ce0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c830 tx=0x7f4a64004430 comp rx=0 tx=0).stop 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 shutdown_connections 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f4a5406dc10 0x7f4a540700c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a68101140 0x7f4a68113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 --2- 172.21.15.94:0/3814723337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a680fb110 0x7f4a68112ce0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 >> 172.21.15.94:0/3814723337 conn(0x7f4a680710b0 msgr2=0x7f4a680f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 shutdown_connections 2024-04-06T03:52:24.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.299+0000 7f4a6faee640 1 -- 172.21.15.94:0/3814723337 wait complete. 2024-04-06T03:52:24.310 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:23 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3446614916' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-06T03:52:24.311 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:23 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2827657060' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-06T03:52:24.312 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-06T03:52:24.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:23 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3446614916' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-06T03:52:24.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:23 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2827657060' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-06T03:52:24.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.410+0000 7f34dccee640 1 -- 172.21.15.94:0/1254432640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d8101140 msgr2=0x7f34d8101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.410+0000 7f34dccee640 1 --2- 172.21.15.94:0/1254432640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d8101140 0x7f34d8101510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f34c80048d0 tx=0x7f34c8009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.410+0000 7f34dccee640 1 -- 172.21.15.94:0/1254432640 shutdown_connections 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.410+0000 7f34dccee640 1 --2- 172.21.15.94:0/1254432640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34d80fb110 0x7f34d80fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.410+0000 7f34dccee640 1 --2- 172.21.15.94:0/1254432640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d8101140 0x7f34d8101510 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.410+0000 7f34dccee640 1 -- 172.21.15.94:0/1254432640 >> 172.21.15.94:0/1254432640 conn(0x7f34d80710b0 msgr2=0x7f34d80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.410+0000 7f34dccee640 1 -- 172.21.15.94:0/1254432640 shutdown_connections 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.410+0000 7f34dccee640 1 -- 172.21.15.94:0/1254432640 wait complete. 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.410+0000 7f34dccee640 1 Processor -- start 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34dccee640 1 -- start start 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34dccee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d80fb110 0x7f34d8111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34dccee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34d8101140 0x7f34d81120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34dccee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d81126f0 con 0x7f34d80fb110 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34dccee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d810bd40 con 0x7f34d8101140 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34d659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d80fb110 0x7f34d8111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34d659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d80fb110 0x7f34d8111bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48664/0 (socket says 172.21.15.94:48664) 2024-04-06T03:52:24.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34d659c640 1 -- 172.21.15.94:0/2880075187 learned_addr learned my addr 172.21.15.94:0/2880075187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34d5d9b640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34d8101140 0x7f34d81120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34d659c640 1 -- 172.21.15.94:0/2880075187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34d8101140 msgr2=0x7f34d81120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34d659c640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34d8101140 0x7f34d81120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34d659c640 1 -- 172.21.15.94:0/2880075187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c8005040 con 0x7f34d80fb110 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34d5d9b640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34d8101140 0x7f34d81120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.411+0000 7f34d659c640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d80fb110 0x7f34d8111bb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f34c80041d0 tx=0x7f34c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.412+0000 7f34c6ffd640 1 -- 172.21.15.94:0/2880075187 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c8030870 con 0x7f34d80fb110 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.412+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d810bfc0 con 0x7f34d80fb110 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.412+0000 7f34c6ffd640 1 -- 172.21.15.94:0/2880075187 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34c80092e0 con 0x7f34d80fb110 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.412+0000 7f34c6ffd640 1 -- 172.21.15.94:0/2880075187 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c8043400 con 0x7f34d80fb110 2024-04-06T03:52:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.412+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d810c4e0 con 0x7f34d80fb110 2024-04-06T03:52:24.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.412+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d80fc800 con 0x7f34d80fb110 2024-04-06T03:52:24.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.413+0000 7f34c6ffd640 1 -- 172.21.15.94:0/2880075187 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f34c803b420 con 0x7f34d80fb110 2024-04-06T03:52:24.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.413+0000 7f34c6ffd640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f34c006dc10 0x7f34c00700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:24.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.413+0000 7f34c6ffd640 1 -- 172.21.15.94:0/2880075187 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f34c80b0420 con 0x7f34d80fb110 2024-04-06T03:52:24.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.413+0000 7f34d5d9b640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f34c006dc10 0x7f34c00700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:24.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.414+0000 7f34d5d9b640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f34c006dc10 0x7f34c00700c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f34cc007fd0 tx=0x7f34cc009e90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:24.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.422+0000 7f34c6ffd640 1 -- 172.21.15.94:0/2880075187 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f34c8079980 con 0x7f34d80fb110 2024-04-06T03:52:24.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.641+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f34d810d220 con 0x7f34d80fb110 2024-04-06T03:52:24.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.641+0000 7f34c6ffd640 1 -- 172.21.15.94:0/2880075187 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v36) v1 ==== 107+0+4357 (secure 0 0 0) 0x7f34c807d040 con 0x7f34d80fb110 2024-04-06T03:52:24.643 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:24.643 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44331,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6825/1500356136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1500356136},{"type":"v1","addr":"172.21.15.136:6825","nonce":1500356136}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:59.289447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:24.643 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 33 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f34c006dc10 msgr2=0x7f34c00700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f34c006dc10 0x7f34c00700c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f34cc007fd0 tx=0x7f34cc009e90 comp rx=0 tx=0).stop 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d80fb110 msgr2=0x7f34d8111bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d80fb110 0x7f34d8111bb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f34c80041d0 tx=0x7f34c8009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 shutdown_connections 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f34c006dc10 0x7f34c00700c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34d8101140 0x7f34d81120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 --2- 172.21.15.94:0/2880075187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34d80fb110 0x7f34d8111bb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 >> 172.21.15.94:0/2880075187 conn(0x7f34d80710b0 msgr2=0x7f34d80f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:24.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 shutdown_connections 2024-04-06T03:52:24.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.643+0000 7f34dccee640 1 -- 172.21.15.94:0/2880075187 wait complete. 2024-04-06T03:52:24.656 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-06T03:52:24.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 -- 172.21.15.94:0/2848069432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c80643a0 msgr2=0x7f80c80647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 --2- 172.21.15.94:0/2848069432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c80643a0 0x7f80c80647d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80b80048d0 tx=0x7f80b8009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 -- 172.21.15.94:0/2848069432 shutdown_connections 2024-04-06T03:52:24.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 --2- 172.21.15.94:0/2848069432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c80643a0 0x7f80c80647d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 --2- 172.21.15.94:0/2848069432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c8102240 0x7f80c8102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 -- 172.21.15.94:0/2848069432 >> 172.21.15.94:0/2848069432 conn(0x7f80c80710b0 msgr2=0x7f80c80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 -- 172.21.15.94:0/2848069432 shutdown_connections 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 -- 172.21.15.94:0/2848069432 wait complete. 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 Processor -- start 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 -- start start 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c80643a0 0x7f80c8112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c8102240 0x7f80c8113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c8113810 con 0x7f80c8102240 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.753+0000 7f80cfcee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c810ce60 con 0x7f80c80643a0 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cd289640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c8102240 0x7f80c8113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cd289640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c8102240 0x7f80c8113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48690/0 (socket says 172.21.15.94:48690) 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cd289640 1 -- 172.21.15.94:0/1622086105 learned_addr learned my addr 172.21.15.94:0/1622086105 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:24.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cda8a640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c80643a0 0x7f80c8112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:24.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cd289640 1 -- 172.21.15.94:0/1622086105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c80643a0 msgr2=0x7f80c8112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cd289640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c80643a0 0x7f80c8112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cd289640 1 -- 172.21.15.94:0/1622086105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b8005040 con 0x7f80c8102240 2024-04-06T03:52:24.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cd289640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c8102240 0x7f80c8113210 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80b8004930 tx=0x7f80b8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:24.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80be7fc640 1 -- 172.21.15.94:0/1622086105 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80b80094c0 con 0x7f80c8102240 2024-04-06T03:52:24.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80be7fc640 1 -- 172.21.15.94:0/1622086105 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80b8032e50 con 0x7f80c8102240 2024-04-06T03:52:24.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c810d0e0 con 0x7f80c8102240 2024-04-06T03:52:24.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80be7fc640 1 -- 172.21.15.94:0/1622086105 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80b80433b0 con 0x7f80c8102240 2024-04-06T03:52:24.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.754+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c810d5a0 con 0x7f80c8102240 2024-04-06T03:52:24.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.755+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c80fd940 con 0x7f80c8102240 2024-04-06T03:52:24.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.756+0000 7f80be7fc640 1 -- 172.21.15.94:0/1622086105 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f80b802f070 con 0x7f80c8102240 2024-04-06T03:52:24.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.756+0000 7f80be7fc640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f80b406dc10 0x7f80b40700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:24.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.756+0000 7f80cda8a640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f80b406dc10 0x7f80b40700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:24.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.756+0000 7f80be7fc640 1 -- 172.21.15.94:0/1622086105 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f80b80af640 con 0x7f80c8102240 2024-04-06T03:52:24.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.757+0000 7f80cda8a640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f80b406dc10 0x7f80b40700c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f80c0004550 tx=0x7f80c0009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:24.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.761+0000 7f80be7fc640 1 -- 172.21.15.94:0/1622086105 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f80b8078b20 con 0x7f80c8102240 2024-04-06T03:52:24.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.992+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f80c8109320 con 0x7f80c8102240 2024-04-06T03:52:24.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.992+0000 7f80be7fc640 1 -- 172.21.15.94:0/1622086105 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v36) v1 ==== 107+0+4354 (secure 0 0 0) 0x7f80b807c1e0 con 0x7f80c8102240 2024-04-06T03:52:24.994 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:24.994 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi094.kvxcqo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6829/1650064917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1650064917},{"type":"v1","addr":"172.21.15.94:6829","nonce":1650064917}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44331,"name":"cephfs.smithi136.hbjgtw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6825/1500356136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1500356136},{"type":"v1","addr":"172.21.15.136:6825","nonce":1500356136}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-06T03:37:49.396246+0000","modified":"2024-04-06T03:48:59.605158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi094.qogyzw","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.94:6827/1447944673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1447944673},{"type":"v1","addr":"172.21.15.94:6827","nonce":1447944673}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-06T03:52:24.995 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 34 2024-04-06T03:52:24.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.994+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f80b406dc10 msgr2=0x7f80b40700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.994+0000 7f80cfcee640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f80b406dc10 0x7f80b40700c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f80c0004550 tx=0x7f80c0009040 comp rx=0 tx=0).stop 2024-04-06T03:52:24.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.994+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c8102240 msgr2=0x7f80c8113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:24.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.994+0000 7f80cfcee640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c8102240 0x7f80c8113210 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f80b8004930 tx=0x7f80b8009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.994+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 shutdown_connections 2024-04-06T03:52:24.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.994+0000 7f80cfcee640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f80b406dc10 0x7f80b40700c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.994+0000 7f80cfcee640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c8102240 0x7f80c8113210 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.994+0000 7f80cfcee640 1 --2- 172.21.15.94:0/1622086105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c80643a0 0x7f80c8112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:24.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.994+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 >> 172.21.15.94:0/1622086105 conn(0x7f80c80710b0 msgr2=0x7f80c80f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:24.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.995+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 shutdown_connections 2024-04-06T03:52:24.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:24.995+0000 7f80cfcee640 1 -- 172.21.15.94:0/1622086105 wait complete. 2024-04-06T03:52:25.007 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-06T03:52:25.018 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-06T03:52:25.018 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:52:25.018 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-06T03:52:25.035 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-06T03:52:25.035 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-06T03:52:25.092 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-06T03:52:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.233+0000 7fe11619d640 1 -- 172.21.15.94:0/2594012946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 msgr2=0x7fe1100faf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.233+0000 7fe11619d640 1 --2- 172.21.15.94:0/2594012946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 0x7fe1100faf30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0048d0 tx=0x7fe10c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.233+0000 7fe11619d640 1 -- 172.21.15.94:0/2594012946 shutdown_connections 2024-04-06T03:52:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.233+0000 7fe11619d640 1 --2- 172.21.15.94:0/2594012946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 0x7fe1100faf30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.233+0000 7fe11619d640 1 --2- 172.21.15.94:0/2594012946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110100f00 0x7fe1101012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.233+0000 7fe11619d640 1 -- 172.21.15.94:0/2594012946 >> 172.21.15.94:0/2594012946 conn(0x7fe1100f6af0 msgr2=0x7fe1100f8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.233+0000 7fe11619d640 1 -- 172.21.15.94:0/2594012946 shutdown_connections 2024-04-06T03:52:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.233+0000 7fe11619d640 1 -- 172.21.15.94:0/2594012946 wait complete. 2024-04-06T03:52:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11619d640 1 Processor -- start 2024-04-06T03:52:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11619d640 1 -- start start 2024-04-06T03:52:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11619d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110100f00 0x7fe11010dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11619d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 0x7fe11010e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11619d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100664b0 con 0x7fe1101018a0 2024-04-06T03:52:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11619d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11010e770 con 0x7fe110100f00 2024-04-06T03:52:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11499a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 0x7fe11010e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11499a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 0x7fe11010e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48714/0 (socket says 172.21.15.94:48714) 2024-04-06T03:52:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11499a640 1 -- 172.21.15.94:0/2854449624 learned_addr learned my addr 172.21.15.94:0/2854449624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11519b640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110100f00 0x7fe11010dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11499a640 1 -- 172.21.15.94:0/2854449624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110100f00 msgr2=0x7fe11010dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11499a640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110100f00 0x7fe11010dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11499a640 1 -- 172.21.15.94:0/2854449624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c005040 con 0x7fe1101018a0 2024-04-06T03:52:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe11499a640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 0x7fe11010e230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe10c004930 tx=0x7fe10c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.234+0000 7fe105ffb640 1 -- 172.21.15.94:0/2854449624 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10c0094c0 con 0x7fe1101018a0 2024-04-06T03:52:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.235+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11010e9f0 con 0x7fe1101018a0 2024-04-06T03:52:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.235+0000 7fe105ffb640 1 -- 172.21.15.94:0/2854449624 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe10c032e50 con 0x7fe1101018a0 2024-04-06T03:52:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.235+0000 7fe105ffb640 1 -- 172.21.15.94:0/2854449624 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10c0433b0 con 0x7fe1101018a0 2024-04-06T03:52:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.235+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe110116f50 con 0x7fe1101018a0 2024-04-06T03:52:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.236+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1100fc670 con 0x7fe1101018a0 2024-04-06T03:52:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.236+0000 7fe105ffb640 1 -- 172.21.15.94:0/2854449624 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fe10c02f070 con 0x7fe1101018a0 2024-04-06T03:52:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.237+0000 7fe105ffb640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe0fc06dc10 0x7fe0fc0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.237+0000 7fe105ffb640 1 -- 172.21.15.94:0/2854449624 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe10c0af460 con 0x7fe1101018a0 2024-04-06T03:52:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.237+0000 7fe11519b640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe0fc06dc10 0x7fe0fc0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.237+0000 7fe11519b640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe0fc06dc10 0x7fe0fc0700c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe100009470 tx=0x7fe100009ed0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:25.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.241+0000 7fe105ffb640 1 -- 172.21.15.94:0/2854449624 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe10c078940 con 0x7fe1101018a0 2024-04-06T03:52:25.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:24 smithi136 ceph-mon[82061]: pgmap v339: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:25.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:24 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/584586099' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-06T03:52:25.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:24 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3814723337' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-06T03:52:25.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:24 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2880075187' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-06T03:52:25.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:24 smithi094 ceph-mon[94053]: pgmap v339: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:25.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:24 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/584586099' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-06T03:52:25.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:24 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3814723337' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-06T03:52:25.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:24 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2880075187' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-06T03:52:25.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.420+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe11004bcc0 con 0x7fe1101018a0 2024-04-06T03:52:25.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.422+0000 7fe105ffb640 1 -- 172.21.15.94:0/2854449624 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16886 (secure 0 0 0) 0x7fe10c07c000 con 0x7fe1101018a0 2024-04-06T03:52:25.423 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:25.423 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":82,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","created":"2024-04-06T03:31:12.411006+0000","modified":"2024-04-06T03:48:51.384943+0000","last_up_change":"2024-04-06T03:47:32.894139+0000","last_in_change":"2024-04-06T03:36:17.627724+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-06T03:35:23.271684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-06T03:37:48.333560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-06T03:37:48.624904+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f51dba0-a74e-444c-b4ab-d5f1fa68ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6803","nonce":95929004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6805","nonce":95929004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6809","nonce":95929004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6807","nonce":95929004}]},"public_addr":"172.21.15.94:6803/95929004","cluster_addr":"172.21.15.94:6805/95929004","heartbeat_back_addr":"172.21.15.94:6809/95929004","heartbeat_front_addr":"172.21.15.94:6807/95929004","state":["exists","up"]},{"osd":1,"uuid":"2dd01097-c6d6-45e9-acee-ca5ea5f4e166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6811","nonce":3698645734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6813","nonce":3698645734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6817","nonce":3698645734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6815","nonce":3698645734}]},"public_addr":"172.21.15.94:6811/3698645734","cluster_addr":"172.21.15.94:6813/3698645734","heartbeat_back_addr":"172.21.15.94:6817/3698645734","heartbeat_front_addr":"172.21.15.94:6815/3698645734","state":["exists","up"]},{"osd":2,"uuid":"9645da7c-ffbc-4415-8fa0-0fcb07c53f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6819","nonce":1970793192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6821","nonce":1970793192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6825","nonce":1970793192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6823","nonce":1970793192}]},"public_addr":"172.21.15.94:6819/1970793192","cluster_addr":"172.21.15.94:6821/1970793192","heartbeat_back_addr":"172.21.15.94:6825/1970793192","heartbeat_front_addr":"172.21.15.94:6823/1970793192","state":["exists","up"]},{"osd":3,"uuid":"43321ebd-f195-4738-a2a5-f4d55a375bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6801","nonce":2267317275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6803","nonce":2267317275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6807","nonce":2267317275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6805","nonce":2267317275}]},"public_addr":"172.21.15.136:6801/2267317275","cluster_addr":"172.21.15.136:6803/2267317275","heartbeat_back_addr":"172.21.15.136:6807/2267317275","heartbeat_front_addr":"172.21.15.136:6805/2267317275","state":["exists","up"]},{"osd":4,"uuid":"572f9cc0-e83c-4354-babb-519be04069c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6809","nonce":3530002524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6811","nonce":3530002524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6815","nonce":3530002524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6813","nonce":3530002524}]},"public_addr":"172.21.15.136:6809/3530002524","cluster_addr":"172.21.15.136:6811/3530002524","heartbeat_back_addr":"172.21.15.136:6815/3530002524","heartbeat_front_addr":"172.21.15.136:6813/3530002524","state":["exists","up"]},{"osd":5,"uuid":"a38e98d8-dfb7-4083-aae8-e5853a896d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6817","nonce":853857034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6819","nonce":853857034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6823","nonce":853857034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6821","nonce":853857034}]},"public_addr":"172.21.15.136:6817/853857034","cluster_addr":"172.21.15.136:6819/853857034","heartbeat_back_addr":"172.21.15.136:6823/853857034","heartbeat_front_addr":"172.21.15.136:6821/853857034","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:25.784070+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:53.656615+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:20.675464+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:44.290139+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:09.122272+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:34.741771+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.136:6826/249801036":"2024-04-07T03:48:51.384914+0000","172.21.15.136:6827/249801036":"2024-04-07T03:48:51.384914+0000","172.21.15.136:6825/838120541":"2024-04-07T03:48:26.996253+0000","172.21.15.136:6824/838120541":"2024-04-07T03:48:26.996253+0000","172.21.15.94:6828/3794043365":"2024-04-07T03:48:03.697599+0000","172.21.15.94:6827/1913624529":"2024-04-07T03:47:37.708879+0000","172.21.15.94:0/2779591805":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6801/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:0/4131567976":"2024-04-07T03:40:06.067489+0000","172.21.15.94:6800/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:0/1084447585":"2024-04-07T03:32:10.456958+0000","172.21.15.94:0/2795266101":"2024-04-07T03:43:05.151475+0000","172.21.15.94:6800/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1319147349":"2024-04-07T03:40:06.067489+0000","172.21.15.94:0/1247691830":"2024-04-07T03:33:12.953920+0000","172.21.15.136:6829/614857857":"2024-04-07T03:42:13.227829+0000","172.21.15.94:0/457112173":"2024-04-07T03:40:06.067489+0000","172.21.15.136:0/2374087532":"2024-04-07T03:42:13.227829+0000","172.21.15.94:6801/4259523022":"2024-04-07T03:43:05.151475+0000","172.21.15.94:6801/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1085015108":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/852522634":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/3195038611":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/2412091380":"2024-04-07T03:33:12.953920+0000","172.21.15.94:6800/4259523022":"2024-04-07T03:43:05.151475+0000","172.21.15.94:0/3423462064":"2024-04-07T03:43:05.151475+0000","172.21.15.136:0/445150575":"2024-04-07T03:42:13.227829+0000","172.21.15.94:0/414290682":"2024-04-07T03:43:05.151475+0000","172.21.15.136:0/299229849":"2024-04-07T03:42:13.227829+0000","172.21.15.94:6800/3627366865":"2024-04-07T03:40:06.067489+0000","172.21.15.94:0/3705686069":"2024-04-07T03:40:06.067489+0000","172.21.15.94:6800/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6829/3794043365":"2024-04-07T03:48:03.697599+0000","172.21.15.94:0/3724179185":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/3039501657":"2024-04-07T03:43:05.151475+0000","172.21.15.136:6828/614857857":"2024-04-07T03:42:13.227829+0000","172.21.15.94:6801/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6801/3627366865":"2024-04-07T03:40:06.067489+0000","172.21.15.136:0/3970723326":"2024-04-07T03:42:13.227829+0000","172.21.15.94:0/1674904333":"2024-04-07T03:43:05.151475+0000","172.21.15.94:0/873173111":"2024-04-07T03:43:05.151475+0000","172.21.15.94:0/1082665650":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6826/1913624529":"2024-04-07T03:47:37.708879+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-06T03:52:25.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe0fc06dc10 msgr2=0x7fe0fc0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe0fc06dc10 0x7fe0fc0700c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe100009470 tx=0x7fe100009ed0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 msgr2=0x7fe11010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 0x7fe11010e230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe10c004930 tx=0x7fe10c009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 shutdown_connections 2024-04-06T03:52:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fe0fc06dc10 0x7fe0fc0700c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1101018a0 0x7fe11010e230 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 --2- 172.21.15.94:0/2854449624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe110100f00 0x7fe11010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 >> 172.21.15.94:0/2854449624 conn(0x7fe1100f6af0 msgr2=0x7fe1100f7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 shutdown_connections 2024-04-06T03:52:25.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.424+0000 7fe11619d640 1 -- 172.21.15.94:0/2854449624 wait complete. 2024-04-06T03:52:25.436 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-06T03:35:23.271684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-06T03:37:48.333560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-06T03:37:48.624904+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-06T03:52:25.437 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.537+0000 7f01576ee640 1 -- 172.21.15.94:0/270626138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01500fb110 msgr2=0x7f01500fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.537+0000 7f01576ee640 1 --2- 172.21.15.94:0/270626138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01500fb110 0x7f01500fb560 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f01400048d0 tx=0x7f0140009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.537+0000 7f01576ee640 1 -- 172.21.15.94:0/270626138 shutdown_connections 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.537+0000 7f01576ee640 1 --2- 172.21.15.94:0/270626138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01500fb110 0x7f01500fb560 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.537+0000 7f01576ee640 1 --2- 172.21.15.94:0/270626138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0150101140 0x7f0150101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.537+0000 7f01576ee640 1 -- 172.21.15.94:0/270626138 >> 172.21.15.94:0/270626138 conn(0x7f01500710b0 msgr2=0x7f01500714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.537+0000 7f01576ee640 1 -- 172.21.15.94:0/270626138 shutdown_connections 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f01576ee640 1 -- 172.21.15.94:0/270626138 wait complete. 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f01576ee640 1 Processor -- start 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f01576ee640 1 -- start start 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f01576ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01500fb110 0x7f015010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:25.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f01576ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0150101140 0x7f015010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f01576ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0150065310 con 0x7f0150101140 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f01576ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015010e930 con 0x7f01500fb110 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f0154c89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0150101140 0x7f015010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f0154c89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0150101140 0x7f015010e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48726/0 (socket says 172.21.15.94:48726) 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f0154c89640 1 -- 172.21.15.94:0/1330179322 learned_addr learned my addr 172.21.15.94:0/1330179322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.538+0000 7f015548a640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01500fb110 0x7f015010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f0154c89640 1 -- 172.21.15.94:0/1330179322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01500fb110 msgr2=0x7f015010deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f0154c89640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01500fb110 0x7f015010deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f0154c89640 1 -- 172.21.15.94:0/1330179322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140005040 con 0x7f0150101140 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f015548a640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01500fb110 0x7f015010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f0154c89640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0150101140 0x7f015010e3f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01400041d0 tx=0x7f0140009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f0145ffb640 1 -- 172.21.15.94:0/1330179322 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0140030870 con 0x7f0150101140 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f0145ffb640 1 -- 172.21.15.94:0/1330179322 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01400092e0 con 0x7f0150101140 2024-04-06T03:52:25.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f0145ffb640 1 -- 172.21.15.94:0/1330179322 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0140043400 con 0x7f0150101140 2024-04-06T03:52:25.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015010ebb0 con 0x7f0150101140 2024-04-06T03:52:25.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.539+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0150117170 con 0x7f0150101140 2024-04-06T03:52:25.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.540+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01500fc800 con 0x7f0150101140 2024-04-06T03:52:25.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.541+0000 7f0145ffb640 1 -- 172.21.15.94:0/1330179322 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f014003b420 con 0x7f0150101140 2024-04-06T03:52:25.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.541+0000 7f0145ffb640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f013c06dc10 0x7f013c0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:25.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.541+0000 7f0145ffb640 1 -- 172.21.15.94:0/1330179322 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f01400b0330 con 0x7f0150101140 2024-04-06T03:52:25.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.541+0000 7f015548a640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f013c06dc10 0x7f013c0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:25.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.541+0000 7f015548a640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f013c06dc10 0x7f013c0700c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f014c009fd0 tx=0x7f014c009e50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:25.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.548+0000 7f0145ffb640 1 -- 172.21.15.94:0/1330179322 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0140079930 con 0x7f0150101140 2024-04-06T03:52:25.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.726+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0150064910 con 0x7f0150101140 2024-04-06T03:52:25.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.726+0000 7f0145ffb640 1 -- 172.21.15.94:0/1330179322 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f014007cff0 con 0x7f0150101140 2024-04-06T03:52:25.728 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-06T03:52:25.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f013c06dc10 msgr2=0x7f013c0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f013c06dc10 0x7f013c0700c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f014c009fd0 tx=0x7f014c009e50 comp rx=0 tx=0).stop 2024-04-06T03:52:25.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0150101140 msgr2=0x7f015010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0150101140 0x7f015010e3f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01400041d0 tx=0x7f0140009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 shutdown_connections 2024-04-06T03:52:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f013c06dc10 0x7f013c0700c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0150101140 0x7f015010e3f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 --2- 172.21.15.94:0/1330179322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01500fb110 0x7f015010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 >> 172.21.15.94:0/1330179322 conn(0x7f01500710b0 msgr2=0x7f01500f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 shutdown_connections 2024-04-06T03:52:25.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.728+0000 7f01576ee640 1 -- 172.21.15.94:0/1330179322 wait complete. 2024-04-06T03:52:25.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-06T03:52:25.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.839+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/1994297762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fb9e0 msgr2=0x7f8d040fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.839+0000 7f8d0ad9d640 1 --2- 172.21.15.94:0/1994297762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fb9e0 0x7f8d040fbdb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40048d0 tx=0x7f8cf4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.839+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/1994297762 shutdown_connections 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.839+0000 7f8d0ad9d640 1 --2- 172.21.15.94:0/1994297762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d040fc2f0 0x7f8d04108570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.839+0000 7f8d0ad9d640 1 --2- 172.21.15.94:0/1994297762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fb9e0 0x7f8d040fbdb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.839+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/1994297762 >> 172.21.15.94:0/1994297762 conn(0x7f8d04071e80 msgr2=0x7f8d04072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.839+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/1994297762 shutdown_connections 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.839+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/1994297762 wait complete. 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0ad9d640 1 Processor -- start 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0ad9d640 1 -- start start 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0ad9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d040fb9e0 0x7f8d04114dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fc2f0 0x7f8d04115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0ad9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d04115990 con 0x7f8d040fc2f0 2024-04-06T03:52:25.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0ad9d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0410ef50 con 0x7f8d040fb9e0 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0959a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fc2f0 0x7f8d04115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0959a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fc2f0 0x7f8d04115300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48732/0 (socket says 172.21.15.94:48732) 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0959a640 1 -- 172.21.15.94:0/3740976070 learned_addr learned my addr 172.21.15.94:0/3740976070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0959a640 1 -- 172.21.15.94:0/3740976070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d040fb9e0 msgr2=0x7f8d04114dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0959a640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d040fb9e0 0x7f8d04114dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.840+0000 7f8d0959a640 1 -- 172.21.15.94:0/3740976070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf4005040 con 0x7f8d040fc2f0 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.841+0000 7f8d0959a640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fc2f0 0x7f8d04115300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800b790 tx=0x7f8cf80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.841+0000 7f8d027fc640 1 -- 172.21.15.94:0/3740976070 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf800c700 con 0x7f8d040fc2f0 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.841+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0410f230 con 0x7f8d040fc2f0 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.841+0000 7f8d027fc640 1 -- 172.21.15.94:0/3740976070 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cf800cd30 con 0x7f8d040fc2f0 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.841+0000 7f8d027fc640 1 -- 172.21.15.94:0/3740976070 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cf8013370 con 0x7f8d040fc2f0 2024-04-06T03:52:25.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.841+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0410f750 con 0x7f8d040fc2f0 2024-04-06T03:52:25.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.842+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d040fb580 con 0x7f8d040fc2f0 2024-04-06T03:52:25.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.843+0000 7f8d027fc640 1 -- 172.21.15.94:0/3740976070 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f8cf8012070 con 0x7f8d040fc2f0 2024-04-06T03:52:25.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.843+0000 7f8d027fc640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8cec06dc10 0x7f8cec0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:25.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.843+0000 7f8d027fc640 1 -- 172.21.15.94:0/3740976070 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f8cf808d280 con 0x7f8d040fc2f0 2024-04-06T03:52:25.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.843+0000 7f8d09d9b640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8cec06dc10 0x7f8cec0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:25.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.844+0000 7f8d09d9b640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8cec06dc10 0x7f8cec0700c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40043d0 tx=0x7f8cf4009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:25.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:25.848+0000 7f8d027fc640 1 -- 172.21.15.94:0/3740976070 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8cf8056760 con 0x7f8d040fc2f0 2024-04-06T03:52:26.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.028+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f8d0410b490 con 0x7f8d040fc2f0 2024-04-06T03:52:26.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.029+0000 7f8d027fc640 1 -- 172.21.15.94:0/3740976070 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8cf8059e20 con 0x7f8d040fc2f0 2024-04-06T03:52:26.030 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-06T03:52:26.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8cec06dc10 msgr2=0x7f8cec0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8cec06dc10 0x7f8cec0700c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40043d0 tx=0x7f8cf4009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fc2f0 msgr2=0x7f8d04115300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fc2f0 0x7f8d04115300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800b790 tx=0x7f8cf80091d0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 shutdown_connections 2024-04-06T03:52:26.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f8cec06dc10 0x7f8cec0700c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d040fc2f0 0x7f8d04115300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 --2- 172.21.15.94:0/3740976070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d040fb9e0 0x7f8d04114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 >> 172.21.15.94:0/3740976070 conn(0x7f8d04071e80 msgr2=0x7f8d04103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:26.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 shutdown_connections 2024-04-06T03:52:26.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.031+0000 7f8d0ad9d640 1 -- 172.21.15.94:0/3740976070 wait complete. 2024-04-06T03:52:26.044 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:25 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1622086105' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-06T03:52:26.044 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:25 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2854449624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:52:26.044 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:25 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/1330179322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:52:26.045 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-06T03:52:26.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.145+0000 7f635b6ee640 1 -- 172.21.15.94:0/2375991258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6354102b80 msgr2=0x7f63540fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.145+0000 7f635b6ee640 1 --2- 172.21.15.94:0/2375991258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6354102b80 0x7f63540fc210 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f63440048d0 tx=0x7f6344009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.145+0000 7f635b6ee640 1 -- 172.21.15.94:0/2375991258 shutdown_connections 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.145+0000 7f635b6ee640 1 --2- 172.21.15.94:0/2375991258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6354102b80 0x7f63540fc210 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.145+0000 7f635b6ee640 1 --2- 172.21.15.94:0/2375991258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63541021e0 0x7f63541025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.145+0000 7f635b6ee640 1 -- 172.21.15.94:0/2375991258 >> 172.21.15.94:0/2375991258 conn(0x7f63540f7dd0 msgr2=0x7f63540fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.145+0000 7f635b6ee640 1 -- 172.21.15.94:0/2375991258 shutdown_connections 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.145+0000 7f635b6ee640 1 -- 172.21.15.94:0/2375991258 wait complete. 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.145+0000 7f635b6ee640 1 Processor -- start 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635b6ee640 1 -- start start 2024-04-06T03:52:26.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635b6ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63541021e0 0x7f635406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:26.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635b6ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6354102b80 0x7f635406fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:26.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635b6ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540716b0 con 0x7f63541021e0 2024-04-06T03:52:26.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635b6ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6354070250 con 0x7f6354102b80 2024-04-06T03:52:26.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635948a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63541021e0 0x7f635406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:26.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635948a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63541021e0 0x7f635406f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48764/0 (socket says 172.21.15.94:48764) 2024-04-06T03:52:26.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635948a640 1 -- 172.21.15.94:0/24161412 learned_addr learned my addr 172.21.15.94:0/24161412 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f6358c89640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6354102b80 0x7f635406fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635948a640 1 -- 172.21.15.94:0/24161412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6354102b80 msgr2=0x7f635406fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635948a640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6354102b80 0x7f635406fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635948a640 1 -- 172.21.15.94:0/24161412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6344005040 con 0x7f63541021e0 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635948a640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63541021e0 0x7f635406f7d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f634c00db00 tx=0x7f634c004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f6349ffb640 1 -- 172.21.15.94:0/24161412 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f634c017070 con 0x7f63541021e0 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f6349ffb640 1 -- 172.21.15.94:0/24161412 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f634c00f040 con 0x7f63541021e0 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.146+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6354070530 con 0x7f63541021e0 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.147+0000 7f6349ffb640 1 -- 172.21.15.94:0/24161412 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f634c003ae0 con 0x7f63541021e0 2024-04-06T03:52:26.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.147+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63541193f0 con 0x7f63541021e0 2024-04-06T03:52:26.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.148+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63540fd950 con 0x7f63541021e0 2024-04-06T03:52:26.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.148+0000 7f6349ffb640 1 -- 172.21.15.94:0/24161412 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f634c003c40 con 0x7f63541021e0 2024-04-06T03:52:26.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.149+0000 7f6349ffb640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f634006dc10 0x7f63400700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:26.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.149+0000 7f6349ffb640 1 -- 172.21.15.94:0/24161412 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f634c08d5f0 con 0x7f63541021e0 2024-04-06T03:52:26.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.149+0000 7f6358c89640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f634006dc10 0x7f63400700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:26.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.149+0000 7f6358c89640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f634006dc10 0x7f63400700c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6354070f00 tx=0x7f6344009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:26.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.153+0000 7f6349ffb640 1 -- 172.21.15.94:0/24161412 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f634c056b50 con 0x7f63541021e0 2024-04-06T03:52:26.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.331+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f635404bcc0 con 0x7f63541021e0 2024-04-06T03:52:26.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.332+0000 7f6349ffb640 1 -- 172.21.15.94:0/24161412 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7f634c05a210 con 0x7f63541021e0 2024-04-06T03:52:26.333 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-06T03:52:26.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f634006dc10 msgr2=0x7f63400700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f634006dc10 0x7f63400700c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6354070f00 tx=0x7f6344009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63541021e0 msgr2=0x7f635406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63541021e0 0x7f635406f7d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f634c00db00 tx=0x7f634c004e60 comp rx=0 tx=0).stop 2024-04-06T03:52:26.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 shutdown_connections 2024-04-06T03:52:26.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f634006dc10 0x7f63400700c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6354102b80 0x7f635406fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 --2- 172.21.15.94:0/24161412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63541021e0 0x7f635406f7d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 >> 172.21.15.94:0/24161412 conn(0x7f63540f7dd0 msgr2=0x7f63540f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:26.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 shutdown_connections 2024-04-06T03:52:26.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.334+0000 7f635b6ee640 1 -- 172.21.15.94:0/24161412 wait complete. 2024-04-06T03:52:26.346 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-06T03:52:26.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:25 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1622086105' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-06T03:52:26.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:25 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2854449624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:52:26.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:25 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/1330179322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:52:26.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.444+0000 7feac24ee640 1 -- 172.21.15.94:0/3238432240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 msgr2=0x7feabc0fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.444+0000 7feac24ee640 1 --2- 172.21.15.94:0/3238432240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 0x7feabc0fc410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feab00048d0 tx=0x7feab0009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.445+0000 7feac24ee640 1 -- 172.21.15.94:0/3238432240 shutdown_connections 2024-04-06T03:52:26.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.445+0000 7feac24ee640 1 --2- 172.21.15.94:0/3238432240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 0x7feabc0fc410 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.445+0000 7feac24ee640 1 --2- 172.21.15.94:0/3238432240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feabc1023e0 0x7feabc1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.445+0000 7feac24ee640 1 -- 172.21.15.94:0/3238432240 >> 172.21.15.94:0/3238432240 conn(0x7feabc0f7fb0 msgr2=0x7feabc0fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:26.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.445+0000 7feac24ee640 1 -- 172.21.15.94:0/3238432240 shutdown_connections 2024-04-06T03:52:26.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.445+0000 7feac24ee640 1 -- 172.21.15.94:0/3238432240 wait complete. 2024-04-06T03:52:26.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.445+0000 7feac24ee640 1 Processor -- start 2024-04-06T03:52:26.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.445+0000 7feac24ee640 1 -- start start 2024-04-06T03:52:26.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feac24ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feabc1023e0 0x7feabc112f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:26.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feac24ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 0x7feabc113470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:26.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feac24ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc113a70 con 0x7feabc102d80 2024-04-06T03:52:26.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feac24ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc10d0c0 con 0x7feabc1023e0 2024-04-06T03:52:26.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feabb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 0x7feabc113470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:26.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feabb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 0x7feabc113470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48790/0 (socket says 172.21.15.94:48790) 2024-04-06T03:52:26.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feabb7fe640 1 -- 172.21.15.94:0/3437632835 learned_addr learned my addr 172.21.15.94:0/3437632835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:26.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feabbfff640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feabc1023e0 0x7feabc112f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feabb7fe640 1 -- 172.21.15.94:0/3437632835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feabc1023e0 msgr2=0x7feabc112f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feabb7fe640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feabc1023e0 0x7feabc112f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feabb7fe640 1 -- 172.21.15.94:0/3437632835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab0005040 con 0x7feabc102d80 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feabb7fe640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 0x7feabc113470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feab0004930 tx=0x7feab0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feab8ff9640 1 -- 172.21.15.94:0/3437632835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feab00094c0 con 0x7feabc102d80 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc10d340 con 0x7feabc102d80 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feab8ff9640 1 -- 172.21.15.94:0/3437632835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feab0032e50 con 0x7feabc102d80 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.446+0000 7feab8ff9640 1 -- 172.21.15.94:0/3437632835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feab00433b0 con 0x7feabc102d80 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.447+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc10d800 con 0x7feabc102d80 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.448+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feabc0fdb50 con 0x7feabc102d80 2024-04-06T03:52:26.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.448+0000 7feab8ff9640 1 -- 172.21.15.94:0/3437632835 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7feab002f070 con 0x7feabc102d80 2024-04-06T03:52:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.448+0000 7feab8ff9640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feaa406dc10 0x7feaa40700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:26.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.448+0000 7feab8ff9640 1 -- 172.21.15.94:0/3437632835 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7feab00af4b0 con 0x7feabc102d80 2024-04-06T03:52:26.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.448+0000 7feabbfff640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feaa406dc10 0x7feaa40700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:26.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.449+0000 7feabbfff640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feaa406dc10 0x7feaa40700c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feaac004ce0 tx=0x7feaac00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:26.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.453+0000 7feab8ff9640 1 -- 172.21.15.94:0/3437632835 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feab0078990 con 0x7feabc102d80 2024-04-06T03:52:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.644+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feabc109540 con 0x7feabc102d80 2024-04-06T03:52:26.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.645+0000 7feab8ff9640 1 -- 172.21.15.94:0/3437632835 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v82) v1 ==== 81+0+2550 (secure 0 0 0) 0x7feab007c050 con 0x7feabc102d80 2024-04-06T03:52:26.646 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6826/249801036 2024-04-07T03:48:51.384914+0000 2024-04-06T03:52:26.646 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6827/249801036 2024-04-07T03:48:51.384914+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6825/838120541 2024-04-07T03:48:26.996253+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6824/838120541 2024-04-07T03:48:26.996253+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6828/3794043365 2024-04-07T03:48:03.697599+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6827/1913624529 2024-04-07T03:47:37.708879+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2779591805 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/682257512 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4131567976 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/682257512 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1084447585 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2795266101 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2299971604 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1319147349 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1247691830 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:26.647 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6829/614857857 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/457112173 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:0/2374087532 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/4259523022 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2299971604 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1085015108 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/852522634 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3195038611 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2412091380 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/4259523022 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3423462064 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:0/445150575 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/414290682 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:0/299229849 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/3627366865 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:26.648 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3705686069 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/3295357679 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6829/3794043365 2024-04-07T03:48:03.697599+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3724179185 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3039501657 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6828/614857857 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/3295357679 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/3627366865 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:0/3970723326 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1674904333 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/873173111 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1082665650 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:26.649 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6826/1913624529 2024-04-07T03:47:37.708879+0000 2024-04-06T03:52:26.650 INFO:teuthology.orchestra.run.smithi094.stderr:listed 43 entries 2024-04-06T03:52:26.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feaa406dc10 msgr2=0x7feaa40700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feaa406dc10 0x7feaa40700c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feaac004ce0 tx=0x7feaac00b040 comp rx=0 tx=0).stop 2024-04-06T03:52:26.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 msgr2=0x7feabc113470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 0x7feabc113470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feab0004930 tx=0x7feab0009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 shutdown_connections 2024-04-06T03:52:26.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7feaa406dc10 0x7feaa40700c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feabc102d80 0x7feabc113470 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 --2- 172.21.15.94:0/3437632835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feabc1023e0 0x7feabc112f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 >> 172.21.15.94:0/3437632835 conn(0x7feabc0f7fb0 msgr2=0x7feabc0f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:26.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 shutdown_connections 2024-04-06T03:52:26.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.647+0000 7feac24ee640 1 -- 172.21.15.94:0/3437632835 wait complete. 2024-04-06T03:52:26.659 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-06T03:52:26.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:26.660 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-06T03:52:26.687 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:52:26.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:26.688 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-06T03:52:26.754 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:52:26.754 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-06T03:52:26.774 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-06T03:52:26.774 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-06T03:52:26.833 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.938+0000 7f67c5eee640 1 -- 172.21.15.94:0/2326270007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101130 msgr2=0x7f67c0101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.938+0000 7f67c5eee640 1 --2- 172.21.15.94:0/2326270007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101130 0x7f67c0101500 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009c50 tx=0x7f67b0004d10 comp rx=0 tx=0).stop 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.938+0000 7f67c5eee640 1 -- 172.21.15.94:0/2326270007 shutdown_connections 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.938+0000 7f67c5eee640 1 --2- 172.21.15.94:0/2326270007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c0101ad0 0x7f67c00fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.938+0000 7f67c5eee640 1 --2- 172.21.15.94:0/2326270007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101130 0x7f67c0101500 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.938+0000 7f67c5eee640 1 -- 172.21.15.94:0/2326270007 >> 172.21.15.94:0/2326270007 conn(0x7f67c00f6ce0 msgr2=0x7f67c00f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.938+0000 7f67c5eee640 1 -- 172.21.15.94:0/2326270007 shutdown_connections 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.938+0000 7f67c5eee640 1 -- 172.21.15.94:0/2326270007 wait complete. 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67c5eee640 1 Processor -- start 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67c5eee640 1 -- start start 2024-04-06T03:52:26.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67c5eee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c0101130 0x7f67c010bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67c5eee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101ad0 0x7f67c010c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67c5eee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00663e0 con 0x7f67c0101ad0 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67c5eee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0066520 con 0x7f67c0101130 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67beffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101ad0 0x7f67c010c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67beffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101ad0 0x7f67c010c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48804/0 (socket says 172.21.15.94:48804) 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67beffd640 1 -- 172.21.15.94:0/3523304615 learned_addr learned my addr 172.21.15.94:0/3523304615 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.939+0000 7f67bf7fe640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c0101130 0x7f67c010bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.940+0000 7f67beffd640 1 -- 172.21.15.94:0/3523304615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c0101130 msgr2=0x7f67c010bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.940+0000 7f67beffd640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c0101130 0x7f67c010bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.940+0000 7f67beffd640 1 -- 172.21.15.94:0/3523304615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b00098b0 con 0x7f67c0101ad0 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.940+0000 7f67beffd640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101ad0 0x7f67c010c1f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f67a800c830 tx=0x7f67a8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.940+0000 7f67b7fff640 1 -- 172.21.15.94:0/3523304615 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67a800a850 con 0x7f67c0101ad0 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.940+0000 7f67b7fff640 1 -- 172.21.15.94:0/3523304615 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67a800ae80 con 0x7f67c0101ad0 2024-04-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.940+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c010c880 con 0x7f67c0101ad0 2024-04-06T03:52:26.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.940+0000 7f67b7fff640 1 -- 172.21.15.94:0/3523304615 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67a8012ea0 con 0x7f67c0101ad0 2024-04-06T03:52:26.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.940+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c01194c0 con 0x7f67c0101ad0 2024-04-06T03:52:26.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.941+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c00fc8a0 con 0x7f67c0101ad0 2024-04-06T03:52:26.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.942+0000 7f67b7fff640 1 -- 172.21.15.94:0/3523304615 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f67a8012400 con 0x7f67c0101ad0 2024-04-06T03:52:26.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.942+0000 7f67b7fff640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f67ac06dc10 0x7f67ac0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:26.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.942+0000 7f67b7fff640 1 -- 172.21.15.94:0/3523304615 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f67a808d7f0 con 0x7f67c0101ad0 2024-04-06T03:52:26.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.942+0000 7f67bf7fe640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f67ac06dc10 0x7f67ac0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:26.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.942+0000 7f67bf7fe640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f67ac06dc10 0x7f67ac0700c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67b0004a30 tx=0x7f67b0037040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:26.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:26.947+0000 7f67b7fff640 1 -- 172.21.15.94:0/3523304615 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f67a8056d50 con 0x7f67c0101ad0 2024-04-06T03:52:27.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.128+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67c004bcc0 con 0x7f67c0101ad0 2024-04-06T03:52:27.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.129+0000 7f67b7fff640 1 -- 172.21.15.94:0/3523304615 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f67a80126b0 con 0x7f67c0101ad0 2024-04-06T03:52:27.131 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-06T03:52:27.131 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":82,"fsid":"c6509676-f3c5-11ee-b649-cb9ed24678a4","created":"2024-04-06T03:31:12.411006+0000","modified":"2024-04-06T03:48:51.384943+0000","last_up_change":"2024-04-06T03:47:32.894139+0000","last_in_change":"2024-04-06T03:36:17.627724+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-06T03:35:23.271684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-06T03:37:48.333560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-06T03:37:48.624904+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f51dba0-a74e-444c-b4ab-d5f1fa68ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6803","nonce":95929004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6805","nonce":95929004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6809","nonce":95929004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":95929004},{"type":"v1","addr":"172.21.15.94:6807","nonce":95929004}]},"public_addr":"172.21.15.94:6803/95929004","cluster_addr":"172.21.15.94:6805/95929004","heartbeat_back_addr":"172.21.15.94:6809/95929004","heartbeat_front_addr":"172.21.15.94:6807/95929004","state":["exists","up"]},{"osd":1,"uuid":"2dd01097-c6d6-45e9-acee-ca5ea5f4e166","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6811","nonce":3698645734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6813","nonce":3698645734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6817","nonce":3698645734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3698645734},{"type":"v1","addr":"172.21.15.94:6815","nonce":3698645734}]},"public_addr":"172.21.15.94:6811/3698645734","cluster_addr":"172.21.15.94:6813/3698645734","heartbeat_back_addr":"172.21.15.94:6817/3698645734","heartbeat_front_addr":"172.21.15.94:6815/3698645734","state":["exists","up"]},{"osd":2,"uuid":"9645da7c-ffbc-4415-8fa0-0fcb07c53f62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6819","nonce":1970793192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6821","nonce":1970793192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6825","nonce":1970793192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":1970793192},{"type":"v1","addr":"172.21.15.94:6823","nonce":1970793192}]},"public_addr":"172.21.15.94:6819/1970793192","cluster_addr":"172.21.15.94:6821/1970793192","heartbeat_back_addr":"172.21.15.94:6825/1970793192","heartbeat_front_addr":"172.21.15.94:6823/1970793192","state":["exists","up"]},{"osd":3,"uuid":"43321ebd-f195-4738-a2a5-f4d55a375bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6801","nonce":2267317275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6803","nonce":2267317275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6807","nonce":2267317275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2267317275},{"type":"v1","addr":"172.21.15.136:6805","nonce":2267317275}]},"public_addr":"172.21.15.136:6801/2267317275","cluster_addr":"172.21.15.136:6803/2267317275","heartbeat_back_addr":"172.21.15.136:6807/2267317275","heartbeat_front_addr":"172.21.15.136:6805/2267317275","state":["exists","up"]},{"osd":4,"uuid":"572f9cc0-e83c-4354-babb-519be04069c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6809","nonce":3530002524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6811","nonce":3530002524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6815","nonce":3530002524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3530002524},{"type":"v1","addr":"172.21.15.136:6813","nonce":3530002524}]},"public_addr":"172.21.15.136:6809/3530002524","cluster_addr":"172.21.15.136:6811/3530002524","heartbeat_back_addr":"172.21.15.136:6815/3530002524","heartbeat_front_addr":"172.21.15.136:6813/3530002524","state":["exists","up"]},{"osd":5,"uuid":"a38e98d8-dfb7-4083-aae8-e5853a896d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6817","nonce":853857034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6819","nonce":853857034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6823","nonce":853857034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":853857034},{"type":"v1","addr":"172.21.15.136:6821","nonce":853857034}]},"public_addr":"172.21.15.136:6817/853857034","cluster_addr":"172.21.15.136:6819/853857034","heartbeat_back_addr":"172.21.15.136:6823/853857034","heartbeat_front_addr":"172.21.15.136:6821/853857034","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:25.784070+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:34:53.656615+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:20.675464+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:35:44.290139+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:09.122272+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-06T03:36:34.741771+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.136:6826/249801036":"2024-04-07T03:48:51.384914+0000","172.21.15.136:6827/249801036":"2024-04-07T03:48:51.384914+0000","172.21.15.136:6825/838120541":"2024-04-07T03:48:26.996253+0000","172.21.15.136:6824/838120541":"2024-04-07T03:48:26.996253+0000","172.21.15.94:6828/3794043365":"2024-04-07T03:48:03.697599+0000","172.21.15.94:6827/1913624529":"2024-04-07T03:47:37.708879+0000","172.21.15.94:0/2779591805":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6801/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:0/4131567976":"2024-04-07T03:40:06.067489+0000","172.21.15.94:6800/682257512":"2024-04-07T03:32:10.456958+0000","172.21.15.94:0/1084447585":"2024-04-07T03:32:10.456958+0000","172.21.15.94:0/2795266101":"2024-04-07T03:43:05.151475+0000","172.21.15.94:6800/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1319147349":"2024-04-07T03:40:06.067489+0000","172.21.15.94:0/1247691830":"2024-04-07T03:33:12.953920+0000","172.21.15.136:6829/614857857":"2024-04-07T03:42:13.227829+0000","172.21.15.94:0/457112173":"2024-04-07T03:40:06.067489+0000","172.21.15.136:0/2374087532":"2024-04-07T03:42:13.227829+0000","172.21.15.94:6801/4259523022":"2024-04-07T03:43:05.151475+0000","172.21.15.94:6801/2299971604":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/1085015108":"2024-04-07T03:33:12.953920+0000","172.21.15.94:0/852522634":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/3195038611":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/2412091380":"2024-04-07T03:33:12.953920+0000","172.21.15.94:6800/4259523022":"2024-04-07T03:43:05.151475+0000","172.21.15.94:0/3423462064":"2024-04-07T03:43:05.151475+0000","172.21.15.136:0/445150575":"2024-04-07T03:42:13.227829+0000","172.21.15.94:0/414290682":"2024-04-07T03:43:05.151475+0000","172.21.15.136:0/299229849":"2024-04-07T03:42:13.227829+0000","172.21.15.94:6800/3627366865":"2024-04-07T03:40:06.067489+0000","172.21.15.94:0/3705686069":"2024-04-07T03:40:06.067489+0000","172.21.15.94:6800/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6829/3794043365":"2024-04-07T03:48:03.697599+0000","172.21.15.94:0/3724179185":"2024-04-07T03:31:39.465850+0000","172.21.15.94:0/3039501657":"2024-04-07T03:43:05.151475+0000","172.21.15.136:6828/614857857":"2024-04-07T03:42:13.227829+0000","172.21.15.94:6801/3295357679":"2024-04-07T03:31:39.465850+0000","172.21.15.94:6801/3627366865":"2024-04-07T03:40:06.067489+0000","172.21.15.136:0/3970723326":"2024-04-07T03:42:13.227829+0000","172.21.15.94:0/1674904333":"2024-04-07T03:43:05.151475+0000","172.21.15.94:0/873173111":"2024-04-07T03:43:05.151475+0000","172.21.15.94:0/1082665650":"2024-04-07T03:32:10.456958+0000","172.21.15.94:6826/1913624529":"2024-04-07T03:47:37.708879+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-06T03:52:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.131+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f67ac06dc10 msgr2=0x7f67ac0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.131+0000 7f67c5eee640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f67ac06dc10 0x7f67ac0700c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67b0004a30 tx=0x7f67b0037040 comp rx=0 tx=0).stop 2024-04-06T03:52:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.131+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101ad0 msgr2=0x7f67c010c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.131+0000 7f67c5eee640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101ad0 0x7f67c010c1f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f67a800c830 tx=0x7f67a8004430 comp rx=0 tx=0).stop 2024-04-06T03:52:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.131+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 shutdown_connections 2024-04-06T03:52:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.131+0000 7f67c5eee640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f67ac06dc10 0x7f67ac0700c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.131+0000 7f67c5eee640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67c0101ad0 0x7f67c010c1f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.131+0000 7f67c5eee640 1 --2- 172.21.15.94:0/3523304615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c0101130 0x7f67c010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.131+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 >> 172.21.15.94:0/3523304615 conn(0x7f67c00f6ce0 msgr2=0x7f67c01056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:27.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.132+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 shutdown_connections 2024-04-06T03:52:27.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.132+0000 7f67c5eee640 1 -- 172.21.15.94:0/3523304615 wait complete. 2024-04-06T03:52:27.144 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-06T03:35:23.271684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-06T03:37:48.333560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-06T03:37:48.624904+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-06T03:52:27.144 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-06T03:52:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.245+0000 7f67809fe640 1 -- 172.21.15.94:0/1526706412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c1029d0 msgr2=0x7f677c0643f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.245+0000 7f67809fe640 1 --2- 172.21.15.94:0/1526706412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c1029d0 0x7f677c0643f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f67640048d0 tx=0x7f6764009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.245+0000 7f67809fe640 1 -- 172.21.15.94:0/1526706412 shutdown_connections 2024-04-06T03:52:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.245+0000 7f67809fe640 1 --2- 172.21.15.94:0/1526706412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c1029d0 0x7f677c0643f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.245+0000 7f67809fe640 1 --2- 172.21.15.94:0/1526706412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c102030 0x7f677c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.245+0000 7f67809fe640 1 -- 172.21.15.94:0/1526706412 >> 172.21.15.94:0/1526706412 conn(0x7f677c071e80 msgr2=0x7f677c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.245+0000 7f67809fe640 1 -- 172.21.15.94:0/1526706412 shutdown_connections 2024-04-06T03:52:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.245+0000 7f67809fe640 1 -- 172.21.15.94:0/1526706412 wait complete. 2024-04-06T03:52:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f67809fe640 1 Processor -- start 2024-04-06T03:52:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f67809fe640 1 -- start start 2024-04-06T03:52:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f67809fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c102030 0x7f677c112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f67809fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1029d0 0x7f677c113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f67809fe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c113780 con 0x7f677c102030 2024-04-06T03:52:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f67809fe640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c10cdd0 con 0x7f677c1029d0 2024-04-06T03:52:27.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f677ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c102030 0x7f677c112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f677ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c102030 0x7f677c112c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48820/0 (socket says 172.21.15.94:48820) 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f677ad9d640 1 -- 172.21.15.94:0/3346759923 learned_addr learned my addr 172.21.15.94:0/3346759923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f6773fff640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1029d0 0x7f677c113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f677ad9d640 1 -- 172.21.15.94:0/3346759923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1029d0 msgr2=0x7f677c113180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f677ad9d640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1029d0 0x7f677c113180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f677ad9d640 1 -- 172.21.15.94:0/3346759923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764005040 con 0x7f677c102030 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.246+0000 7f6773fff640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1029d0 0x7f677c113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.247+0000 7f677ad9d640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c102030 0x7f677c112c40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f676c002a50 tx=0x7f676c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.247+0000 7f67737fe640 1 -- 172.21.15.94:0/3346759923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f676c007c30 con 0x7f677c102030 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.247+0000 7f67737fe640 1 -- 172.21.15.94:0/3346759923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f676c00f040 con 0x7f677c102030 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.247+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c10d0b0 con 0x7f677c102030 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.247+0000 7f67737fe640 1 -- 172.21.15.94:0/3346759923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f676c00db50 con 0x7f677c102030 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.247+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c10d5d0 con 0x7f677c102030 2024-04-06T03:52:27.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.248+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677c0fd7b0 con 0x7f677c102030 2024-04-06T03:52:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.248+0000 7f67737fe640 1 -- 172.21.15.94:0/3346759923 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f676c01c400 con 0x7f677c102030 2024-04-06T03:52:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.249+0000 7f67737fe640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f676806dc10 0x7f67680700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:27.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.249+0000 7f67737fe640 1 -- 172.21.15.94:0/3346759923 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f676c08cf30 con 0x7f677c102030 2024-04-06T03:52:27.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.249+0000 7f6773fff640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f676806dc10 0x7f67680700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:27.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.249+0000 7f6773fff640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f676806dc10 0x7f67680700c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f677c10e500 tx=0x7f6764009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:27.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.253+0000 7f67737fe640 1 -- 172.21.15.94:0/3346759923 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f676c0c1280 con 0x7f677c102030 2024-04-06T03:52:27.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:26 smithi136 ceph-mon[82061]: pgmap v340: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:27.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:26 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3740976070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-06T03:52:27.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:26 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/24161412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-06T03:52:27.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:26 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3437632835' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-06T03:52:27.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:26 smithi094 ceph-mon[94053]: pgmap v340: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:27.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:26 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3740976070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-06T03:52:27.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:26 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/24161412' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-06T03:52:27.417 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:26 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3437632835' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-06T03:52:27.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.527+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f677c1091b0 con 0x7f677c102030 2024-04-06T03:52:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.527+0000 7f67737fe640 1 -- 172.21.15.94:0/3346759923 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f676c056410 con 0x7f677c102030 2024-04-06T03:52:27.529 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-06T03:52:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f676806dc10 msgr2=0x7f67680700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f676806dc10 0x7f67680700c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f677c10e500 tx=0x7f6764009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c102030 msgr2=0x7f677c112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c102030 0x7f677c112c40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f676c002a50 tx=0x7f676c004600 comp rx=0 tx=0).stop 2024-04-06T03:52:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 shutdown_connections 2024-04-06T03:52:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f676806dc10 0x7f67680700c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1029d0 0x7f677c113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 --2- 172.21.15.94:0/3346759923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f677c102030 0x7f677c112c40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 >> 172.21.15.94:0/3346759923 conn(0x7f677c071e80 msgr2=0x7f677c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 shutdown_connections 2024-04-06T03:52:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.529+0000 7f67809fe640 1 -- 172.21.15.94:0/3346759923 wait complete. 2024-04-06T03:52:27.542 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-06T03:52:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.650+0000 7fc08cd9d640 1 -- 172.21.15.94:0/839985341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc088100f10 msgr2=0x7fc0881012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.650+0000 7fc08cd9d640 1 --2- 172.21.15.94:0/839985341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc088100f10 0x7fc0881012e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc0780048d0 tx=0x7fc078009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.650+0000 7fc08cd9d640 1 -- 172.21.15.94:0/839985341 shutdown_connections 2024-04-06T03:52:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.650+0000 7fc08cd9d640 1 --2- 172.21.15.94:0/839985341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0881018b0 0x7fc0880faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.650+0000 7fc08cd9d640 1 --2- 172.21.15.94:0/839985341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc088100f10 0x7fc0881012e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.650+0000 7fc08cd9d640 1 -- 172.21.15.94:0/839985341 >> 172.21.15.94:0/839985341 conn(0x7fc088071e80 msgr2=0x7fc088072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.650+0000 7fc08cd9d640 1 -- 172.21.15.94:0/839985341 shutdown_connections 2024-04-06T03:52:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.650+0000 7fc08cd9d640 1 -- 172.21.15.94:0/839985341 wait complete. 2024-04-06T03:52:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.650+0000 7fc08cd9d640 1 Processor -- start 2024-04-06T03:52:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc08cd9d640 1 -- start start 2024-04-06T03:52:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc08cd9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc088100f10 0x7fc08810dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:27.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc08cd9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0881018b0 0x7fc08810e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc08cd9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0880652c0 con 0x7fc0881018b0 2024-04-06T03:52:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc08cd9d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08810e7f0 con 0x7fc088100f10 2024-04-06T03:52:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc07effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0881018b0 0x7fc08810e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc07effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0881018b0 0x7fc08810e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56542/0 (socket says 172.21.15.94:56542) 2024-04-06T03:52:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc07effd640 1 -- 172.21.15.94:0/2010415357 learned_addr learned my addr 172.21.15.94:0/2010415357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc0877fe640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc088100f10 0x7fc08810dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc07effd640 1 -- 172.21.15.94:0/2010415357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc088100f10 msgr2=0x7fc08810dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc07effd640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc088100f10 0x7fc08810dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc07effd640 1 -- 172.21.15.94:0/2010415357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078005040 con 0x7fc0881018b0 2024-04-06T03:52:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.651+0000 7fc07effd640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0881018b0 0x7fc08810e2b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc07000eb70 tx=0x7fc07000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.652+0000 7fc084ff9640 1 -- 172.21.15.94:0/2010415357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc070007c30 con 0x7fc0881018b0 2024-04-06T03:52:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.652+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08810ea20 con 0x7fc0881018b0 2024-04-06T03:52:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.652+0000 7fc084ff9640 1 -- 172.21.15.94:0/2010415357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc07000f040 con 0x7fc0881018b0 2024-04-06T03:52:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.652+0000 7fc084ff9640 1 -- 172.21.15.94:0/2010415357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0700095b0 con 0x7fc0881018b0 2024-04-06T03:52:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.652+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc088116fd0 con 0x7fc0881018b0 2024-04-06T03:52:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.653+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0880fc680 con 0x7fc0881018b0 2024-04-06T03:52:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.653+0000 7fc084ff9640 1 -- 172.21.15.94:0/2010415357 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7fc070009730 con 0x7fc0881018b0 2024-04-06T03:52:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.654+0000 7fc084ff9640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc07406dc10 0x7fc0740700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.654+0000 7fc0877fe640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc07406dc10 0x7fc0740700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.654+0000 7fc084ff9640 1 -- 172.21.15.94:0/2010415357 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fc07008d3c0 con 0x7fc0881018b0 2024-04-06T03:52:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.654+0000 7fc0877fe640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc07406dc10 0x7fc0740700c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc078003080 tx=0x7fc078009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:27.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.658+0000 7fc084ff9640 1 -- 172.21.15.94:0/2010415357 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc0700568a0 con 0x7fc0881018b0 2024-04-06T03:52:27.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.847+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fc08810ebb0 con 0x7fc0881018b0 2024-04-06T03:52:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.847+0000 7fc084ff9640 1 -- 172.21.15.94:0/2010415357 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc070056030 con 0x7fc0881018b0 2024-04-06T03:52:27.849 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-06T03:52:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc07406dc10 msgr2=0x7fc0740700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc07406dc10 0x7fc0740700c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc078003080 tx=0x7fc078009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0881018b0 msgr2=0x7fc08810e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0881018b0 0x7fc08810e2b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc07000eb70 tx=0x7fc07000b040 comp rx=0 tx=0).stop 2024-04-06T03:52:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 shutdown_connections 2024-04-06T03:52:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7fc07406dc10 0x7fc0740700c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0881018b0 0x7fc08810e2b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 --2- 172.21.15.94:0/2010415357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc088100f10 0x7fc08810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 >> 172.21.15.94:0/2010415357 conn(0x7fc088071e80 msgr2=0x7fc088103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 shutdown_connections 2024-04-06T03:52:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.850+0000 7fc08cd9d640 1 -- 172.21.15.94:0/2010415357 wait complete. 2024-04-06T03:52:27.862 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.962+0000 7faedb2ee640 1 -- 172.21.15.94:0/748188406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 msgr2=0x7faed40fb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.962+0000 7faedb2ee640 1 --2- 172.21.15.94:0/748188406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 0x7faed40fb760 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faed00048d0 tx=0x7faed0009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.962+0000 7faedb2ee640 1 -- 172.21.15.94:0/748188406 shutdown_connections 2024-04-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.962+0000 7faedb2ee640 1 --2- 172.21.15.94:0/748188406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 0x7faed40fb760 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.962+0000 7faedb2ee640 1 --2- 172.21.15.94:0/748188406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faed41010d0 0x7faed41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.962+0000 7faedb2ee640 1 -- 172.21.15.94:0/748188406 >> 172.21.15.94:0/748188406 conn(0x7faed40714a0 msgr2=0x7faed40f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.962+0000 7faedb2ee640 1 -- 172.21.15.94:0/748188406 shutdown_connections 2024-04-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.962+0000 7faedb2ee640 1 -- 172.21.15.94:0/748188406 wait complete. 2024-04-06T03:52:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faedb2ee640 1 Processor -- start 2024-04-06T03:52:27.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faedb2ee640 1 -- start start 2024-04-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faedb2ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faed41010d0 0x7faed4111c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faedb2ee640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 0x7faed4112180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:27.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faedb2ee640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed4112810 con 0x7faed4101a70 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faedb2ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed410bdd0 con 0x7faed41010d0 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faed8889640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 0x7faed4112180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faed8889640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 0x7faed4112180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56560/0 (socket says 172.21.15.94:56560) 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faed8889640 1 -- 172.21.15.94:0/2243232215 learned_addr learned my addr 172.21.15.94:0/2243232215 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faed8889640 1 -- 172.21.15.94:0/2243232215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faed41010d0 msgr2=0x7faed4111c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faed8889640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faed41010d0 0x7faed4111c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.963+0000 7faed8889640 1 -- 172.21.15.94:0/2243232215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0005040 con 0x7faed4101a70 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.964+0000 7faed8889640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 0x7faed4112180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faed0004930 tx=0x7faed0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.964+0000 7faec5ffb640 1 -- 172.21.15.94:0/2243232215 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed00094c0 con 0x7faed4101a70 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.964+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed410c050 con 0x7faed4101a70 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.964+0000 7faec5ffb640 1 -- 172.21.15.94:0/2243232215 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faed0032e50 con 0x7faed4101a70 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.964+0000 7faec5ffb640 1 -- 172.21.15.94:0/2243232215 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed00433b0 con 0x7faed4101a70 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.964+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed410c510 con 0x7faed4101a70 2024-04-06T03:52:27.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.965+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed40fc980 con 0x7faed4101a70 2024-04-06T03:52:27.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.965+0000 7faec5ffb640 1 -- 172.21.15.94:0/2243232215 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7faed002f070 con 0x7faed4101a70 2024-04-06T03:52:27.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.966+0000 7faec5ffb640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7faec006dc10 0x7faec00700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:27.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.966+0000 7faec5ffb640 1 -- 172.21.15.94:0/2243232215 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7faed00af5a0 con 0x7faed4101a70 2024-04-06T03:52:27.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.966+0000 7faed908a640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7faec006dc10 0x7faec00700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:27.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.966+0000 7faed908a640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7faec006dc10 0x7faec00700c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faec80097e0 tx=0x7faec8006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:27.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:27.971+0000 7faec5ffb640 1 -- 172.21.15.94:0/2243232215 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faed0078a80 con 0x7faed4101a70 2024-04-06T03:52:28.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:27 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3523304615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:52:28.166 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:27 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/3346759923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:52:28.167 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:27 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2010415357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-06T03:52:28.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.199+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7faed404bcc0 con 0x7faed4101a70 2024-04-06T03:52:28.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.199+0000 7faec5ffb640 1 -- 172.21.15.94:0/2243232215 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v82) v1 ==== 107+0+11 (secure 0 0 0) 0x7faed007c140 con 0x7faed4101a70 2024-04-06T03:52:28.201 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7faec006dc10 msgr2=0x7faec00700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7faec006dc10 0x7faec00700c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faec80097e0 tx=0x7faec8006d20 comp rx=0 tx=0).stop 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 msgr2=0x7faed4112180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 0x7faed4112180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faed0004930 tx=0x7faed0009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 shutdown_connections 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7faec006dc10 0x7faec00700c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faed4101a70 0x7faed4112180 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 --2- 172.21.15.94:0/2243232215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faed41010d0 0x7faed4111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 >> 172.21.15.94:0/2243232215 conn(0x7faed40714a0 msgr2=0x7faed4103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 shutdown_connections 2024-04-06T03:52:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.201+0000 7faedb2ee640 1 -- 172.21.15.94:0/2243232215 wait complete. 2024-04-06T03:52:28.214 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-06T03:52:28.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.314+0000 7f3d899ff640 1 -- 172.21.15.94:0/1123753826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d840643a0 msgr2=0x7f3d840647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.314+0000 7f3d899ff640 1 --2- 172.21.15.94:0/1123753826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d840643a0 0x7f3d840647f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3d780048d0 tx=0x7f3d78009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.314+0000 7f3d899ff640 1 -- 172.21.15.94:0/1123753826 shutdown_connections 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.314+0000 7f3d899ff640 1 --2- 172.21.15.94:0/1123753826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d840643a0 0x7f3d840647f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.314+0000 7f3d899ff640 1 --2- 172.21.15.94:0/1123753826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d841022c0 0x7f3d84102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.314+0000 7f3d899ff640 1 -- 172.21.15.94:0/1123753826 >> 172.21.15.94:0/1123753826 conn(0x7f3d84071220 msgr2=0x7f3d84071620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.314+0000 7f3d899ff640 1 -- 172.21.15.94:0/1123753826 shutdown_connections 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.314+0000 7f3d899ff640 1 -- 172.21.15.94:0/1123753826 wait complete. 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d899ff640 1 Processor -- start 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d899ff640 1 -- start start 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d899ff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d840643a0 0x7f3d840fb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d899ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d841022c0 0x7f3d840fb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d899ff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840fd170 con 0x7f3d841022c0 2024-04-06T03:52:28.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d899ff640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840fbd10 con 0x7f3d840643a0 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d827fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d841022c0 0x7f3d840fb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d827fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d841022c0 0x7f3d840fb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56578/0 (socket says 172.21.15.94:56578) 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d827fc640 1 -- 172.21.15.94:0/2639256060 learned_addr learned my addr 172.21.15.94:0/2639256060 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d82ffd640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d840643a0 0x7f3d840fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d827fc640 1 -- 172.21.15.94:0/2639256060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d840643a0 msgr2=0x7f3d840fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d827fc640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d840643a0 0x7f3d840fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.315+0000 7f3d827fc640 1 -- 172.21.15.94:0/2639256060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d78005040 con 0x7f3d841022c0 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.316+0000 7f3d827fc640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d841022c0 0x7f3d840fb7d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d78004930 tx=0x7f3d78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.316+0000 7f3d6b7fe640 1 -- 172.21.15.94:0/2639256060 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d780094c0 con 0x7f3d841022c0 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.316+0000 7f3d6b7fe640 1 -- 172.21.15.94:0/2639256060 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d78032e50 con 0x7f3d841022c0 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.316+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d840fbf90 con 0x7f3d841022c0 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.316+0000 7f3d6b7fe640 1 -- 172.21.15.94:0/2639256060 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d780433b0 con 0x7f3d841022c0 2024-04-06T03:52:28.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.316+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d84119400 con 0x7f3d841022c0 2024-04-06T03:52:28.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.317+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d840fd9c0 con 0x7f3d841022c0 2024-04-06T03:52:28.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.317+0000 7f3d6b7fe640 1 -- 172.21.15.94:0/2639256060 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 39) v1 ==== 91859+0+0 (secure 0 0 0) 0x7f3d7802f070 con 0x7f3d841022c0 2024-04-06T03:52:28.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.318+0000 7f3d6b7fe640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d6c06dc10 0x7f3d6c0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-06T03:52:28.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.318+0000 7f3d6b7fe640 1 -- 172.21.15.94:0/2639256060 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f3d780af330 con 0x7f3d841022c0 2024-04-06T03:52:28.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.318+0000 7f3d82ffd640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d6c06dc10 0x7f3d6c0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-06T03:52:28.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.318+0000 7f3d82ffd640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d6c06dc10 0x7f3d6c0700c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d74004ed0 tx=0x7f3d7400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-06T03:52:28.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.327+0000 7f3d6b7fe640 1 -- 172.21.15.94:0/2639256060 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3d780788c0 con 0x7f3d841022c0 2024-04-06T03:52:28.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:27 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3523304615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-06T03:52:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:27 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/3346759923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-06T03:52:28.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:27 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2010415357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-06T03:52:28.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.515+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3d8404bcc0 con 0x7f3d841022c0 2024-04-06T03:52:28.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.515+0000 7f3d6b7fe640 1 -- 172.21.15.94:0/2639256060 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v82) v1 ==== 81+0+2550 (secure 0 0 0) 0x7f3d7807bf80 con 0x7f3d841022c0 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6826/249801036 2024-04-07T03:48:51.384914+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6827/249801036 2024-04-07T03:48:51.384914+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6825/838120541 2024-04-07T03:48:26.996253+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6824/838120541 2024-04-07T03:48:26.996253+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6828/3794043365 2024-04-07T03:48:03.697599+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6827/1913624529 2024-04-07T03:47:37.708879+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2779591805 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/682257512 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/4131567976 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/682257512 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:28.517 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1084447585 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2795266101 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2299971604 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1319147349 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1247691830 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6829/614857857 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/457112173 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:0/2374087532 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/4259523022 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2299971604 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1085015108 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/852522634 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3195038611 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2412091380 2024-04-07T03:33:12.953920+0000 2024-04-06T03:52:28.518 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/4259523022 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3423462064 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:0/445150575 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/414290682 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:0/299229849 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/3627366865 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3705686069 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/3295357679 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6829/3794043365 2024-04-07T03:48:03.697599+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3724179185 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3039501657 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:6828/614857857 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/3295357679 2024-04-07T03:31:39.465850+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/3627366865 2024-04-07T03:40:06.067489+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.136:0/3970723326 2024-04-07T03:42:13.227829+0000 2024-04-06T03:52:28.519 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1674904333 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:28.520 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/873173111 2024-04-07T03:43:05.151475+0000 2024-04-06T03:52:28.520 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1082665650 2024-04-07T03:32:10.456958+0000 2024-04-06T03:52:28.520 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6826/1913624529 2024-04-07T03:47:37.708879+0000 2024-04-06T03:52:28.521 INFO:teuthology.orchestra.run.smithi094.stderr:listed 43 entries 2024-04-06T03:52:28.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d6c06dc10 msgr2=0x7f3d6c0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:28.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d6c06dc10 0x7f3d6c0700c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d74004ed0 tx=0x7f3d7400b040 comp rx=0 tx=0).stop 2024-04-06T03:52:28.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d841022c0 msgr2=0x7f3d840fb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-06T03:52:28.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d841022c0 0x7f3d840fb7d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d78004930 tx=0x7f3d78009da0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 shutdown_connections 2024-04-06T03:52:28.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:6800/3117305505,v1:172.21.15.94:6801/3117305505] conn(0x7f3d6c06dc10 0x7f3d6c0700c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d841022c0 0x7f3d840fb7d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 --2- 172.21.15.94:0/2639256060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d840643a0 0x7f3d840fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-06T03:52:28.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 >> 172.21.15.94:0/2639256060 conn(0x7f3d84071220 msgr2=0x7f3d840f9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-06T03:52:28.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 shutdown_connections 2024-04-06T03:52:28.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-06T03:52:28.517+0000 7f3d899ff640 1 -- 172.21.15.94:0/2639256060 wait complete. 2024-04-06T03:52:28.533 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-06T03:52:28.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:28.533 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-06T03:52:28.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2024-04-06T03:52:28.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:28.571 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-06T03:52:28.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:28.647 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-06T03:52:28.662 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-06T03:52:28.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:28.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-06T03:52:28.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:28.731 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-06T03:52:28.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:28.838 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2024-04-06T03:52:28.856 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-06T03:52:28.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:28.857 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-06T03:52:28.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-06T03:52:28.931 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link delete ceph-brx 2024-04-06T03:52:29.023 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-06T03:52:29.040 INFO:tasks.cephadm:Teardown begin 2024-04-06T03:52:29.040 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-06T03:52:29.065 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:28 smithi094 ceph-mon[94053]: pgmap v341: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:29.065 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:28 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2243232215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-06T03:52:29.065 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:28 smithi094 ceph-mon[94053]: from='client.? 172.21.15.94:0/2639256060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-06T03:52:29.069 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-06T03:52:29.099 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-06T03:52:29.099 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-06T03:52:29.125 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-06T03:52:29.154 INFO:tasks.cephadm:Stopping all daemons... 2024-04-06T03:52:29.154 INFO:tasks.cephadm.mon.smithi094:Stopping mon.smithi094... 2024-04-06T03:52:29.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094 2024-04-06T03:52:29.347 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:28 smithi136 ceph-mon[82061]: pgmap v341: 65 pgs: 65 active+clean; 194 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-06T03:52:29.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:28 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2243232215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-06T03:52:29.348 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:28 smithi136 ceph-mon[82061]: from='client.? 172.21.15.94:0/2639256060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-06T03:52:29.416 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:29 smithi094 systemd[1]: Stopping Ceph mon.smithi094 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:52:29.916 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:29 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[94049]: 2024-04-06T03:52:29.492+0000 7f71bb413700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi094 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:52:29.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:29 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094[94049]: 2024-04-06T03:52:29.492+0000 7f71bb413700 -1 mon.smithi094@0(leader) e3 *** Got Signal Terminated *** 2024-04-06T03:52:29.917 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:29 smithi094 podman[151005]: 2024-04-06 03:52:29.645834964 +0000 UTC m=+0.273873905 container died 02f9626f65141f687879b102d918bfd3bc3788255fa35fc4015ba2554477628d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-06T03:52:30.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:30 smithi094 podman[151005]: 2024-04-06 03:52:30.026584577 +0000 UTC m=+0.654623524 container cleanup 02f9626f65141f687879b102d918bfd3bc3788255fa35fc4015ba2554477628d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-06T03:52:30.262 INFO:journalctl@ceph.mon.smithi094.smithi094.stdout:Apr 06 03:52:30 smithi094 bash[151005]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi094 2024-04-06T03:52:30.408 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi094.service' 2024-04-06T03:52:30.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-06T03:52:30.463 INFO:tasks.cephadm.mon.smithi094:Stopped mon.smithi094 2024-04-06T03:52:30.463 INFO:tasks.cephadm.mon.smithi136:Stopping mon.smithi136... 2024-04-06T03:52:30.463 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi136 2024-04-06T03:52:30.783 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:30 smithi136 systemd[1]: Stopping Ceph mon.smithi136 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:52:31.097 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:30 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136[82056]: 2024-04-06T03:52:30.781+0000 7f355192e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi136 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:52:31.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:30 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136[82056]: 2024-04-06T03:52:30.781+0000 7f355192e700 -1 mon.smithi136@1(peon) e3 *** Got Signal Terminated *** 2024-04-06T03:52:31.098 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:30 smithi136 podman[118214]: 2024-04-06 03:52:30.965125183 +0000 UTC m=+0.304367078 container died 46983eca715e307fac8e88e12b99cee98a2f947f2be64f0c4997abb73d432c8e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-06T03:52:31.597 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:31 smithi136 podman[118214]: 2024-04-06 03:52:31.415326782 +0000 UTC m=+0.754568683 container cleanup 46983eca715e307fac8e88e12b99cee98a2f947f2be64f0c4997abb73d432c8e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9f19235, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:52:31.598 INFO:journalctl@ceph.mon.smithi136.smithi136.stdout:Apr 06 03:52:31 smithi136 bash[118214]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-mon-smithi136 2024-04-06T03:52:31.829 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@mon.smithi136.service' 2024-04-06T03:52:31.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-06T03:52:31.876 INFO:tasks.cephadm.mon.smithi136:Stopped mon.smithi136 2024-04-06T03:52:31.876 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-06T03:52:31.876 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.0 2024-04-06T03:52:32.156 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:31 smithi094 systemd[1]: Stopping Ceph osd.0 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:52:32.417 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:32 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0[103784]: 2024-04-06T03:52:32.154+0000 7f393de86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:52:32.418 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:32 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0[103784]: 2024-04-06T03:52:32.154+0000 7f393de86700 -1 osd.0 82 *** Got signal Terminated *** 2024-04-06T03:52:32.418 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:32 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0[103784]: 2024-04-06T03:52:32.154+0000 7f393de86700 -1 osd.0 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:52:37.916 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:37 smithi094 podman[151136]: 2024-04-06 03:52:37.665402199 +0000 UTC m=+5.603902864 container died 38a0da79352f0159085a51d209f86b338f72ced672cf7a5ee5ef6cf18813bb2a (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True) 2024-04-06T03:52:38.318 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:38 smithi094 podman[151136]: 2024-04-06 03:52:38.122525467 +0000 UTC m=+6.061026148 container cleanup 38a0da79352f0159085a51d209f86b338f72ced672cf7a5ee5ef6cf18813bb2a (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, io.buildah.version=1.33.5) 2024-04-06T03:52:38.318 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:38 smithi094 bash[151136]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0 2024-04-06T03:52:38.666 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:38 smithi094 podman[151164]: 2024-04-06 03:52:38.457430445 +0000 UTC m=+0.782883387 container remove 38a0da79352f0159085a51d209f86b338f72ced672cf7a5ee5ef6cf18813bb2a (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-06T03:52:39.166 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:38 smithi094 podman[151258]: 2024-04-06 03:52:38.744655345 +0000 UTC m=+0.017930260 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:52:39.167 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:38 smithi094 podman[151258]: 2024-04-06 03:52:38.850004933 +0000 UTC m=+0.123279847 container create 7acb474cd9e17f5818035f933fd7875e69d2f90db2bd5b3cbbbb093d121bbbbf (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-06T03:52:39.167 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:39 smithi094 podman[151258]: 2024-04-06 03:52:39.077350368 +0000 UTC m=+0.350625284 container init 7acb474cd9e17f5818035f933fd7875e69d2f90db2bd5b3cbbbb093d121bbbbf (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-06T03:52:39.167 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:39 smithi094 podman[151258]: 2024-04-06 03:52:39.080408082 +0000 UTC m=+0.353683000 container start 7acb474cd9e17f5818035f933fd7875e69d2f90db2bd5b3cbbbb093d121bbbbf (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:52:39.666 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:39 smithi094 podman[151258]: 2024-04-06 03:52:39.169109281 +0000 UTC m=+0.442384213 container attach 7acb474cd9e17f5818035f933fd7875e69d2f90db2bd5b3cbbbb093d121bbbbf (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-06T03:52:39.667 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:39 smithi094 podman[151258]: 2024-04-06 03:52:39.370110046 +0000 UTC m=+0.643384965 container died 7acb474cd9e17f5818035f933fd7875e69d2f90db2bd5b3cbbbb093d121bbbbf (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-9f19235, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-06T03:52:39.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.0.service' 2024-04-06T03:52:39.981 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:39 smithi094 podman[151285]: 2024-04-06 03:52:39.939380692 +0000 UTC m=+0.564428926 container remove 7acb474cd9e17f5818035f933fd7875e69d2f90db2bd5b3cbbbb093d121bbbbf (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-0-deactivate, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:52:39.981 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:39 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.0.service: Deactivated successfully. 2024-04-06T03:52:39.981 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:39 smithi094 systemd[1]: Stopped Ceph osd.0 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:52:39.981 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 06 03:52:39 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.0.service: Consumed 8.757s CPU time. 2024-04-06T03:52:39.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-06T03:52:39.997 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-06T03:52:39.997 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-06T03:52:39.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.1 2024-04-06T03:52:40.318 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:40 smithi094 systemd[1]: Stopping Ceph osd.1 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:52:40.666 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:40 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[109656]: 2024-04-06T03:52:40.316+0000 7f96e4395700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:52:40.666 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:40 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[109656]: 2024-04-06T03:52:40.316+0000 7f96e4395700 -1 osd.1 82 *** Got signal Terminated *** 2024-04-06T03:52:40.667 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:40 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1[109656]: 2024-04-06T03:52:40.316+0000 7f96e4395700 -1 osd.1 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:52:46.144 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:45 smithi094 podman[151363]: 2024-04-06 03:52:45.86072881 +0000 UTC m=+5.631852386 container died 108ce348aea2d29a8084bba6747871db98a4e77470ac441a6599b15b3d234511 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, GIT_CLEAN=True, ceph=True, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-06T03:52:46.416 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:46 smithi094 podman[151363]: 2024-04-06 03:52:46.333830427 +0000 UTC m=+6.104954005 container cleanup 108ce348aea2d29a8084bba6747871db98a4e77470ac441a6599b15b3d234511 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1, RELEASE=squid-9f19235, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-06T03:52:46.416 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:46 smithi094 bash[151363]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1 2024-04-06T03:52:46.865 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:46 smithi094 podman[151381]: 2024-04-06 03:52:46.593611033 +0000 UTC m=+0.737736459 container remove 108ce348aea2d29a8084bba6747871db98a4e77470ac441a6599b15b3d234511 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-06T03:52:47.118 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:46 smithi094 podman[151469]: 2024-04-06 03:52:46.861760094 +0000 UTC m=+0.018747449 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:52:47.119 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:46 smithi094 podman[151469]: 2024-04-06 03:52:46.989441758 +0000 UTC m=+0.146429112 container create 3d44b7381aa97bac2466a36e9ef3826125a6b2db2eb59271bca9873d26813bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, ceph=True, RELEASE=squid-9f19235, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-06T03:52:47.416 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:47 smithi094 podman[151469]: 2024-04-06 03:52:47.221975911 +0000 UTC m=+0.378963255 container init 3d44b7381aa97bac2466a36e9ef3826125a6b2db2eb59271bca9873d26813bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-06T03:52:47.416 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:47 smithi094 podman[151469]: 2024-04-06 03:52:47.226211611 +0000 UTC m=+0.383198956 container start 3d44b7381aa97bac2466a36e9ef3826125a6b2db2eb59271bca9873d26813bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-9f19235, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-06T03:52:47.417 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:47 smithi094 podman[151469]: 2024-04-06 03:52:47.296995982 +0000 UTC m=+0.453983342 container attach 3d44b7381aa97bac2466a36e9ef3826125a6b2db2eb59271bca9873d26813bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-06T03:52:47.842 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:47 smithi094 podman[151469]: 2024-04-06 03:52:47.55993655 +0000 UTC m=+0.716923900 container died 3d44b7381aa97bac2466a36e9ef3826125a6b2db2eb59271bca9873d26813bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240403) 2024-04-06T03:52:48.102 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 06 03:52:48 smithi094 podman[151495]: 2024-04-06 03:52:48.0835093 +0000 UTC m=+0.518229955 container remove 3d44b7381aa97bac2466a36e9ef3826125a6b2db2eb59271bca9873d26813bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-1-deactivate, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-06T03:52:48.108 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.1.service' 2024-04-06T03:52:48.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-06T03:52:48.148 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-06T03:52:48.148 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-06T03:52:48.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.2 2024-04-06T03:52:48.393 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:48 smithi094 systemd[1]: Stopping Ceph osd.2 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:52:48.666 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:48 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[115533]: 2024-04-06T03:52:48.502+0000 7f4ad7197700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:52:48.666 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:48 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[115533]: 2024-04-06T03:52:48.502+0000 7f4ad7197700 -1 osd.2 82 *** Got signal Terminated *** 2024-04-06T03:52:48.667 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:48 smithi094 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2[115533]: 2024-04-06T03:52:48.502+0000 7f4ad7197700 -1 osd.2 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:52:54.305 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:54 smithi094 podman[151572]: 2024-04-06 03:52:54.016388138 +0000 UTC m=+5.646483724 container died eb8d21d9c0dc91216688cfdbce37f2c697a26d034c1ee39d123dc49eb62296f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, org.label-schema.build-date=20240403, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-06T03:52:54.666 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:54 smithi094 podman[151572]: 2024-04-06 03:52:54.402834037 +0000 UTC m=+6.032929623 container cleanup eb8d21d9c0dc91216688cfdbce37f2c697a26d034c1ee39d123dc49eb62296f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:52:54.666 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:54 smithi094 bash[151572]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2 2024-04-06T03:52:55.041 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:54 smithi094 podman[151590]: 2024-04-06 03:52:54.695853421 +0000 UTC m=+0.675707572 container remove eb8d21d9c0dc91216688cfdbce37f2c697a26d034c1ee39d123dc49eb62296f8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-06T03:52:55.325 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:55 smithi094 podman[151682]: 2024-04-06 03:52:54.944025371 +0000 UTC m=+0.021264658 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:52:55.325 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:55 smithi094 podman[151682]: 2024-04-06 03:52:55.046789826 +0000 UTC m=+0.124029104 container create e00e3f186e72155b591a12193c9e2aaefabd7a8aad813d9c59f4719f9fbb8e4d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-06T03:52:55.325 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:55 smithi094 podman[151682]: 2024-04-06 03:52:55.249259902 +0000 UTC m=+0.326499187 container init e00e3f186e72155b591a12193c9e2aaefabd7a8aad813d9c59f4719f9fbb8e4d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-04-06T03:52:55.325 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:55 smithi094 podman[151682]: 2024-04-06 03:52:55.253154321 +0000 UTC m=+0.330393607 container start e00e3f186e72155b591a12193c9e2aaefabd7a8aad813d9c59f4719f9fbb8e4d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, GIT_CLEAN=True) 2024-04-06T03:52:55.325 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:55 smithi094 podman[151682]: 2024-04-06 03:52:55.324283691 +0000 UTC m=+0.401522987 container attach e00e3f186e72155b591a12193c9e2aaefabd7a8aad813d9c59f4719f9fbb8e4d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:52:55.666 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:55 smithi094 podman[151682]: 2024-04-06 03:52:55.546726329 +0000 UTC m=+0.623965607 container died e00e3f186e72155b591a12193c9e2aaefabd7a8aad813d9c59f4719f9fbb8e4d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:52:56.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.2.service' 2024-04-06T03:52:56.141 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:56 smithi094 podman[151708]: 2024-04-06 03:52:56.094327254 +0000 UTC m=+0.542525730 container remove e00e3f186e72155b591a12193c9e2aaefabd7a8aad813d9c59f4719f9fbb8e4d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-2-deactivate, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, RELEASE=squid-9f19235, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-06T03:52:56.141 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:56 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.2.service: Deactivated successfully. 2024-04-06T03:52:56.141 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:56 smithi094 systemd[1]: Stopped Ceph osd.2 for c6509676-f3c5-11ee-b649-cb9ed24678a4. 2024-04-06T03:52:56.141 INFO:journalctl@ceph.osd.2.smithi094.stdout:Apr 06 03:52:56 smithi094 systemd[1]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.2.service: Consumed 7.477s CPU time. 2024-04-06T03:52:56.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-06T03:52:56.157 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-06T03:52:56.157 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-06T03:52:56.157 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.3 2024-04-06T03:52:56.459 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:52:56 smithi136 systemd[1]: Stopping Ceph osd.3 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:52:56.847 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:52:56 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:52:56.457+0000 7fa77f9be700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:52:56.848 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:52:56 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:52:56.457+0000 7fa77f9be700 -1 osd.3 82 *** Got signal Terminated *** 2024-04-06T03:52:56.848 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:52:56 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3[91963]: 2024-04-06T03:52:56.457+0000 7fa77f9be700 -1 osd.3 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:53:02.098 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:01 smithi136 podman[118337]: 2024-04-06 03:53:01.840678543 +0000 UTC m=+5.497383631 container died 0bdf47d78dc3cbb4550a4b621db935a574c2aa5ca9ddebc735d5e442c8dd4851 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-06T03:53:02.497 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:02 smithi136 podman[118337]: 2024-04-06 03:53:02.308201893 +0000 UTC m=+5.964906980 container cleanup 0bdf47d78dc3cbb4550a4b621db935a574c2aa5ca9ddebc735d5e442c8dd4851 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:53:02.497 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:02 smithi136 bash[118337]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3 2024-04-06T03:53:02.847 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:02 smithi136 podman[118367]: 2024-04-06 03:53:02.601417879 +0000 UTC m=+0.757141409 container remove 0bdf47d78dc3cbb4550a4b621db935a574c2aa5ca9ddebc735d5e442c8dd4851 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.build-date=20240403, io.buildah.version=1.33.5) 2024-04-06T03:53:02.848 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:02 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:02.495+0000 7f3d2aa20700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.978470+0000 front 2024-04-06T03:52:36.978503+0000 (oldest deadline 2024-04-06T03:53:01.678423+0000) 2024-04-06T03:53:03.175 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:02 smithi136 podman[118461]: 2024-04-06 03:53:02.824424386 +0000 UTC m=+0.017625174 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:53:03.175 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:02 smithi136 podman[118461]: 2024-04-06 03:53:02.95239463 +0000 UTC m=+0.145595449 container create fd231a29b83802053916a0a4595bff43cd0be801bfb8f3d9206356e6b1ba000d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-9f19235) 2024-04-06T03:53:03.175 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:03 smithi136 podman[118461]: 2024-04-06 03:53:03.171541893 +0000 UTC m=+0.364742666 container init fd231a29b83802053916a0a4595bff43cd0be801bfb8f3d9206356e6b1ba000d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240403) 2024-04-06T03:53:03.175 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:03 smithi136 podman[118461]: 2024-04-06 03:53:03.174609223 +0000 UTC m=+0.367810002 container start fd231a29b83802053916a0a4595bff43cd0be801bfb8f3d9206356e6b1ba000d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, GIT_BRANCH=HEAD) 2024-04-06T03:53:03.472 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:03 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:03.411+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:03.473 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:03 smithi136 podman[118461]: 2024-04-06 03:53:03.263353031 +0000 UTC m=+0.456553804 container attach fd231a29b83802053916a0a4595bff43cd0be801bfb8f3d9206356e6b1ba000d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-06T03:53:03.772 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:03 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:03.532+0000 7f3d2aa20700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.978470+0000 front 2024-04-06T03:52:36.978503+0000 (oldest deadline 2024-04-06T03:53:01.678423+0000) 2024-04-06T03:53:03.773 INFO:journalctl@ceph.osd.3.smithi136.stdout:Apr 06 03:53:03 smithi136 podman[118461]: 2024-04-06 03:53:03.472377031 +0000 UTC m=+0.665577820 container died fd231a29b83802053916a0a4595bff43cd0be801bfb8f3d9206356e6b1ba000d (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, GIT_CLEAN=True) 2024-04-06T03:53:04.074 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.3.service' 2024-04-06T03:53:04.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-06T03:53:04.115 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-06T03:53:04.115 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-06T03:53:04.115 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.4 2024-04-06T03:53:04.347 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:04 smithi136 systemd[1]: Stopping Ceph osd.4 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:53:04.847 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:04 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:04.393+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:04.848 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:04 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:04.453+0000 7f3d2dc2b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:53:04.848 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:04 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:04.453+0000 7f3d2dc2b700 -1 osd.4 82 *** Got signal Terminated *** 2024-04-06T03:53:04.848 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:04 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:04.453+0000 7f3d2dc2b700 -1 osd.4 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:53:04.848 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:04 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:04.529+0000 7f3d2aa20700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.978470+0000 front 2024-04-06T03:52:36.978503+0000 (oldest deadline 2024-04-06T03:53:01.678423+0000) 2024-04-06T03:53:05.847 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:05 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:05.512+0000 7f3d2aa20700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.978470+0000 front 2024-04-06T03:52:36.978503+0000 (oldest deadline 2024-04-06T03:53:01.678423+0000) 2024-04-06T03:53:05.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:05 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:05.410+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:06.847 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:06 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:06.530+0000 7f3d2aa20700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.978470+0000 front 2024-04-06T03:52:36.978503+0000 (oldest deadline 2024-04-06T03:53:01.678423+0000) 2024-04-06T03:53:06.910 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:06 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:06.412+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:07.847 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:07 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:07.567+0000 7f3d2aa20700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.978470+0000 front 2024-04-06T03:52:36.978503+0000 (oldest deadline 2024-04-06T03:53:01.678423+0000) 2024-04-06T03:53:07.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:07 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:07.367+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:07.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:07 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:07.367+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:08.847 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:08 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4[97114]: 2024-04-06T03:53:08.552+0000 7f3d2aa20700 -1 osd.4 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.978470+0000 front 2024-04-06T03:52:36.978503+0000 (oldest deadline 2024-04-06T03:53:01.678423+0000) 2024-04-06T03:53:08.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:08 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:08.397+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:08.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:08 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:08.397+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:09.846 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:09 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:09.430+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:09.846 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:09 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:09.430+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:10.097 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:09 smithi136 podman[118566]: 2024-04-06 03:53:09.84456495 +0000 UTC m=+5.502885753 container died c31367e1c2d53d57c59534dfa2bcf245ed64c52613f8cbe7b906e624fe08fef0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-06T03:53:10.411 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:10 smithi136 podman[118566]: 2024-04-06 03:53:10.211605494 +0000 UTC m=+5.869926294 container cleanup c31367e1c2d53d57c59534dfa2bcf245ed64c52613f8cbe7b906e624fe08fef0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-9f19235, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:53:10.412 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:10 smithi136 bash[118566]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4 2024-04-06T03:53:10.775 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:10 smithi136 podman[118583]: 2024-04-06 03:53:10.463335516 +0000 UTC m=+0.614751808 container remove c31367e1c2d53d57c59534dfa2bcf245ed64c52613f8cbe7b906e624fe08fef0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-9f19235, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:53:10.775 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:10 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:10.410+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:10.775 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:10 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:10.410+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:11.092 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:10 smithi136 podman[118676]: 2024-04-06 03:53:10.677475179 +0000 UTC m=+0.017488905 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:53:11.092 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:10 smithi136 podman[118676]: 2024-04-06 03:53:10.789035806 +0000 UTC m=+0.129049525 container create 4142ff18ebec8a82e3c33393167be26c0bdb9f39554c26109626aef35a41f130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:53:11.092 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:11 smithi136 podman[118676]: 2024-04-06 03:53:11.008402553 +0000 UTC m=+0.348416271 container init 4142ff18ebec8a82e3c33393167be26c0bdb9f39554c26109626aef35a41f130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403) 2024-04-06T03:53:11.093 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:11 smithi136 podman[118676]: 2024-04-06 03:53:11.011719768 +0000 UTC m=+0.351733489 container start 4142ff18ebec8a82e3c33393167be26c0bdb9f39554c26109626aef35a41f130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-9f19235) 2024-04-06T03:53:11.093 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:11 smithi136 podman[118676]: 2024-04-06 03:53:11.091538493 +0000 UTC m=+0.431552222 container attach 4142ff18ebec8a82e3c33393167be26c0bdb9f39554c26109626aef35a41f130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403) 2024-04-06T03:53:11.347 INFO:journalctl@ceph.osd.4.smithi136.stdout:Apr 06 03:53:11 smithi136 podman[118676]: 2024-04-06 03:53:11.305876649 +0000 UTC m=+0.645890369 container died 4142ff18ebec8a82e3c33393167be26c0bdb9f39554c26109626aef35a41f130 (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-4-deactivate, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-9f19235, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:53:11.847 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:11 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:11.416+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:11.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:11 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:11.416+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:11.935 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.4.service' 2024-04-06T03:53:11.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-06T03:53:11.976 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-06T03:53:11.976 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-06T03:53:11.976 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.5 2024-04-06T03:53:12.282 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:12 smithi136 systemd[1]: Stopping Ceph osd.5 for c6509676-f3c5-11ee-b649-cb9ed24678a4... 2024-04-06T03:53:12.597 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:12 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:12.281+0000 7fc0a9382700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-06T03:53:12.598 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:12 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:12.281+0000 7fc0a9382700 -1 osd.5 82 *** Got signal Terminated *** 2024-04-06T03:53:12.598 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:12 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:12.281+0000 7fc0a9382700 -1 osd.5 82 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-06T03:53:12.598 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:12 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:12.375+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:12.598 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:12 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:12.375+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:13.847 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:13 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:13.390+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:13.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:13 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:13.390+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:14.597 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:14 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:14.368+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:14.598 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:14 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:14.368+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:14.598 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:14 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:14.368+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6822 osd.2 since back 2024-04-06T03:52:49.623072+0000 front 2024-04-06T03:52:49.623088+0000 (oldest deadline 2024-04-06T03:53:14.323041+0000) 2024-04-06T03:53:15.847 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:15 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:15.412+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:15.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:15 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:15.412+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:15.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:15 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:15.412+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6822 osd.2 since back 2024-04-06T03:52:49.623072+0000 front 2024-04-06T03:52:49.623088+0000 (oldest deadline 2024-04-06T03:53:14.323041+0000) 2024-04-06T03:53:16.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:16 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:16.436+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-06T03:52:36.722035+0000 front 2024-04-06T03:52:36.721891+0000 (oldest deadline 2024-04-06T03:53:02.621963+0000) 2024-04-06T03:53:16.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:16 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:16.436+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6814 osd.1 since back 2024-04-06T03:52:42.622414+0000 front 2024-04-06T03:52:42.622390+0000 (oldest deadline 2024-04-06T03:53:07.322287+0000) 2024-04-06T03:53:16.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:16 smithi136 ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5[102112]: 2024-04-06T03:53:16.436+0000 7fc0a216f700 -1 osd.5 82 heartbeat_check: no reply from 172.21.15.94:6822 osd.2 since back 2024-04-06T03:52:49.623072+0000 front 2024-04-06T03:52:49.623088+0000 (oldest deadline 2024-04-06T03:53:14.323041+0000) 2024-04-06T03:53:17.875 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:17 smithi136 podman[118781]: 2024-04-06 03:53:17.613169035 +0000 UTC m=+5.423200653 container died 16386e8f8583d884ca32bdbeccc2c5dcf72e51a2b42696e719da0d39095b151e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-9f19235) 2024-04-06T03:53:18.166 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:17 smithi136 podman[118781]: 2024-04-06 03:53:17.95578733 +0000 UTC m=+5.765818946 container cleanup 16386e8f8583d884ca32bdbeccc2c5dcf72e51a2b42696e719da0d39095b151e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-9f19235, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-06T03:53:18.166 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:17 smithi136 bash[118781]: ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5 2024-04-06T03:53:18.571 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:18 smithi136 podman[118800]: 2024-04-06 03:53:18.165303798 +0000 UTC m=+0.546846576 container remove 16386e8f8583d884ca32bdbeccc2c5dcf72e51a2b42696e719da0d39095b151e (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-9f19235, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-06T03:53:18.844 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:18 smithi136 podman[118893]: 2024-04-06 03:53:18.473982149 +0000 UTC m=+0.018133599 image pull a5511c7deba913038976ce236d2528051568e0aa5cd4df4989ad160a0a4c3880 quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c 2024-04-06T03:53:18.844 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:18 smithi136 podman[118893]: 2024-04-06 03:53:18.59956437 +0000 UTC m=+0.143715864 container create 54823a8eea9abcb47c498dc5b4bf3256f7efca24da5b957faab22759e0e18bce (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, RELEASE=squid-9f19235, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-06T03:53:18.844 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:18 smithi136 podman[118893]: 2024-04-06 03:53:18.843833374 +0000 UTC m=+0.387984823 container init 54823a8eea9abcb47c498dc5b4bf3256f7efca24da5b957faab22759e0e18bce (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=squid-9f19235) 2024-04-06T03:53:19.097 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:18 smithi136 podman[118893]: 2024-04-06 03:53:18.846928563 +0000 UTC m=+0.391080021 container start 54823a8eea9abcb47c498dc5b4bf3256f7efca24da5b957faab22759e0e18bce (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-9f19235, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-06T03:53:19.098 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:18 smithi136 podman[118893]: 2024-04-06 03:53:18.910424911 +0000 UTC m=+0.454576377 container attach 54823a8eea9abcb47c498dc5b4bf3256f7efca24da5b957faab22759e0e18bce (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-9f19235) 2024-04-06T03:53:19.438 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 06 03:53:19 smithi136 podman[118893]: 2024-04-06 03:53:19.147020103 +0000 UTC m=+0.691171560 container died 54823a8eea9abcb47c498dc5b4bf3256f7efca24da5b957faab22759e0e18bce (image=quay.ceph.io/ceph-ci/ceph@sha256:a592ef0f2cc60f3f7bee762d48a80bb237e35213427028fdb29a2b5d27584e3c, name=ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-9f19235, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-06T03:53:19.713 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6509676-f3c5-11ee-b649-cb9ed24678a4@osd.5.service' 2024-04-06T03:53:19.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-06T03:53:19.755 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-06T03:53:19.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 --force --keep-logs 2024-04-06T03:53:43.307 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 --force --keep-logs 2024-04-06T03:53:56.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-06T03:53:56.433 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-06T03:53:56.463 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-06T03:53:56.464 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/crash to /home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944/remote/smithi094/crash 2024-04-06T03:53:56.465 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/crash -- . 2024-04-06T03:53:56.503 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-06T03:53:56.503 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-04-06T03:53:56.505 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/crash to /home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944/remote/smithi136/crash 2024-04-06T03:53:56.507 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/crash -- . 2024-04-06T03:53:56.534 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-06T03:53:56.534 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-04-06T03:53:56.535 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-06T03:53:56.535 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-06T03:53:56.584 INFO:teuthology.orchestra.run.smithi094.stdout:1712374934.1108444 mon.smithi094 (mon.0) 914 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-06T03:53:56.584 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-06T03:53:56.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[SEC\]' /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-06T03:53:56.658 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]' /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-06T03:53:56.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[WRN\]' /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-06T03:53:56.804 INFO:teuthology.orchestra.run.smithi094.stdout:1712374934.1108444 mon.smithi094 (mon.0) 914 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-06T03:53:56.805 INFO:tasks.cephadm:Compressing logs... 2024-04-06T03:53:56.805 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-06T03:53:56.847 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-06T03:53:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-06T03:53:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:: No such file or directory 2024-04-06T03:53:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.audit.log 2024-04-06T03:53:56.871 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.1.log 2024-04-06T03:53:56.872 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mgr.smithi094.purlzp.log 2024-04-06T03:53:56.872 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.audit.log: /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi094.qogyzw.log 2024-04-06T03:53:56.873 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.0.log 2024-04-06T03:53:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mgr.smithi094.purlzp.log: /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi094.qogyzw.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.cephadm.log 2024-04-06T03:53:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mon.smithi094.log 2024-04-06T03:53:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log 2024-04-06T03:53:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mon.smithi094.log: 85.4% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-06T03:53:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.2.log 2024-04-06T03:53:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log: 91.0% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.audit.log.gz 2024-04-06T03:53:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-volume.log 2024-04-06T03:53:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.2.log: 87.7% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log.gz 2024-04-06T03:53:56.881 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi094.kvxcqo.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:: No such file or directory 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi136.trdonf.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mgr.smithi136.jagvwt.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi136.trdonf.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.audit.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mgr.smithi136.jagvwt.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.4.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.5.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.3.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mon.smithi136.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.cephadm.log 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-06T03:53:56.883 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mon.smithi136.log: gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log 2024-04-06T03:53:56.884 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-06T03:53:56.884 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-volume.log 2024-04-06T03:53:56.884 INFO:teuthology.orchestra.run.smithi136.stderr: 91.2% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.audit.log.gz 2024-04-06T03:53:56.884 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi136.hbjgtw.log 2024-04-06T03:53:56.884 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-volume.log: /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log: 87.7% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph.log.gz 2024-04-06T03:53:56.901 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-volume.log: /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi094.kvxcqo.log: 91.1% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-volume.log.gz 2024-04-06T03:53:56.902 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi136.hbjgtw.log: 91.0% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-volume.log.gz 2024-04-06T03:53:56.910 INFO:teuthology.orchestra.run.smithi094.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-06T03:53:56.944 INFO:teuthology.orchestra.run.smithi136.stderr: 89.6% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mgr.smithi136.jagvwt.log.gz 2024-04-06T03:53:57.137 INFO:teuthology.orchestra.run.smithi136.stderr: 92.6% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mon.smithi136.log.gz 2024-04-06T03:53:57.235 INFO:teuthology.orchestra.run.smithi094.stderr: 88.3% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mgr.smithi094.purlzp.log.gz 2024-04-06T03:53:57.927 INFO:teuthology.orchestra.run.smithi094.stderr: 90.5% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mon.smithi094.log.gz 2024-04-06T03:54:01.582 INFO:teuthology.orchestra.run.smithi094.stderr: 93.5% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.2.log.gz 2024-04-06T03:54:02.029 INFO:teuthology.orchestra.run.smithi136.stderr: 94.9% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi136.trdonf.log.gz 2024-04-06T03:54:02.065 INFO:teuthology.orchestra.run.smithi136.stderr: 94.9% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi136.hbjgtw.log.gz 2024-04-06T03:54:02.268 INFO:teuthology.orchestra.run.smithi094.stderr: 94.9% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi094.kvxcqo.log.gz 2024-04-06T03:54:02.550 INFO:teuthology.orchestra.run.smithi094.stderr: 93.4% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.1.log.gz 2024-04-06T03:54:03.079 INFO:teuthology.orchestra.run.smithi136.stderr: 93.3% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.4.log.gz 2024-04-06T03:54:03.556 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.3.log.gz 2024-04-06T03:54:03.752 INFO:teuthology.orchestra.run.smithi094.stderr: 93.4% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.0.log.gz 2024-04-06T03:54:04.459 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-osd.5.log.gz 2024-04-06T03:54:04.461 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-06T03:54:04.462 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m7.600s 2024-04-06T03:54:04.462 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m30.120s 2024-04-06T03:54:04.462 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m1.020s 2024-04-06T03:55:29.300 INFO:teuthology.orchestra.run.smithi094.stderr: 92.6% -- replaced with /var/log/ceph/c6509676-f3c5-11ee-b649-cb9ed24678a4/ceph-mds.cephfs.smithi094.qogyzw.log.gz 2024-04-06T03:55:29.302 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-06T03:55:29.302 INFO:teuthology.orchestra.run.smithi094.stderr:real 1m32.443s 2024-04-06T03:55:29.302 INFO:teuthology.orchestra.run.smithi094.stderr:user 1m52.693s 2024-04-06T03:55:29.302 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m3.542s 2024-04-06T03:55:29.302 INFO:tasks.cephadm:Archiving logs... 2024-04-06T03:55:29.303 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944/remote/smithi094/log 2024-04-06T03:55:29.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-06T03:55:47.379 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944/remote/smithi136/log 2024-04-06T03:55:47.380 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-06T03:55:51.428 INFO:tasks.cephadm:Removing cluster... 2024-04-06T03:55:51.428 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 --force 2024-04-06T03:55:52.037 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6509676-f3c5-11ee-b649-cb9ed24678a4 --force 2024-04-06T03:55:52.572 INFO:tasks.cephadm:Removing cephadm ... 2024-04-06T03:55:52.572 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-06T03:55:52.589 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-06T03:55:52.604 INFO:tasks.cephadm:Teardown complete 2024-04-06T03:55:52.604 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-06T03:55:52.616 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-06T03:55:52.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-06T03:55:52.631 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-06T03:55:52.717 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-06T03:55:52.728 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-06T03:55:52.728 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-06T03:55:52.731 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-06T03:55:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-06T03:55:52.744 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-04-06T03:55:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-06T03:55:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-06T03:55:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 105 -319us[ -333us] +/- 81ms 2024-04-06T03:55:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -187us[ -195us] +/- 64ms 2024-04-06T03:55:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +519us[ +511us] +/- 68ms 2024-04-06T03:55:52.747 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:55:52.749 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-06T03:55:52.749 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-04-06T03:55:52.749 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 -337us[ -322us] +/- 82ms 2024-04-06T03:55:52.749 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -188us[ -174us] +/- 64ms 2024-04-06T03:55:52.749 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +514us[ +528us] +/- 68ms 2024-04-06T03:55:52.749 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-06T03:55:52.749 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-06T03:55:52.764 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-06T03:55:52.764 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-06T03:55:52.780 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-06T03:55:52.874 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-06T03:55:52.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-06T03:55:52.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:55:52.982 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-06T03:55:53.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:55:53.013 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-06T03:55:53.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-06T03:55:53.040 INFO:teuthology.task.internal:Duration was 2081.070789 seconds 2024-04-06T03:55:53.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-06T03:55:53.057 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-06T03:55:53.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-06T03:55:53.059 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-06T03:55:53.100 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-06T03:55:53.103 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-06T03:55:53.457 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-06T03:55:53.457 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-04-06T03:55:53.457 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-06T03:55:53.517 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-04-06T03:55:53.517 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-06T03:55:53.542 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-06T03:55:53.543 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-06T03:55:53.559 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-06T03:55:53.611 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-06T03:55:53.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-06T03:55:53.629 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-06T03:55:54.260 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-06T03:55:54.273 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-06T03:55:54.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-06T03:55:54.300 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-06T03:55:54.330 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-06T03:55:54.348 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-06T03:55:54.350 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-06T03:55:54.373 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-04-06T03:55:54.397 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-04-06T03:55:54.413 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-06T03:55:54.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:55:54.444 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-06T03:55:54.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-06T03:55:54.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-06T03:55:54.480 INFO:teuthology.task.internal:Transferring archived files... 2024-04-06T03:55:54.481 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944/remote/smithi094 2024-04-06T03:55:54.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-06T03:55:54.536 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944/remote/smithi136 2024-04-06T03:55:54.536 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-06T03:55:54.577 INFO:teuthology.task.internal:Removing archive directory... 2024-04-06T03:55:54.578 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-06T03:55:54.580 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-06T03:55:54.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-06T03:55:54.640 INFO:teuthology.task.internal:Not uploading archives. 2024-04-06T03:55:54.640 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-06T03:55:54.658 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-06T03:55:54.658 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-06T03:55:54.661 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-06T03:55:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 6 03:55 /home/ubuntu/cephtest 2024-04-06T03:55:54.683 INFO:teuthology.orchestra.run.smithi136.stdout: 265917 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 6 03:55 /home/ubuntu/cephtest 2024-04-06T03:55:54.684 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-06T03:55:54.700 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-06T03:55:54.731 INFO:teuthology.nuke:Checking targets against current locks 2024-04-06T03:55:54.758 DEBUG:teuthology.nuke:shortname: smithi094 2024-04-06T03:55:54.759 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-06T03:55:54.782 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-06 03:12:52.092289', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBP6SMWH+MXi6ihK37d/buMQ5Cj8s0JaeT5CJ0ALb5Dbkd42VTFVza4obvVBO52QC1WMGi8VAbe0+QrbvRwpL8='} 2024-04-06T03:55:54.787 DEBUG:teuthology.nuke:shortname: smithi136 2024-04-06T03:55:54.787 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-06T03:55:54.812 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-05_21:08:03-orch-squid-distro-default-smithi/7641944', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-06 03:12:52.090642', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5WSDxOO7mZqpUabI27URiunB+kqa+EEogb7FWfw54oyIZo6dzpfK23aXc6x9GGFPahLiwO82CZZOIRr99ALdU='} 2024-04-06T03:55:54.842 INFO:teuthology.orchestra.console.smithi094:Power off 2024-04-06T03:55:54.842 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-06T03:55:54.865 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2024-04-06T03:55:54.865 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-06T03:55:54.881 INFO:teuthology.orchestra.console.smithi136:Power off 2024-04-06T03:55:54.882 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-06T03:55:54.892 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-04-06T03:55:54.905 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2024-04-06T03:55:54.905 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-06T03:55:54.951 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-04-06T03:55:58.893 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-06T03:55:58.952 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-06T03:55:59.004 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-04-06T03:55:59.062 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-04-06T03:56:03.006 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-06T03:56:03.063 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-06T03:56:03.119 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-04-06T03:56:03.173 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-04-06T03:56:03.221 INFO:teuthology.orchestra.console.smithi094:Power off completed 2024-04-06T03:56:07.177 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-06T03:56:07.287 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2024-04-06T03:56:07.388 INFO:teuthology.orchestra.console.smithi136:Power off completed 2024-04-06T03:56:07.492 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2081.070789337158 failure_reason: '"1712374934.1108444 mon.smithi094 (mon.0) 914 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-04-06T03:56:07.493 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-06T03:56:07.591 INFO:teuthology.run:FAIL